Commit Graph

284 Commits

Author SHA1 Message Date
Colin Walters
13f307da36 livefs: Update /usr/lib/{passwd,group}, run systemd-tmpfiles
This gets us to supporting live installs of `httpd` and many
other services.  I also tried `postgresql-server` for example.

Closes: https://github.com/projectatomic/rpm-ostree/issues/962

Closes: #996
Approved by: jlebon
2017-09-25 15:10:15 +00:00
Colin Walters
e066dc93c8 tests: Skip running introspection tests if built with ASAN
This is the rpm-ostree equivalent of
<47b4dd1b38>

Unfortunately, introspection uses `dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.

Prep for syncing PAPR config with ostree.

Closes: #1000
Approved by: jlebon
2017-09-24 13:32:00 +00:00
Jonathan Lebon
167fd4c1dd libpriv/scripts: use rofiles-fuse on /etc too
We were directly bind mounting the checked out `/usr/etc` onto `/etc`
which was exposing us to corruption from scriptlets. Since we already
have an rofiles-fuse mount for `/usr`, let's just re-use its `etc/`
subdir and bind mount that instead.

Closes: #1003
Approved by: cgwalters
2017-09-24 13:11:21 +00:00
Jonathan Lebon
f089b8de1f libpriv/postprocess: also delete semanage lock files
We don't need those in the tree, so let's nuke them. This also fixes
subtle compatibility issues between hardlinks and lock files (see #999).

Closes: #1002
Approved by: cgwalters
2017-09-21 15:28:55 +00:00
Colin Walters
80de17623f tests/livefs: Make sure /usr/bin/ls is executable
Oh man was I confused how this happened; thought it was a bug in some livefs
changes I was working on.

Closes: #995
Approved by: jlebon
2017-09-15 19:10:24 +00:00
Ruixin
56ef177235 tests/core: allow merging identical files during installation
This PR uses https://github.com/ostreedev/ostree/pull/1156
to ensure that when installing packages containing files with
exact same content, the files will be merged.

When installing packages containing conflicting files,
the error will still be detected at the ostree side.

The checkout overwrite option at rpm-ostree side is also
modified to accomodate the changes made in ostree side.

A test is added for regression
2017-09-13 20:41:15 +00:00
Jonathan Lebon
4ad3ea58c6 app: smarter deployment change detection
Commands like `upgrade` and `deploy` need to know if a new deployment
was actually laid down so that it may print a pkg diff if so. This is
implemented by listening for changes to the DefaultDeployment D-Bus
property. D-Bus emits a signal when the deployment variant changes
value.

However, in #595, with the introduction of `pending-*` related keys, the
deployment variant no longer represents data solely tied to that
specific deployment. In this case, because `deploy` operations currently
set the ref to the resolved checksum, it can happen that deploying the
same base commit when the current refspec *isn't* pointing to that base
commit will result in the `pending-*` keys dropping out and a default
deployment change notification going out.

In this patch, we strengthen how we determine whether a new deployment
was laid down by actually looking at the deployment id, rather than just
assuming that a change to the property implies a new deployment.

Closes: #981

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Jonathan Lebon
077d7c1a9e daemon/deploy: allow redeploying livefs booted commit
Also something I noticed while working on #981. When sitting on a livefs
commit, once a user does `rpm-ostree cleanup --pending --rollback`, it's
impossible to redeploy the same booted commit. Let's allow users to do
this.

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Jonathan Lebon
c08ca8f922 daemon/upgrader: make use of override-commit-ids
Currently, when setting the `override-commit` key in the origin, the
upgrader pulls that commit checksum directly and then updates the
refspec to point to it. This behaviour was inherited from its ostree
version; at the time it was implemented, the pull code didn't support
passing a specific commit for a given refspec. However, we now have
the override-commit-ids option, which will make libostree update the ref
for us.

We change the code here to make use of it and simplify the function.
This also fixes the corner case of local branches: we shouldn't change
the ref if we're on a local branch. This is actually what drove me to
this patch as I was debugging #981.

(Aside: I'm still not convinced updating the refspec is always the
correct thing to do even in the remote case, though it's a bit messy to
disentangle).

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Colin Walters
f113fc5e27 Rework treecompose kernel processing
Prep for changing `boot_location: new` to use `/usr/lib/ostree-boot`
and `/usr/lib/modules`.  Rework our kernel postprocessing
so that we unify the `boot_location` handling with initramfs generation.

Instead of doing the initramfs first in postprocessing, we do it nearly last,
after e.g. `etc` is renamed to `usr/etc`. This has some consequences, such as
the fact that `run_bwrap_mutably()` is now called in both situations. In
general, our handling of `etc` is inconsistent, although understandably so.

As part of this, I finally got around to implementing the bit from
https://github.com/systemd/systemd/pull/4174 however suboptimal it is; need the
unified core so we can cleanly ignore the posttrans like we do others.  We
intentionally keep the file around in the generated tree so that installing a
kernel RPM per client doesn't try to do any of this either.

This all gets folded together so that the logic for handling the bootloader gets
simpler - in the Fedora case, we now know to find kernels in `/usr/lib/modules`
and can ignore `/boot`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
93d3fc6617 tests/compose: Add env variables to force caching and fast fail
This helped me debug/fix the tests faster.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
150de0087e scripts: Support overriding lua, use for glibc-all-langpacks
Today in Fedora the `glibc-all-langpacks.posttrans` is implemented
in lua, for no good reason.  See:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Since that's stalled out, let's add support for overrides.  This
is obviously a much bigger step with more long term maintenance
implications over our current "ignore scripts" list.  But we can't
block either.

This is needed for unified core work:
https://github.com/projectatomic/rpm-ostree/issues/729

(We also override `fedora-release-atomichost` but I'll likely
 submit a patch for that upstream)

Closes: #980
Approved by: jlebon
2017-09-08 18:02:22 +00:00
Colin Walters
e40eaebbe0 core: Use fd-relative access to rpmdb
I was linking to this code from elsewhere and noticed that
for our hardlink breaks we were not using fd-relative even
though we can.  Down the line if we fork librpm into a separate
process and do e.g. `--dbpath=.` it'll do it too.

(Side note, I verified that commenting out the hardlink breaking
 here was caught by the `ostree fsck` I added to the test suite)

Closes: #979
Approved by: jlebon
2017-09-07 22:54:40 +00:00
Colin Walters
b5c98ec4f1 daemon: Add a policy for reload-config
This fixes `rpm-ostree reload` as root, and supports configuring
it to be enabled for other users as well.  This was overlooked
in the polkit work originally.

Closes: https://github.com/projectatomic/rpm-ostree/issues/976

Closes: #977
Approved by: jlebon
2017-09-07 15:11:00 +00:00
Jonathan Lebon
b0c75ae900 vmcheck: use --selinux-policy when committing
Let's make SELinux and atomic-host-tests happy with trees we concoct
ourselves by using the new --selinux-policy. (Specifically, we want our
sync'ed binaries to have install_exec_t).

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Jonathan Lebon
50ecc04cb6 overlay.sh: drop hack for backup passwd files
We shouldn't need this anymore.

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Jonathan Lebon
b22d7d764d vmcheck: rename build.sh to install.sh
I had meant for this to be in the other PR#968. I originally did both
`make` and `make install` there, but now it only does `make install`, so
let's just rename it to make that more obvious.

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Colin Walters
c093a587a2 tests: Add a test case for lua scripts
This is currently unsupported, let's add a test case. Prep for support for
script overrides.

Closes: #973
Approved by: jlebon
2017-09-07 14:09:47 +00:00
Colin Walters
20abc85243 unpacker: Translate /boot → /usr/lib/ostree-boot
At one point `rpm-ostree install libvirt` dragged in libguestfs which in turn
brought in `syslinux-extlinux-nonlinux` which has files in `/boot/extlinux`,
which we rejected.  (That dependency chain appears to have been fixed currently)

For the general case, this is just a partial fix in that we haven't nailed down
the semantics of how updates for `/boot` work.  But in this particular case,
we'll just break libguestfs' `extlinux` verb, which I'm OK with.

Another case is `fwupdate-efi` - we require manual intervention to copy the
data into `/boot` after installing the package.

This is also preparation for [unified core](https://github.com/projectatomic/rpm-ostree/issues/729)
in that we now ensure imported kernels don't end up in `/boot` unless
explicitly configured.

Closes: https://github.com/projectatomic/rpm-ostree/issues/853

Closes: #969
Approved by: jlebon
2017-09-07 13:30:36 +00:00
Ruixin Bao
d6b3217a86 lib/core: Detect file name conflict before rpmdb
This PR uses https://github.com/ostreedev/ostree/pull/1116
to ensure that when installing packages containing conflicting files, the
error will be detected at the ostree side, rather than when
writing the rpm database.

Closes: https://github.com/projectatomic/rpm-ostree/issues/365

Closes: #974
Approved by: cgwalters
2017-09-07 13:13:12 +00:00
Jonathan Lebon
7fbf9f32d4 hacking: make it easier to use a custom tree
Let's make using a custom install tree easier and document the process.
We split out the insttree step into `build.sh` so that we no longer have
to `flock(1)` around it, and also share between `overlay.sh` and
`sync.sh`.

Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Colin Walters
213a656ddf postprocess: Unlink our treecompose-post out of the final /bin
Another variant of <https://github.com/projectatomic/rpm-ostree/pull/822> 😭

Closes: #964
Approved by: jlebon
2017-08-31 03:06:11 +00:00
Colin Walters
3047513509 core: Run %post before %posttrans
While working on unified core and the Fedora Atomic Host content set, I hit a
dependency between `docker.posttrans` which tries to read `/etc/os-release`, and
`fedora-release-atomichost.post` which creates that symlink.

It seems best practice to me to run `%post`s strictly before
`%posttrans`; we're not likely to do parallelization anytime
soon anyways.

While here I cleaned things up by having an enum for the script kind,
rather than multiple functions, otherwise we would have had another
wrapper in core.c.

Closes: #963
Approved by: jlebon
2017-08-30 15:50:09 +00:00
Jonathan Lebon
2f6f8a910d libvm: set up ControlPath socket in /var/tmp
It turns out there's a much smaller limit than PATH_MAX for Unix
sockets. On Linux, it's 108 characters. It took me some time to figure
out why `vmcheck` would sometimes fail depending on where the src
directory is and how ${topsrcdir} is defined. Let's just make things
safer by just using /var/tmp.

Closes: #949
Approved by: cgwalters
2017-08-24 22:12:17 +00:00
Colin Walters
82f95e7761 daemon/upgrader: Use new libostree timestamp-check option
Since we have a copy of this libostree code, pick up the new
changes from <https://github.com/ostreedev/ostree/pull/1055>.

Note the added test doesn't really test our logic since
we're only doing local pulls, but at least we have something.

Closes: #932
Approved by: jlebon
2017-08-22 16:08:21 +00:00
Colin Walters
932520a0c7 tests/compose: Be a bit more verbose
This would have helped me debug an issue that caused the first compose to fail.

Closes: #931
Approved by: jlebon
2017-08-22 01:02:52 +00:00
Colin Walters
8bb604ce67 tree-wide: Port to ostree_repo_{open,create}_at()
A lot of code gets nicer.

Closes: #922
Approved by: jlebon
2017-08-17 15:28:14 +00:00
Colin Walters
b6705f3feb daemon: Check for updated rpms when upgrading
This closes a longstanding bug - since package layering first
landed, we only checked for newer RPMs if the base tree changed.
In some scenarios like RHELAH, this doesn't matter much by default
since they move at the same cadence.  Except if you use EPEL for example.
In Fedora, today the FAH releases are async of the rpm-md repos, and
there's also COPR which can update more than once a day even.

We should check for both update sources. Luckily we'd already introduced logic
for this in the treecompose case (checksumming the depsolved package sack). We
just need to start using it for client side assembly too.

Closes: https://github.com/projectatomic/rpm-ostree/issues/391

Closes: #911
Approved by: jlebon
2017-08-16 15:33:11 +00:00
Jonathan Lebon
8d92273f3d vmcheck: create vmcheck branch and rebase onto it
I'm working on getting the vmcheck suite working as part of Fedora's new
CI pipeline. In that context, we want to test the deliverable as it is,
i.e. with SKIP_VMOVERLAY=1. For compatibility with the testsuite, we
ensure that the machine is on the vmcheck branch before starting the
tests.

Eventually, we should try to make the vmcheck suite runnable outside of
a configured build directory to make it easier to re-use in such
contexts.

Closes: #917
Approved by: cgwalters
2017-08-15 15:40:35 +00:00
Jonathan Lebon
188b45146c multitest.py: error if no hosts given
Otherwise it'll just hang waiting for a non-existent host.

Closes: #917
Approved by: cgwalters
2017-08-15 15:40:35 +00:00
Colin Walters
787c880b64 bin/rebase: Add -b and -m options
The rebase command syntax has confused people a lot.  Let's follow
git here and add a `-b/--branch` option and encourage people to use
that.  The case of switching remotes is `-m/--remote`; it's definitely
unfortunate that `-r` is already taken for `--reboot`.

One thing I'm a little bit unhappy about is how we're doing logic
on the client side here.  Changing the DBus API for this would
also be awkward though.

Closes: https://github.com/projectatomic/rpm-ostree/issues/886

Closes: #890
Approved by: jlebon
2017-08-08 16:02:15 +00:00
Jonathan Lebon
dc24dd3105 rollback: allow users to undo a rollback
The new API to find pending and rollback deployments do so relative to
the booted deployment. This caused an interesting behaviour: the first
time a user uses "rpm-ostree rollback", it would (as expected) move the
previous deployment first. but the second call to "rpm-ostree rollback"
would fail since there were now no more rollback deployments.

We fine tune the logic here to allow this, as well as the more general
case of putting the booted deployment back on top.

This fixes a subtle regression from b7cf58e
(https://github.com/projectatomic/rpm-ostree/pull/767).

Closes: https://github.com/projectatomic/rpm-ostree/issues/906

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Jonathan Lebon
0c9f77ca78 vmsync: overlay /etc only if it exists
This is the same as ddc0f40 but for `vmsync`.

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Jonathan Lebon
95d3f36fc4 vmcheck: fix sync and overlay
Fix regression from https://github.com/projectatomic/rpm-ostree/pull/870
which caused `vmsync` and `vmoverlay` to no longer actually overlay
ostree bits.

We go back to using `--files-from`, but just make sure to filter out the
stuff that we don't need (and which previously caused issues).

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Ruixin
53c39632d4 Deployment: add endoflife
When commit metadata contains ostree.endoflife attribute,
its information will be added to the deployment Variant,
which will later be shown as a red & bold message when
'rpm-ostree status' command is called.

A test is added for future regression

Closes: #889
Approved by: cgwalters
2017-08-02 13:53:10 +00:00
Colin Walters
0d4d6be94f Implement file triggers (%transfiletriggerin) for layered pkgs
File triggers are a post-RHEL7 thing; more information at
http://rpm.org/user_doc/file_triggers.html

There are two notable users I've been testing this with;
`glib2` and `vagrant`.  The `vagrant` one is more immediately urgent,
since it makes `vagrant-libvirt` work, which I currently rely on
for my workstation dev.

I've tested things successfully with `vagrant`, and I did verify that we run the
`glib2` ones when doing `rpm-ostree ex container`.

Long term, more transaction file triggers are likely to live in
"base" packages like `glib2`.  We don't implement those yet, but
extending this to do that shouldn't be too hard.

There was *significant* what I'd call reverse engineering of the
implementation in librpm.  The file triggers code there is spread out
and abstracted in a few different places in the code.  I found
trying to understand what header values were involved to be quite
tricky.

There are some corner cases like multiple patterns that I *think*
this does correctly, but could use more validation.  The main
question I had was - is it required that the patterns for e.g.
`%transfiletriggerin` and `%transfiletriggerun` be identical?

Closes: https://github.com/projectatomic/rpm-ostree/issues/648

Closes: #869
Approved by: jlebon
2017-07-27 20:58:09 +00:00
Jonathan Lebon
d4effe8f47 dbus: don't register if not active
Follow-up tweak to #894. Make the client smarter so we only register
when we know we can. We could be more sophisticated here and e.g.
introduce the concept of "read-only" clients in the daemon to only allow
access to non-mutating methods, though let's delay that discussion at
least until the daemon learns to auto-exit.

Closes: #898

Closes: #900
Approved by: cgwalters
2017-07-27 20:34:22 +00:00
Colin Walters
34b5a004a8 daemon: Add a sanitycheck(/bin/true) before we deploy a tree
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.

We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).

There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.

Closes: #892
Approved by: jlebon
2017-07-27 17:58:58 +00:00
Jonathan Lebon
ddc0f40355 overlay.sh: also overlay /etc to /usr/etc
This is needed to test changes to the D-Bus config file.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
95bc93205e vmcheck/test-basic.sh: adjust for new polkit policy
Now that we require clients to have an active session to RegisterClient,
we can't use runuser to check for non-root functionality. Add a new
vm_cmd_as() function to allow connecting as a different user. While
we're there, do some minor cleanups to consistently use `local` when
possible.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
f6c422a6d5 vmcheck/test-basic.sh: strengthen pkg-add test
This test would actually fail even if the bin user were allowed to
install a package because there are no enabled repos to install. Fix it
so that we know we have foo there and explicitly check that the error
message is what we expect.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
53978f6f03 sync.sh: also copy out /etc files
We do install one thing there: the dbus config file.

Closes: #896
Approved by: cgwalters
2017-07-26 19:22:51 +00:00
Colin Walters
2082b3f8fb lib: Add version macros and version checking function
The version checking function in particular is really useful for people doing
`from gi.repository import RpmOstree`, which we'd like at least some things like
Anaconda and Pungi to do.

Closes: #891
Approved by: jlebon
2017-07-21 20:35:26 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Colin Walters
2f3ca1bc8e tests: Add a test case for a %post that does rm -rf /
I was thinking today about our script handling, and I realized
an excellent way to showcase the advancement rpm-ostree makes
over traditional package managers is the fact that we survive a
`%post` script that does `rm -rf /`!

See e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858521

It's been this way ever since we first implemented package layering;
the fact that we construct a new root and use bubblewrap to sandbox
makes us very resilient to this type of thing.

But, let's add a test case for this to be sure we preserve this behavior; for
example, if in the future we for some reason we decide to leak some host state
into the scripts.

Closes: #888
Approved by: jlebon
2017-07-21 14:45:30 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Jonathan Lebon
cf19d83502 ci: unite testsuites and run vmcheck on centos
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.

Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.

This commit also switches to Fedora 26 as the primary test base.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Jonathan Lebon
114d53a02d vmcheck: fixes for CentOS 7
The `install` command in CentOS 7 is too old to understand that
`-Dt foo/bar` means creating both `foo` and `bar`, which is useful so
that we avoid an explicit `mkdir` before. But we can't do that here.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Matthew Barnes
2934799741 tests: Support older versions of rsync
Older versions of rsync (< 3.1) lack an --ignore-missing-args option.
Use this workaround [1] to allow "make vmsync" to work on CentOS 7.

[1] https://stackoverflow.com/questions/43391493/rsync-simulate-ignore-missing-args-on-old-server-version

Closes: #870
Approved by: cgwalters
2017-07-13 21:35:47 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
3b0b9fb98a vmcheck: handle livefs reruns stronger
Make sure that we wipe out any leftover configuration files from a
previous run before layering the test pkg, or we'll get false positive.
Also make sure to correctly clean up the VM in the case the livefs test
errors out.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
970a20974b build_rpm: print out nevra rather than just name
It's more precise and makes it possible to find out if e.g. pkg
upgrades made it to the tree.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Jonathan Lebon
2b547c2d0a tests: transition to build_rpm
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
5cb1fa5460 libtest: add build_rpm function
Having to prepare RPMs in advance separately from the tests that use
them severely limits our ability to test various cases and to iterate
quickly when creating tests.

Add a new `build_rpm` function which can basically build the RPM on the
fly and update the yum repo afterwards. It makes it trivial to test
things like package updates:

    build_rpm foo 1.0 1
    <stuff>
    build_rpm foo 1.0 2
    <stuff>

The RPMs are all created inside the temporary test directory and thus
cleaned up on exit.

I'm doing this in a separate commit because it's the most important diff
of the transition and might be easy to lose in the larger diff where we
move all the tests to make use of this.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
47cc5f60ed tests/check: remove test-compose.sh
Nowadays, we have a much more in-depth and realistic compose test suite
that obsoletes this test.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
4445df2c01 tests: add libtest.c for binary tests
Import libostreetest.c from ostreedev/ostree as libtest.c. This is just
a really useful and outrageous way of using libtest.sh from C.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
4c831e619c check/test-utils.c: assert no error first
Minor tweak; use g_assert_no_error() before using g_assert(ret) so that
we actually get a printout of the error the test fails.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
853737f667 unpacker: encode NEVRA info in commit metadata
It seems silly that to find out more detailed information about the
NEVRA of a cached pkg, we have to resort to write out the header to
disk, then reading it back in with librpm in order to tease out the info
we want. Let's just encode that information directly in the commit
metadata and provide a helper to fetch it.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
44bcb9d60a tests: merge cache_branch_to_nevra into test-utils
I didn't realize at the time I wrote the cache_branch_to_nevra test that
the already existing test-utils.c would be the perfect place to add this
test. Merge the two together now.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:21 +00:00
Jonathan Lebon
0b1c5eda17 override remove: allow inactive removals
The property of removal overrides dropping out if the package was
removed from the base layer felt a bit too magical and hacky. We really
should remember that wish and re-apply it if the pkg comes back. This is
similar to package layering: requests can become inactive (seems like a
better word than "dormant") if the package is already part of the base
layer, but they don't really go away.

This patch reworks the logic so that removal overrides work the same
way. In the status output, we now have both "RemovedBasePackages" and
"InactiveBaseRemovals" (which is only printed in verbose mode),
similarly to how we have "LayeredPackages" and "InactiveRequests". And
similarly, we also print out in the upgrader during a transaction all
the inactive base removals.

Another cool thing is that we now allow any pattern to be specified at
the CLI. E.g. `ex override remove /usr/bin/strace` will resolve to
strace.

Closes: #836
Approved by: cgwalters
2017-06-20 21:24:05 +00:00
Jonathan Lebon
5803fbc807 vmcheck: also clear repo metadata between tests
Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
6b8f0f6b23 core: fix error message construction
In the error path when trying to remove a base package, we would try to
print a DnfPackage as char*, which of course didn't result in any
coherent output.

Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
9907f9b283 vmoverlay: overlay on default checksum, not booted
This allows us to save one reboot if we want the overlay to happen on
the pending base checksum.

Closes: #829
Approved by: cgwalters
2017-06-13 13:25:24 +00:00
Colin Walters
6b74f440dc tests: Use G_DEBUG=fatal-warnings for daemon
To catch things like https://github.com/projectatomic/rpm-ostree/issues/826
during testing.

(Really IMO this should be the system-wide default except GNOME and
 lots of apps would start dumping core all over the place... 😢)

Closes: #828
Approved by: jlebon
2017-06-09 20:42:44 +00:00
Colin Walters
1977e3cfba scripts: Unlink our temporary post scripts
I just happened to do an `ls /usr` on my workstation. Sigh 😣

Closes: #822
Approved by: jlebon
2017-06-08 15:55:57 +00:00
Jonathan Lebon
b9f6d09f91 vmcheck: add new test for override remove
Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Jonathan Lebon
ec34eec6db vmcheck: create new test-basic.sh test
And move tests from test-layering-basic.sh that aren't related to
pkglayering there.

Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Jonathan Lebon
4297bdf263 vmsync: also pull ostree from build container
Copy the bits from `overlay.sh` that takes the ostree files from the
build container so that we also get updated ostree bits when we
`vmsync`.

Also make sure to `daemon-reload` before restarting the daemon.

Closes: #803
Approved by: cgwalters
2017-05-30 14:47:31 +00:00
Jonathan Lebon
ac9c3c1635 compose: use test env fedora.repo file instead
During provisioning, PAPR injects a fedora.repo pointing at a much
better & faster mirror than dl.fp.o. Let's use that to make the compose
test less flaky. Hoping to make these sorts of optimizations more
discoverable in upstream PAPR.

Closes: #799
Approved by: cgwalters
2017-05-30 14:17:38 +00:00
Colin Walters
d2bd8500da compose: Add tmp-is-dir option to make /tmp a directory
There are a few reasons to do this. First, systemd changed to refuse mounts on
symlinks, and hence if one *wants* "/tmp-on-tmpfs", one would need to write a
different `sysroot-tmp.mount` unit.

Second, the original rationale for having this symlink was that if you had
multiple ostree stateroots ("osnames"), it's nicer if they had the same `/tmp`
to avoid duplication. But in practice today that's already an issue due to
`/var/tmp`, and further the multiple-stateroot case is pretty unusual. And that
case is *further* broken by SELinux (if one wanted to have e.g. an Ubuntu and
Fedora) stateroots.  So let's fully decouple this and make `/tmp` a plain
old directory by default, so systemd's `tmp.mount` can become useful.

Now, things get interesting for the case where someone wants a physical `/tmp`
that *does* persist across reboots. Right now, if one just did a `systemctl mask
tmp.mount` as we do in Fedora Atomic Host's cloud images, you'd get a semantic
where `/tmp` stays per-deployment, which is weird.  Our recommendation for
that should likely be to set up a bind mount for `/tmp` → `/var/tmp`.

For now, this stays an option to ensure compatibility; if FAH Cloud images
want to stay with "physical /tmp", then we'd have to change the kickstart.

Closes: https://github.com/projectatomic/rpm-ostree/issues/669

Closes: #778
Approved by: jlebon
2017-05-24 17:49:36 +00:00
Colin Walters
2adc1cf246 core: Ignore overrides for nonexistent %ghost files in /etc
As seen in e.g. `ipa-client-common`.  We expect ghosts 👻 to not
exist.

Closes: https://github.com/projectatomic/rpm-ostree/issues/784

Closes: #785
Approved by: jlebon
2017-05-24 15:53:07 +00:00
Colin Walters
a18c2ecbf7 livefs: Fix etc merge with subdirectories
This was a kind of last-minute bug introduced when I tweaked the
checkout to use `.` to avoid a `mkdir()` for files.  But there were
multiple bugs with that; for files that are in subdirectories of `/etc`
we obviously need to get the right subdir and not use `/etc`.

Second, we need to handle the case where we're adding new subdirectories.

This change fixes `rpm-ostree install firewalld + rpm-ostree ex livefs`.

Closes: #783
Approved by: jlebon
2017-05-23 20:34:47 +00:00
Colin Walters
8ee6e86e38 tests: Use libtest-core.sh from ostree
Reduces drift.  In the future we may want to explicitly share
more test suite code too.

See https://github.com/ostreedev/ostree/pull/877

Closes: #782
Approved by: jlebon
2017-05-19 21:37:53 +00:00
Jonathan Lebon
174510fc5d core: allow no enabled repos for local pkgs
This is a valid case when layering local RPMs. Otherwise (e.g. if
specific non-local packages are requested), give the nicer error rather
than letting libdnf throw something less useful.

Closes: #780

Closes: #781
Approved by: cgwalters
2017-05-19 16:08:18 +00:00
Colin Walters
2ada15aff3 libtest: Fix bugs in assert_file_has_content()
First, we need to preserve whitespace in the regexps.  Second,
in the error path, all of our variables were wrong.

Commit 376a2cc3f5 is an excellent
poster child for how many bugs one can introduce in a single commit
for a ~5 line function.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
69d8bfa042 compose-tests: Consolidate some misc options into test-misc-tweaks.sh
I was about to add another one of these but it feels like a bit
overkill to run through a recompose for trivial tweaks like turning off docs.

We can do a compose with multiple options at once and test the result as a unit,
at least for the smaller/less invasive options.

This change is prep for adding a switch to do `/tmp` as a regular dir.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
376a2cc3f5 libtest: Support matching multiple regexps in files
For a future patch.

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Jonathan Lebon
27c525b706 vmcheck: stronger post-test cleanup
Now that rpm-ostree operations are cumulative, we need to make sure we
also clean up pending deployments. Also clean up the rollback for
consistency.

We also strengthen our ref handling. Always nuke the vmcheck ref (which
might not even exist anymore if a test rebased without --skip-purge) and
recreate it from the original ref.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
caafaf6c1b vmcheck: add test for pkgcache chksum invalidation
Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Colin Walters
e66ea0865a libtest: Adapt xattr disabling for newer ostree
Matches the test changes in e8efd1c8dc

Closes: #770
Approved by: jlebon
2017-05-09 13:32:25 +00:00
Jonathan Lebon
752368eb5b rebase: allow rebasing to a local branch
This is a follow-up to commit 77acf62. There, we added support for
rebasing from a local branch to another local branch. But in testing,
it's also really useful to be able to rebase from a remote-based refspec
to a local branch. We allow this here by slightly expanding the syntax
of allowed refspecs.

Now, we can use rpm-ostree all the time rather than fallback to `ostree
admin deploy`, which isn't pkg-aware.

Closes: #764
Approved by: cgwalters
2017-05-05 21:01:26 +00:00
Colin Walters
a284b64479 ci: Use FAHC for build container
Conceptually: we're going to move rpm-ostree and ostree at the same
cadence most of the time; for both releases *and* for git master.
The problem so far has been the latter part.  Reusing FAHC
for the build gets us half of the problem.

The other trick I realized we can do - just pull ostree out from the build
container. This avoids fetching it from the internet, and makes my workflow for
hacking on both nicer - I just `sudo make install` in my build container for
ostree.

It's tempting to make the whole thing symmetric and require `sudo make install`
for rpm-ostree and not do the insttree thing but...perhaps after.

Closes: #758
Approved by: jlebon
2017-05-01 19:10:05 +00:00
Colin Walters
11309cab97 core: Ignore %pretrans
This was argued on list a while ago: http://lists.rpm.org/pipermail/rpm-ecosystem/2016-August/000391.html
Basically, every use of `%pretrans` is a workaround for some in-place
upgrade problem.  But we never do inplace updates, we always assemble
a new tree.  Hence, there is no point for us to ever execute these.

Let's stop erroring out on them. If there does turn out to be some problem with
a particular package (e.g. the `openjdk-copy-configs` or whatever), we would
likely end up needing to fix that package's `%post`, not start executing its
`%pretrans`.

Closes: #763
Approved by: jlebon
2017-05-01 18:07:06 +00:00
Colin Walters
e4b5e77e64 tests: Add unit tests for varsubst
Trying to debug https://bugzilla.redhat.com/show_bug.cgi?id=1381357

I don't see the problem yet, but let's exercise the code a bit.

Closes: #755
Approved by: jlebon
2017-04-27 18:57:10 +00:00
Colin Walters
a180a35a48 status: Do show Commit with --verbose, don't show StateRoot without it
`Commit` is useful in cases where one wants to e.g. run `rpm-ostree db diff`.
`StateRoot` however is going to be in the vast majority of cases identical
across deployments.

Closes: #743
Approved by: jlebon
2017-04-21 13:31:21 +00:00
Colin Walters
9ab46f5221 Require ostree 2017.5
This is preparatory for future changes which will make use the newer sysroot
writing API. But in this change, we can drop all of our version ifdef'd code.

Closes: #740
Approved by: jlebon
2017-04-20 20:58:51 +00:00
Colin Walters
b21b08c770 app: Only call RegisterClient() if we're root
Otherwise we can't do e.g. `rpm-ostree status`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/738

Closes: #739
Approved by: dustymabe
2017-04-14 02:30:19 +00:00
Colin Walters
fbdbddb3d4 vmcheck: Support a VMCHECK_DEBUG to not reset the system
I want to be able to debug a test by sshing in with the same setup;
right now the suite resets and reboots by default.

This also required fixing how we deal with `/etc/yum.repos.d`, since at the
start of the test it could be either pristine or not.

Closes: #578
Approved by: jlebon
2017-04-11 16:04:29 +00:00
Colin Walters
378216b5a1 tests/ucontainer: Also disable xattrs here
This actually only worked before because of
https://github.com/ostreedev/ostree/pull/775

Closes: #723
Approved by: jlebon
2017-04-04 17:48:26 +00:00
Jonathan Lebon
9c023429b6 vmcheck: add tests for unified txn support
Closes: #711
Approved by: cgwalters
2017-03-31 14:58:38 +00:00
Colin Walters
e666a5b350 vmcheck: Support VMCHECK_INSTTREE
I need to build ostree from git too.  So now my workflow is:

```
export insttree=/srv/walters/tmp/rootfs
cd ostree
make && make install DESTDIR=${insttree}
cd rpm-ostree
make && make install DESTDIR=${insttree}
env VMCHECK_INSTTREE=${insttree} make vmoverlay
```

Closes: #705
Approved by: jlebon
2017-03-27 17:48:39 +00:00
Colin Walters
fa0af8b317 ci: Build ostree from git temporarily
See https://github.com/ostreedev/ostree/issues/758

Closes: #707
Approved by: jlebon
2017-03-27 16:35:43 +00:00
Colin Walters
3e833659b7 daemon: Add a separate OSExperimental interface
Just like `rpm-ostree ex`, for things like `ex livefs` that have DBus
interfaces, we should segregate these off so that people know they're unstable.
And conversely that they can test for the presence of the method on the main
interface for stability.

I initially tried having the same `RpmostreeOS` object implement both
but couldn't work out how to do that; see https://mail.gnome.org/archives/gtk-app-devel-list/2017-March/msg00161.html

Closes: #701
Approved by: jlebon
2017-03-23 19:24:41 +00:00
Colin Walters
1c421029d5 vmcheck: Drop hack for updating ostree
The version in the current release has the fix.

Closes: #698
Approved by: jlebon
2017-03-22 15:35:46 +00:00
Colin Walters
eaced1be7c vmcheck/overlay: Delete passwd backup files here for now
Until we land <https://github.com/projectatomic/rpm-ostree/pull/693> and it
makes its way into all of our streams.

Necessary for livefs.

Closes: #698
Approved by: jlebon
2017-03-22 15:35:46 +00:00
Colin Walters
8e8c26f9fb vmcheck: Don't throw if test.sh fails early
If test.sh failed even before writing to the log file, opening will fail.

Closes: #698
Approved by: jlebon
2017-03-22 15:35:46 +00:00
Colin Walters
8b6c55ca60 core: Also apply mode overrides to symlinks
I originally thought there was a bug here, but missed we were skipping
symlinks earlier above.  See [previous discussion][1].

Now, I'm not aware right now of something this patch actively fixes, but I think
it makes sense to be consistent in our handling of things here with respect to
symlinks.

1: 29dd1bd801..8158dcfb47 (r95017893)

Closes: #689
Approved by: jlebon
2017-03-21 21:35:37 +00:00
Colin Walters
19b093e4a0 vmcheck: Honor TESTS=
Minor regression from the multitest reimplementation, but it's really handy for
the "debug and fix a test" case.

Closes: #692
Approved by: jlebon
2017-03-20 20:41:30 +00:00