Commit Graph

243 Commits

Author SHA1 Message Date
Colin Walters
34b5a004a8 daemon: Add a sanitycheck(/bin/true) before we deploy a tree
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.

We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).

There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.

Closes: #892
Approved by: jlebon
2017-07-27 17:58:58 +00:00
Jonathan Lebon
ddc0f40355 overlay.sh: also overlay /etc to /usr/etc
This is needed to test changes to the D-Bus config file.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
95bc93205e vmcheck/test-basic.sh: adjust for new polkit policy
Now that we require clients to have an active session to RegisterClient,
we can't use runuser to check for non-root functionality. Add a new
vm_cmd_as() function to allow connecting as a different user. While
we're there, do some minor cleanups to consistently use `local` when
possible.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
f6c422a6d5 vmcheck/test-basic.sh: strengthen pkg-add test
This test would actually fail even if the bin user were allowed to
install a package because there are no enabled repos to install. Fix it
so that we know we have foo there and explicitly check that the error
message is what we expect.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
53978f6f03 sync.sh: also copy out /etc files
We do install one thing there: the dbus config file.

Closes: #896
Approved by: cgwalters
2017-07-26 19:22:51 +00:00
Colin Walters
2f3ca1bc8e tests: Add a test case for a %post that does rm -rf /
I was thinking today about our script handling, and I realized
an excellent way to showcase the advancement rpm-ostree makes
over traditional package managers is the fact that we survive a
`%post` script that does `rm -rf /`!

See e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858521

It's been this way ever since we first implemented package layering;
the fact that we construct a new root and use bubblewrap to sandbox
makes us very resilient to this type of thing.

But, let's add a test case for this to be sure we preserve this behavior; for
example, if in the future we for some reason we decide to leak some host state
into the scripts.

Closes: #888
Approved by: jlebon
2017-07-21 14:45:30 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Jonathan Lebon
114d53a02d vmcheck: fixes for CentOS 7
The `install` command in CentOS 7 is too old to understand that
`-Dt foo/bar` means creating both `foo` and `bar`, which is useful so
that we avoid an explicit `mkdir` before. But we can't do that here.

Closes: #871
Approved by: cgwalters
2017-07-18 13:58:38 +00:00
Matthew Barnes
2934799741 tests: Support older versions of rsync
Older versions of rsync (< 3.1) lack an --ignore-missing-args option.
Use this workaround [1] to allow "make vmsync" to work on CentOS 7.

[1] https://stackoverflow.com/questions/43391493/rsync-simulate-ignore-missing-args-on-old-server-version

Closes: #870
Approved by: cgwalters
2017-07-13 21:35:47 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
3b0b9fb98a vmcheck: handle livefs reruns stronger
Make sure that we wipe out any leftover configuration files from a
previous run before layering the test pkg, or we'll get false positive.
Also make sure to correctly clean up the VM in the case the livefs test
errors out.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
970a20974b build_rpm: print out nevra rather than just name
It's more precise and makes it possible to find out if e.g. pkg
upgrades made it to the tree.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Jonathan Lebon
2b547c2d0a tests: transition to build_rpm
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
0b1c5eda17 override remove: allow inactive removals
The property of removal overrides dropping out if the package was
removed from the base layer felt a bit too magical and hacky. We really
should remember that wish and re-apply it if the pkg comes back. This is
similar to package layering: requests can become inactive (seems like a
better word than "dormant") if the package is already part of the base
layer, but they don't really go away.

This patch reworks the logic so that removal overrides work the same
way. In the status output, we now have both "RemovedBasePackages" and
"InactiveBaseRemovals" (which is only printed in verbose mode),
similarly to how we have "LayeredPackages" and "InactiveRequests". And
similarly, we also print out in the upgrader during a transaction all
the inactive base removals.

Another cool thing is that we now allow any pattern to be specified at
the CLI. E.g. `ex override remove /usr/bin/strace` will resolve to
strace.

Closes: #836
Approved by: cgwalters
2017-06-20 21:24:05 +00:00
Jonathan Lebon
5803fbc807 vmcheck: also clear repo metadata between tests
Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
6b8f0f6b23 core: fix error message construction
In the error path when trying to remove a base package, we would try to
print a DnfPackage as char*, which of course didn't result in any
coherent output.

Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
9907f9b283 vmoverlay: overlay on default checksum, not booted
This allows us to save one reboot if we want the overlay to happen on
the pending base checksum.

Closes: #829
Approved by: cgwalters
2017-06-13 13:25:24 +00:00
Colin Walters
6b74f440dc tests: Use G_DEBUG=fatal-warnings for daemon
To catch things like https://github.com/projectatomic/rpm-ostree/issues/826
during testing.

(Really IMO this should be the system-wide default except GNOME and
 lots of apps would start dumping core all over the place... 😢)

Closes: #828
Approved by: jlebon
2017-06-09 20:42:44 +00:00
Colin Walters
1977e3cfba scripts: Unlink our temporary post scripts
I just happened to do an `ls /usr` on my workstation. Sigh 😣

Closes: #822
Approved by: jlebon
2017-06-08 15:55:57 +00:00
Jonathan Lebon
b9f6d09f91 vmcheck: add new test for override remove
Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Jonathan Lebon
ec34eec6db vmcheck: create new test-basic.sh test
And move tests from test-layering-basic.sh that aren't related to
pkglayering there.

Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Jonathan Lebon
4297bdf263 vmsync: also pull ostree from build container
Copy the bits from `overlay.sh` that takes the ostree files from the
build container so that we also get updated ostree bits when we
`vmsync`.

Also make sure to `daemon-reload` before restarting the daemon.

Closes: #803
Approved by: cgwalters
2017-05-30 14:47:31 +00:00
Colin Walters
a18c2ecbf7 livefs: Fix etc merge with subdirectories
This was a kind of last-minute bug introduced when I tweaked the
checkout to use `.` to avoid a `mkdir()` for files.  But there were
multiple bugs with that; for files that are in subdirectories of `/etc`
we obviously need to get the right subdir and not use `/etc`.

Second, we need to handle the case where we're adding new subdirectories.

This change fixes `rpm-ostree install firewalld + rpm-ostree ex livefs`.

Closes: #783
Approved by: jlebon
2017-05-23 20:34:47 +00:00
Jonathan Lebon
174510fc5d core: allow no enabled repos for local pkgs
This is a valid case when layering local RPMs. Otherwise (e.g. if
specific non-local packages are requested), give the nicer error rather
than letting libdnf throw something less useful.

Closes: #780

Closes: #781
Approved by: cgwalters
2017-05-19 16:08:18 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Jonathan Lebon
27c525b706 vmcheck: stronger post-test cleanup
Now that rpm-ostree operations are cumulative, we need to make sure we
also clean up pending deployments. Also clean up the rollback for
consistency.

We also strengthen our ref handling. Always nuke the vmcheck ref (which
might not even exist anymore if a test rebased without --skip-purge) and
recreate it from the original ref.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
caafaf6c1b vmcheck: add test for pkgcache chksum invalidation
Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Colin Walters
a284b64479 ci: Use FAHC for build container
Conceptually: we're going to move rpm-ostree and ostree at the same
cadence most of the time; for both releases *and* for git master.
The problem so far has been the latter part.  Reusing FAHC
for the build gets us half of the problem.

The other trick I realized we can do - just pull ostree out from the build
container. This avoids fetching it from the internet, and makes my workflow for
hacking on both nicer - I just `sudo make install` in my build container for
ostree.

It's tempting to make the whole thing symmetric and require `sudo make install`
for rpm-ostree and not do the insttree thing but...perhaps after.

Closes: #758
Approved by: jlebon
2017-05-01 19:10:05 +00:00
Colin Walters
9ab46f5221 Require ostree 2017.5
This is preparatory for future changes which will make use the newer sysroot
writing API. But in this change, we can drop all of our version ifdef'd code.

Closes: #740
Approved by: jlebon
2017-04-20 20:58:51 +00:00
Colin Walters
b21b08c770 app: Only call RegisterClient() if we're root
Otherwise we can't do e.g. `rpm-ostree status`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/738

Closes: #739
Approved by: dustymabe
2017-04-14 02:30:19 +00:00
Colin Walters
fbdbddb3d4 vmcheck: Support a VMCHECK_DEBUG to not reset the system
I want to be able to debug a test by sshing in with the same setup;
right now the suite resets and reboots by default.

This also required fixing how we deal with `/etc/yum.repos.d`, since at the
start of the test it could be either pristine or not.

Closes: #578
Approved by: jlebon
2017-04-11 16:04:29 +00:00
Jonathan Lebon
9c023429b6 vmcheck: add tests for unified txn support
Closes: #711
Approved by: cgwalters
2017-03-31 14:58:38 +00:00
Colin Walters
e666a5b350 vmcheck: Support VMCHECK_INSTTREE
I need to build ostree from git too.  So now my workflow is:

```
export insttree=/srv/walters/tmp/rootfs
cd ostree
make && make install DESTDIR=${insttree}
cd rpm-ostree
make && make install DESTDIR=${insttree}
env VMCHECK_INSTTREE=${insttree} make vmoverlay
```

Closes: #705
Approved by: jlebon
2017-03-27 17:48:39 +00:00
Colin Walters
1c421029d5 vmcheck: Drop hack for updating ostree
The version in the current release has the fix.

Closes: #698
Approved by: jlebon
2017-03-22 15:35:46 +00:00
Colin Walters
eaced1be7c vmcheck/overlay: Delete passwd backup files here for now
Until we land <https://github.com/projectatomic/rpm-ostree/pull/693> and it
makes its way into all of our streams.

Necessary for livefs.

Closes: #698
Approved by: jlebon
2017-03-22 15:35:46 +00:00
Colin Walters
8e8c26f9fb vmcheck: Don't throw if test.sh fails early
If test.sh failed even before writing to the log file, opening will fail.

Closes: #698
Approved by: jlebon
2017-03-22 15:35:46 +00:00
Colin Walters
8b6c55ca60 core: Also apply mode overrides to symlinks
I originally thought there was a bug here, but missed we were skipping
symlinks earlier above.  See [previous discussion][1].

Now, I'm not aware right now of something this patch actively fixes, but I think
it makes sense to be consistent in our handling of things here with respect to
symlinks.

1: 29dd1bd801..8158dcfb47 (r95017893)

Closes: #689
Approved by: jlebon
2017-03-21 21:35:37 +00:00
Colin Walters
19b093e4a0 vmcheck: Honor TESTS=
Minor regression from the multitest reimplementation, but it's really handy for
the "debug and fix a test" case.

Closes: #692
Approved by: jlebon
2017-03-20 20:41:30 +00:00
Jonathan Lebon
7233db8a74 multitest.py: don't use os.wait()
We can't use os.wait(), since it will conflict with the subprocess'
module's poll() and wait().

Closes: #694
Approved by: cgwalters
2017-03-20 16:59:00 +00:00
Colin Walters
ab7431c458 core: Apply rpmfi overrides for /etc to /usr/etc
This fixes installing `mock`, which for some reason has files in `/etc/mock`
owned by `root:mock`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/671

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Jonathan Lebon
29840bcecb vmcheck: add support for parallel runs
Allow the `make vmcheck` target to take a HOSTS var, which is simply a
space-separated list of hosts on which we can run testsuites. Add a
multitest.py script that takes care of monitoring and scheduling the
tests onto the nodes.

The script itself is "dumb": we don't know how long each test can take,
so we can't do any smart/heuristic scheduling that could save more time.

Closes: #675
Approved by: cgwalters
2017-03-13 00:42:13 +00:00
Jonathan Lebon
9d9042d835 vmcheck: prep for parallel runs
1. Don't require an ssh-config

In the case of redhat-ci, the VMs are already fully configured for the
system (injected in the hosts file, host key accepted, etc...). So
there's no need to have an ssh-config there. In general, it should be
acceptable to run the vmcheck suite against a resolvable host without
having to create an ssh-config for it.

2. Make the host name configurable

Rather than hardcoding "vmcheck" as the hostname, allow overridding it
by specifying a VM env var directly. We also prepare the various scripts
to make use of the $VM variable whenever host-specific dirs/files are
created so that parallel runs won't step on each other.

Closes: #675
Approved by: cgwalters
2017-03-13 00:42:13 +00:00
Jonathan Lebon
fd64084b9b vmcheck: add basic test for local RPM installs
Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Jonathan Lebon
49cbdb739a status: always include the packages entries
Pull #646 introduced a subtle regression: we went from always including
a "packages" entry to only including it if there are packages present.
Albeit it's easy to guard against, though to be nice, let's make it
easier for consumers by always including it.

Reported-by: Micah Abbott <miabbott@redhat.com>

Closes: #670
Approved by: cgwalters
2017-03-10 14:54:45 +00:00
Colin Walters
66425c3161 core: Do GPG verification before importing
While reading a recent conversation about GPG checking at treecompose
time, I had a sudden thought - were we actually doing verification
client side?  Turned out, we aren't.  That happens as part of
`dnf_transaction_commit()` which we don't use.

That function verifies every package at one go, but for us I think it's better
to do it before "importing". We shouldn't have untrusted bits that we've
unpacked (they might have suid binaries, for one thing).

This is an embarassing problem, but it's worth emphasizing that everyone should
be retrieving repodata at a minimum over TLS, which sets a baseline. On RHEL, we
already do pinned TLS, and there are discussions about extending that elsewhere.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1422157

Closes: #656
Approved by: jlebon
2017-03-06 15:19:43 +00:00
Colin Walters
22048b25a7 scripts: Use tmpfs for /var/tmp, not the host's /tmp
We don't want to expose the host's `/tmp` since that means scripts could
potentially find things like the X11 socket or whatever.

To debug things better, add a quick bash script to run bwrap like the C code
does. Perhaps down the line we can add `rpm-ostree internals run-bwrap` or so.

Closes: #647
Approved by: jlebon
2017-02-28 23:37:15 +00:00
Jonathan Lebon
7fd474f8e1 vmcheck: adjust for new behaviour
Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Jonathan Lebon
a3ee87d67d vmcheck/test.sh: output reboot details to log
Clean up the output of vmcheck a bit by printing out ssh debug details
and rpm-ostree status output to the log file. This will help make it
easier to quickly see which tests failed (as well as helping satisfy my
OCD for a nice output).

Closes: #645
Approved by: cgwalters
2017-02-24 19:50:25 +00:00
Colin Walters
f573354324 ci: Hotfix ostree temporarily
For https://github.com/ostreedev/ostree/pull/705

Closes: #642
Approved by: jlebon
2017-02-23 20:03:06 +00:00
Jonathan Lebon
8eb746f533 test-initramfs.sh: fix for centos
Fetch the osname rather than hardcoding it.

Closes: #636
Approved by: cgwalters
2017-02-21 14:13:42 +00:00
Colin Walters
ddbaf19f1e importer: Error importing RPMs which install to /opt (outside of /usr)
See https://github.com/projectatomic/rpm-ostree/issues/233 - for RPMs which
place files in e.g. `/opt`, we have different behavior in the treecompose case
(silently drop it) versus package layering (does the wrong thing).

Since the unpacker right now is only used in the layering case, this just
ensures we'll get a consistent error there.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Jonathan Lebon
8cf071127e unpacker: support root-owned /var & /run files
I'm not sure why we weren't doing this before, but we need to also
support files in /var and /run that are owned by root.

Related: RHBZ#1421781

Closes: #622
Approved by: cgwalters
2017-02-13 16:49:59 +00:00
Colin Walters
a66d27230d Add a cleanup command and DBus API
We sometimes talk about using `ostree admin undeploy`, but that
doesn't know about the pkgcache, and hence space there leaks
until the next rpm-ostree operation.

Just for this, we need to expose a cleanup command (and API).  But
we also need to support cleaning:

 - repomd
 - downloads (repo/tmp)

So let's start implementing that.

Closes: #614
Approved by: jlebon
2017-02-13 15:28:05 +00:00
Colin Walters
0a1289ae38 upgrader: Use "pending" deployment for origin by default
Until now, we always used the booted deployment, and would
garbage collect the "pending" deployment.  This is the
way OSTree was designed, but I think for rpm-ostree given
how mutable we are on the client side, there's a much stronger
argument for being more stateful too.

This is a relatively simple code change to split the "merge deployment"
concept into two.  There's now the "config merge deployment" and the
"origin merge deployment".

Basically, `rpm-ostree install foo; rpm-ostree install bar` will
now install both `foo` and `bar`.  But we will still use the booted
deployment for `/etc`.

Down the line, I think I'd like to drive into OSTree the concept of
a "staged" deployment, that has the hardlink checkout done, but doesn't
have the config merge.

But we don't need to change the OSTree core for this yet; we can
do it here in rpm-ostree, and this relatively simple code change
fixes many issues at once.

For example, `rpm-ostree upgrade && rpm-ostree install foo` now
does what you expect as well.

Obviously, we want to enable doing multiple things in *one* transaction,
and we're not far away, but I think this is also the right thing to do now.

I'm relatively confident it won't break anyone's workflow, as what
we did before wasn't generally that useful.  However, people will
need to learn to `ostree admin undeploy 0` if they *don't* want
this behavior.  (We need to have `rpm-ostree cleanup`).

Closes: https://github.com/projectatomic/rpm-ostree/issues/406

Closes: #611
Approved by: jlebon
2017-02-08 19:55:44 +00:00
Colin Walters
a52cb7d78e core: Add rpmostree.repo metadata to imported packages
I'm watching https://github.com/rpm-software-management/libdnf/pull/199 and I
really don't like it. We already have a place to put out-of-rpmdb metadata,
which is in the ostree commit for imported packages. No need to involve a
relational database for this (and further, one that would need to learn about
multiple ostrees).

We're not yet *using* this information in the UI, but we could; imagine
changing the `status` `Packages:` to show packages-per-repo or so.  We
could also expose an `rpm-ostree pkg-info foo`.

But for now, let's just start recording this.

Closes: #610
Approved by: jlebon
2017-02-08 17:52:16 +00:00
Jonathan Lebon
283b915ecf check: use jq for asserting json elements
Closes: #609
Approved by: cgwalters
2017-02-08 17:11:06 +00:00
Colin Walters
ace223acf8 Add pending-base-commit to status
One thing that's very confusing about OSTree is there are two layers -
deployments and the refs/commits. If one does an `rpm-ostree upgrade`, but then
e.g. `ostree admin undeploy 0`, you still have the new revision in the repo.

We don't do a good job of displaying this state, or helping people clean
it up.

Down the line, I also want to better support something like `rpm-ostree pull` to
cache updates explicitly *without* deploying.

This commit just adds a bit of information to the status display. We might want
to have better formatting, but I think this an OK start.

Closes: #595
Approved by: jlebon
2017-02-08 13:59:48 +00:00
Colin Walters
687567d3ee Merge daemon binary into main binary
The actual problem I am trying to fix with this is fallout from the
introduction of `/usr/libexec/rpm-ostreed`, which required a SELinux
policy change.  Specifically for CentOS, the base policy is rev'd
slowly.

My hope was that by merging the daemon code back into `/usr/bin/rpm-ostree`
which is labeled `install_exec_t`, starting via systemd would do
the right thing.  It turns out that doesn't happen.

Now later, I'm picking this patch back up because I want to do multprocessing in
the daemon (and in the core), and it makes sense to share code between them,
because multiprocessing will need to go through a re-exec path.

Another benefit is we avoid duplicated text (libglnx, internal helpers) between
the two binaries.

Closes: #292
Approved by: jlebon
2017-02-07 16:07:09 +00:00
Jonathan Lebon
566367ca3f upgrader: let ostree create the commit dir
When we checked out the base tree for package layering, we would create
the directory in which ostree did the checkout. This meant however that
ostree wouldn't apply xattrs on the root directory itself. This would
cause the directory to be mislabeled (as system_conf_t instead of
root_t), which in turn cause SELinux violations on reboot when systemd
tried to make the root mount shared.

This patch fixes this by first settling on a permanent directory in
which to do checkouts -- really, we'll never have multiple package
layering operations going on at the same time. Once we know that we have
a reserved path, we can safely let ostree create it for us with the
proper xattrs.

Resolves: RHBZ#1318547

Closes: #605
Approved by: cgwalters
2017-02-04 11:08:27 +00:00
Colin Walters
cac4522e5b Support "system/regenerate-initramfs=true" flag in origin
Currently we push for a model where the initramfs is
generated (in non-hostonly mode), and merely replicated.

However, to support a few unfortunate corner cases like dm-multipath which wants
to inject a config file into the initramfs, we need to support regenerating it
client side too.

Down the line, we'll need this to support overriding the kernel too.

This changes things in the core to add the concept of an "empty"
`RpmOstreeContext`. I initially tried skipping it, but that was too much
duplication. We still want all of the core ostree-related logic that lives in
that code too.

The treespec bits barfed if the spec didn't have a `tree/packages` key. It was
simplest to change that to allow it - and because that was the only case where
we errored out in parsing, I dropped the error handling.

There was another place in the upgrader that now needed to be fixed to handle
transitioning from just regenerating initramfs to not.

Closes: #574
Approved by: jlebon
2017-01-20 16:17:51 +00:00
Colin Walters
d277205c13 tests: Actually install the new binaries
I think since I landed a change to `--enable-new-name`, `/usr/bin/rpm-ostree`
became a symlink, and without the `-l` switch, rsync skips over them.

Hence, we have only been testing in vmcheck the old binaries, not new ones.
Oops.

Closes: #585
Approved by: jlebon
2017-01-19 18:55:15 +00:00
Colin Walters
59f34e8a03 tests: Disable ASAN leak checks
Right now we're far from leak free, and I want to use ASAN
as a sanity checker, not a leak checker.

Closes: #576
Approved by: jlebon
2017-01-18 16:52:46 +00:00
Colin Walters
c6f3c43295 tests: Brutally hack in copying in sanitizer shared libraries
Yes, there are better ways to do this, but there are also
better things to do...

Closes: #576
Approved by: jlebon
2017-01-18 16:52:46 +00:00
Colin Walters
4a511fa011 app: Check for root privileges where required early
Rather than sending a dbus message that gets denied, which
is ugly.

Closes: #565

Closes: #570
Approved by: jlebon
2017-01-13 19:46:24 +00:00
Colin Walters
533ff648a6 vmcheck: Note how many tests were skipped
I was initially confused as to the pattern matching, and this will make things a
bit clearer.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
dee0807147 vmcheck: Scrape out the journal on failure
So we can debug things more easily.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
d3bf60d373 tests: Add a pkg with rofiles violation that should fail
To ensure we don't accidentally allow mutations.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Jonathan Lebon
9f8136e097 vmcheck: add test-layering-non-root-caps.sh
This new test exercises the new support for non-root file paths and
files with capabilities.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
c5fa202378 sync.sh: tweak for container workflow
In the container build + vagrant test workflow, we have to run `make
vmsync` from the env in which autoconf was run, which is the container.

However, when using git worktrees (which is useful in this scenario to
avoid having to overwrite your local builds) and a CentOS build
container, the git inside is too old to understand worktrees, so let's
avoid invoking git in any of the test harness scripts.

Also make sure to restorecon after installing the new rpm-ostreed.

Closes: #555
Approved by: cgwalters
2016-12-21 20:00:43 +00:00
Jonathan Lebon
68acb1d2be compose-tests: add more tests
Add a few more tests to exercise some of the treefile options. We do
need to also expand test-basic.sh itself to sanity-check the structure
of a normal ostree compose. That's up next on the list.

Closes: #548
Approved by: cgwalters
2016-12-09 16:18:11 +00:00
Jonathan Lebon
a21c05ab10 vmcheck: disable default repos to speed it up
Closes: #543
Approved by: cgwalters
2016-12-07 21:47:58 +00:00
Jonathan Lebon
12c4b5b16f vmcheck: adapt for non-vagrant hosts
This is just the final bit required to make sure the vagrant and
non-vagrant paths can work happily together. It's mostly minor fixes,
though the most major change which also affects vagrant is that we now
sync to the root home dir, rather than ~vagrant.

Closes: #524
Approved by: cgwalters
2016-11-22 17:15:20 +00:00
Colin Walters
2f9d586bdc Redo vmcheck to abstract over ssh-config, drop all building
My development environment is now using "pet" docker containers.
I use VMs for testing things that require that (like rpm-ostree).

This patch builds on work from @jlebon in
https://github.com/projectatomic/rpm-ostree/pull/509
to rework `vmcheck` such that it can work on any `ssh-config`.  By
default we expect this to be Vagrant.

However, I go a lot farther and delete the `vmbuild` code that was
trying to do builds in a container on the target VM.  I think this is
still worth pursuing at some point, but for now I think it's
reasonable to assume that the rpm-ostree developer audience uses Linux
as their host workstation and hence has containers.

(There's another important point here in that for developing lower
 level things like rpm-ostree, there's a strong push to make the VM
 disposable and not a pet)

Closes: #516
Approved by: jlebon
2016-11-16 18:14:23 +00:00
Colin Walters
a5379a2520 tests: Test pinned package version
It should work for both layering and treecompose.

Closes: #518
Approved by: jlebon
2016-11-14 21:31:54 +00:00
Jonathan Lebon
accd2b9f6f vmcheck: add a test for RPM scripts
Thought it'd be fun to write a test for verifying proper handling of
scriptlets during package layering. There's obviously a lot more that
could go in here (patches welcome!), but it's a start.

Closes: #434
Approved by: cgwalters
2016-08-17 21:28:01 +00:00
Jonathan Lebon
194c2bf5cd libvm.sh: add vm_setup and vm_rsync
In preparation for being vagrant independent, let's factor out some
things directly into the libvm.

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
8942268ecb test.sh: write test result to log for easier grepping
Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
ab6fe314aa vmcheck/test.sh: support external ssh-config
Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
2da585023d vmcheck: remove test-basic.sh
This is a useless test for now since we should be able to test on a VM
even if there are updates available.

I initially had it just as an example of the vmcheck test harness,
without actually thinking too much into what it was testing. But we do
need a nice test-basic.sh that will give rpm-ostree a good general
workout.

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
f8907b6d53 vmcheck: add more pkg layering tests
- Rename test-layering.sh to test-layering-basic.sh and make it test
  both pkg-add and pkg-remove.
- Add test-layering-relayer.sh, which verifies that pkgs are properly
  relayered during the creation of new deployments (e.g. upgrades,
  rebases, deploys).
- Add test-layering-rpmdb.sh, which verifies that packages respect the
  rpm requirements before being overlayed.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
ba05972bbb vmcheck/test.sh: make more silent and add some bling
Print out a better summary that sums up the passes, failures, and skips.
Silence the restoring step at the end of the testsuite by only echo'ing
to the log file. Finally, add some colouring because colours are
awesome.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
7697702aaa vmcheck: strengthen test harness
We now make the test harness handle restoring the VM to the original
state. The wonderful thing about ostree here is that it's a perfect
shoo-in for this. We make a 'backup' of the current ref, and just have
to make sure that the VM is back on that ref after running each test.

This will allow us to write tests without worrying as much about
cleaning up in the event of an error.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
d718b820ef Makefile-tests.am: more cleanups and add pkg bar
The new package bar will be used to test conflicting packages.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
65f5678ecb vmcheck: add timeout option for vm_ssh_wait
Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
812c431856 test-layering.sh: basic package layering test
This is a preliminary package layering test which simply installs the
foo package and verifies that it functions properly. A bunch of
primitives are added to libvm.sh to facilitate this and future tests.

Closes: #344
Approved by: cgwalters
2016-06-23 00:05:09 +00:00
Jonathan Lebon
c0924b8997 vmcheck: add a basic test harness
This is a very simple test harness inspired by the atomic one. It's a
simple bash script that sets up a permanent ssh connection to the host
and runs the test scripts. Also add a "demo" test-basic.sh test to make
sure that it works.

Closes: #344
Approved by: cgwalters
2016-06-23 00:05:09 +00:00
Jonathan Lebon
9587ab6597 vagrant: move helper files to vagrant/
These files are not only for vmcheck, but may be used as well for
setting up for development.

Closes: #344
Approved by: cgwalters
2016-06-23 00:05:09 +00:00
Jonathan Lebon
38c8040879 vmcheck: clean before building and adapt to new json
If the dir that was rsync'ed already contains build artifacts, we don't
want those contaminating our build process.

Closes: #336
Approved by: cgwalters
2016-06-20 19:57:37 +00:00
Jonathan Lebon
816194cd58 Makefile-tests: add toplevel vm* targets
The `make vmshell` target makes it even easier to immediately try out
your changes in a live running Atomic Host. It will automatically
provision the VM, sync your latest changes, build, install in a new
deployment onto which the VM is rebooted, and drop you in the shell.

Closes: #321
Approved by: cgwalters
2016-06-15 13:38:34 +00:00
Jonathan Lebon
be3a677125 vmcheck: create a new deployment instead
In order for vmcheck to be useful for testing, we need the machine to be
in a "clean" state. That is, sitting on a commit, ready for being
manipulated.

This is a small step towards this goal. Instead of overwriting the
files, we use rofiles-fuse to safely install and create a new
deployment.

Closes: #321
Approved by: cgwalters
2016-06-15 13:38:34 +00:00
Colin Walters
a3769a7219 vmcheck: Experiment with the name nxs
I find myself not liking the name rpm-ostree anymore; it's
descriptive, but unfortunately we compete with other projects with
easier and sexier names.

Also, people continually find the ostree and rpm-ostree layering
unclear.  It's *much* easier to say "nxs depends on ostree", even
though textually it's obvious "rpm-ostree depends on ostree".

Anyways, just an experiment for now.

Closes: #307
Approved by: giuseppe
2016-06-08 14:26:08 +00:00
Colin Walters
77eb3e2b5b tests: Introduce "vmcheck"
This is infrastructure that's oriented around Vagrant, we do builds
inside the target VM (actually inside a Docker container), but then
directly `make install DESTDIR=/host/`.

The goal here is to have a convenient workflow for:

  - dev -> hand testing
  - dev -> run destructive tests

Closes: #303
Approved by: jlebon
2016-06-07 18:39:46 +00:00