Commit Graph

101 Commits

Author SHA1 Message Date
Colin Walters
9df2428a1a tests: Move upgrade/rebase tests from unit to vmcheck/test-upgrades
Now we stop running rpm-ostreed as non-root, which is going to be
a maintenance pain going forward.  If we do introduce non-VM based
tests I think we should look to doing in-container testing.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
a50a6cb959 tests: Move more update-generation bits to libvm.sh
The autoupdate test had a lot of useful infrastructure.  Prep
for removing `tests/check/test-upgrade-rebase.sh`.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
73fa64eb12 tests: Move ostree-update creation functions into libvm.sh
So they can be reused outside of just the autoupdate test.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Colin Walters
4f42b94da7 tests: Migrate some basic unit tests that start daemon to vmcheck
The unit tests run an rpm-ostree daemon as non-root, which worked
surprisingly well for quite a while.  But it started failing when
working on a patch which adds caching that writes to `/var`.

Since we have the vmcheck system now, let's switch over to that.
This PR moves the random "basic" tests we'd accumulated like
one to verify `StateRoot` is only in `status --verbose`, but not
the tests for the `rebase` command etc.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Jonathan Lebon
3a1fd2c139 tests/libvm: Support SSH_CONFIG env var
When juggling multiple test VMs for different purposes, it's useful to
be able to easily e.g. `make vmcheck` a specific one by overriding the
ssh-config file to use, rather than editing the latter each time.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
2eb424b9ce vmcheck: Add vm_shell_inline
Initially suggested in #1298. Wanted to try this out for an upcoming
patch.

Closes: #1304
Approved by: cgwalters
2018-03-14 21:49:16 +00:00
Jonathan Lebon
3e39fda4aa vmcheck: Make vm_ansible_inline more verbose
Because otherwise, there's no way to see the output of the script.

Also, turn off `gather_facts` since in the majority of cases, we don't
need it, so let's avoid the overhead. We can make this an opt-in flag
later on if needed.

Closes: #1304
Approved by: cgwalters
2018-03-14 21:49:16 +00:00
Colin Walters
63f545a6f4 tests: Replace some more libvm bits with Ansible
`vm_cmdfile()` was clearly calling out for this. I also replaced `vm_send()`
calls.

Closes: #1298
Approved by: jlebon
2018-03-14 14:13:07 +00:00
Colin Walters
bb86912de9 tests: Dip our toes into using Ansible
This adds a shell primitive to make it easy to execute a playbook
task list.

The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.

Closes: #1297
Approved by: jlebon
2018-03-09 19:04:32 +00:00
Jonathan Lebon
b54b8744b9 tests/libvm: Handle transient service already existing
We would error out when trying to start the transient httpd service if
it already exists, e.g. from a previous test.

Depending on how we exit, the `vm_stop_httpd` trap for the previous test
might not have been able to kick in. I think this happens when we exit
using `fatal`, which just does an `exit 1`. It's not strictly an error,
so doesn't trip the `ERR` handler.

Let's just go the extra mile and explicitly delete transient services if
they already exist.

Closes: #1284
Approved by: cgwalters
2018-03-01 23:29:17 +00:00
Jonathan Lebon
0729487ae5 Check and display pending security advisories
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.

This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.

Closes: #1249
Approved by: cgwalters
2018-02-15 15:30:26 +00:00
Jonathan Lebon
8b72dcbde2 tests/libtest: Fix bash syntax error
I think the logic there just worked because we don't actually use
`noarch` RPMs in the vmcheck suite.

Closes: #1248
Approved by: cgwalters
2018-02-14 19:02:02 +00:00
Jonathan Lebon
6c933bbf3c lib/package: fix package diffs for multilib
Our complicated heuristics for handling multiple packages of the same
name comes back to bite us. In the multilib case, we can have packages
of the same NEVR, but different arch, sitting in the same tree.

Previously, even if the arch was different, we would still mark it as an
upgrade or downgrade. But that complicates things in the case of
multiple packages of the same name in the same tree.

We greatly simplify things here by making the diff algorithm dumber. We
now only consider a package as "modified" (i.e. upgraded/downgraded) if
it has the same NA (but different EVR). This makes handling multilib
cases natural and seems worth it overall vs trying to handle the odd
e.g. noarch <--> archful pkg transitions that could happen.

Closes: #1230
Approved by: cgwalters
2018-02-06 21:30:48 +00:00
Jonathan Lebon
51fb641305 Initial support for automatic updates
This patch introduces a new `AutomaticUpdatePolicy` configuration. This
was a long time coming for rpm-ostree, given that its update model makes
it extremely apt for such a feature.

The config supports a `check` mode, which should be very useful to
Atomic Workstation users, as well as a `reboot` mode, which could be
used in its present form in simple single node Atomic Host situations.

There is still a lot of work to be done, including integrating
advisories, and supporting a `deploy` mode. This feature hopefully will
be leveraged as well by higher-level projects like GNOME Software and
Cockpit.

Closes: #1147
Approved by: cgwalters
2018-01-27 23:52:43 +00:00
Colin Walters
3e9c6cf230 Fix "releasever" option, test it by default
In #875 AKA b46fc35901 we
added support for the `releasever` option in treefiles.  I am
pretty sure it worked at the time...but I didn't add tests.

Either it never worked or some refactoring broke it. The whole chain of
`GKeyFile` → `GVariant` is so confusing. Anyways fix it by copying the string.
Now let's use it by default in the compose tests, and while we're here bump
those to F27.

I'm doing this patch now as I was playing with doing a compose from
the `/usr/share/rpm-ostree/treefile.json` and wanted to use the stock
`.repo` files.

Closes: #1220
Approved by: jlebon
2018-01-23 15:18:52 +00:00
Colin Walters
48e0cac9fb scripts: Ignore %verifyscript
It obviously shouldn't block the ability to install, and anyways
the single use in `ksh` is not compelling.  If someone comes up with
one we can revisit supporting this.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1216

Closes: #1218
Approved by: jlebon
2018-01-19 15:32:09 +00:00
Jonathan Lebon
9e24e9c793 Revert "tests/libvm: write rpm build logs to file"
This reverts commit 1ef259ed76.

Basically, for historical POSIX compatibility, `errexit` doesn't work at
all inside functions called from an if-statement/boolean context. This
is something I had already learned (and forgotten) when hacking on PAPR.

There are ways around this, but it's just not worth the added complex
shell goop for what it gives us.

http://mywiki.wooledge.org/BashFAQ/105

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
1ef259ed76 tests/libvm: write rpm build logs to file
The output from `rpmbuild` makes test results harder to comb through
when debugging. Let's just dump all that to file and only output it if
something goes wrong.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Colin Walters
bc5237ebf8 tests: Add a test case for epoch 0 in rpmdb pkglist
Basically we're choosing the libdnf behavior of omitting `Epoch: 0` which IMO is
the sane thing to do even though in *theory* they're different RPMs but
who would ever ship both without incrementing release/etc?

More information in: https://github.com/projectatomic/rpm-ostree/pull/187

Closes: #1198
Approved by: cgwalters
2018-01-11 21:58:49 +00:00
Jonathan Lebon
738fbc633b tests/libtest-core: support multiple literal checks
`grep` supports checking multiple fixed strings separated by newlines,
but it's mostly just easier to pass them as separate arguments, so let's
support that. This is now at parity with the similar
`assert_file_has_content`.

Will upstream this to ostree as well once reviewed.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
37273705dd app/db-diff: make use of new db API
This is the first step towards unifying how we introspect packages from
a specific commit. We currently do this in three ways: libdnf, librpm,
and now `rpmostree.rpmdb.pkglist`. I'd like to get to a point where we
only have `rpmostree.rpmdb.pkglist` and libdnf, the latter only when
more complex queries are required.

This patch teaches the `db diff` command to make use of the new db diff
API so that it can work even on metadata-only commits. This is relevant
for use cases mentioned in #558.

I didn't get rid of the `rpmhdrs_diff` functions right now because of
the `--changelogs` option: libdnf currently does not expose this, so we
fall back to the previous API in that case. OTOH, I wonder how much it's
actually used in the wild; maybe we could just nix it?

Closes: #1162
Approved by: cgwalters
2017-12-30 11:32:38 +00:00
Colin Walters
42282c0a84 tests/libtest: Fix logic error in creation test-repo file
`test -n` would always be true.  Also while I have the patient open
let's make writing the file atomic.  Maybe someday I'll be motivated
enough to write an `O_TMPFILE` patch for bash.

Closes: #1103
Approved by: jlebon
2017-12-04 14:24:53 +00:00
Jonathan Lebon
7056e6b726 daemon: Unify pkgcache with system repo
We originally needed the pkgcache to be a separate repo due to ostree's
overzealous pruning policies. The idea was to maintain multiple commits
in each pkg branch for different SELinux policies. In practice, there's
not much use in maintaining old copies and it's just easier to always
relabel on the fly. So then, the need for a separate repo completely
melts away.

This helps simplify the mental model a bit and allows us to avoid subtle
issues like #1047. Note however that the core is still capable of
handling split repos for the `--ex-unified-core` compose use case. Once
that and the jigdo work are a bit more settled, we can have a clearer
picture of how to simplify the core further.

The tricky bit is migrating the cache. When deploying, we check if a
pkgcache repo exists and migrate its refs if so. We then leave behind a
symlink to the system repo to remain compatible with older rpm-ostrees.

Closes: #1055
Approved by: cgwalters
2017-12-01 15:41:40 +00:00
Jonathan Lebon
e5e9f1f0b8 libvm: factor out helpers to start httpd
This will be used in other tests. Plus, it makes it much nicer to use.

Closes: #1053
Approved by: cgwalters
2017-10-13 15:00:25 +00:00
Jonathan Lebon
dd7c748ce1 tests/vmcheck: add test for refresh-md and --cache-only
Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Jonathan Lebon
dc08ebda0e tests/libvm: use rsync and add yumrepo mode
I've been lazy about actually using using rsync instead of scp when
copying new RPMs over to the VM. We do this here. Also make
`vm_send_test_repo` take a mode argument that allows callers to
completely skip the sending of the repo file itself. This will be needed
for the `makecache` test, in which we *don't* want the repo to be local.
It looks cleaner anyway for the gpgcheck use case as well.

Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Colin Walters
3f367dbce2 scripts+bwrap: Make script execution cancellable
Prep for implementing `rpm-ostree cancel`, but this works with the way we handle
`Ctrl-C` interactively on a client as well. Being able to cancel a script
execution is quite nice; some of them are expensive, and having one loop forever
has been known to happen.

Closes: #1025
Approved by: jlebon
2017-10-03 14:24:07 +00:00
Jonathan Lebon
a9c38d33b8 vmcheck: add SELinux labeling tests
There's a lot of paths in the core related to SELinux policy changes and
relabeling packages. We currently have no test coverage for them. We add
support in the test libraries here to build such packages.

We also add a test that checks both that we correctly relabel RPMs when
the policy changes and that we handle layered packages that install
SELinux packages properly.

Closes: #999
Approved by: cgwalters
2017-10-03 01:01:14 +00:00
Jonathan Lebon
2c94b11b68 libvm: don't rsync $VMCHECK_INSTTREE
We pretty standardized on insttree/ being the installation tree, and
we don't actually support $VMCHECK_INSTTREE anymore.

Closes: #1032
Approved by: cgwalters
2017-10-02 20:36:07 +00:00
Colin Walters
1acd834104 Add test infra for ex container, and one test for bash
We have some unit-style tests that run `ex container`, but
they aren't "real"; they don't use scripts for example.  Let's
add tests for this similar to `tests/compose`.

We use a 26 base, but the target repos need to be 27
to pick up the fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1478172

Add some bits to share infra between `tests/compose` and `tests/ex-container`;
basically handling the rpmmd repos. I tweaked things to be more streamlined
there between the `.papr.yml` and the test script.

Right now this is just one test for `bash`, but lays some of the infrastructure
for doing more. One thing that we need to do to improve more here is to better
cache RPMs, a bit like the compose tests do.

Closes: #1024
Approved by: jlebon
2017-10-02 18:04:24 +00:00
Colin Walters
57937d0e63 libtest: Squash a race in temp webserver setup
Saw this in one of the PR contexts.

Closes: #1031
Approved by: jlebon
2017-10-02 18:02:40 +00:00
Jonathan Lebon
cb7e84c4a6 libpriv/scripts: redirect scriptlet output to journal
Rather than just letting the scriptlets inherit the daemon's
stdout/stderr, redirect their outputs so that we can set a customized
identifier to make it easier to distinguish from the daemon output.

Also print out the `journalctl` command needed so that users can
investigate the output themselves.

Closes: #998
Approved by: cgwalters
2017-09-27 19:01:45 +00:00
Colin Walters
e066dc93c8 tests: Skip running introspection tests if built with ASAN
This is the rpm-ostree equivalent of
<47b4dd1b38>

Unfortunately, introspection uses `dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.

Prep for syncing PAPR config with ostree.

Closes: #1000
Approved by: jlebon
2017-09-24 13:32:00 +00:00
Jonathan Lebon
2f6f8a910d libvm: set up ControlPath socket in /var/tmp
It turns out there's a much smaller limit than PATH_MAX for Unix
sockets. On Linux, it's 108 characters. It took me some time to figure
out why `vmcheck` would sometimes fail depending on where the src
directory is and how ${topsrcdir} is defined. Let's just make things
safer by just using /var/tmp.

Closes: #949
Approved by: cgwalters
2017-08-24 22:12:17 +00:00
Colin Walters
0d4d6be94f Implement file triggers (%transfiletriggerin) for layered pkgs
File triggers are a post-RHEL7 thing; more information at
http://rpm.org/user_doc/file_triggers.html

There are two notable users I've been testing this with;
`glib2` and `vagrant`.  The `vagrant` one is more immediately urgent,
since it makes `vagrant-libvirt` work, which I currently rely on
for my workstation dev.

I've tested things successfully with `vagrant`, and I did verify that we run the
`glib2` ones when doing `rpm-ostree ex container`.

Long term, more transaction file triggers are likely to live in
"base" packages like `glib2`.  We don't implement those yet, but
extending this to do that shouldn't be too hard.

There was *significant* what I'd call reverse engineering of the
implementation in librpm.  The file triggers code there is spread out
and abstracted in a few different places in the code.  I found
trying to understand what header values were involved to be quite
tricky.

There are some corner cases like multiple patterns that I *think*
this does correctly, but could use more validation.  The main
question I had was - is it required that the patterns for e.g.
`%transfiletriggerin` and `%transfiletriggerun` be identical?

Closes: https://github.com/projectatomic/rpm-ostree/issues/648

Closes: #869
Approved by: jlebon
2017-07-27 20:58:09 +00:00
Jonathan Lebon
95bc93205e vmcheck/test-basic.sh: adjust for new polkit policy
Now that we require clients to have an active session to RegisterClient,
we can't use runuser to check for non-root functionality. Add a new
vm_cmd_as() function to allow connecting as a different user. While
we're there, do some minor cleanups to consistently use `local` when
possible.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
970a20974b build_rpm: print out nevra rather than just name
It's more precise and makes it possible to find out if e.g. pkg
upgrades made it to the tree.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Jonathan Lebon
2b547c2d0a tests: transition to build_rpm
Nuke all the previous goop that was used to create RPMs at `make check`
time and transition all the tests to use the new `build_rpm` function.
It definitely feels cleaner to use. It's also really nice to have the
spec live in the same file as the test that uses it.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
5cb1fa5460 libtest: add build_rpm function
Having to prepare RPMs in advance separately from the tests that use
them severely limits our ability to test various cases and to iterate
quickly when creating tests.

Add a new `build_rpm` function which can basically build the RPM on the
fly and update the yum repo afterwards. It makes it trivial to test
things like package updates:

    build_rpm foo 1.0 1
    <stuff>
    build_rpm foo 1.0 2
    <stuff>

The RPMs are all created inside the temporary test directory and thus
cleaned up on exit.

I'm doing this in a separate commit because it's the most important diff
of the transition and might be easy to lose in the larger diff where we
move all the tests to make use of this.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
4445df2c01 tests: add libtest.c for binary tests
Import libostreetest.c from ostreedev/ostree as libtest.c. This is just
a really useful and outrageous way of using libtest.sh from C.

Closes: #854
Approved by: cgwalters
2017-06-29 16:16:54 +00:00
Jonathan Lebon
b9f6d09f91 vmcheck: add new test for override remove
Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Colin Walters
2adc1cf246 core: Ignore overrides for nonexistent %ghost files in /etc
As seen in e.g. `ipa-client-common`.  We expect ghosts 👻 to not
exist.

Closes: https://github.com/projectatomic/rpm-ostree/issues/784

Closes: #785
Approved by: jlebon
2017-05-24 15:53:07 +00:00
Colin Walters
a18c2ecbf7 livefs: Fix etc merge with subdirectories
This was a kind of last-minute bug introduced when I tweaked the
checkout to use `.` to avoid a `mkdir()` for files.  But there were
multiple bugs with that; for files that are in subdirectories of `/etc`
we obviously need to get the right subdir and not use `/etc`.

Second, we need to handle the case where we're adding new subdirectories.

This change fixes `rpm-ostree install firewalld + rpm-ostree ex livefs`.

Closes: #783
Approved by: jlebon
2017-05-23 20:34:47 +00:00
Colin Walters
8ee6e86e38 tests: Use libtest-core.sh from ostree
Reduces drift.  In the future we may want to explicitly share
more test suite code too.

See https://github.com/ostreedev/ostree/pull/877

Closes: #782
Approved by: jlebon
2017-05-19 21:37:53 +00:00
Colin Walters
2ada15aff3 libtest: Fix bugs in assert_file_has_content()
First, we need to preserve whitespace in the regexps.  Second,
in the error path, all of our variables were wrong.

Commit 376a2cc3f5 is an excellent
poster child for how many bugs one can introduce in a single commit
for a ~5 line function.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
376a2cc3f5 libtest: Support matching multiple regexps in files
For a future patch.

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00