Commit Graph

42 Commits

Author SHA1 Message Date
Jonathan Lebon
fd64084b9b vmcheck: add basic test for local RPM installs
Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Colin Walters
66425c3161 core: Do GPG verification before importing
While reading a recent conversation about GPG checking at treecompose
time, I had a sudden thought - were we actually doing verification
client side?  Turned out, we aren't.  That happens as part of
`dnf_transaction_commit()` which we don't use.

That function verifies every package at one go, but for us I think it's better
to do it before "importing". We shouldn't have untrusted bits that we've
unpacked (they might have suid binaries, for one thing).

This is an embarassing problem, but it's worth emphasizing that everyone should
be retrieving repodata at a minimum over TLS, which sets a baseline. On RHEL, we
already do pinned TLS, and there are discussions about extending that elsewhere.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1422157

Closes: #656
Approved by: jlebon
2017-03-06 15:19:43 +00:00
Colin Walters
22048b25a7 scripts: Use tmpfs for /var/tmp, not the host's /tmp
We don't want to expose the host's `/tmp` since that means scripts could
potentially find things like the X11 socket or whatever.

To debug things better, add a quick bash script to run bwrap like the C code
does. Perhaps down the line we can add `rpm-ostree internals run-bwrap` or so.

Closes: #647
Approved by: jlebon
2017-02-28 23:37:15 +00:00
Jonathan Lebon
7fd474f8e1 vmcheck: adjust for new behaviour
Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Colin Walters
229fdfa7f9 scripts: Bind rather than symlink /usr/etc → /etc
This fixes resolution of relative symlinks, which fixes installation
of `firewalld` in Fedora 25.

Closes: https://github.com/projectatomic/rpm-ostree/issues/638

Closes: #640
Approved by: jlebon
2017-02-24 20:43:05 +00:00
Jonathan Lebon
88548f2e1a libvm: don't call rpm-ostree status on first time
On the first call to vmsync/vmoverlay, we do an immediate vm_ssh_wait()
to check if we have a live VM. However, we don't necessarily have a
working rpm-ostree in there if we're hacking on stuff. It gets annoying
to wait for the timeout there.

Let's just work around that by instead only calling status if we got
there through vm_reboot_cmd(), which is most likely when we're most
interested in the status output anyway.

Closes: #645
Approved by: cgwalters
2017-02-24 19:50:25 +00:00
Colin Walters
ddbaf19f1e importer: Error importing RPMs which install to /opt (outside of /usr)
See https://github.com/projectatomic/rpm-ostree/issues/233 - for RPMs which
place files in e.g. `/opt`, we have different behavior in the treecompose case
(silently drop it) versus package layering (does the wrong thing).

Since the unpacker right now is only used in the layering case, this just
ensures we'll get a consistent error there.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Jonathan Lebon
8cf071127e unpacker: support root-owned /var & /run files
I'm not sure why we weren't doing this before, but we need to also
support files in /var and /run that are owned by root.

Related: RHBZ#1421781

Closes: #622
Approved by: cgwalters
2017-02-13 16:49:59 +00:00
Jonathan Lebon
283b915ecf check: use jq for asserting json elements
Closes: #609
Approved by: cgwalters
2017-02-08 17:11:06 +00:00
Colin Walters
ace223acf8 Add pending-base-commit to status
One thing that's very confusing about OSTree is there are two layers -
deployments and the refs/commits. If one does an `rpm-ostree upgrade`, but then
e.g. `ostree admin undeploy 0`, you still have the new revision in the repo.

We don't do a good job of displaying this state, or helping people clean
it up.

Down the line, I also want to better support something like `rpm-ostree pull` to
cache updates explicitly *without* deploying.

This commit just adds a bit of information to the status display. We might want
to have better formatting, but I think this an OK start.

Closes: #595
Approved by: jlebon
2017-02-08 13:59:48 +00:00
Colin Walters
cac4522e5b Support "system/regenerate-initramfs=true" flag in origin
Currently we push for a model where the initramfs is
generated (in non-hostonly mode), and merely replicated.

However, to support a few unfortunate corner cases like dm-multipath which wants
to inject a config file into the initramfs, we need to support regenerating it
client side too.

Down the line, we'll need this to support overriding the kernel too.

This changes things in the core to add the concept of an "empty"
`RpmOstreeContext`. I initially tried skipping it, but that was too much
duplication. We still want all of the core ostree-related logic that lives in
that code too.

The treespec bits barfed if the spec didn't have a `tree/packages` key. It was
simplest to change that to allow it - and because that was the only case where
we errored out in parsing, I dropped the error handling.

There was another place in the upgrader that now needed to be fixed to handle
transitioning from just regenerating initramfs to not.

Closes: #574
Approved by: jlebon
2017-01-20 16:17:51 +00:00
Colin Walters
b099680d1d build: Capture git describe on build if available, add to --version
Many projects do this, and it really helps debugging to know the
exact hash.

(Of course this is broken in traditional rpm builds from a tarball,
 and rpmdistro-gitoverlay injects it into the Version field,
 but it will help me for vmcheck debugging)

Closes: #584
Approved by: jlebon
2017-01-19 19:49:58 +00:00
Colin Walters
d0a4a258a8 tests: Make failure of assert_file_empty() actually fatal
I just noticed this while reading the tests.  The original
ostree version looks OK, so not sure how that happened.

Closes: #580
Approved by: jlebon
2017-01-18 20:24:31 +00:00
Colin Walters
f02130b19e tests: Hard require jq
I don't see a reason not to require more stuff to execute tests,
and having it be optional reduces our coverage unnecessarily.

Closes: #580
Approved by: jlebon
2017-01-18 20:24:31 +00:00
Colin Walters
30bed29230 daemon/upgrader: Rework layer tracking
Working on initramfs, I hit a subtle issue with the fact that
I was trying to "redeploy", but with the origin file changed
during the process.

Previously, it was a bit unclear which parts of the upgrader logic are operating
on the *new* origin versus the "original origin".

The package layering code in the upgrader explicitly carries a delta on top in
the "add/remove" hash sets, which means it isn't visible to
`rpmostree_origin_is_locally_assembled()`.

Whereas for initramfs, I set a new origin. This broke things since we were
expecting to find a parent commit, but the original origin wasn't locally
assembled.

When looking more at this, I realized there's a far simpler model -
rather than keeping track of commit + origin, and using the origin
to try to determine whether or not the commit is layered, we can
keep track of `base_revision` and `final_revision`, and the latter
is only set if we're doing layering.

The diff speaks for itself here - a lot of fragile logic looking at the origin
drops away.

The next step here is probably to drop away the package layering hash sets, but
I'm trying to not change everything at once.

Closes: #579
Approved by: jlebon
2017-01-18 20:05:19 +00:00
Colin Walters
59f34e8a03 tests: Disable ASAN leak checks
Right now we're far from leak free, and I want to use ASAN
as a sanity checker, not a leak checker.

Closes: #576
Approved by: jlebon
2017-01-18 16:52:46 +00:00
Colin Walters
178ec03154 tests: Honor TEST_SKIP_CLEANUP like ostree
We seem to have substantial infrastructure drift here between the two, and I was
*really* confused why `TEST_SKIP_CLEANUP` wasn't being honored. Turned out in
rpm-ostree we have tmpdir code in two places, not entirely sure why. Punting
full cleanup.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
d3bf60d373 tests: Add a pkg with rofiles violation that should fail
To ensure we don't accidentally allow mutations.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
55caec269d libvm: Set -o User=root for rsync too
Without fuse-sshfs, we also need to set the root user, which
is necessary for the Vagrant case.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
be7dd69b7b libvm: Make it easier to debug login failures
First try to log in, and if we fail, retry with SSH debugging,
so we have a better idea what might be going wrong.  This helped
me figure out that vmcheck's `-o User=root` assumption wasn't
working with my vagrant setup.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Jonathan Lebon
9f8136e097 vmcheck: add test-layering-non-root-caps.sh
This new test exercises the new support for non-root file paths and
files with capabilities.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
b68209b6d4 daemon: start with one commit only when resolving versions
During a deploy operation, we would fetch commit objects from the remote
to resolve the version string. If gpg-verify was turned on, we would
fail to pull them if some of the commits were not signed. This is
because we pulled them in batches. We partially address this by only
fetching the HEAD commit on the first pass. This allows `upgrade`
operations to work just as well as `deploy` operations.

Though there is still an issue if we have to traverse farther back than
when signed commits become unsigned (unless they happen to fall on a
batch boundary). We leave that unsolved for now, since that would likely
require a more complex solution and it's not clear whether it's a real
world issue (signers can just retroactively sign commits).

Copy the gpghome from ostree so that we can test GPG-related cases in
our suite.

Closes: #527

Closes: #557
Approved by: cgwalters
2016-12-24 12:28:48 +00:00
Jonathan Lebon
e10c97007f rebase: add support for rebasing to a specific rev
Expand the available options in the Rebase() D-Bus method to also have a
"revision" key. Its value has the same semantics as the "revision" key
in the Deploy() method (e.g. the "revision=" and "version=" prefixes are
also supported). Also expand the rebase CLI to allow for specifying the
revision as an additional argument.

This allows users to rebase to a specific version or checksum, rather
than only to the latest. Conceptually, this is the equivalent of doing a
rebase followed by a deploy. I.e. we specify an override-commit in the
origin and expect the same behaviours that apply after a deploy to also
apply here.

Closes: #212

Closes: #555
Approved by: cgwalters
2016-12-21 20:00:43 +00:00
Jonathan Lebon
5bf7a040a5 libvm.sh: strengthen reboot
Unsurprisingly, rebooting machines that are running in OpenStack is
not as reliable or as fast as a local VM, which is what vmcheck was
originally written for and tested against.

Replace the:

  sleep 2 # give time for port to go down

which is rife with raciness, with a stronger boot_id-based check to
ensure we're in a new boot. Run "sync" before rebooting which sometimes
helps (though I didn't fully investigate why or whether it always helps,
there's probably something more subtle going on underneath). Increase
the timeout to 120s.

Closes: #543
Approved by: cgwalters
2016-12-07 21:47:58 +00:00
Jonathan Lebon
68a9719696 libvm.sh: increase reboot timeout
Closes: #538
Approved by: cgwalters
2016-12-06 21:00:23 +00:00
Colin Walters
b81c0cdfda tests: Add ./tests/compose
Our current compose tests only use a synthetic `empty.rpm`, but
this really limits usefulness.

Let's make a test suite that requires an internet connection and
downloads Fedora RPMs and does "real" tree composes.

See the updated `tests/README.md` for more information.

This is still a WIP.

Closes: #531
Approved by: jlebon
2016-12-06 19:05:05 +00:00
Jonathan Lebon
12c4b5b16f vmcheck: adapt for non-vagrant hosts
This is just the final bit required to make sure the vagrant and
non-vagrant paths can work happily together. It's mostly minor fixes,
though the most major change which also affects vagrant is that we now
sync to the root home dir, rather than ~vagrant.

Closes: #524
Approved by: cgwalters
2016-11-22 17:15:20 +00:00
Colin Walters
2f9d586bdc Redo vmcheck to abstract over ssh-config, drop all building
My development environment is now using "pet" docker containers.
I use VMs for testing things that require that (like rpm-ostree).

This patch builds on work from @jlebon in
https://github.com/projectatomic/rpm-ostree/pull/509
to rework `vmcheck` such that it can work on any `ssh-config`.  By
default we expect this to be Vagrant.

However, I go a lot farther and delete the `vmbuild` code that was
trying to do builds in a container on the target VM.  I think this is
still worth pursuing at some point, but for now I think it's
reasonable to assume that the rpm-ostree developer audience uses Linux
as their host workstation and hence has containers.

(There's another important point here in that for developing lower
 level things like rpm-ostree, there's a strong push to make the VM
 disposable and not a pet)

Closes: #516
Approved by: jlebon
2016-11-16 18:14:23 +00:00
Jonathan Lebon
313a832d7c vagrant: generate ssh-config
We further split libvm from vagrant. It no longer does 'vagrant
ssh-config'. Instead, it always assumes that an ssh-config is provided.
We now have complete separation of libvm from vagrant.

We change the ansible provisioner as follows:
  - Allow passing in a VAGRANT_BOX env var to override the default
    CentOS box.
  - No longer assume that the root user account is unlocked and has a
    valid 'vagrant' password. This worked for the centos box but isn't
    sure to work on every box. Instead, we now just run ansible as the
    default vagrant user, and during provisioning set up the root
    account and generate an ssh-config so that libvm can connect
    directly as root.
  - No longer build the buildimg during provisioning. This actually
    stopped working a while ago since the default rsync is disabled. We
    can just let the buildimg get created on the first compilation. In
    practice, the bigger issue isn't creating the buildimg, but being
    able to easily update the host and buildimg pkgs.

Closes: #516
Approved by: jlebon
2016-11-16 18:14:23 +00:00
Colin Walters
a5379a2520 tests: Test pinned package version
It should work for both layering and treecompose.

Closes: #518
Approved by: jlebon
2016-11-14 21:31:54 +00:00
Colin Walters
0d720e46e0 Add .redhat-ci.yml
Copied from ostree, tweaked a little bit.  We can't yet use
`-Werror=unused-variable` until
https://github.com/rpm-software-management/libhif/pull/205

Closes: #506
Approved by: jlebon
2016-10-31 13:17:26 +00:00
Jonathan Lebon
accd2b9f6f vmcheck: add a test for RPM scripts
Thought it'd be fun to write a test for verifying proper handling of
scriptlets during package layering. There's obviously a lot more that
could go in here (patches welcome!), but it's a start.

Closes: #434
Approved by: cgwalters
2016-08-17 21:28:01 +00:00
Jonathan Lebon
194c2bf5cd libvm.sh: add vm_setup and vm_rsync
In preparation for being vagrant independent, let's factor out some
things directly into the libvm.

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
ce2a434104 libtest.sh: output file in assert_file_has_content
This is the same patch as:
https://github.com/ostreedev/ostree/pull/264

Closes: #394
Approved by: cgwalters
2016-07-15 19:14:19 +00:00
Jonathan Lebon
f8907b6d53 vmcheck: add more pkg layering tests
- Rename test-layering.sh to test-layering-basic.sh and make it test
  both pkg-add and pkg-remove.
- Add test-layering-relayer.sh, which verifies that pkgs are properly
  relayered during the creation of new deployments (e.g. upgrades,
  rebases, deploys).
- Add test-layering-rpmdb.sh, which verifies that packages respect the
  rpm requirements before being overlayed.

Closes: #371
Approved by: cgwalters
2016-07-05 15:26:45 +00:00
Jonathan Lebon
7697702aaa vmcheck: strengthen test harness
We now make the test harness handle restoring the VM to the original
state. The wonderful thing about ostree here is that it's a perfect
shoo-in for this. We make a 'backup' of the current ref, and just have
to make sure that the VM is back on that ref after running each test.

This will allow us to write tests without worrying as much about
cleaning up in the event of an error.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
d718b820ef Makefile-tests.am: more cleanups and add pkg bar
The new package bar will be used to test conflicting packages.

Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
65f5678ecb vmcheck: add timeout option for vm_ssh_wait
Closes: #360
Approved by: cgwalters
2016-06-29 19:17:38 +00:00
Jonathan Lebon
812c431856 test-layering.sh: basic package layering test
This is a preliminary package layering test which simply installs the
foo package and verifies that it functions properly. A bunch of
primitives are added to libvm.sh to facilitate this and future tests.

Closes: #344
Approved by: cgwalters
2016-06-23 00:05:09 +00:00
Jonathan Lebon
5479ed5240 tests: refactor rpm building and add package foo
We're going to be create many different packages in the future. Let's
refactor and generalize the targets so that they can build anything.

We also add a package foo, which will be used soon for a basic test of
package layering.

Closes: #344
Approved by: cgwalters
2016-06-23 00:05:09 +00:00
Jonathan Lebon
c0924b8997 vmcheck: add a basic test harness
This is a very simple test harness inspired by the atomic one. It's a
simple bash script that sets up a permanent ssh connection to the host
and runs the test scripts. Also add a "demo" test-basic.sh test to make
sure that it works.

Closes: #344
Approved by: cgwalters
2016-06-23 00:05:09 +00:00
Jonathan Lebon
acd3c8ff41 tests: restructure dirs
- Move all the uninstalled, non-destructive tests to check/
- Add a README
- Move utilities to utils/
- Move common files to common/

Closes: #304
Approved by: cgwalters
2016-06-06 21:14:07 +00:00