Commit Graph

82 Commits

Author SHA1 Message Date
Colin Walters
2f71136eec tests: Alias assert_not_reached() -> fatal()
We had a lot of copies of the "echo something 1>&2; exit 1" code even though
`assert_not_reached()` was it.  Hence, I think we need a shorter alias for that.

Doing this particularly since I noticed a missing `1` in an `exit 1` call in the
rpm-ostree copy of this.

Closes: #648
Approved by: jlebon
2017-01-18 14:28:29 +00:00
Colin Walters
ced22f6c9b Split trivial-httpd into separate binary
Working on the libcurl backend, I hit the issue that the trivial-httpd program
depends on libsoup. I briefly considered having two versions, but libcurl is
client only, and moreover trivial-httpd is no longer trivial - it has various
features which are used by the test suite extensively.

Hence, what we'll do is build it as a separate binary which links to libsoup,
and use it during the tests. We *also* currently still provide `ostree
trivial-httpd` since some things use it like `rpm-ostree-toolbox` and the
Cockpit tests.

After those are ported to use some other webserver, I plan to add a build-time
option to drop it.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
359ea9b58f tests: Tweak installed tests to deal with ASAN
We need to disable readdir-rand there too.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Simon McVittie
7091d288f7 Force C.UTF-8 or C locale for tests
Otherwise several tests fail, for example in this build done in a French
locale by Debian's reproducible builds initiative, to check whether
the resulting binaries are identical to what was produced in an
English locale:

<https://tests.reproducible-builds.org/debian/logs/unstable/amd64/ostree_2016.11-1.build2.log.gz>

(test-basic)
# error: Cannot write to repository: Permission non accordée
...
File 'error-message' doesn't match regexp 'Permission denied'

(test-help)
# Utilisation :
#   ostree [OPTION...] COMMAND
...
File 'out' doesn't match regexp '[Uu]sage'

(test-pull-metalink)
# error: Erreur à la ligne 1, caractère 1 : Le document doit commencer avec un élément (par ex. <book>)
...
File 'err.txt' doesn't match regexp 'Document must begin with an element'

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #558
Approved by: cgwalters
2016-10-29 18:12:42 +00:00
Jonathan Lebon
c4c030cc79 libtest: add has_gpgme() helper function
Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
William Manley
94e50ca5aa tests: Add basic tests for ostree-prepare-root
These tests use unshare and mount to prepare a fake initrd/early boot
directory structure so we can then test ostree-prepare-root.

Things that are tested:

* Running in an initrd environment
* Running without initrd
* /var and /sysroot being mounted correctly
* /usr being mounted read-only

Things not tested (yet):

* Running as init - this could be accomplished by unsharing the pid
  namespace too.
* mounting/unmounting `/proc` if `/proc/cmdline` isn't available
* Persistent overlayfs for `/usr`
* Probably other things

The tests are basic but can be extended in the future as we do more work
on `ostree-prepare-root`.

These tests must be run as root as they require the ability to `mount`
and to `unshare` the mount namespace. Perhaps in the future we can use
user namespaces for this test once they are more widely available.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
Colin Walters
87c9e227cb Revert "tests/libtest.sh: Print non-matching file on failure"
This reverts commit 71301d1824.

I was confused by the fact that the non-matching content was
empty and forgot we already did print it.

Reported-by: smcv

Closes: #390
Approved by: jlebon
2016-07-11 19:30:31 +00:00
Colin Walters
71301d1824 tests/libtest.sh: Print non-matching file on failure
We clean up the temporary directory on failure, which means it's hard
to know *why* a regex didn't match.  Print it when we hit an error.

Closes: #383
Approved by: mbarnes
2016-07-08 18:38:11 +00:00
Colin Walters
d262fc2e0f tests: Support OT_SKIP_READDIR_RAND
For some reason I'm really not inclined to debug right now, `libtool`
is eating the `-fsanitize=address` argument when linking
`libreaddir-rand.so`, which causes failures since we're trying to
`LD_PRELOAD`.

Closes: #351
Approved by: jlebon
2016-06-17 14:43:44 +00:00
Simon McVittie
8011d8c627 libtest: make gpghome in tmpdir writeable
Otherwise, during "make distcheck" it will be read-only (because it
is a copy of the read-only ${srcdir} with permissions preserved), and
deletion will fail during cleanup.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Alexander Larsson
d258b124a2 tests: Exit valgrind tests if a leak is detected
This fails the test if OT_TESTS_VALGRIND is set and valgrind
detects a leak.

Closes: #334
Approved by: cgwalters
2016-06-10 12:14:40 +00:00
Colin Walters
70af1d26b1 tests: Modernize valgrind infrastructure
The recent memleak fixes motivated me to look at the bitrotted code to
run invocations of `ostree` in the test suite underneath valgrind.

There are a few things here.  First, update suppressions file from
libhif, since I recently worked on it.

When running *uninstalled* as we now support, we need
`libtool --mode=execute` in the mix so it expands out to
the uninstalled binary and we don't valgrind the intermediate shell.

However, it's harder than that because we chdir into a tmpdir,
which defeats the libtool logic.  AFAICS, the only fix for this
is to determine the realbin path before we chdir, and then unfortunately
we need to change every use of `ostree` to `${OSTREE}` =(

Then this immediately breaks for me on RHEL7 because my ancient
copy of `valgrind-3.10.0-16.el7.x86_64` is unaware of syscall 306, i.e.
`syncfs`.

But let's do this first before I dive into that.

Closes: #292
Approved by: krnowak
2016-06-09 21:10:35 +00:00
Jonathan Lebon
5ef4898afa tests/test-libarchive.sh: add more test
- Test both tar and cpio archives
- Test more hardlink corner cases
- Test symlinks more rigorously
- Test stat override
- Test skip list

Closes: #275
Approved by: cgwalters
2016-05-06 14:44:55 +00:00
Colin Walters
e3ec83a934 repo: Add OSTREE_BOOTID override envvar for debugging
It's useful for test cases to be able to influence this.

Conflicts:
	src/libostree/ostree-repo.c

Closes: #170
Approved by: jlebon
2016-05-02 18:44:44 +00:00
Colin Walters
d5498a8b16 libtest: Skip FUSE if CAP_SYS_ADMIN not in bounding set
I'm not sure if something changed or there was a race condition , or
if extraterrestrials intervened to make the tests pass before, but
in the Docker container `/dev/fuse` does exist and since we appear
to be running as uid 0 it'll be writable.

We previously discussed that another requirement for FUSE (and most
setuid binaries) is having `CAP_SYS_ADMIN`, so let's check for that in
the bounding set too.

Closes: #266
Approved by: jlebon
2016-04-18 19:53:13 +00:00
Simon McVittie
24cab773a2 libtest: show files' contents when assertions about them fail
I've seen an intermittent test failure in an autobuilder (sbuild)
environment where logs from failed builds cannot be retrieved,
but I can no longer reproduce it. Put the contents of the offending
file in the test's failing output so that if the failure comes back,
it can be debugged.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #264
Approved by: cgwalters
2016-04-17 14:16:34 +00:00
Gatis Paeglis
4e81548447 Introducing ostree-grub-generator
ostree-grub-generator can be used to customize
the generated grub.cfg file. Compile time
decision ostree-grub-generator vs grub2-mkconfig
can be overwritten with the OSTREE_GRUB2_EXEC
envvar - useful for auto tests and OS installers.

Why this alternative approach:

1) The current approach is less flexible than using a
   custom 'ostree-grub-generator' script. Each system can
   adjust this script for its needs, instead of using the
   hardcoded values from ostree-bootloader-grub2.c.

2) Too much overhead on embedded to generate grub.cfg
   via /etc/grub.d/ configuration files. It is still
   possible to do so, even with this patch applied.
   No need to install grub2 package on a target device.

3) The grub2-mkconfig code path has other issues:
   https://bugzilla.gnome.org/show_bug.cgi?id=761180

Task: https://bugzilla.gnome.org/show_bug.cgi?id=762220

Closes: #228
Approved by: cgwalters
2016-04-04 14:19:35 +00:00
Simon McVittie
18e9169d7a libtest.sh: only check whether $(pwd) is empty once
test-sysroot.js runs libtestExec() twice, one of which is after
creating non-hidden directories in $(pwd), so this check needs to be
skipped the second time.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #234
Approved by: cgwalters
2016-03-31 18:54:13 +00:00
Simon McVittie
efb86ba9d3 libtest.sh: use G_TEST_SRCDIR, G_TEST_BUILDDIR to find resources
This fixes the bug that in installed-tests that run testlib.sh under
"bash -c" (i.e. the C and JS tests), $(dirname $0) is "." and we can't do
the LD_PRELOAD correctly:

ERROR: ld.so: object './libreaddir-rand.so' from LD_PRELOAD cannot be
preloaded (cannot open shared object file): ignored.

Similarly, those tests can't copy gpghome correctly.

This also removes the confusing situation that SRCDIR in libtest.sh
(which is ${top_srcdir}/tests) does not mean the same thing as SRCDIR
in test-abi.sh (which is just ${top_srcdir}).

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #234
Approved by: cgwalters
2016-03-31 18:54:13 +00:00
Colin Walters
bdf24cdc04 tests: Make failing to kill the GPG agent non-fatal
It's not working for me in `make check` on a RHEL 7 Workstation,
apparently because no GPG agent is spawned.  I'm guessing this has
something to do with the GPG version?

The downside of this is we will be less likely to notice if GPG
changes again and we start leaking agents like we're in The Matrix
Reloaded.  But the real solution to that is containers anyways.

Closes: #233
Approved by: smcv
2016-03-31 18:43:31 +00:00
Simon McVittie
e1ce859368 Skip tests that run rofiles-fuse if /dev/fuse or /etc/mtab unavailable
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #232
Approved by: cgwalters
2016-03-31 14:22:06 +00:00
Simon McVittie
3e3755c497 various tests: skip if temp directory lacks xattr support
Some autobuilder environments place the entire build chroot on tmpfs, so
even /var/tmp might not have this.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #232
Approved by: cgwalters
2016-03-31 14:22:06 +00:00
Colin Walters
a02b425fc2 tests: Strengthen test tmpdir sanity check, be compat with ginsttest saving
GNOME Continuous uses ginstest-runner --report-directory, which causes
the tests to save their tmpdirs persistently.  This also means the
result directories didn't match the `/(var/)?tmp` regexp, which broke
the ostree tests in GContinuous.

Fix this by simply asserting that the tmpdir either has `.tmpdir` or
nothing.
2016-03-18 08:45:25 -04:00
Colin Walters
ebd0370976 tests: Unify some tmpdir code, add ability for C to use libtest.sh
I want to be able to easily test the C API on actual data in an OSTree
repo.  The shell `libtest.sh` has code to generate it.  Bridge the two
worlds by introducing a little `libostreetest` library which has a C
API which spawns a shell that runs things in `libtest.sh`.

Yes, this is about as beautiful as it sounds, which is to say, it's
not.  But it works!

Note while we were here, I realized we were actually now creating
*two* tmpdirs per test in `make check` because the tap driver was
already doing that.  Unify it so we know the C code can rely on it.
2016-03-03 18:03:32 -05:00
Colin Walters
d25212f04a tests: Port to glib-tap.mk, make make check run all of the tests
OSTree's code for testing predates the `glib-tap.mk` making its
way into GLib.  Let's switch to it, as it provides a number
of advantages.

By far the biggest advantage is that `make check` can start to run
most of the tests *in addition* to having them work installed.

This commit keeps the installed tests working, but `make check` turns
out to be really broken because...our TAP usage has bitrotted to say
the least.  Fix that all up.

Do some hacks so that the tests work uninstalled as well - in
particular, `glib-tap.mk` and the bits encoded into
`g_test_build_filename()` assume *recursive* Automake (blah).  Work
around that by creating a symlink when installed to loop back.
2016-03-03 08:50:19 -05:00
Jonathan Lebon
eba7df0da9 ostree-sysroot: add debug option to help testing
If ostree is run in a test setup where it operates as root in a tmp
directory, it might cause issues to flag the deployments as immutable.
The test harness might simply be doing an `rm -rf` (effectively the case
for gnome-desktop-testing-runner), which will then fail.

We add a new debug option to the ostree_sysroot object using GLib's
GDebugKey functionality to allow our tests to communicate to ostree that
we don't want immutable deployments.
2016-02-19 11:58:08 -05:00
Colin Walters
5ebe43859d tests: Use "bash strict mode"
I noticed in the static deltas tests, there were some tests that
should have been under `-o pipefail` to ensure we properly propagate
errors.

There were a few places where we were referencing undefined variables.

Overall, this is clearly a good idea IMO.
2016-01-27 11:44:10 -05:00
Matthew Barnes
df0cae4375 tests: Export OSTREE_SYSROOT in setup_os_repository
Eliminates the need for constantly passing --sysroot=sysroot, but
also makes ostree place remote configs for sysroot/ostree/repo in
sysroot/etc/ostree/remotes.d where they should have been all along.
2015-07-16 12:49:34 -04:00
Colin Walters
92deafec46 tests: Run all tests through a randomized readdir()
Having undefined (but in practice rarely changing) ordering for
`readdir()` ended up screwing us over for bootloader config
generation; see https://bugzilla.redhat.com/show_bug.cgi?id=1226520

Let's make things significantly more likely to fail more quickly in
the future if similar bugs are introduced.  We accomplish this by
introducing a little `LD_PRELOAD` library that randomizes the results
of `readdir()`.
2015-06-02 12:02:59 -04:00
Colin Walters
5ec148ec4d Revert "tests: Run all tests through a randomized readdir()"
Unintentionally pushed.

This reverts commit ce49264157.
2015-06-02 12:02:28 -04:00
Colin Walters
ce49264157 tests: Run all tests through a randomized readdir()
Having undefined ordering (but in practice rarely changing)
ordering for `readdir()` ended up screwing us over with respect
to bootloader config file read ordering.

Let's make things significantly more likely to fail more quickly in
the future if similar bugs are introduced.  We accomplish this by
introducing a little `LD_PRELOAD` library that randomizes the results
of `readdir()`.
2015-06-01 22:34:14 -04:00
Giuseppe Scrivano
6aeeba4280 tests: add a test for signed summary file
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-05-07 21:58:04 +02:00
Matthew Barnes
9b8bad3c2e tests: Update test-gpg-signed-commit.sh
Utilize and test new CLI capabilities:

  - Signature count in 'ostree show' result
  - Duplicate signatures now rejected
  - Ability to delete signatures
2015-03-18 11:53:26 -04:00
Giuseppe Scrivano
aea173b3f8 tests: enforce ${CMD_PREFIX} on all ostree processes
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 18:32:24 -05:00
Colin Walters
2f205331f4 tests: Move test gpg keyring into writable tmpdir
At least the version of gpgme in RHEL7 wants to create a lock file,
work around this by copying it into the writable test temporary
directory.
2015-02-23 15:21:23 -05:00
Colin Walters
b43ce2329e tests: Add some versioning metadata and test it appears in status output
Followup to previous commits.
2014-10-23 11:24:44 -04:00
Giuseppe Scrivano
7b0e70b72f tests: do not leave running httpd after tests exit
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2014-10-07 12:05:59 +02:00
Owen W. Taylor
7fce7e0338 Add test for the behavior of --help
Recursive over ostree and all subcommands, and check that --help
is supported, properly outputs to standard out, and exits
with a 0 exit status. Check that for commands with subcommands,
they produce the help output to standard error when run with no arguments.

https://bugzilla.gnome.org/show_bug.cgi?id=737194
2014-09-25 14:28:58 -04:00
Colin Walters
f2e0162846 upgrade: Refuse chronologically older commits unless --allow-downgrade
We don't want to allow MITM attackers to intercept upgrade requests
and provide clients with older OS versions vulnerable to security
flaws.

Only "ostree admin upgrade" gets this behavior for now - whether we
want to do it for "ostree admin switch" is another question.
2014-02-20 18:25:56 -05:00
Colin Walters
26d5db696b tests: Fix up GPG tests for more strict EL7 GPG
These GPG tests were failing for me on EL7 - it appears to be because
we had only one directory for both private and public keys, and we
were giving that to ostree for verification, which passed them onto
gpgv.

In EL7 beta at least, gpgv now barfs if it finds a private key where
it is just expecting to find public keys.

Fix this by splitting out the public trusted directory from the
private key directory.  Except now for signing, we still need the
public key there, so symlink it.  Whee!
2014-02-10 09:40:35 -05:00
Colin Walters
878a43411e admin/switch: New builtin to switch between trees
This is something I want to make easier, as it better showcases the
flexibility of OSTree.
2014-01-18 17:47:16 -05:00
Colin Walters
b2a98f4ca3 sysroot: Handle deleting directories in /etc
We need to use the full shutil_rm_rf() in order to actually delete
complete directories.

Test suite code based on a patch from Sjoerd Simons <sjored@luon.net>

https://bugzilla.gnome.org/show_bug.cgi?id=710097
2013-10-15 14:56:43 -04:00
Colin Walters
af1c9b8721 tests: Extend test-sysroot.js further
Now covers a lot more of the API.
2013-10-04 10:52:05 -04:00
Jeremy Whiting
7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Jeremy Whiting
7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters
298625d7f8 deploy: Correctly swap bootloader version with new boot checksums
If we had two deployments with different boot checksums, and were
trying to remove the one that was the same and add a new one (the
normal case), we'd end up assuming due to comparison with 0 that
we only needed to do the fast subbootversion swap.

Fix this by actually putting 1 where we really mean 1.

And update the tests to verify the fix; I have double-verified by
undoing the fix, and noting that the test fails.

https://bugzilla.gnome.org/show_bug.cgi?id=708351
2013-09-20 11:21:08 -04:00
Javier Martinez Canillas
ae61abb029 tests: allow setup_os_repository to create U-Boot stub config
Currently OSTree supports two bootloader backends: syslinux
and U-Boot; allow to create a stub configuration for both.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Javier Martinez Canillas
f19ba12360 tests: don't create a syslinux stub unconditionally in libtest
libtest.sh has an setup_os_repository() helper function tha is
used by many tests to setup an OSTree initial repository.

This function creates an syslinux configuration unconditionally
but OSTree supports other bootloader backends besides syslinux.

So, is better to conditionally create a syslinux configuration
only when it is needed.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Colin Walters
b2625268aa tests: Drop kernel modules from checksums
We can actually share these across different kernels.  Thanks to
Tobias Hunger <tobias.hunger@gmail.com> for pointing this out.
2013-08-30 18:12:31 -04:00
Jeremy Whiting
499df2a90b pull: Add support for resuming downloads via range requests
Use a consistent temporary filename to download uri's.
Check for downloaded files before fetching from uri.
Download to hash.part file, then copy/move to hash.done when complete.
Add argument support to setup_fake_remote_repo1 function.
Add test for pull resume.
To implement this, pass --force-range-requests into the trivial-httpd,
which will only serve half of the objects to clients at a time.

https://bugzilla.gnome.org/show_bug.cgi?id=706344
2013-08-28 14:35:54 -04:00