Commit Graph

153 Commits

Author SHA1 Message Date
Colin Walters
e3291ccce9 tests: Rework detection of trivial-httpd
Because it's now at the toplevel.
2023-09-19 16:52:13 -04:00
Colin Walters
9d5ccfefff Add ostree admin stateroot-init as alias for os-init
To further help deprecate the confusing "osname" terminology.
2023-08-17 16:46:02 -04:00
Colin Walters
0c36e8143d Drop "ostree trivial-httpd" CLI, move to tests directory
See https://github.com/ostreedev/ostree/issues/1593

Basically this makes it easier for people packaging, as the trivial-httpd
is only for tests, and this way the binary will live with the tests.

Also at this point nothing should depend on `ostree trivial-httpd`.
2023-07-04 08:18:24 -04:00
Simon McVittie
3500ad0443 tests: Ensure non-root users have access to libcap tools
On Debian systems, by default only root has /{usr/,}sbin in PATH.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2023-02-27 12:40:05 +00:00
Colin Walters
647f7fb7f1 tests: Require OSTREE_TEST_SUDO to be set to use sudo
IMO, tools invoking `sudo` automatically is a huge anti-pattern.
Require this to be opt-in.

In practice also...these tests really want to be *integration*
and not unit tests because involving `sudo` alongside uninstalled
binaries just creates a giant mess.

Closes: https://github.com/ostreedev/ostree/issues/2761
2022-11-22 10:27:28 -05:00
Colin Walters
23446a0218 Remove readdir-rand
This was only there to reproduce a bug we hit long ago
with bootloader file ordering.  We're extremely unlikely
to reintroduce such a bug, and it's not worth carrying around
this code.
2022-11-01 16:29:17 -04:00
Miguel Angel Ajo
e234b630f8 Support overlayfs whiteouts on checkout
Introduces an intermediate format for overlayfs storage, where
.wh-ostree. prefixed files will be converted into char 0:0
whiteout devices used by overlayfs to mark deletions across layers.

The CI scripts now uses a volume for the scratch directories
previously in /var/tmp otherwise we cannot create whiteout
devices into an overlayfs mounted filesystem.

Related-Issue: #2712
2022-09-28 12:26:31 +02:00
Joseph Marrero
581a58067b Update FSF license notices to use URL instead of address 2021-12-07 08:34:25 -05:00
Luca BRUNO
27fcee861a
libtest: tweak selinux/relabel message 2021-09-06 07:41:00 +00:00
Simon McVittie
2c5fa2cdb6 tests: Unset SOURCE_DATE_EPOCH
Some distributions set this during build in order to have reproducible
builds from the same source code: for example, Debian uses the date
from debian/changelog.

However, some of our tests assume that `ostree commit` will result in
a commit with the current date/time, and SOURCE_DATE_EPOCH breaks that
assumption. Unset it for our build-time tests.

Resolves: https://github.com/ostreedev/ostree/issues/2405
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-07-26 11:48:45 +01:00
Simon McVittie
3cc4f8710f libtest.sh: Remove duplicate ERR trap and report_err()
Since #2377 was merged, this is in libtest-core.sh, which is sourced by
libtest.sh.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-22 10:37:39 +01:00
Timothée Ravier
02527f115e *: rename master to main in tests & examples 2021-05-07 16:55:03 +02:00
Dan Nicholson
e660855796 tests: Test without a cache directory by default
Several tests generate summaries and then expect to use the generated
summary immediately. However, this can cause intermittent test failures
when they inadvertantly get a cached summary file. This typically
happens when the test is run on a filesystem that doesn't support user
extended attributes. In that case, the caching code can only use the
last modified time, which only has 1 second granularity. If tests don't
carefully manage the summary modification times or the repo cache then
they are likely subject to races in some test environments.

This introduces an environment variable `OSTREE_SKIP_CACHE` that
prevents the repo from using a cache directory. This is enabled by
default in tests and disabled for tests that are a explicitly trying to
test the caching behavior.

Fixes: #2313
Fixes: #2351
2021-04-19 11:09:44 -06:00
Simon McVittie
9d48d39a4b libtest: On failure, make it clearer what has happened
If we fail as a result of `set -x`, It's often not completely obvious
which command failed or how. Use a trap on ERR to show the command that
failed, and its exit status.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-04-18 12:17:00 +01:00
Colin Walters
a90d59cf2d build-sys: Remove --enable-experimental-api
It was added for the collections bits, but we made that stable.
It's now just cruft and we're very unlikely to reuse the infrastructure
again.

Motivated by a unit test failure when running from a tarball:
https://github.com/ostreedev/ostree/issues/2313
2021-04-07 19:19:05 +00:00
Colin Walters
857587615d Add an API+CLI to inject metadata for bootable OSTree commits
I was doing some rpm-ostree work and I wanted to compare two
OSTree commits to see if the kernel has changed.  I think
this should be a lot more natural.

Add `ostree commit --bootable` which calls into a new generic
library API `ostree_commit_metadata_for_bootable()` that
discovers the kernel version and injects it as an `ostree.linux`
metadata key.  And for extra clarity, add an `ostree.bootable`
key.

It's interesting because the "core" OSTree layer is all about
generic files, but this is adding special APIs around bootable
OSTree commits (as opposed to e.g. flatpak as well as
things like rpm-ostree's pkgcache refs).

Eventually, I'd like to ensure everyone is using this and
hard require this metadata key for the `ostree admin deploy`
flow - mainly to prevent accidents.
2021-03-12 19:01:42 +00:00
Colin Walters
d49f3291ad Add --enable-sanitizers, fix make check with it
It's cleaner if this is an build option rather than being
kludged into the CI layer.

Notably we can't use `LD_PRELOAD` anymore with ASAN, so update
our tests to check for `ASAN_OPTIONS`.
2021-02-03 20:33:19 +00:00
Frédéric Danis
96bcc25632 tests/libtest.sh: Add skip_without_sign_ed25519() function
Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Colin Walters
f14aa894d3 ci: Test for clock skew
I saw `tests/test-admin-deploy.none.sh` fail in one CI run, and
I want to check if it was because of clock skew, so fail
fast if we detect that.

xref https://github.com/ostreedev/ostree/pull/2099#issuecomment-629805375
2020-05-18 18:48:28 +00:00
Colin Walters
d1e5ed379e Use sign-ed25519 for the feature name
`libsodium` is an implementation detail.  In particular, I'd like
to consider using OpenSSL for ed25519 (if libsodium isn't configured
and openssl is).

So switch the name of the exposed feature and adjust the tests.
2020-04-16 16:43:43 +00:00
Denis Pynkin
a9df634c47 tests/libtest: add functions for ed25519 tests
Add functions for keys generation to be used in signing-related tests:
- gen_ed25519_keys initializing variables ED25519PUBLIC, ED25519SEED and
  ED25519SECRET with appropriate base64-encoded keys
- gen_ed25519_random_public print a random base64 public key (used in
  tests with wrong keys)

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
9e8f0f4ca0 tests: add test for commits sign/verification
Add tests checking:
- sign mechanism is in working state
- module 'dummy' is able to sign/verify commit
- module 'ed25519' is able to sign/verify commit
- both modules could be used for the same commit
- 'ostree sign' builtin works with commits
- 'ostree commit' builtin able to sign commits

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Dan Nicholson
63414e85c3 tests/libtest: Make temporary gpghome private
gpg prints a warning about unsafe permissions if the homedir is group or
world readable. This is just noise in the test logs, so appease it by
making the homedir 700.
2020-01-24 13:02:09 -07:00
Dan Nicholson
7f04c5d764 tests/libtest: Record long GPG key IDs and fingerprints
Use long GPG key IDs as it's safer and matches the format used by gpg
and gpgme. Add the associated fingerprints since these are needed by gpg
when manipulating keys.
2020-01-24 13:02:09 -07:00
Alex Kiernan
87ccb400a2 tests: Skip /var test if running with systemd and libmount
If running with systemd and libmount then /var mounting is deferred for
systemd. Skip the relevant tests in this case as it will always fail.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 14:18:36 +00:00
Alex Kiernan
abf1a7392b tests: Avoid musl failure with cp -a
When copying the tree, using musl and GNU coreutils, something gets confused
when setting the ownership of symlinks and the copy fails with:

  cp: failed to preserve ownership for osdata-devel/bin: Not supported

Rework using tar to avoid the problem.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-11-01 05:05:53 +00:00
Jonathan Lebon
7ae8da08b9 lib/deploy: Also install HMAC file into /boot
To allow for FIPS mode, we need to also install the HMAC file from
`/usr/lib/modules` to `/boot` alongside the kernel image where the
`fips` dracut module will find it. For details, see:

https://github.com/coreos/fedora-coreos-tracker/issues/302

Note I didn't include the file in the boot checksum since it's itself a
checksum of the kernel, so we don't really gain much here other than
potentially causing an unnecessary bootcsum bump.
2019-10-29 16:45:29 -04:00
Denis Pynkin
453ce83379 tests/gpg: fix the check of GPG support
Shell function `has_gpgme` shouldn't exit if GPG support is not detected
since it stop any test with error.

Added function `skip_without_gpgme` to skip the whole test if it is
useless without GPG support

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Dan Nicholson
abb173352d tests: Always cleanup gpg-agent when exiting
Add `libtest_cleanup_gpg()` to the array of commands to run when
exiting. This provides 2 improvements:

1. You don't need to worry about whether the test will spawn a gpg-agent
   and therefore require adding a call to `libtest_cleanup_gpg()`.

2. All the existing users were calling `libtest_cleanup_gpg()` at the
   end of the script. If there was a failure and the script exited
   early, then it wouldn't cleanup and there may be a stray gpg-agent
   hanging around.

Closes: #1799
Approved by: cgwalters
2019-06-19 17:30:24 +00:00
Dan Nicholson
0dd27bbf4b tests/libtest: Allow appending actions to be run on EXIT
Currently if a test script adds a trap on `EXIT` to run some cleanup, it
will stomp on the existing trap to run `save_core()`. Allow for scripts
to append actions that will run on exit by introducing an array that
will be iterated over by a single exit runner.

Closes: #1799
Approved by: cgwalters
2019-06-19 17:30:24 +00:00
Dan Nicholson
0c63f30783 tests/test-gpg-signed-commit: Test more key states
Extend test-gpg-signed-commit.sh to test various key states. If gpg is
found that supports the required options, keys will be generated on the
fly and changed in various ways to exercise the output from
`ostree_gpg_verify_result_describe_variant` used in `ostree show`.

I tested this using gnupg 2.2.12, so I hope it works well enough on
various gpgs found in the wild.

Closes: #1872
Approved by: cgwalters
2019-06-19 17:17:11 +00:00
Dan Nicholson
d8649f4d56 tests/libtest: Allow specifying GPG homedir to cleanup
In case the tests want to use a custom GPG homedir, allow passing in the
homedir to use when cleaning up a running gpg-agent.

Closes: #1872
Approved by: cgwalters
2019-06-19 17:17:11 +00:00
Jonathan Lebon
978cffed70 lib/deploy: Don't include version twice in bootmenu title
In Silverblue right now, the boot menu title looks like this:

    Fedora 29.20190301.0 (Workstation Edition) 29.20190301.0 (ostree)

This is because RPM-OSTree's `mutate-os-release` feature is enabled,
which injects the OSTree version string directly into `VERSION` and
`PRETTY_NAME`. So appending the version string again is a bit redundant.
Let's just do a simple substring check here before adding the version to
the title.

Closes: #1829
Approved by: cgwalters
2019-03-09 19:44:15 +00:00
Robert Fairley
21ebc7d21e Add sysroot.bootloader repo config key
The sysroot.bootloader key configures the bootloader
that OSTree uses when deploying a sysroot. Having this key
allows specifying behavior not to use the default bootloader
backend code, which is preferable when creating a first
deployment from the sysroot (#1774).

As of now, the key can take the values "auto" or "none". If
the key is not given, the value defaults to "auto".

"auto" causes _ostree_sysroot_query_bootloader() to be used
when writing a new deployment, which is the original behavior
that dynamically detects which bootloader to use.

"none" avoids querying the bootloader dynamically. The BLS
config fragments are still written to
sysroot/boot/loader/entries for use by higher-level software.

More values can be supported in future to specify a single
bootloader, different behavior for the bootloader code, or
a list of bootloaders to try.

Resolves: #1774

Closes: #1814
Approved by: jlebon
2019-03-01 21:20:35 +00:00
Robert Fairley
4f096c8f11 tests: Move assert_fail function to tests/libtest.sh
This moves the assert_fail function definition which was defined
and called in tests/test-remote-headers.sh. Done in preparation
for use of the assert_fail function in other test files.

Closes: #1669
Approved by: jlebon
2018-07-05 17:59:09 +00:00
William Manley
bab3b2bd4c tests: Save corefiles back to tests/ directory if one exists
Makes it easier to debug failures from the tests.

Closes: #1657
Approved by: cgwalters
2018-06-28 15:07:14 +00:00
Jonathan Lebon
dc4aa346a3 lib/deploy: Also compare deployment csum versions
When comparing deployments to determine whether we need a new
bootversion, we should also check whether the commit "version" metadata
is the same. Otherwise, we may end up with the a bootconfig whose
`title` includes a version that doesn't match the one from the
deployment checksum.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1343

Closes: #1556
Approved by: cgwalters
2018-04-24 17:04:27 +00:00
Jonathan Lebon
4e4436beec lib/fetcher: Allow clients to append to User-Agent
We do already have `http-headers`, which potentially could be used to
allow clients to completely override the field, but it seems like the
more common use case is simply to append.

Closes: #1496
Approved by: cgwalters
2018-03-16 19:21:31 +00:00
Marcus Folkesson
6bf4b3e1d8 Add SPDX-License-Identifier to source files
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.

Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
  text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
  environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
  license for their code is respected

See http://spdx.org for further reading.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1439
Approved by: cgwalters
2018-01-30 20:03:42 +00:00
Colin Walters
9d82d258ba tests: Add a "pull-test2" that uses slightly more realistic content
This one actually exercises the delta paths more; I pulled the
data from some local F26AH builds.  One thing I noticed while doing
that is that we try bsdiff on xz'd data, but that won't be useful.
https://github.com/projectatomic/rpm-ostree/issues/470
https://bugzilla.redhat.com/show_bug.cgi?id=1367496

Anyways I verified this test fails without the fixup for `pread()`
in https://github.com/ostreedev/ostree/pull/1312

Closes: #1314
Approved by: jlebon
2017-11-01 17:38:26 +00:00
Colin Walters
62445166fc lib/pull: Delete unused "exampleos" code
We never ended up using this, and I'm going to revisit this in another patch
with a different approach that has useful *content* and not just a lot of files.

Closes: #1314
Approved by: jlebon
2017-11-01 17:38:26 +00:00
Jonathan Lebon
a06bd82cd4 tests: check for relabeling rather than overlay
Instead of checking for overlayfs, let's explicitly check for our
ability to relabel files since we now have a `libtest` function to do
this. Also port that logic to `libostreetest`.

Note that overlayfs *does* allow manipulating user xattrs. So ideally,
we should break down `OSTREE_NO_XATTRS` further to distinguish between
tests that use bare repos from other modes.

We check the current directory instead of `/` so that developers can
just point `TEST_TMPDIR` to a non-overlayfs mount point when hacking
from a container.

Closes: #1170
Approved by: cgwalters
2017-09-30 00:05:07 +00:00
Jonathan Lebon
e4a90caeb9 tests/libtest.sh: always nuke repo and files
This allows users to easily re-initialize the test repo.

Closes: #1170
Approved by: cgwalters
2017-09-30 00:05:07 +00:00
Philip Withnall
9546e6795e create-usb: Add a create-usb command to complement OstreeRepoFinderMount
This can be used to put OSTree repositories on USB sticks in a format
recognised by OstreeRepoFinderMount.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1182
Approved by: cgwalters
2017-09-27 14:44:00 +00:00
Colin Walters
5963d5a2a9 tests,ci: Move "test-basic" (bare mode) to installed test
Our CI uses default Docker, which has SELinux labeling but is rather
evil in returning `EOPNOTSUPP` to any attempts to set `security.selinux`,
even if to the same value.

The previous fire 🔥 for this was: https://github.com/ostreedev/ostree/pull/759

The `bare` repo mode really only makes sense as uid 0, so our installed
test framework is a good match for this.  However, the unit tests *do*
work in a privileged container even as non-root, and *also* should
work on SELinux-disabled systems.  So let's teach the test framework
how to skip in those situations.

I tested this both in a priv container (my default builder) and an unpriv
container (like our CI).

At the same time, start executing the `test-basic.sh` from an installed test,
so we get better coverage than before.

This is just the start - all of the sysroot tests really need the
same treatment.

Closes: #1217
Approved by: jlebon
2017-09-27 13:13:14 +00:00
Jonathan Lebon
95bfe6b862 tests/libtest: check that we have setfattr
We use `setfattr` to determine whether the filesystem we're on supports
xattrs, but we need to check that `setfattr` itself is available. We
just make it a hard requirement but only if trying to run tests that ask
about xattr support.

Closes: #1207
Approved by: cgwalters
2017-09-21 21:50:40 +00:00
Colin Walters
75150fe04a lib/repo: Don't syncfs or fsync() dirs if fsync opt is disabled
There are use cases for not syncing at all; think build cache repos, etc. Let's
be consistent here and make sure if fsync is disabled we do no sync at all.

I chose this opportunity to add tests using the shiny new strace fault
injection.  I can forsee using this for a lot more things, so I made
the support for detecting things generic.

Related: https://github.com/ostreedev/ostree/issues/1184

Closes: #1186
Approved by: jlebon
2017-09-21 13:21:59 +00:00
Colin Walters
7a8511e0ca tests/libtest: Factor out user xattr detection
Make the equivalent of a `GOnce` or Rust `lazy_static!` detecting this and share
it between the two callers.

Prep for a future similar patch for strace fault injection.

Closes: #1186
Approved by: jlebon
2017-09-21 13:21:59 +00:00
Simon McVittie
e3c3ec5dd9 tests: Reset umask to 022 while creating test repository
In test-basic-root.sh we make assertions about the permissions
of files like baz/cow, which were created without an explicit chmod.
We can't do that unless we control the permissions.

For some reason the "debomatic" autobuilder used to do some Debian
archive rebuilds does the entire build including build-time tests
as uid 0 with umask 002, which broke those assertions. This seems
a weird thing to do, and I've opened a bug, but it also seems
reasonable to fix this test.

This also lets us remove a couple of existing workarounds for the
same issue.

Bug-Debian: https://bugs.debian.org/876138
Signed-off-by: Simon McVittie <smcv@collabora.com>

Closes: #1192
Approved by: cgwalters
2017-09-19 20:41:36 +00:00
Simon McVittie
a7686a0c40 libtest: Allow skipping single checks without user xattrs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1120
Approved by: cgwalters
2017-08-29 19:08:59 +00:00