1117 Commits

Author SHA1 Message Date
Dan Nicholson
5135a1e58a tests/core: Really pick C.UTF-8 locale
The case-ignoring regex `^(C|en_US)` will match any locale that starts
with `c`. On my system this is `ca_AD.utf8`, which breaks the test
suite. Instead, use a single regex that includes the joining `.` rather
than 2 separate regexes. This also changes `head` to use the `-n`
option, which has been preferred for at least 10 years in the coreutils
version and is supported by busybox as well.
2020-01-21 10:29:41 -07:00
Dan Nicholson
97c831dd5f bin/show: Add --print-sizes option to show sizes metadata
Use the new `ostree_commit_get_object_sizes()` API to read the
`ostree.sizes` commit metadata and print a summary.
2020-01-20 20:46:31 -07:00
Dan Nicholson
1bbe674d91 libarchive: Support commit sizes metadata
Call the helper to set the generate_sizes boolean so that object size
data is stored while writing the mtree.
2020-01-20 20:42:27 -07:00
John Hiesey
291e9da258 lib/commit: Include object type in sizes metadata
Append a byte encoding the OSTree object type for each object in the
metadata. This allows the commit metadata to be fetched and then for the
program to see which objects it already has for an accurate calculation
of which objects need to be downloaded.

This slightly breaks the `ostree.sizes` `ay` metadata entries. However,
it's unlikely anyone was asserting the length of the entries since the
array currently ends in 2 variable length integers. As far as I know,
the only users of the sizes metadata are the ostree test suite and
Endless' eos-updater[1]. The former is updated here and the latter
already expects this format.

1. https://github.com/endlessm/eos-updater/
2020-01-20 20:42:27 -07:00
Dan Nicholson
a4592678aa tests/sizes: Check duplicate file doesn't add sizes entry
A duplicate file will resolve to the same object, so it shouldn't add
any entries to the sizes metadata.
2020-01-20 20:42:27 -07:00
Dan Nicholson
4f1b991246 tests/sizes: Test that sizes metadata is not reused
Ensure that the object sizes hash table is cleared after a commit and
not only when the repo is closed.
2020-01-20 20:42:27 -07:00
Dan Nicholson
1ea719b76b tests/sizes: Test sizes metadata with existing objects
Repeat the commit to make sure that the files are enumerated again for
the size metadata.
2020-01-20 20:42:27 -07:00
Dan Nicholson
694b741a36 tests/sizes: Improve metadata validation
Ensure all 3 of the checksum, compressed size and uncompressed size are
correct. For repeatable objects, skip xattrs and use canonical
permissions for the commit. For the sizes, read a varint rather than
assuming they will be a single byte. To work around bugs in gjs with
byte array unpacking, manually build the array byte by byte. Split out
some helper functions to use in subsequent tests.
2020-01-20 20:42:27 -07:00
OpenShift Merge Robot
20d84f40fa
Merge pull request #1987 from akiernan/us-switchroot-tests
Skip /var test if running with systemd and libmount
2020-01-16 16:47:41 +01:00
Alex Kiernan
d61183ce43 fixup! test-switchroot.sh: Find ostree-prepare-root in installed tests 2019-12-30 13:28:35 +00:00
Alex Kiernan
e4db245bec test-switchroot.sh: Find ostree-prepare-root in installed tests
When running with installed tests, ostree-prepare-root (probably)
exists in /usr/lib. Add heuristics to look for it based on the directory
we're running from.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 14:24:30 +00:00
Alex Kiernan
87ccb400a2 tests: Skip /var test if running with systemd and libmount
If running with systemd and libmount then /var mounting is deferred for
systemd. Skip the relevant tests in this case as it will always fail.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 14:18:36 +00:00
Alex Kiernan
f6867358e2 test-switchroot.sh: Exclude /proc from file list
Since we're not interested in any file inside /proc, exclude it from the
file listing in our fake root thus avoiding failures when processes die
during our execution and find(1) can't then look inside those
directories.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 12:37:10 +00:00
Colin Walters
978bd19f97 Release 2019.6
Nothing really big here, but let's get a release out
so some bigger things like ro-sysroot, signing, sizes can
bake in master for a bit.
2019-12-09 16:18:01 +00:00
Colin Walters
36039759e2 pull: Add support for basic auth
This has been requested a few times for people delivering
OSTree content and wanting to do access control.
2019-12-02 22:48:33 +00:00
Colin Walters
16bb294384 tests/repo-finder: Run realpath() on /tmp
This fixes running this test case inside
https://github.com/cgwalters/coretoolbox
2019-11-23 18:04:58 +00:00
Robert Fairley
aadc4db012 lib/kernel-args: Store kernel args as key/value entries
Define an `OstreeKernelArgsEntry` structure, which holds
both the key and the value. The kargs order array stores
entries for each key/value pair, instead of just the keys.
The hash table is used to locate entries, by storing
entries in a pointer array for each key. The same public
interface is preserved, while maintaining ordering
information of each key/value pair when
appending/replacing/deleting kargs.

Fixes: #1859
2019-11-07 23:39:10 -05:00
Colin Walters
58980a717a lib/keyfile: Treat "group not found" the same as "key not found"
Prep for fsverity, where I want to create a new group
`[fsverity]` in the keyfile that has default values.  We should
treat the absence of a group the same as absence of a key
in these "with defaults" APIs.
2019-11-07 21:15:41 +00:00
Alex Kiernan
abf1a7392b tests: Avoid musl failure with cp -a
When copying the tree, using musl and GNU coreutils, something gets confused
when setting the ownership of symlinks and the copy fails with:

  cp: failed to preserve ownership for osdata-devel/bin: Not supported

Rework using tar to avoid the problem.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-11-01 05:05:53 +00:00
Alex Kiernan
3d48021fb0 tests/core: Assume C.UTF-8 if locale isn't found
When building with musl there's no locale command, also its default
locale is C.UTF-8, so just get C.UTF-8 if we can't find locale.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-11-01 05:05:53 +00:00
Alex Kiernan
fb519f0361 tests: Handle EPIPE failures when head terminates
When using musl, it appears that the default is line buffered output, so
when `head -1` reads from a pipe we have to handle the source end of the
pipe getting EPIPE.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-11-01 05:05:53 +00:00
Alex Kiernan
4d17cd917f tests/core: Fallback to en_US.UTF-8 locale
A number of tests expect explicit left/right single quotes in their
messages, which will never happen in the C locale. Change so we pick a
likely UTF-8 locale, or fail if we can't find one.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-10-31 21:28:30 +00:00
Colin Walters
980ca07b03 Release 2019.5 2019-10-30 17:08:00 +00:00
Jonathan Lebon
7ae8da08b9 lib/deploy: Also install HMAC file into /boot
To allow for FIPS mode, we need to also install the HMAC file from
`/usr/lib/modules` to `/boot` alongside the kernel image where the
`fips` dracut module will find it. For details, see:

https://github.com/coreos/fedora-coreos-tracker/issues/302

Note I didn't include the file in the boot checksum since it's itself a
checksum of the kernel, so we don't really gain much here other than
potentially causing an unnecessary bootcsum bump.
2019-10-29 16:45:29 -04:00
Alex Kiernan
e314b31ec9 tests/export: Guard with check for libarchive
If we are built without libarchive support, this test fails:

  error: This version of ostree is not compiled with libarchive support
  ...
  ERROR: tests/test-export.sh - too few tests run (expected 5, got 0)
  ERROR: tests/test-export.sh - exited with status 1

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-10-24 19:08:59 +01:00
Colin Walters
01a3a65525 tests: [scan-build] Initialize a variable
False positive.
2019-10-18 15:07:39 +00:00
Colin Walters
0a808ffe20 tests: Port keyfile test to new style
Just noticed in passing.
2019-10-18 15:07:39 +00:00
Colin Walters
a982dc97ea tree-wide: [scan-build] Fix some dead stores
No real issues, just quieting the scanner.
2019-10-18 15:07:31 +00:00
Jonathan Lebon
9d39e7d91e Release 2019.4
Tiny release. Just want to get out the important bugfixes instead of
backporting patches (notably the gpg-agent stuff and
`ostree-finalize-staged.service` ordering).

Closes: #1927
Approved by: cgwalters
2019-09-25 13:43:28 +00:00
Jason Wessel
bdbce9d042 fsck: Add test for --delete corruption, fix repair, and partial commit checks
The ostree fsck test is aimed to check that it will still fail an fsck
if the repository has been repaired by fsck.  It also checks that a
pull operation corrects the error and ostree fsck will exit with zero.

The test was modeled after the following script:

rm -rf ./f1
mkdir -p ./f1
./ostree --repo=./f1 init --mode=archive-z2
mkdir -p ./trial
echo test > ./trial/test
./ostree --repo=./f1 commit --tree=dir=./trial --skip-if-unchanged --branch=exp1 --subject="test Commit"

rm -rf ./f2
mkdir -p ./f2
./ostree --repo=./f2 init
./ostree --repo=./f2 pull-local  ./f1

echo whoops > `find ./f2 |grep objects |grep \\.file `
./ostree fsck --repo=./f2 ; echo Exit: $?
./ostree fsck --delete --repo=./f2 ; echo Exit: $?
./ostree fsck --repo=./f2 ; echo Exit: $?
./ostree --repo=./f2 pull-local  ./f1
./ostree fsck --repo=./f2 ; echo Exit: $?

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>

fsck: Update test so that it will pass on fs without xattrs

The fsck test does not require xattrs to prove that it works.  It is
simple enough to change it to use an archvie instead of a bare type
repository.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>

Closes: #1910
Approved by: cgwalters
2019-09-09 13:40:36 +00:00
Javier Martinez Canillas
d42f970ed3 lib/bootconfig-parser: Always include deployment index in BLS title
If there are different deployments for the same commit version, the BLS
snippets will have the same title fields (but different version fields):

$ grep title *
ostree-1-testos.conf:title TestOS 42 20190902.0 (ostree)
ostree-2-testos.conf:title TestOS 42 20190902.0 (ostree)
ostree-3-testos.conf:title TestOS 42 20190902.0 (ostree)

But bootloaders could expect the title field to be unique for BLS files.
For example, the zipl bootloader used in the s390x architecture uses the
field to name the boot sections that are created from the BLS snippets.

So two BLS snippets having the same title would lead to zipl failing to
create the IPL boot sections because they would have duplicated names:

$ zipl
Using config file '/etc/zipl.conf'
Using BLS config file '/boot/loader/entries/ostree-3-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-2-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-1-testos.conf'
Error: Config file '/etc/zipl.conf': Line 0: section name 'TestOS 42 20190902.0 (ostree)' already specified

Avoid this by always including the deployment index along with the commit
version in the title field, so this will be unique even if there are BLS
files for deployments that use the same commit version:

$ grep title *
ostree-1-testos.conf:title TestOS 42 20190902.0 (ostree:2)
ostree-2-testos.conf:title TestOS 42 20190902.0 (ostree:1)
ostree-3-testos.conf:title TestOS 42 20190902.0 (ostree:0)

$ zipl
Using config file '/etc/zipl.conf'
Using BLS config file '/boot/loader/entries/ostree-3-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-2-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-1-testos.conf'
Building bootmap in '/boot'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section 'TestOS 42 20190902.0 (ostree:0)' (default)
Adding #2: IPL section 'TestOS 42 20190902.0 (ostree:1)'
Adding #3: IPL section 'TestOS 42 20190902.0 (ostree:2)'
Preparing boot device: dasda (0120).
Done.

Closes: #1911
Approved by: cgwalters
2019-09-02 21:08:58 +00:00
Colin Walters
b15ed42111 Release 2019.3
It's been a while, and we need the new kargs API for rpm-ostree.

Closes: #1902
Approved by: rfairley
2019-08-22 18:37:38 +00:00
Jonathan Lebon
0c48769de3 bin: Better handle --
We would stop passing through `--` and args after it to the underlying
command in `ostree_run`. This made it impossible to use `--` to tell the
parser that following args starting with `-` really are positional.

AFAICT, that logic for `--` here came from a time when we parse options
manually in a big loop, in which case breaking out made sense (see
97558276e4f855442337be01abc8f90cf0dd1810).

There's an extra step here, which is that glib by default leaves the
`--` in the list of args, so we need to take care to remove it from the
list after parsing.

Closes: #1898

Closes: #1899
Approved by: rfairley
2019-08-14 12:31:10 +00:00
Colin Walters
ab73d9f525 admin/init-fs: Add a --modern switch
This skips creating the default stuff in the physical sysroot.
I don't recall why I did that to be honest; it originated with
the first commit of this file.  It might not have ever been
necessary.

In any case, it's not necessary now with Fedora CoreOS, so
prune it and let's have a clean `/`.

Keep the old behavior by default though to avoid breaking anyone.

Closes: #1894
Approved by: ajeddeloh
2019-08-08 20:02:11 +00:00
Denis Pynkin
4c8b0ac255 tests/test-repo-finder-mount: skip some tests if GPG is not supported
`ostree_repo_resolve_keyring_for_collection()` function fail the tests
if there is no GPG support.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Denis Pynkin
3767d87107 tests/gpg: fix GPG-dependent shell tests if no GPG support
Skip tests or run them without GPG-related functionality if GPGME
wasn't enabled in a build time.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Denis Pynkin
453ce83379 tests/gpg: fix the check of GPG support
Shell function `has_gpgme` shouldn't exit if GPG support is not detected
since it stop any test with error.

Added function `skip_without_gpgme` to skip the whole test if it is
useless without GPG support

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Jonathan Lebon
9332830e09 lib/pull: Allow downgrade protection with overrides
There's a valid use case for enabling the timestamp downgrade check
while still also using override commits.

We'll make use of this in Fedora CoreOS, where the agent specifies the
exact commit to upgrade to, while still enforcing that it be newer.

Closes: #1891
Approved by: cgwalters
2019-07-29 13:58:29 +00:00
Dan Nicholson
d2a9c8604e tests/gpg: Skip tests when subkeys can't be expired
The ability to expire subkeys using gpg's --quick-set-expire is only
available on gnupg 2.1.22. If expiring a subkey fails, assume this is
why and skip the tests that require it but run the actions that the
subsequent tests depend on. This was failing on the Debian Stretch CI
tests since stretch has gnupg 2.1.18.

Closes: #1892
Approved by: jlebon
2019-07-27 03:20:21 +00:00
Dan Nicholson
8ac348d19a tests/gpg: Use exit hook to kill agent in temporary GPG homedir
This wasn't available when I originally wrote this, but it ensures that
the running gpg-agent in tmpgpghome is killed in case the tests exit
early.

Closes: #1892
Approved by: jlebon
2019-07-27 03:20:21 +00:00
Dan Nicholson
615861443b tests/sizes.js: Fix byte array unpacking
Recent GJS changed how byte arrays are unpacked with some assumptions
that they are likely strings. Manually use get_child_value() and
get_byte() to ensure the correct value is parsed when checking the
`ostree.sizes` metadata.

The upstream test is currently passing fine with GJS 1.56.2, but at
Endless we (unfortunately) have a downstream change that adds the object
type as an additional byte in the array. This is parsed incorrectly by
`deep_unpack()`. We can carry this patch downstream, but this change
makes the test more robust regardless.

Closes: #1884
Approved by: cgwalters
2019-07-23 12:49:26 +00:00
Dan Nicholson
2312caad76 tests/sizes: Fix call to commit_transaction()
The GIR for commit_transaction() only has a single argument for the
GCancellable. Calling it with 2 arguments prints a GJS warning:

Gjs-Message: 15:37:40.287: JS WARNING: [/home/dan/src/ostree/tests/test-sizes.js 56]: Too many arguments to method OSTree.Repo.commit_transaction: expected 1, got 2

Currently this is harmless, but it could become a hard error in GJS at
some point.

Closes: #1884
Approved by: cgwalters
2019-07-23 12:49:26 +00:00
Dan Nicholson
abb173352d tests: Always cleanup gpg-agent when exiting
Add `libtest_cleanup_gpg()` to the array of commands to run when
exiting. This provides 2 improvements:

1. You don't need to worry about whether the test will spawn a gpg-agent
   and therefore require adding a call to `libtest_cleanup_gpg()`.

2. All the existing users were calling `libtest_cleanup_gpg()` at the
   end of the script. If there was a failure and the script exited
   early, then it wouldn't cleanup and there may be a stray gpg-agent
   hanging around.

Closes: #1799
Approved by: cgwalters
2019-06-19 17:30:24 +00:00
Dan Nicholson
0dd27bbf4b tests/libtest: Allow appending actions to be run on EXIT
Currently if a test script adds a trap on `EXIT` to run some cleanup, it
will stomp on the existing trap to run `save_core()`. Allow for scripts
to append actions that will run on exit by introducing an array that
will be iterated over by a single exit runner.

Closes: #1799
Approved by: cgwalters
2019-06-19 17:30:24 +00:00
Dan Nicholson
0c63f30783 tests/test-gpg-signed-commit: Test more key states
Extend test-gpg-signed-commit.sh to test various key states. If gpg is
found that supports the required options, keys will be generated on the
fly and changed in various ways to exercise the output from
`ostree_gpg_verify_result_describe_variant` used in `ostree show`.

I tested this using gnupg 2.2.12, so I hope it works well enough on
various gpgs found in the wild.

Closes: #1872
Approved by: cgwalters
2019-06-19 17:17:11 +00:00
Dan Nicholson
d8649f4d56 tests/libtest: Allow specifying GPG homedir to cleanup
In case the tests want to use a custom GPG homedir, allow passing in the
homedir to use when cleaning up a running gpg-agent.

Closes: #1872
Approved by: cgwalters
2019-06-19 17:17:11 +00:00
Dan Nicholson
d714e622b8 lib/gpg: Show information for expired keys
Introduce a new signature attribute for the key expiration timestamp and
display it when the key has a non-zero expiration time. Without this,
the error shown is `BAD signature`, which isn't correct.

Closes: #1872
Approved by: cgwalters
2019-06-19 17:17:11 +00:00
Allen Bai
be2572bf68 lib/kargs: Make API public and upstream new rpm-ostree APIs
This change makes public the current kargs API in src/libostree/ostree-kernel-args.c
and adds documentations.

Upstreams the new kargs API from rpm-ostree/src/libpriv/rpmostree-kargs-process.c

Merges libostree_kernel_args_la_SOURCES to libostree_1_la_SOURCES in Makefile-libostree.am

Upstreams tests/check/test-kargs.c from rpm-ostree.

Closes: #1833

Closes: #1869
Approved by: jlebon
2019-06-18 14:29:14 +00:00
Rafael Fonseca
2db79fb398 admin: allow multiple args for pin subcommand
Fixes #1816

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>

Closes: #1874
Approved by: rfairley
2019-06-17 15:06:09 +00:00
Dan Nicholson
d916383953 bin/refs: Allow overwriting existing ref
Currently if you want to update a non-alias ref, you need to first check
if it exists and use either `ostree refs --create` or `ostree reset` as
appropriate. That's unnecessarily complicated and is much less
convenient than the old `write-refs` builtin that simply called
`ostree_repo_set_ref_immediate()` without any checks.

Add a `--force` option to be used with `--create` that does not raise an
error when the destination ref already exists.

Closes: #1870
Approved by: jlebon
2019-06-12 15:00:19 +00:00