Commit Graph

234 Commits

Author SHA1 Message Date
Colin Walters
d260d918dd tests: Add a payload link unit test
Motivated by changing this code.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-07-29 17:08:03 -04:00
Joseph Marrero Corchado
cb70e4063f ostree-sysroot-deploy: check if deployments are in the same stateroot. 2024-05-03 19:57:31 -04:00
Colin Walters
f81b9fa166 sysroot: Rework /var handling to act like Docker VOLUME /var
We've long struggled with semantics for `/var`.  Our stance of
"/var should start out empty and be managed by the OS" is a strict
one, that pushes things closer to the original systemd upstream
ideal of the "OS state is in /usr".

However...well, a few things.  First, we had some legacy bits
here which were always populating the deployment `/var`.  I don't
think we need that if systemd is in use, so detect if the tree
has `usr/lib/tmpfiles.d`, and don't create that stuff at
`ostree admin stateroot-init` time if so.

Building on that then, we have the stateroot `var` starting out
actually empty.

When we do a deployment, if the stateroot `var` is empty,
make a copy (reflink if possible of course) of the commit's `/var`
into it.

This matches the semantics that Docker created with volumes,
and this is sufficiently simple and easy to explain that I think
it's closer to the right thing to do.

Crucially...it's just really handy to have some pre-existing
directories in `/var` in container images, because Docker (and podman/kube/etc)
don't run systemd and hence don't run `tmpfiles.d` on startup.

I really hit on the fact that we need `/var/tmp` in our container
images by default for example.

So there's still some overlap here with e.g. `/usr/lib/tmpfiles.d/var.conf`
as shipped by systemd, but that's fine - they don't actually conflict
per se.
2024-02-09 17:46:12 -05:00
Colin Walters
cae4ceb6c5 deploy: Honor prepare-root.conf at deploy time
I want to try to get away from the "repository global" configuration
in the repo config.

A major problem is that there's not an obvious way to configure
it as part of an ostree commit/container build - it needs
to be managed "out of band".

With this change, we parse the `usr/lib/ostree/prepare-root.conf`
in the deployment root, and if composefs is enabled there,
then we honor it.

We do still honor `ex-integrity.composefs` but that I think
we can schedule to remove.
2024-02-08 19:53:23 -05:00
Colin Walters
d36bf35c10 switchroot: Lower config parser to otcore, add unit tests
Part of the continuation of unit testing coverage.
2023-08-25 15:59:34 -04:00
Colin Walters
e3f0c4d49c tests: Add otcore unit tests
This just stubs out the basic infrastructure, to be expanded upon.
2023-08-22 13:04:46 -04:00
Colin Walters
0406fd3912 deploy: Support an empty /etc and populated /usr/etc
In preparation for support for a transient `/etc`:
https://github.com/ostreedev/ostree/issues/2868
particularly in combination with composefs.

Basically it's just much more elegant if we can directly mount
an overlayfs on the *empty* `etc` directory, using `usr/etc` as
the lower.

In the composefs case, we'd have to mount the composefs overlayfs
itself writable (and call `mkdir`) *just* so we can make that
empty `etc` directory which is ugly.
2023-07-29 08:47:54 -04:00
Colin Walters
0c36e8143d Drop "ostree trivial-httpd" CLI, move to tests directory
See https://github.com/ostreedev/ostree/issues/1593

Basically this makes it easier for people packaging, as the trivial-httpd
is only for tests, and this way the binary will live with the tests.

Also at this point nothing should depend on `ostree trivial-httpd`.
2023-07-04 08:18:24 -04:00
Colin Walters
a6f2d053c8 lib: Rework composefs metadata, drop custom signatures
We will be switching to handling signature verification of
the target ostree commit.
2023-06-17 17:08:46 -04:00
Daniel Kolesa
d0ea2db430 fetcher: add libsoup3 backend
The default is still soup2, you can use --with-soup3 to enable
the soup3 backend instead.
2023-04-12 22:33:08 -06:00
Dan Nicholson
3ec7b5db18 tests: Ensure real GIO backends aren't used
Set a few environment variables during tests to ensure fake GIO backends
are used. This is particularly important with the soup fetcher backend
as it can cause strange test errors in containerized test environments.
Upstream soup has been setting these 3 environment variables for their
tests since 2015.
2023-04-12 21:42:33 -06:00
Joseph Marrero Corchado
4f0c13fb8b
Merge pull request #2705 from cgwalters/always-boot
Add `sysroot.bootprefix` option
2023-03-17 11:09:19 -04:00
Dan Nicholson
008c5a9540 bin/remote-refs: Add option to print revisions
The only other way to get the remote ref revision from the CLI is to
scrape the output of `ostree remote summary` or pull the commit. The
revision is already there in the summary's ref map, so might as well add
an option to show it.
2023-02-07 22:59:30 -07:00
Colin Walters
23446a0218 Remove readdir-rand
This was only there to reproduce a bug we hit long ago
with bootloader file ordering.  We're extremely unlikely
to reintroduce such a bug, and it's not worth carrying around
this code.
2022-11-01 16:29:17 -04:00
Miguel Angel Ajo
e234b630f8 Support overlayfs whiteouts on checkout
Introduces an intermediate format for overlayfs storage, where
.wh-ostree. prefixed files will be converted into char 0:0
whiteout devices used by overlayfs to mark deletions across layers.

The CI scripts now uses a volume for the scratch directories
previously in /var/tmp otherwise we cannot create whiteout
devices into an overlayfs mounted filesystem.

Related-Issue: #2712
2022-09-28 12:26:31 +02:00
Colin Walters
10e465c3c4 Add sysroot.bootprefix option
This is a follow up to
0ced9fde76
"sysroot: Support /boot on root or as seperate filesystem for syslinux and u-boot"

What we should have done at the time is changed our bootloader entries
to be prefixed with `/boot`.  This means that the GRUB2 BLS support
will Just Work.

For now, I'm making this option default to off out of a lot of
conservatism.  I think in the future we should flip this on by default.
2022-08-31 16:30:31 -04:00
Huijing Hei
3bc59a5206 RFE: Add a hidden option to ostree admin kargs edit-in-place to
update all existing deployments in place

Example:
$ sudo ostree admin kargs edit-in-place --append-if-missing=rw
See https://github.com/ostreedev/ostree/issues/2617

This will not add duplicate key, if there is `TESTARG=VAL1` in the
kernel arguments, `--append-if-missing=TESTARG=VAL2` will be ignored.
2022-06-23 22:31:39 +08:00
Luca BRUNO
7e83726faa
tests: move fixture to fix installed tests
This moves around the packed archive for bare-split-xattrs mode,
in order to fix installed tests.
2022-04-12 09:53:54 +00:00
Luca BRUNO
aca5671eb1
tests/basic-bare-split-xattrs: add fixture, check read logic 2022-03-03 11:11:59 +00:00
Luca BRUNO
08e98e9042
lib/core: introduce 'bare-split-xattrs' mode 2022-03-02 16:45:00 +00:00
Luca BRUNO
513b3c09a5
main: add support for CLI extensions via external binaries
This adds some logic to detect and dispatch unknown subcommands to
extensions available in `$PATH`. Additional commands can be
implemented by adding relevant `ostree-$verb` binaries to the system.

As an example, if a `/usr/bin/ostree-extcommand` extension is provided,
the execution of `ostree extcommand --help` will be dispatched to that
as `ostree-extcommand extcommand --help`.
2021-12-20 10:00:02 +00:00
Joseph Marrero
581a58067b Update FSF license notices to use URL instead of address 2021-12-07 08:34:25 -05:00
Colin Walters
359435de84 Add an API to verify a commit signature explicitly
We have a bunch of APIs to do GPG verification of a commit,
but that doesn't generalize to signapi.  Further, they
require the caller to check the signature status explicitly
which seems like a trap.

This much higher level API works with both GPG and signapi.
The intention is to use this in things that are doing "external
pulls" like the ostree-ext tar import support.  There we will
get the commitmeta from the tarball and we want to verify it
at the same time we import the commit.
2021-08-30 13:27:38 -04:00
Dan Nicholson
58a683f8f0 bin/remote: Rename list-gpg-keys to gpg-list-keys
As pointed out in the original review, `gpg-list-keys` fits better
alongside the existing `gpg-import`.

Changes were done with:

```
git grep -l list-gpg-keys | xargs sed -i 's/list-gpg-keys/gpg-list-keys/'
for src in $(git ls-files '*list-gpg-keys*'); do
  dst=${src/list-gpg-keys/gpg-list-keys}
  git mv "$src" "$dst"
done
```
2021-08-23 15:30:31 -06:00
Dan Nicholson
74fb0c5f78 bin/remote: Add list-gpg-keys subcommand
This provides a wrapper for the `ostree_repo_remote_get_gpg_keys`
function to show the GPG keys associated with a remote. This is
particularly useful for validating that GPG key updates have been
applied. Tests are added, which checks the
`ostree_repo_remote_get_gpg_keys` API by extension.
2021-07-15 15:50:04 -06:00
Luca BRUNO
70a8f56ce1
lib/commit: respect SOURCE_DATE_EPOCH for commit timestamp
This tweaks `ostree_repo_write_commit` so that it checks for the
envinroment variable `SOURCE_DATE_EPOCH` as a way to override
the current time, which is used as the commit timestamp.

Ref: https://reproducible-builds.org/docs/source-date-epoch/
Ref: https://reproducible-builds.org/specs/source-date-epoch/
2021-06-22 12:15:18 +00:00
Colin Walters
7310203ce9 tests/inst: Make nondestructive tests runnable as unit tests
Ideally in the future we change more of our unit tests to
support running installed; we've tried this in the past with
https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests

I'd like to pick that back up again.  This takes a step
towards that by having our Rust tests.

To make this even easier, add a `tests/run-installed`
which runs the installed tests (uninstalled, confusingly
but conveniently for now).
2021-04-15 12:33:34 -04:00
Colin Walters
a90d59cf2d build-sys: Remove --enable-experimental-api
It was added for the collections bits, but we made that stable.
It's now just cruft and we're very unlikely to reuse the infrastructure
again.

Motivated by a unit test failure when running from a tarball:
https://github.com/ostreedev/ostree/issues/2313
2021-04-07 19:19:05 +00:00
Philip Withnall
0974a7faf1 tests: Split RFC 2616 date parsing code out and add tests
This makes it testable, and increases its test coverage too 100% of
lines, as measured by `make coverage`.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-22 21:03:34 +01:00
Philip Withnall
a522bf7628 tests: Add simple test for summary file caching
This test would have actually passed before the summary file caching
changes (in the previous few commits) were added, as the `summary.sig`
essentially acted as the ETag for the summary file, and itself wasn’t
updated on disk if it didn’t change when querying the server.

Actually testing that the HTTP caching headers are working to reduce
HTTP traffic would require test hooks into the pull code or the
trivial-httpd server, neither of which I have the time to add at the
moment.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-22 21:03:34 +01:00
OpenShift Merge Robot
e6f673a8ed
Merge pull request #2202 from cgwalters/bootcsum-dtb
deploy: Remove deployment bootcsum assertion
2020-09-24 22:14:31 +00:00
Colin Walters
aa2a2783ea deploy: Remove deployment bootcsum assertion
When support for devicetree was added, it created a problem
because old and new ostree versions would compute different
checksums for the "boot data".  The scenario here is:

- Have system with ostree < 2020.4
- Reboot into system with ostree 2020.5
- Try to perform an operation that would retain
  that previous booted deployment (common)

Currently ostree iterates over all the deployments
that will be retained and calls `install_deployment_kernel()`,
even for the booted one (which is a bit silly), but
just to verify that all boot data for the targeted
deployments are installed.

This then re-computes the checksum and we'd trip this
assertion.

In practice though, we don't strictly require them to match;
the only thing that will happen if they don't is that we'll
end up with another copy of the kernel/initramfs - and
that only temporarily until the previous deployment
gets GC'd.

Longer term, I think what we really want to do anyways
is probably closer to like a little ostree repo for `/boot`
so that we can e.g. still hardlink kernels there even if
the initramfs changes, or hardlink both kernel/initramfs
if just the devicetree changes, etc.

Closes: https://github.com/ostreedev/ostree/issues/2154
2020-09-24 21:24:48 +00:00
Frédéric Danis
869dbc037e tests/delta: new tests for 'ed25519' signed deltas
Add tests to generate signed deltas and verify them using 'ed25519'
signature engine.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis
c98a993c99 tests/delta: new tests for signed deltas
Add tests to generate signed deltas and verify them using 'dummy'
signature engine.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Colin Walters
40d6f6b5ee tests: Add a pre-signed-pull.sh test
I'm thinking about adding an implementation of ed25519 signatures
with OpenSSL (so we can ship the feature with Fedora CoreOS
without requiring an additional library) and in preparation for
that it's essential that we validate that libsodium-generated
signatures and OpenSSL-generated signatures are compatible.

I don't know if they are yet actually, but the goal of this
new test is to add a pre-generated repository with a signed
commit generated by libsodium.

This will catch if e.g. there's ever a change in libsodium,
or if existing libsodium implementation versions (e.g. the
one in Debian) might differ from what we ship here.
2020-06-11 18:56:35 +00:00
Colin Walters
48f5a1885e ci: Install kola tests
This builds on
https://github.com/coreos/coreos-assembler/pull/1441
to install our tests rather than running them from the source
directory.  This model will more cleanly allow us to ship
our tests along with a test container or elsewhere, separate
from the source directory.

Also prep for https://github.com/ostreedev/ostree/pull/2048
2020-05-15 19:40:54 +00:00
Colin Walters
368574b657 deploy: Add --no-merge
All of the underlying libostree APIs have supported passing `NULL`
for a merge deployment for...a long time.  But we never plumbed
it up into the CLI.

Add a `--no-merge` option to aid people who want to do a "factory reset":
https://github.com/ostreedev/ostree/issues/1793
2020-04-22 14:09:54 +00:00
Denis Pynkin
1de2efa2ed tests/sign: new test for summary file verification
Add test for signature verification of summary file during the pull.
Adopted version of GPG tests from `test-pull-summary-sigs.sh`.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:55 +03:00
Denis Pynkin
82c773710c tests/sign: add initial test for pulling
Test if we pull signed commits from remote.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
9e8f0f4ca0 tests: add test for commits sign/verification
Add tests checking:
- sign mechanism is in working state
- module 'dummy' is able to sign/verify commit
- module 'ed25519' is able to sign/verify commit
- both modules could be used for the same commit
- 'ostree sign' builtin works with commits
- 'ostree commit' builtin able to sign commits

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
OpenShift Merge Robot
8a9a496501
Merge pull request #1957 from dbnicholson/commit-sizes
Upstream Endless sizes metadata changes
2020-01-27 08:25:59 -08:00
Dan Nicholson
7fe265b087 tests/gpg-verify-data: Split out signature data
The full block with all 5 signatures remains, but this allows passing
individual signatures through the GPG verification APIs. The split was
done with `gpgsplit`, and looking at the output of `gpg --list-packets`
of the split and unsplit files appears correct.
2020-01-24 13:02:09 -07:00
Dan Nicholson
b825083549 tests/gpghome: Create revocation certificates for keys
These can then be imported during a test to revoke a key without trying
to go through the gpg --generate-revocation dialog. Note that these need
to go in a subdirectory of the homedir since `gpgkeypath` will try to
import every regular file in the homedir.
2020-01-24 13:02:09 -07:00
Dan Nicholson
97c831dd5f bin/show: Add --print-sizes option to show sizes metadata
Use the new `ostree_commit_get_object_sizes()` API to read the
`ostree.sizes` commit metadata and print a summary.
2020-01-20 20:46:31 -07:00
Dan Nicholson
260bcd1193 core: Add ostree_commit_get_object_sizes API
This function parses the object listing in the `ostree.sizes` metadata
and returns an array of `OstreeCommitSizesEntry` structures.

Unfortunately, for reasons I don't understand, the linker wants to
resolve `_ostree_read_varuint64` from `ostree-core.c` even though it's
not used by `test-checksum.c` at all.
2020-01-20 20:46:29 -07:00
Colin Walters
36039759e2 pull: Add support for basic auth
This has been requested a few times for people delivering
OSTree content and wanting to do access control.
2019-12-02 22:48:33 +00:00
Jason Wessel
bdbce9d042 fsck: Add test for --delete corruption, fix repair, and partial commit checks
The ostree fsck test is aimed to check that it will still fail an fsck
if the repository has been repaired by fsck.  It also checks that a
pull operation corrects the error and ostree fsck will exit with zero.

The test was modeled after the following script:

rm -rf ./f1
mkdir -p ./f1
./ostree --repo=./f1 init --mode=archive-z2
mkdir -p ./trial
echo test > ./trial/test
./ostree --repo=./f1 commit --tree=dir=./trial --skip-if-unchanged --branch=exp1 --subject="test Commit"

rm -rf ./f2
mkdir -p ./f2
./ostree --repo=./f2 init
./ostree --repo=./f2 pull-local  ./f1

echo whoops > `find ./f2 |grep objects |grep \\.file `
./ostree fsck --repo=./f2 ; echo Exit: $?
./ostree fsck --delete --repo=./f2 ; echo Exit: $?
./ostree fsck --repo=./f2 ; echo Exit: $?
./ostree --repo=./f2 pull-local  ./f1
./ostree fsck --repo=./f2 ; echo Exit: $?

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>

fsck: Update test so that it will pass on fs without xattrs

The fsck test does not require xattrs to prove that it works.  It is
simple enough to change it to use an archvie instead of a bare type
repository.

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>

Closes: #1910
Approved by: cgwalters
2019-09-09 13:40:36 +00:00
Denis Pynkin
e187f240fc build: Conditionally build the GPG-related tests
Do not build GPG signing/verification tests if `--without-gpgme`
option is used during configuration.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Allen Bai
be2572bf68 lib/kargs: Make API public and upstream new rpm-ostree APIs
This change makes public the current kargs API in src/libostree/ostree-kernel-args.c
and adds documentations.

Upstreams the new kargs API from rpm-ostree/src/libpriv/rpmostree-kargs-process.c

Merges libostree_kernel_args_la_SOURCES to libostree_1_la_SOURCES in Makefile-libostree.am

Upstreams tests/check/test-kargs.c from rpm-ostree.

Closes: #1833

Closes: #1869
Approved by: jlebon
2019-06-18 14:29:14 +00:00
Robert Fairley
21ebc7d21e Add sysroot.bootloader repo config key
The sysroot.bootloader key configures the bootloader
that OSTree uses when deploying a sysroot. Having this key
allows specifying behavior not to use the default bootloader
backend code, which is preferable when creating a first
deployment from the sysroot (#1774).

As of now, the key can take the values "auto" or "none". If
the key is not given, the value defaults to "auto".

"auto" causes _ostree_sysroot_query_bootloader() to be used
when writing a new deployment, which is the original behavior
that dynamically detects which bootloader to use.

"none" avoids querying the bootloader dynamically. The BLS
config fragments are still written to
sysroot/boot/loader/entries for use by higher-level software.

More values can be supported in future to specify a single
bootloader, different behavior for the bootloader code, or
a list of bootloaders to try.

Resolves: #1774

Closes: #1814
Approved by: jlebon
2019-03-01 21:20:35 +00:00