1357 Commits

Author SHA1 Message Date
Colin Walters
1dd64eb264 Release 2025.2
Signed-off-by: Colin Walters <walters@verbum.org>
2025-03-21 08:13:36 -04:00
Colin Walters
75f2f7d50d tests/auto-prune: Don't go negative
- First I kept wondering what the magic of 10000 was here before
  looking above and noticing it matched the number of dtb files.
  Make a shared variable so the connection is more obvious
- Next, I *believe* the intention of this test was to test
  the edge case of bytes vs blocks, but we ended up subtracting
  blocks and I think recent FCOS images happened to get small
  enough that we started going negative here.
  Fix this to convert the bytes into blocks.
2025-03-21 08:13:36 -04:00
Daiki Ueno
79470298bf sign: Support generic "spki" type of commit signatures
The current "ed25519" signing type assumes raw Ed25519 key format for
both public and private keys. This patch generalizes it by adding a
new signature type "spki" which uses the X.509 SubjectPublicKeyInfo
format for public keys. Keys in this format can easily be created with
openssl tools and provide crypto agility[1] as the format embeds
algorithm identifier.

The supposed use-case of this feature is to attach multiple signatures
with different algorithms to a single commit, so even if an algorithm
turned vulnerable, the signatures made with other algorithms can still
be used as a fallback. For instance, signer can create an Ed25519
signature along with a quantum-resistent ML-DSA signature.

The following are a couple of implementation notes:

- The private keys shall be stored in the PKCS#8 format, though future
  extensions may support other format such as opaque key handles on a
  hardware token.

- The "spki" signature type prefers the keys to be encoded in the PEM
  format on disk, while it still accepts base64 encoded keys when given
  through the command-line.

1. https://en.wikipedia.org/wiki/Cryptographic_agility

Signed-off-by: Daiki Ueno <dueno@redhat.com>
2025-03-18 16:22:16 +09:00
Daiki Ueno
86a7a53d86 tests: Use tap_ok/tap_end in test-signed-commit.sh
Signed-off-by: Daiki Ueno <dueno@redhat.com>
2025-03-18 16:21:25 +09:00
Daiki Ueno
1a6165e3ee sign: Add PEM reading facility
This adds a new class OstreePemReader, which reads PEM blocks from an
input stream.  This would be useful for the "x509" signing backend, as
the keys are typically stored in the PEM format.

Signed-off-by: Daiki Ueno <dueno@redhat.com>
2025-03-18 16:21:25 +09:00
Daiki Ueno
dc1f9fb49b sign: Fix typo in error messages and comments
Signed-off-by: Daiki Ueno <dueno@redhat.com>
2025-03-18 16:21:00 +09:00
Colin Walters
a433e44bba
Merge pull request #3387 from smcv/debug3386
test-gpg-verify-result: Show what the result was before asserting about it
2025-03-17 09:21:43 -04:00
Colin Walters
8df797d355
Merge pull request #3389 from jlebon/pr/drop-finalize-staged-path
boot: Drop ostree-finalize-staged.path
2025-03-10 15:57:54 -04:00
Jonathan Lebon
2b9912e9f9
boot: Drop ostree-finalize-staged.path
This effectively reverts ac1a919f ("boot: Add
ostree-finalize-staged.path").

A bug came in on the OCP side that demonstrates that the way things are
setup right now is racy. If a reboot is triggered quickly after staging
a deployment, the whole pipeline of:

- ostree-finalize-staged.path, which triggers
- ostree-finalize-staged.service, which triggers
- ostree-finalize-staged-hold.service,

may not fully have happened before systemd isolates to `reboot.target`
which will want to kill all pending jobs.

Just directly starting the systemd unit is less elegant but much more
explicit and gets rid of any possible race because it's directly part of
the staging operation.

Fixes: https://issues.redhat.com/browse/OCPBUGS-51150
2025-03-07 17:11:44 -05:00
Simon McVittie
3bdcf24b28 test-gpg-verify-result: Show what the result was before asserting about it
Helps: https://github.com/ostreedev/ostree/issues/3386
Signed-off-by: Simon McVittie <smcv@debian.org>
2025-02-26 19:58:31 +00:00
Colin Walters
0c55f68006 tests/basic: Add lots of user. xattrs
This exercises our requirement for xattr sorting.

Signed-off-by: Colin Walters <walters@verbum.org>
2025-02-25 14:58:35 -05:00
Joseph Marrero Corchado
372c4444dd Release 2025.1 2025-01-15 13:11:04 -05:00
Colin Walters
20ff0865c7 Always generate composefs blob, don't enable runtime by default
Followup to 9a0acd7249

Basically our composefs enablement flag has long had a tension between
trying to do two things:

- Enable generating the composefs blob (at deployment time)
- Enable at runtime in prepare-root

And we've hit issues in "ratcheting" enabling composefs
across upgrades because of this.

This change builds on the previous one, and now it's really
simple to talk about:

- If composefs is enabled at build time, we *always*
  generate a composefs blob at deplyment time
- Configuring the prepare-root config now mostly
  only affects the runtime state.

There is one detail though: in order to handle the
verity requirement at deploy time, we do still parse
the config then.

But for the basic "is composefs enabled at all at runtime"
that is now fully keyed off the config, not the build time
or (worse) whether the deployment happened to have a composefs
blob.

For users who want composefs on, they need to do so in the base
image configuration.

Signed-off-by: Colin Walters <walters@verbum.org>
2025-01-10 08:04:50 -05:00
Colin Walters
64a38aec8c
Merge pull request #3353 from jlebon/pr/composefs-maybe
libostree/deploy: enable composefs by default
2024-12-17 15:34:37 -05:00
Jonathan Lebon
9a0acd7249
libostree/deploy: enable composefs by default
The composefs libostree integration has been supported for a while now
and is actively in use in various ostree/bootc-based systems. Let's
turn it on by default.

This has no effect if composefs support is not compiled in. Note also
that this does not change the default value of the `composefs.enabled`
tristate to `true`. The default is still `maybe`, but the deploy API
will now also create composefs images for `maybe`.

The reason for doing it this way is so that systems upgrading from
old libostree versions (which may either not have composefs support or
may have composefs-related bugs) will still be able to upgrade and not
trip `ostree-prepare-root` in the new deployment (which allows missing
composefs images for `maybe`).

We may in the future change the default value to `true`.

See also: https://github.com/ostreedev/ostree/issues/2867
2024-12-17 13:30:08 -05:00
Colin Walters
1858d3d300 core: Always sort incoming xattrs
When recomputing selinux attrs during commit, we weren't sorting,
which could cause various issues like fsck failures.

This is a big hammer; change things so we always canonicalize
(i.e. sort) the incoming xattrs when creating a file header
and directory metadata.

I think almost all places in the code were already keeping
things sorted, but it's better to ensure correctness first.
If we ever have some performance issue (I'm doubtful) we
could add something like `_ostree_file_header_known_canonicalized`
or so.

Closes: https://github.com/ostreedev/ostree/issues/3343

Signed-off-by: Colin Walters <walters@verbum.org>
2024-12-02 10:40:17 -05:00
Colin Walters
9e0d778df3 bootupd-static: Drop this test
It breaks due to https://bugzilla.redhat.com/show_bug.cgi?id=2308594
2024-11-04 14:28:13 -05:00
Colin Walters
6ed1f83ab8 checkout: Only verify digest if repo requires fsverity
Fixes a regression from the previous commit; in
the case where the target repo doesn't have composefs in
signed mode there's no reason to verify the digest
at checkout time because we aren't verifying it at
boot time either.

The regression is in cases that use rpm-ostree e.g.
where as of recently we unconditionally add the composefs
digest, but for e.g. FCOS we aren't deploying with fsverity
enabled.

Closes: https://github.com/ostreedev/ostree/issues/3330

Signed-off-by: Colin Walters <walters@verbum.org>
2024-11-04 13:01:55 -05:00
Simon McVittie
1d916231a4 tests: Work around GPG 2.2.45 error behaviour when revoking an expired key
In GPG 2.2.45, a diagnostic message about the only trusted key having
already expired causes this import to produce exit status 2, but the
import still succeeds (the key is still revoked).

Bug: https://dev.gnupg.org/T7351
Bug-Debian: https://bugs.debian.org/1086140
2024-10-31 10:54:23 +00:00
Colin Walters
80c7b86551 tests: Skip checking for immutable bit on composefs
Needed changing after FCOS switch.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-10-30 13:07:01 -04:00
Colin Walters
a6d07b6cc3 deploy: Don't recompute verity checksums if not enabled
This fixes a truly horrific performance bug when
composefs is enabled, but fsverity is not supported
by the filesystem. We'd fall back to doing *userspace*
checksumming of all files at deployment time which was absolutely
not expected or required.

There's really an immense amount of technical debt
here, such as the confusion between `ex-integity.composefs`
vs the prepare-root config, how we handle "torn" states
where some objects don't have verity enabled but some do,
etc.

The ostree composefs state has two modes:

- signed: We need to enforce fsverity
- unsigned: Best effort resilience

So we fix this by making the deploy path to make verity
"opportunistic" - if the ioctl gives us the data, then we
add it to the composefs.

However, this code path is also invoked when we're
computing the expected composefs digest to inject
as commit metadata, and *that* API must work regardless
of whether the target repo has fsverity enabled as
it may operate on a build server.

One lucky thing in all of this: When I went to add
the "checkout composefs" API I added a stub `GVariant`
for options extensibility, which we now use.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-10-28 09:31:34 -04:00
Timothée Ravier
508a8b61ac bootloader/grub2: Handle empty static configs
In #3205, we introduced a check to skip re-generating the GRUB config if
we detect that static configs are in used by looking at bootupd's state.

Unfortunately this check is incomplete and does not account for present
but null entries in the JSON state file.

A proper fix would be to parse the JSON but this requires a larger code
change.

Fixes: https://github.com/ostreedev/ostree/issues/3295
Fixes: https://github.com/ostreedev/ostree/pull/3205
2024-09-14 00:34:24 +02:00
Colin Walters
413b0ad00e
Merge pull request #3292 from dbnicholson/var-slave-shared
switchroot: Stop making /sysroot mount private
2024-09-06 19:35:19 -04:00
Dan Nicholson
2973ec5910 switchroot: Stop making /sysroot mount private
Back in 2b8d586c5, /sysroot was changed to be a private mount so that
submounts of /var do not propagate back to the stateroot /var. That's
laudible, but it makes /sysroot different than every other shared mount
in the root namespace. In particular, it means that submounts of
/sysroot do not propagate into separate mount namespaces.

Rather than make /sysroot private, make /var a slave+shared mount so
that it receives mount events from /sysroot but not vice versa. That
achieves the same effect of preventing /var submount events from
propagating back to /sysroot while allowing /sysroot mount events to
propagate forward like every other system mount. See
mount_namespaces(7)[1] and the linux shared subtrees[2] documentation
for details on slave+shared mount propagation.

When /var is mounted in the initramfs, this is accomplished with
mount(2) syscalls. When /var is mounted after switching to the real
root, the mount propagation flags are applied as options in the
generated var.mount unit. This depends on a mount(8) feature that has
been present since util-linux 2.23. That's available in RHEL 7 and every
non-EOL Debian and Ubuntu release. Applying the propagation from
var.mount fixes a small race, too. Previously, if a /var submount was
added before /sysroot was made private, it would have propagated back
into /sysroot. That was possible since ostree-remount.service orders
itself after var.mount but not before any /var submounts.

1. https://man7.org/linux/man-pages/man7/mount_namespaces.7.html
2. https://docs.kernel.org/filesystems/sharedsubtree.html

Fixes: #2086
2024-09-06 15:49:49 -06:00
Dan Nicholson
fae8941196 tests: Add mount propagation test
This tests the current behavior of making /sysroot a private mount so
that submounts on /var do not propagate back to /sysroot. It also shows
how submounts of /sysroot do not propagate into separate mount
namespaces for the same reason.
2024-09-06 15:49:43 -06:00
Colin Walters
4f46d06ba8 deploy: Log to journal for boot space, not stderr
bootc doesn't run as a systemd unit, and this pollutes stderr.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-08-30 15:00:27 -04:00
Colin Walters
d260d918dd tests: Add a payload link unit test
Motivated by changing this code.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-07-29 17:08:03 -04:00
Colin Walters
4941d9cd0d Release 2024.7
Signed-off-by: Colin Walters <walters@verbum.org>
2024-07-18 17:55:21 -04:00
Colin Walters
e226c87614 prepare-root: Add ostree.prepare-root.composefs
We have a use case for overriding the composefs state via
the kernel commandline; see e.g.
https://gitlab.com/fedora/bootc/tracker/-/issues/27

Signed-off-by: Colin Walters <walters@verbum.org>
2024-07-11 17:04:09 -04:00
Colin Walters
26abca7126 keyfile-utils: Add API to parse tristate strings
Prep for using this in multiple places. Add unit tests.
2024-07-10 17:52:28 -04:00
Daiki Ueno
a0bd18945f libostree: Remove compatibility code with GLib < 2.44
As the build system has required GLib 2.44 since commit
eb09207e1abd7499bd92866cce1de6148d659a4a, the manual expansion of
G_DECLARE_INTERFACE and G_DECLARE_FINAL_TYPE is no longer necessary.

Signed-off-by: Daiki Ueno <dueno@redhat.com>
2024-07-10 10:22:07 +09:00
Colin Walters
e19f732481 core: Validate that xattr names aren't empty
In the ostree-ext codebase the test fixture was generating xattrs
without the trailing NUL byte.  This caused confusing errors
later.  Change the dirmeta validator to catch this.

The way GVariant represents bytestrings, the trailing NUL is there
on wire/disk so it can be there in memory too, but `g_variant_get_bytestring()`
will just return an empty `""` string if actually the value
has a missing NUL.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-06-01 14:46:23 -04:00
Colin Walters
c0a715f1e2 checkout: Add API to directly checkout composefs
We were missing the simple, obvious API and CLI to go
from ostree commit -> composefs.

Internally, we had `ostree_repo_checkout_composefs`
with the right "shape" mostly, except it had more code
in the deploy path to turn that into a composefs.

Add a straightforward public API that does what
the deploy code did before, and then the old
API becomes an explicitly internal helper with an `_`
prefix.

Goals:

- Lead towards a composefs-oriented future
- This makes the composefs logic more testable directly

Signed-off-by: Colin Walters <walters@verbum.org>
2024-05-22 20:38:24 -04:00
Colin Walters
ee066d799d sysroot: Handle /ostree/deploy having epoch 0
Ironically we break if the timestamp there is zero.

Closes: https://github.com/ostreedev/ostree/issues/3022
Signed-off-by: Colin Walters <walters@verbum.org>
2024-05-15 09:59:50 -04:00
Joseph Marrero Corchado
cb70e4063f ostree-sysroot-deploy: check if deployments are in the same stateroot. 2024-05-03 19:57:31 -04:00
Colin Walters
3d3f0b8d31 ci: Also skip if we detect /run/.containerenv 2024-05-02 14:18:55 -04:00
Dan Nicholson
60f4592b2c repo: Make summary and signature mtime match
HTTP servers derive Last-Modified from the modification time of the
file. When used in combination with a Cache-Control max-age value,
having the modification times match means that caches will consider them
expired at the same time. This helps make it more likely that clients
won't receive a cached summary and fresh signature or vice versa.

This makes more sense to do now that the summary and signature are
created in a temporary directory and renamed into place. In the old days
where they were created directly in the repo root, it would be strange
to change the summary mtime when it wasn't actually modified.
2024-04-25 09:38:50 -06:00
Dan Nicholson
ffbeff6cd7 tests: Correctly skip single fsverity test
The skip shell function is for skipping an entire test plan. To skip a
single test result, a directive is needed[1]. Without this change, the
test suite errors claiming that 2 test plans were provided when fsverity
isn't available.

1. https://testanything.org/tap-specification.html#skipping-tests
2024-04-24 21:57:23 -06:00
Colin Walters
99ef9806e2
Merge pull request #3230 from cgwalters/initfs-epoch-2
init-fs: Add --epoch=2
2024-04-12 19:56:35 -04:00
Colin Walters
3647395fe5 tests: Skip composefs tests without the feature
Previously this was masked by us shipping composefs vendored.
2024-04-12 16:16:21 -04:00
Colin Walters
74a3d2da9c init-fs: Add --epoch=2
We want to start switching things so that the toplevel `/ostree`
repository is mode 0700, to close off unprivileged code
from being able to access it.  Previous deployment roots
may have setuid binaries, etc.  The `/var/lib/containers/storage`
directory is mode 0700 for this reason I believe.

Closes: https://github.com/ostreedev/ostree/issues/3211
2024-04-12 16:10:36 -04:00
Colin Walters
38f6b9b3ab init-fs: Add --epoch
I want to add another variant here, and `--modern` is now old.  Let's
acknowledge that we may want to make even more changes in the
future.  So `--modern == --epoch=1` but I will add `--epoch=2` after
this.
2024-04-12 08:56:13 -04:00
Colin Walters
bb6dfd3ab0 checkout: Always replace existing content in overlay mode
The combination of the "honor whiteout" and "union" flags
are intended to basically be "merge trees like overlayfs does".
But we were missing this case in order to support e.g. replacing
a symlink with a directory.
2024-03-13 14:07:54 -04:00
HuijingHei
abc7d5b9a0 kargs: parse spaces in kargs input and keep quotes
According to Jonathan's suggestion, should fix the code from
ostree repo.

With this patch:
- kargs input like "init_on_alloc=1 init_on_free=1", will be
parsed as 2 seperated args `init_on_alloc=1` and `init_on_free=1`,
instead of whole;
- According to https://www.kernel.org/doc/html/v4.14/admin-guide/kernel-parameters.html,
need to keep spaces in double-quotes, like `param="spaces in here"`
will be parsed as whole instead of 3.

Fixes https://github.com/coreos/rpm-ostree/issues/4821
2024-03-08 10:01:06 +08:00
Colin Walters
2fb7a04838 sysroot: Turn on bootloader-naming-2 by default
I think it's about time we flipped this on by default;
like the bootprefix I was a bit too chicken.  We still have
a `bootloader-naming-1` that can be flipped on in case of
some regression.

Closes: https://github.com/ostreedev/ostree/issues/2961
2024-02-28 09:40:34 -05:00
Colin Walters
e47b370963 bootloader/grub2: Don't do anything if we have static configs
This builds on top of fa9924e4fe
(But in a very hacky way because we don't currently link to a JSON library)

Basically, bootupd supports injecting static configs, and this
is the currently least hacky way for us to detect this and understand
that we shouldn't try to run `grub2-mkconfig`.

A further patch I'd like to do here is also change the probing
logic to gracefully no-op if `grub2-mkconfig` doesn't exist,
but that has a bit more risk and involvement.
2024-02-27 14:21:26 -05:00
Eric Curtin
e48cdb90d3 sysroot: Reword comment and use gboolean over bool, error handling
Be more explicit in the comment, and use gboolean over bool. Less header
inclusions when we use gboolean. Although bool is used in some places.
Write a separate _ostree_sysroot_parse_bootlink_aboot function for
aboot. Make is_aboot optional. Handle invalid androidboot karg and no
ostree and androidboot kargs differently.

Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
2024-02-23 14:58:48 +00:00
Eric Curtin
4a71845b12 generator: Fixes for Android Boot environment
In Android Boot environment we do not parse ostree= karg to determine
what directory to boot into, alternatively we do this based on the
androidboot.slot_suffix= karg. But we do set ostree=true karg to denote
that we are indeed booting an ostree environment (required for some
systemd unit files). This change accounts for this approach in the
systemd generator. In this case androidboot.slot_suffix= points you to
/ostree/root.[a|b] and then that points you to the directory to boot
into in /ostree/deploy... Here is what a cmdline may look like in this
type of environment:

androidboot.slot_suffix=_a androidboot.bootdevice=*.ufshc root=PARTLABEL=system_a root=UUID=76a22bf4-f153-4541-b6c7-0332c0dfaeac rw ostree=true loglevel=4 acpi=off console=ttyAMA0 systemd.show_status=auto libahci.ignore_sss=1 slub_debug=FPZ fsck.mode=skip rcupdate.rcu_normal_after_boot=0 rcupdate.rcu_expedited=1

Signed-off-by: Eric Curtin <ecurtin@redhat.com>
2024-02-21 20:54:55 +00:00
Simon McVittie
035b2c1647 test-admin-deploy-var: Don't rely on OSTREE_FEATURES
This is set during build-time testing, but unset during "as-installed"
tests.

Resolves: https://github.com/ostreedev/ostree/issues/3183
Signed-off-by: Simon McVittie <smcv@debian.org>
2024-02-19 21:01:15 +00:00
Simon McVittie
37466ec35b tests: Use skip_without_ostree_feature to detect libarchive, composefs
This avoids false negatives from `ostree --version | grep -q ...`
exiting with failure under `set -o pipefail` because `grep -q` can exit
as soon as it sees the desired string, leaving `ostree --version` to be
terminated by `SIGPIPE` next time it writes to stdout.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2024-02-19 21:01:07 +00:00