3578 Commits

Author SHA1 Message Date
Colin Walters
5fa364d399 Release 2022.4 2022-06-06 11:04:44 -04:00
Joseph Marrero Corchado
8bd1c88013
Merge pull request #2625 from cgwalters/drop-features-introspection
Drop `OSTREE_BUILT_FEATURES` from introspection
2022-06-03 09:22:48 -04:00
Colin Walters
3e645ad950 Drop OSTREE_BUILT_FEATURES from introspection
It inherently depends on the individual build, and can't
really be an official stable API for introspection users.
I've noticed the value of this flip flop when doing local builds.

I'm fairly certain no one is trying to use it from a higher level
language.

It'd probably make sense to even drop from the official C API,
but I'm trying to be conservative with that.
2022-06-02 19:44:40 -04:00
Colin Walters
32ee82133b Rename ostree-cmdprivate to drop out of introspection
I was looking at our `.gir` and noticed we had the cmdprivate bits
because the pattern for excluding headers is `-private.h`, which
didn't match `cmdprivate.h`.
2022-06-02 17:13:17 -04:00
James Westman
03d592b116
Fix annotations on ostree_mutable_tree_lookup()
(nullable) and (optional) were missing on lookup()'s out parameters,
which caused the rust bindings for the function to not work. Due to the
missing (nullable), it would return a Result<(GString, MutableTree), _>,
not a Result<(Option<GString>, Option<MutableTree>), _>, which led to
panics.
2022-05-30 17:36:07 -05:00
Nikita Dubrovskii
972f00e483 s390x: do not unpack existing initrd, just append LUKS keys to its copy
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-05-27 12:06:33 +02:00
Jonathan Lebon
9aca8816c6
Merge pull request #2615 from nikita-dubrovskii/handle_errors 2022-05-25 13:38:23 -04:00
Colin Walters
2363d3ad98
Merge pull request #2605 from saqibali-2k/pr/bls-append
ostree-systroot-deploy: parse bls-append-except-default key
2022-05-25 11:33:10 -04:00
Nikita Dubrovskii
b03fa626f1 s390x: fail on error during reading of SecureExecution sysfs flag 2022-05-25 11:00:34 -04:00
Huijing Hei
b024ce5eeb Remove dup "--os=OSNAME" line in output of "ostree admin config-diff --help"
Small fix for https://github.com/ostreedev/ostree/issues/2594
2022-05-25 11:15:39 +08:00
Huijing Hei
606459ad26 Fix GNU style of two spaces for indent in https://github.com/ostreedev/ostree/pull/2612 2022-05-25 11:12:47 +08:00
Colin Walters
ea0be27bd3
Merge pull request #2612 from HuijingHei/deploy-karg-delete
Teach --karg-delete option to ostree admin deploy
2022-05-24 11:18:54 -04:00
Nikita Dubrovskii
d000569859 s390x: ensure SecureExecution is enabled before sd-boot generation
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-05-23 18:24:43 +02:00
Huijing Hei
0ba3bfe4fd Teach --karg-delete option to ostree admin deploy
See https://github.com/ostreedev/ostree/issues/2586
2022-05-19 22:50:09 +08:00
Saqib Ali
326d89752a ostree-systroot-deploy: parse bls-append-except-default key
We want to parse a new "bls-append-except-default" key from ostree config. The
key-value pairs specified by this key will be added to the generated
BLS fragments of non-default deployments. They must follow the format
"key1,value1;key2,value2" and so on.

This change will allow us to land GRUB password support in FCOS.

Relevant: https://github.com/coreos/fedora-coreos-tracker/issues/134
2022-05-19 09:59:38 -04:00
Nikita Dubrovskii
7a5c604ca3 s390x: generate sd-boot at its own partition
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-05-18 13:06:29 +02:00
Colin Walters
6a055f802f sysroot: Strengthen several g_return_if_fail into g_assert()
Similar to prior changes; motivated by testing out an updated
`gcc -fanalyzer`.
2022-05-16 11:00:08 -04:00
Colin Walters
e27fed45fd sysroot: Quiet one gcc -fanalyzer warning
I don't think this can happen in practice, but add an assertion
to verify it.
2022-05-15 16:03:01 -04:00
Colin Walters
f79b2cea91 Add APIs to get xattrs from disk
I'm aiming to do some more work on the Rust side around `fsck`
like functionality, and this is a useful primitive.  There isn't
a great Rust crate for xattrs, and I think it's better to share this
code.
2022-05-09 12:33:38 -04:00
Luca BRUNO
4b7fc4f1e9
lib/ostree-sysroot-deploy: add a missing include for stdint.h 2022-05-06 10:07:23 +00:00
Simon McVittie
71304e854c repo: Correctly initialize refcount of temporary transaction
Previously, the reference count was left uninitialized as a result of
bypassing the constructor, and the intended abort-on-error usually
wouldn't have happened.

Fixes: 8a9737a "repo/private: move OstreeRepoAutoTransaction to a boxed type"
Resolves: https://github.com/ostreedev/ostree/issues/2592
Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-04-30 12:53:42 +01:00
Simon McVittie
540e60c3e3 repo: Factor out _ostree_repo_auto_transaction_new()
This will allow the direct allocation in
ostree_repo_prepare_transaction() to be replaced with a call to this
function, avoiding breaking encapsulation.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-04-30 12:20:11 +01:00
Colin Walters
d3d3e4ea13 Add an ostree-boot-complete.service to propagate staging failures
Quite a while ago we added staged deployments, which solved
a bunch of issues around the `/etc` merge.  However...a persistent
problem since then is that any failures in that process that
happened in the *previous* boot are not very visible.

We ship custom code in `rpm-ostree status` to query the previous
journal.  But that has a few problems - one is that on systems
that have been up a while, that failure message may even get
rotated out.  And second, some systems may not even have a persistent
journal at all.

A general thing we do in e.g. Fedora CoreOS testing is to check
for systemd unit failures.  We do that both in our automated tests,
and we even ship code that displays them on ssh logins.  And beyond
that obviously a lot of other projects do the same; it's easy via
`systemctl --failed`.

So to make failures more visible, change our `ostree-finalize-staged.service`
to have an internal wrapper around the process that "catches" any
errors, and copies the error message into a file in `/boot/ostree`.

Then, a new `ostree-boot-complete.service` looks for this file on
startup and re-emits the error message, and fails.

It also deletes the file.  The rationale is to avoid *continually*
warning.  For example we need to handle the case when an upgrade
process creates a new staged deployment.  Now, we could change the
ostree core code to delete the warning file when that happens instead,
but this is trying to be a conservative change.

This should make failures here much more visible as is.
2022-04-26 13:02:46 -04:00
Colin Walters
4fd0d6b4c1 fetcher/curl: Consistently check return value curl_easy_setopt
Static analyzers don't like when we only check it sometimes.

And we definitely want to know if any of these are failing.
2022-04-20 15:16:22 -04:00
Colin Walters
2346d5f4d5 libarchive: Handle archive_entry_symlink() returning NULL
The `archive_entry_symlink()` API can definitely return `NULL`,
reading through the libarchive sources.

I hit this in the wild when using old ostree-ext to try to unpack
a chunked archive.

I didn't try to characterize this more, and sorry no unit test right
now.
2022-04-04 10:25:35 -04:00
Colin Walters
c58a4fe661 deploy: Be a bit more verbose about SELinux bits
Let's log when we don't find the expected CLI argument which
will help debug things.
2022-03-28 17:46:59 -04:00
Ondrej Mosnacek
edb4f38934 deploy: Try to rebuild policy in new deployment if needed
Whenever the user has SELinux enabled and has any local
modules/modifications installed, it is necessary to rebuild the policy
in the final deployment, otherwise ostree will leave the binary policy
files unchanged from last deployment as it detects difference against
the base content (in rpm-ostree case this is the RPM content).

To avoid the situation where the policy binaries go stale once any local
customization of the policy is made, try to rebuild the policy as part
of sysroot_finalize_deployment(). Use the special
--rebuild-if-modules-changed switch, which detects if the input module
files have changed relative to last time the policy was built and skips
the most time-consuming part of the rebuild process if modules are
unchanged (thus making this a relatively cheap operation if the user
hasn't made any modifications to the shipped policy).

As suggested by Jonathan Lebon, this uses bubblewrap (via
g_spawn_sync()) to perform the rebuild inside the deployment's
filesystem tree, which also means that ostree will have a runtime
dependency on bubblewrap.

Partially addresses: https://github.com/coreos/fedora-coreos-tracker/issues/701

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2022-03-28 17:18:03 -04:00
Daniel Kolesa
aca9e8e6a7 glib: bump glib requirement to 2.66 and port to GUri
This removes the old SoupURI copypasta from previous generation
of libsoup and opens up a path for a simple libsoup3 port.
2022-03-12 04:44:18 +01:00
Colin Walters
0d020a7145 tmpfiles: Create /run/ostree
This is referenced by 9645cee4f2/lib/src/globals.rs (L16)
specifically used for the (container image) pull secret in
`/run/ostree/auth.json`.

Let's pre-create the directory so users don't have to.

Motivated by https://github.com/openshift/machine-config-operator/pull/3007#discussion_r824172564
2022-03-11 13:08:23 -05:00
Philip Withnall
8cc8e68768 ostree-repo-pull: Take correct out path on error
Like every other error return path in this function, jump to the `out`
label on error here. Returning directly will cause leaks.

Spotted by reading the code, not actually necessarily encountered in the
wild.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2022-03-08 15:36:27 +00:00
Simon McVittie
a9a0110f44 s390x-se-luks-gencpio: Use interoperable path for bash
On OSs that do not consistently merge /usr/bin with /bin, the path to
bash has traditionally been /bin/bash.

Signed-off-by: Simon McVittie <smcv@debian.org>
2022-03-07 22:49:33 +00:00
Simon McVittie
084f8913f0 s390x-se-luks-gencpio: Fix shebang syntax
An indented `#!` is technically meaningless, although many shells will
run text files with the shell if asked to execute them.

Signed-off-by: Simon McVittie <smcv@debian.org>
2022-03-07 22:49:30 +00:00
Colin Walters
fbc6d21c2f Release 2022.2 2022-03-03 18:44:30 -05:00
Luca BRUNO
7e79b82ff8
lib/commit: disallow writing content in 'bare-split-xattrs' mode
This prevents writing content into 'bare-split-xattrs` repository,
while carving some space for experimenting via a temporary
`OSTREE_EXP_WRITE_BARE_SPLIT_XATTRS` environment flag.
2022-03-02 16:45:02 +00:00
Luca BRUNO
14a6e6d8d0
lib/repo: read split xattrs content from file-xattrs-link objects 2022-03-02 16:45:01 +00:00
Luca BRUNO
08e98e9042
lib/core: introduce 'bare-split-xattrs' mode 2022-03-02 16:45:00 +00:00
Luca BRUNO
2c60f302f9
lib/core: introduce two new object types for split xattrs
This adds two new object types for storing xattrs separately from
content objects.

`.file-xattrs` are regular files storing xattrs content, encoded as
GVariant. Each object is keyed by the checksum of its content, allowing
for multiple references.

`.file-xattrs-link` are hardlinks which are associated to file objects.
Each object is keyed by the same checksum of the corresponding file
object. The target of the hardlink is an existing file-xattrs object.
In case of reaching the limit of too many links, this object could be
a plain file too.
2022-03-02 16:44:59 +00:00
Saqib Ali
725d50a3b5 src/ostree: Add --commit-only option to ostree prune
Recently we have noticed exceedingly long execution times
for multiple invocations of ostree prune. This is a result of
calculating full reachability on each invocation.

The --commit-only flag provides an alternative strategy. It will only
traverse and delete commit objects to avoid the more expensive
reachability calculations. This allows us to chain multiple --commit-only
commands cheaply, and then follow with a more expensive ostree prune
invocation at the end to clean up orphaned meta and content objects.
2022-02-25 18:32:25 -05:00
Jonathan Lebon
00e39ebedc
Merge pull request #2548 from cgwalters/mtree-load-ensured 2022-02-22 15:20:22 -05:00
Phaedrus Leeds
c213dd3a84 lib/repo-refs: Remove misleading newline 2022-02-19 14:46:02 -06:00
Phaedrus Leeds
5d3b1ca37a Fix marking static delta commits as partial
This patch makes it so that we mark the .commit file from a static delta
as partial before writing the commit to the staging directory. This
exactly mirrors what we do in meta_fetch_on_complete() when writing the
commit on that codepath, which should lend some credibility to the
correctness of this patch.

I have checked that this fixes an issue Flatpak users have been
encountering (https://github.com/flatpak/flatpak/issues/3479) which
results in error messages like "error: Failed to install
org.freedesktop.Sdk.Extension.texlive: Failed to read commit
c7958d966cfa8b80a42877d1d6124831d7807f93c89461a2a586956aa28d438a: No
such metadata object
8bdaa943b957f3cf14d19301c59c7eec076e57389e0fbb3ef5d30082e47a178f.dirtree"

Here's the sequence of events that lead to the error:
1. An install operation is started that fetches static deltas.
2. The fetch is interrupted for some reason such as network connectivity
   dropping.
3. The .commit and .commitmeta files for the commit being pulled are
   left in the staging dir, e.g.
   "~/.local/share/flatpak/repo/tmp/staging-dfe862b2-13fc-49a2-ac92-5a59cc0d8e18-RURckd"
4. There is no `.commitpartial` file for the commit in
   "~/.local/share/flatpak/repo/state/"
5. The next time the user attempts the install, libostree reuses the
   existing staging dir, pulls the commit and commitmeta objects into
   the repo from the staging dir on the assumption that it's a complete
   commit.
6. Flatpak then tries to deploy the commit but fails in
   ostree_repo_read_commit() in flatpak_dir_deploy(), leading to the
   error message "Failed to read commit ..."
7. This happens again any subsequent time the user attempts the install,
   until the incomplete commit is removed with "flatpak repair --user".

I will try to also add a workaround in Flatpak so this is fixed even
when Flatpak links against affected versions of libostree.
2022-02-19 08:11:23 -06:00
Colin Walters
f36a940ed0 mtree: Load traversed subdirs when creating parents
I'm working on enhancing the ostree-rs-ext test suite and I hit
a bug where walking a mtree and creating a parent would fail to
load lazy intermediate directories, e.g.:

/ -> usr -> bin

If we walked we'd load `/` but keep `usr` lazy, and then invalidation
would crash because it wasn't loaded.

If we're going to mutate a subdir, we need to have all the parents
loaded.

I know this is missing tests, but...it's a bit tedious to do with
the existing C tests.  Eventually soon we'll execute on merging
all 3 repos, and better share test suites.
2022-02-18 10:30:27 -05:00
Colin Walters
929f62c59e mtree: Use declare-and-initialize style
Prep for further work.
2022-02-18 10:22:26 -05:00
Colin Walters
48104f76ee lib/tar: Add some error prefixing
We're trying to debug a problem with a tar stream with hardlinks,
and I think this will be helpful.
2022-02-15 15:33:04 -05:00
Colin Walters
d61fdc7123
Merge pull request #2539 from chergert/main
lib/bootloader: use ot_journal_print() instead of sd-journal
2022-02-12 09:04:43 -05:00
Christian Hergert
6419c32334 lib/bootloader: use ot_journal_print() instead of sd-journal
This needs to use the helper so that USE_LIBSYSTEMD still works as
expected.
2022-02-11 15:40:26 -08:00
Christian Hergert
5d08032aec lib/util: add syslog.h for ot_journal_print()
If we aren't including sd-journal, we may need this too.
2022-02-11 15:40:19 -08:00
Colin Walters
bea8d9a2b4
Merge pull request #2538 from cgwalters/dirmeta-not-floating
core: Mark `ostree_create_directory_metadata` as `(not nullable)`
2022-02-11 18:01:49 -05:00
Colin Walters
5f2ab097d5
Merge pull request #2491 from nikita-dubrovskii/secure-execution
s390x: add secure-execution support
2022-02-11 15:34:55 -05:00
Colin Walters
ea5f7b0f38 core: Mark ostree_create_directory_metadata as (not nullable)
So I can drop an unnecessary use of `unwrap()` in Rust.
2022-02-11 14:09:49 -05:00