Commit Graph

5427 Commits

Author SHA1 Message Date
Colin Walters
dd194eca72 deploy: Be a bit more verbose about SELinux bits
Let's log when we don't find the expected CLI argument which
will help debug things.

(cherry picked from commit c58a4fe661)
2022-05-04 16:28:31 -04:00
Ondrej Mosnacek
62e62bcfd8 deploy: Try to rebuild policy in new deployment if needed
Whenever the user has SELinux enabled and has any local
modules/modifications installed, it is necessary to rebuild the policy
in the final deployment, otherwise ostree will leave the binary policy
files unchanged from last deployment as it detects difference against
the base content (in rpm-ostree case this is the RPM content).

To avoid the situation where the policy binaries go stale once any local
customization of the policy is made, try to rebuild the policy as part
of sysroot_finalize_deployment(). Use the special
--rebuild-if-modules-changed switch, which detects if the input module
files have changed relative to last time the policy was built and skips
the most time-consuming part of the rebuild process if modules are
unchanged (thus making this a relatively cheap operation if the user
hasn't made any modifications to the shipped policy).

As suggested by Jonathan Lebon, this uses bubblewrap (via
g_spawn_sync()) to perform the rebuild inside the deployment's
filesystem tree, which also means that ostree will have a runtime
dependency on bubblewrap.

Partially addresses: https://github.com/coreos/fedora-coreos-tracker/issues/701

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
(cherry picked from commit edb4f38934)
2022-05-04 16:28:22 -04:00
Simon McVittie
51c7960bea repo: Correctly initialize refcount of temporary transaction
Previously, the reference count was left uninitialized as a result of
bypassing the constructor, and the intended abort-on-error usually
wouldn't have happened.

Fixes: 8a9737a "repo/private: move OstreeRepoAutoTransaction to a boxed type"
Resolves: https://github.com/ostreedev/ostree/issues/2592
Signed-off-by: Simon McVittie <smcv@collabora.com>
(cherry picked from commit 71304e854c)
2022-05-04 16:27:44 -04:00
Simon McVittie
4a997ae086 repo: Factor out _ostree_repo_auto_transaction_new()
This will allow the direct allocation in
ostree_repo_prepare_transaction() to be replaced with a call to this
function, avoiding breaking encapsulation.

Signed-off-by: Simon McVittie <smcv@collabora.com>
(cherry picked from commit 540e60c3e3)
2022-05-04 16:27:33 -04:00
Colin Walters
e5b45f861a libarchive: Handle archive_entry_symlink() returning NULL
The `archive_entry_symlink()` API can definitely return `NULL`,
reading through the libarchive sources.

I hit this in the wild when using old ostree-ext to try to unpack
a chunked archive.

I didn't try to characterize this more, and sorry no unit test right
now.
2022-05-04 16:27:02 -04:00
Colin Walters
a6d45dc165 Add an ostree-boot-complete.service to propagate staging failures
Quite a while ago we added staged deployments, which solved
a bunch of issues around the `/etc` merge.  However...a persistent
problem since then is that any failures in that process that
happened in the *previous* boot are not very visible.

We ship custom code in `rpm-ostree status` to query the previous
journal.  But that has a few problems - one is that on systems
that have been up a while, that failure message may even get
rotated out.  And second, some systems may not even have a persistent
journal at all.

A general thing we do in e.g. Fedora CoreOS testing is to check
for systemd unit failures.  We do that both in our automated tests,
and we even ship code that displays them on ssh logins.  And beyond
that obviously a lot of other projects do the same; it's easy via
`systemctl --failed`.

So to make failures more visible, change our `ostree-finalize-staged.service`
to have an internal wrapper around the process that "catches" any
errors, and copies the error message into a file in `/boot/ostree`.

Then, a new `ostree-boot-complete.service` looks for this file on
startup and re-emits the error message, and fails.

It also deletes the file.  The rationale is to avoid *continually*
warning.  For example we need to handle the case when an upgrade
process creates a new staged deployment.  Now, we could change the
ostree core code to delete the warning file when that happens instead,
but this is trying to be a conservative change.

This should make failures here much more visible as is.
2022-05-04 16:26:37 -04:00
Colin Walters
fbc6d21c2f Release 2022.2 2022-03-03 18:44:30 -05:00
Colin Walters
06d13ff16c
Merge pull request #2532 from lucab/ups/repo-mode-bare-split-xattrs
lib/core: introduce 'bare-split-xattrs' mode
2022-03-03 09:41:45 -05:00
Luca BRUNO
aca5671eb1
tests/basic-bare-split-xattrs: add fixture, check read logic 2022-03-03 11:11:59 +00:00
Luca BRUNO
7e79b82ff8
lib/commit: disallow writing content in 'bare-split-xattrs' mode
This prevents writing content into 'bare-split-xattrs` repository,
while carving some space for experimenting via a temporary
`OSTREE_EXP_WRITE_BARE_SPLIT_XATTRS` environment flag.
2022-03-02 16:45:02 +00:00
Luca BRUNO
14a6e6d8d0
lib/repo: read split xattrs content from file-xattrs-link objects 2022-03-02 16:45:01 +00:00
Luca BRUNO
08e98e9042
lib/core: introduce 'bare-split-xattrs' mode 2022-03-02 16:45:00 +00:00
Luca BRUNO
2c60f302f9
lib/core: introduce two new object types for split xattrs
This adds two new object types for storing xattrs separately from
content objects.

`.file-xattrs` are regular files storing xattrs content, encoded as
GVariant. Each object is keyed by the checksum of its content, allowing
for multiple references.

`.file-xattrs-link` are hardlinks which are associated to file objects.
Each object is keyed by the same checksum of the corresponding file
object. The target of the hardlink is an existing file-xattrs object.
In case of reaching the limit of too many links, this object could be
a plain file too.
2022-03-02 16:44:59 +00:00
Luca Bruno
6ad4a3457f
Merge pull request #2554 from ostreedev/dependabot/submodules/libglnx-c71f7ae
build(deps): bump libglnx from `88da8dd` to `c71f7ae`
2022-03-02 16:44:31 +00:00
dependabot[bot]
65ccf2951d
build(deps): bump libglnx from 88da8dd to c71f7ae
Bumps libglnx from `88da8dd` to `c71f7ae`.

---
updated-dependencies:
- dependency-name: libglnx
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-02 12:30:34 +00:00
Colin Walters
999c70346a
Merge pull request #2536 from saqibali-2k/pr/prune-commit-only
src/ostree: Add --commit-only option to ostree prune
2022-02-28 08:47:09 -05:00
Saqib Ali
cf66eaccee tests/test-prune.sh: Use TAP API
Change tests to use the newer TAP API introduced
in https://github.com/ostreedev/ostree/pull/2440
2022-02-25 18:32:25 -05:00
Saqib Ali
18ab5361b9 tests/test-prune.sh: expand testing for --commit-only
Let's add additional tests to expand the test
suite for the new --commit-only functionality.
2022-02-25 18:32:25 -05:00
Saqib Ali
ce44b1907e man/prune, bash: Add --commit-only flag for ostree prune
Update the man page and the auto-complete script
to include the --commit-only flag
2022-02-25 18:32:25 -05:00
Saqib Ali
725d50a3b5 src/ostree: Add --commit-only option to ostree prune
Recently we have noticed exceedingly long execution times
for multiple invocations of ostree prune. This is a result of
calculating full reachability on each invocation.

The --commit-only flag provides an alternative strategy. It will only
traverse and delete commit objects to avoid the more expensive
reachability calculations. This allows us to chain multiple --commit-only
commands cheaply, and then follow with a more expensive ostree prune
invocation at the end to clean up orphaned meta and content objects.
2022-02-25 18:32:25 -05:00
Jonathan Lebon
00e39ebedc
Merge pull request #2548 from cgwalters/mtree-load-ensured 2022-02-22 15:20:22 -05:00
Luca Bruno
4a38ac7b8d
Merge pull request #2552 from ostreedev/dependabot/submodules/libglnx-88da8dd
build(deps): bump libglnx from `803adaf` to `88da8dd`
2022-02-22 16:32:08 +00:00
dependabot[bot]
f73434f1b3
build(deps): bump libglnx from 803adaf to 88da8dd
Bumps libglnx from `803adaf` to `88da8dd`.

---
updated-dependencies:
- dependency-name: libglnx
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-22 15:51:55 +00:00
Colin Walters
4304505af8
Merge pull request #2549 from ostreedev/mwleeds/fix-partial-delta-fetches
Fix marking static delta commits as partial
2022-02-21 14:23:39 -05:00
Phaedrus Leeds
c213dd3a84 lib/repo-refs: Remove misleading newline 2022-02-19 14:46:02 -06:00
Phaedrus Leeds
5d3b1ca37a Fix marking static delta commits as partial
This patch makes it so that we mark the .commit file from a static delta
as partial before writing the commit to the staging directory. This
exactly mirrors what we do in meta_fetch_on_complete() when writing the
commit on that codepath, which should lend some credibility to the
correctness of this patch.

I have checked that this fixes an issue Flatpak users have been
encountering (https://github.com/flatpak/flatpak/issues/3479) which
results in error messages like "error: Failed to install
org.freedesktop.Sdk.Extension.texlive: Failed to read commit
c7958d966cfa8b80a42877d1d6124831d7807f93c89461a2a586956aa28d438a: No
such metadata object
8bdaa943b957f3cf14d19301c59c7eec076e57389e0fbb3ef5d30082e47a178f.dirtree"

Here's the sequence of events that lead to the error:
1. An install operation is started that fetches static deltas.
2. The fetch is interrupted for some reason such as network connectivity
   dropping.
3. The .commit and .commitmeta files for the commit being pulled are
   left in the staging dir, e.g.
   "~/.local/share/flatpak/repo/tmp/staging-dfe862b2-13fc-49a2-ac92-5a59cc0d8e18-RURckd"
4. There is no `.commitpartial` file for the commit in
   "~/.local/share/flatpak/repo/state/"
5. The next time the user attempts the install, libostree reuses the
   existing staging dir, pulls the commit and commitmeta objects into
   the repo from the staging dir on the assumption that it's a complete
   commit.
6. Flatpak then tries to deploy the commit but fails in
   ostree_repo_read_commit() in flatpak_dir_deploy(), leading to the
   error message "Failed to read commit ..."
7. This happens again any subsequent time the user attempts the install,
   until the incomplete commit is removed with "flatpak repair --user".

I will try to also add a workaround in Flatpak so this is fixed even
when Flatpak links against affected versions of libostree.
2022-02-19 08:11:23 -06:00
Colin Walters
f36a940ed0 mtree: Load traversed subdirs when creating parents
I'm working on enhancing the ostree-rs-ext test suite and I hit
a bug where walking a mtree and creating a parent would fail to
load lazy intermediate directories, e.g.:

/ -> usr -> bin

If we walked we'd load `/` but keep `usr` lazy, and then invalidation
would crash because it wasn't loaded.

If we're going to mutate a subdir, we need to have all the parents
loaded.

I know this is missing tests, but...it's a bit tedious to do with
the existing C tests.  Eventually soon we'll execute on merging
all 3 repos, and better share test suites.
2022-02-18 10:30:27 -05:00
Colin Walters
929f62c59e mtree: Use declare-and-initialize style
Prep for further work.
2022-02-18 10:22:26 -05:00
Dan Nicholson
4066b0ab45
Merge pull request #2546 from cgwalters/drop-aggregate-return
build-sys: Drop `-Werror=aggregate-return`
2022-02-17 14:30:21 -07:00
Colin Walters
71ae1c35e3 build-sys: Drop -Werror=aggregate-return
This is failing for me as of recently but only when I build
without optimization.  I don't think we've ever written any
code that returned a large structure by value.

Let's just drop this one.
2022-02-17 14:22:27 -05:00
Colin Walters
12cafbcf8d
Merge pull request #2545 from dbnicholson/lgtm-deps
.lgtm.yml: Fix gpgme dependency
2022-02-17 12:45:19 -05:00
Dan Nicholson
b91f29ca39 .lgtm.yml: Fix gpgme dependency
Since Ubuntu 18.04, libgpgme-dev is the real package and libgpgme11-dev
is a virtual package provided by it. Apparently LGTM running on Ubuntu
20.04 no longer resolves the virtual package:

```
WARNING: Package 'libgpgme11-dev' requested by configuration file was not found
```

That ends up causing the build to fail:

```
configure: error: Need GPGME_PTHREAD version 1.1.8 or later
```
2022-02-17 10:16:13 -07:00
Colin Walters
188b18795a
Merge pull request #2542 from cgwalters/tar-error-prefixing
lib/tar: Add some error prefixing
2022-02-15 18:22:01 -05:00
Colin Walters
48104f76ee lib/tar: Add some error prefixing
We're trying to debug a problem with a tar stream with hardlinks,
and I think this will be helpful.
2022-02-15 15:33:04 -05:00
Colin Walters
e6a734ab25
Merge pull request #2541 from melix99/find-remotes-fix-typo
man: Fix typo in ostree-find-remotes
2022-02-13 10:54:30 -05:00
Colin Walters
65f318a80a
Merge pull request #2540 from melix99/switch-fix-typo
man: Fix typo in ostree-admin-switch
2022-02-13 10:53:33 -05:00
Marco Melorio
6264c6deca
man: Fix typo in ostree-find-remotes 2022-02-12 19:00:28 +01:00
Marco Melorio
99e01c8b7b
man: Fix typo in ostree-admin-switch 2022-02-12 18:48:35 +01:00
Colin Walters
d61fdc7123
Merge pull request #2539 from chergert/main
lib/bootloader: use ot_journal_print() instead of sd-journal
2022-02-12 09:04:43 -05:00
Christian Hergert
6419c32334 lib/bootloader: use ot_journal_print() instead of sd-journal
This needs to use the helper so that USE_LIBSYSTEMD still works as
expected.
2022-02-11 15:40:26 -08:00
Christian Hergert
5d08032aec lib/util: add syslog.h for ot_journal_print()
If we aren't including sd-journal, we may need this too.
2022-02-11 15:40:19 -08:00
Colin Walters
bea8d9a2b4
Merge pull request #2538 from cgwalters/dirmeta-not-floating
core: Mark `ostree_create_directory_metadata` as `(not nullable)`
2022-02-11 18:01:49 -05:00
Colin Walters
5f2ab097d5
Merge pull request #2491 from nikita-dubrovskii/secure-execution
s390x: add secure-execution support
2022-02-11 15:34:55 -05:00
Colin Walters
ea5f7b0f38 core: Mark ostree_create_directory_metadata as (not nullable)
So I can drop an unnecessary use of `unwrap()` in Rust.
2022-02-11 14:09:49 -05:00
Nikita Dubrovskii
d1ab18f327 s390x: add LUKS keyfile to 'sd-boot'
This allows to use Secure Execution with LUKS encrypted boot disk,
key and cryptab are stored only in 'sd-boot' encrypted image.

Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-02-11 09:02:42 +01:00
Nikita Dubrovskii
c27b98a2cb s390x: add "IBM Secure Execution for Linux" support
If system contains ibm-z-hostkey (fetched during ignition), than
ostree generates 'sd-boot' image and reboots into Secure Execution

Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-02-11 09:00:38 +01:00
Colin Walters
d8e33e3a49
Merge pull request #2537 from jlebon/pr/fix-floating
lib/gpg-verify-result: Add missing floating annotation
2022-02-09 17:15:26 -05:00
Jonathan Lebon
c5ecef0aa4 lib/gpg-verify-result: Add missing floating annotation
I think I'm hitting issues due to this while using the Rust bindings:
https://github.com/coreos/rpm-ostree/pull/3406#issuecomment-1033084956

The bindings for those APIs use `from_glib_full` which says:

> Because ownership can only be transferred if something is already
> referenced, this is unsuitable for floating references.
2022-02-09 14:35:31 -05:00
Colin Walters
0eaad8762b
Merge pull request #2535 from dbnicholson/summary-commit-version
lib/repo: Add commit version metadata to summary metadata
2022-02-04 17:51:02 -05:00
Dan Nicholson
6fbf759279 lib/repo: Add commit version metadata to summary metadata
The commit metadata `version` key is well established but getting it for
a remote commit is cumbersome since the commit object needs to be
fetched and loaded. Including it in the summary additional metadata
allows a much more convenient view of what each of the remote refs
represents.
2022-02-04 14:28:11 -07:00