Commit Graph

142 Commits

Author SHA1 Message Date
Colin Walters
c0a715f1e2 checkout: Add API to directly checkout composefs
We were missing the simple, obvious API and CLI to go
from ostree commit -> composefs.

Internally, we had `ostree_repo_checkout_composefs`
with the right "shape" mostly, except it had more code
in the deploy path to turn that into a composefs.

Add a straightforward public API that does what
the deploy code did before, and then the old
API becomes an explicitly internal helper with an `_`
prefix.

Goals:

- Lead towards a composefs-oriented future
- This makes the composefs logic more testable directly

Signed-off-by: Colin Walters <walters@verbum.org>
2024-05-22 20:38:24 -04:00
Colin Walters
28cc761806 sysroot: Stabilize deployment finalization, add API and CLI
It's about time we do this; deployment finalization locking
is a useful feature.  An absolutely key thing here is that
we've slowly been moving towards the deployments as the primary
"source of truth".

Specifically in bootc for example, we will GC container images
not referenced by a deployment.

This is then neecessary to support a "pull but don't apply automatically" model.

This stabilizes the existing `ostree admin deploy --lock-finalization`
CLI, and adds a new `ostree admin unlock-finalization`.

We still check the old lock file path, but there's a new boolean
value as part of the staged deployment data which is intended
to be the source of truth in the future.  At some point then we
can drop the rpm-ostree lockfile handling.

Closes: https://github.com/ostreedev/ostree/issues/3025
2023-11-27 10:59:56 -05:00
Alexander Larsson
accb1f0602 Add ostree admin post-copy command
This command will apply fs-verity on all objects that need it and
needs to be called when an ostree deployment has been copied on a
file-by-file basis, which would loose information such as fs-verity.

This is needed by osbuild which works by creating the final image in a
rootfs, and then separately copying that rootfs file-by-file to a
loopback mounted filesystem image.
2023-11-14 10:15:19 +01:00
Colin Walters
a6f2d053c8 lib: Rework composefs metadata, drop custom signatures
We will be switching to handling signature verification of
the target ostree commit.
2023-06-17 17:08:46 -04:00
Dan Nicholson
1384267973 repo: Add API to regenerate and sign repo metadata
Currently this is just a wrapper around regenerating and signing the
summary in one call, but later it will be used to also generate the
`ostree-metadata` commit if the repo has a collection ID.
2023-02-07 14:50:47 -07:00
Colin Walters
599ffefe7f sysroot: Add an API to initialize with mountns
This lowers down into the C library some logic we
have in the binary/app logic, in prep for having more Rust-native
CLI code in https://github.com/ostreedev/ostree-rs-ext/pull/412

Basically we want to *ensure* a mount namespace by invoking
`unshare()` if necessary, instead of requiring our callers
to do this dance.

This also helps fix e.g.
Closes: https://github.com/ostreedev/ostree/issues/2769
2022-11-22 08:45:13 -05:00
Rafael Garcia Ruiz
f6d308f130 Idempotent delete operation for OstreeKernelArgs
ostree_kernel_args_delete_if_present checks if an argument is present
in OstreeKernelArgs and delete it.

Signed-off-by: Rafael Garcia Ruiz <rafael.garcia@collabora.com>
2022-11-18 20:26:28 +01:00
Rafael Garcia Ruiz
c4db171dae ostree_kernel_args_contains for OstreeKernelArgs
Check if an argument is present in OstreeKernelArgs. This is a way to
make easier idempotent append and delete operations.
ostree_kernel_args_append_if_missing uses it to avoid inserting a
duplicate key.

Closes #2329

Signed-off-by: Rafael Garcia Ruiz <rafael.garcia@collabora.com>
2022-11-18 13:52:08 +01:00
Huijing Hei
7b7b6d741b Fix ostree admin kargs edit-in-place fails issue
Add func to set kernel arguments in place, instead of create new
deployment
Fix https://github.com/ostreedev/ostree/issues/2664
2022-07-12 16:27:00 +08:00
Huijing Hei
3bc59a5206 RFE: Add a hidden option to ostree admin kargs edit-in-place to
update all existing deployments in place

Example:
$ sudo ostree admin kargs edit-in-place --append-if-missing=rw
See https://github.com/ostreedev/ostree/issues/2617

This will not add duplicate key, if there is `TESTARG=VAL1` in the
kernel arguments, `--append-if-missing=TESTARG=VAL2` will be ignored.
2022-06-23 22:31:39 +08:00
Colin Walters
32ee82133b Rename ostree-cmdprivate to drop out of introspection
I was looking at our `.gir` and noticed we had the cmdprivate bits
because the pattern for excluding headers is `-private.h`, which
didn't match `cmdprivate.h`.
2022-06-02 17:13:17 -04:00
Colin Walters
f79b2cea91 Add APIs to get xattrs from disk
I'm aiming to do some more work on the Rust side around `fsck`
like functionality, and this is a useful primitive.  There isn't
a great Rust crate for xattrs, and I think it's better to share this
code.
2022-05-09 12:33:38 -04:00
Luca BRUNO
ca0bb75954
apidoc: add missing page includes
This fixes some missing sections in API reference, adding all the
relevant includes.
2022-03-07 08:33:08 +00:00
Saqib Ali
725d50a3b5 src/ostree: Add --commit-only option to ostree prune
Recently we have noticed exceedingly long execution times
for multiple invocations of ostree prune. This is a result of
calculating full reachability on each invocation.

The --commit-only flag provides an alternative strategy. It will only
traverse and delete commit objects to avoid the more expensive
reachability calculations. This allows us to chain multiple --commit-only
commands cheaply, and then follow with a more expensive ostree prune
invocation at the end to clean up orphaned meta and content objects.
2022-02-25 18:32:25 -05:00
Joseph Marrero
581a58067b Update FSF license notices to use URL instead of address 2021-12-07 08:34:25 -05:00
Colin Walters
a73a28634d Remove OstreeTlsCertInteraction bits from introspection
We filter out everything named `-private.h` from scanning,
which differs from the gtk-doc exclude.  Eventually this will
be solved when we switch to the new gir-based docs.

Came up in https://github.com/ostreedev/ostree-rs/pull/34#discussion_r723337772
2021-10-06 12:42:56 -04:00
Colin Walters
868776a296 lib: Add an API to construct a MutableTree from a commit
This is nicer than having the caller parse the commit
object, or indirect via the `OstreeRepoFile*` object of the root.

Will be used in ostree-rs-ext around tar parsing.
2021-10-01 09:34:19 -04:00
Colin Walters
1b9e3a9375 repo: Add an API to init OstreeSePolicy from commit directly
This is part of `OstreeCommitModifier`, but I'm not using
that in some of the ostree-ext Rust code.

It just makes more sense as a direct policy API, where it should
have been in the first place.  There's already support for
setting a policy object on a commit modifier, so that's all the
old API needs to do now.
2021-09-30 15:54:48 -04:00
Colin Walters
359435de84 Add an API to verify a commit signature explicitly
We have a bunch of APIs to do GPG verification of a commit,
but that doesn't generalize to signapi.  Further, they
require the caller to check the signature status explicitly
which seems like a trap.

This much higher level API works with both GPG and signapi.
The intention is to use this in things that are doing "external
pulls" like the ostree-ext tar import support.  There we will
get the commitmeta from the tarball and we want to verify it
at the same time we import the commit.
2021-08-30 13:27:38 -04:00
Dan Nicholson
a50f6d0b9f lib/repo: Add ostree_repo_remote_get_gpg_keys()
This function enumerates the trusted GPG keys for a remote and returns
an array of `GVariant`s describing them. This is useful to see which
keys are collected by ostree for a particular remote. The same
information can be gathered with `gpg`. However, since ostree allows
multiple keyring locations, that's only really useful if you have
knowledge of how ostree collects GPG keyrings.

The format of the variants is documented in
`OSTREE_GPG_KEY_GVARIANT_FORMAT`. This format is primarily a copy of
selected fields within `gpgme_key_t` and its subtypes. The fields are
placed within vardicts rather than using a more efficient tuple of
concrete types. This will allow flexibility if more components of
`gpgme_key_t` are desired in the future.
2021-07-15 15:50:04 -06:00
Colin Walters
0f36d8c221 repo: Make locking APIs public
Doing anything even somewhat sophisticated requires this;
turns out our own `ostree prune` CLI wants this, e.g.
https://github.com/ostreedev/ostree/issues/2337

Closes: https://github.com/ostreedev/ostree/issues/2286
2021-06-05 09:00:21 -06:00
Colin Walters
6f84aff0ae repo: Add ostree_repo_write_regfile
This API is push rather than pull, which makes it much more
suitable to use cases like parsing a tar file from external
code.

Now, we have a large mess in this area internally because
the original file writing code was pull based, but static
deltas hit the same problem of wanting a push API, so I added
this special `OstreeRepoBareContent` just for writing regular
files from a push API.

Eventually...I'd like to deprecate the pull based API,
and rework things so that for regular files the push API
is the default, and then `write_content_object()` would
be split up into archive/bare cases.

In this world the `ostree_repo_write_content()` API would
then need to hackily bridge pull to push and it'd be
less efficient.

Anyways for now due to this bifurcation, this API only
works on non-archive repositories, but that's fine for
now because that's what I want for the `ostree-ext-container`
bits.
2021-04-09 21:54:44 +00:00
Colin Walters
fce69cdf70 repo: Add ostree_repo_write_symlink
Continuation of the addition of `ostree_repo_write_regfile_inline()`.
This will be helpful for ostree-rs-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.
2021-04-08 21:10:00 +00:00
Colin Walters
9332955b5f
Merge pull request #2327 from cgwalters/writing-apis
repo: Add ostree_repo_write_regfile_inline
2021-04-08 17:09:11 -04:00
Colin Walters
4e2a14eb0c repo: Add ostree_repo_write_regfile_inline
When working on ostree-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.

Plus the gtk-rs binding API to map from `impl Read` to Gio
https://docs.rs/gio/0.9.1/gio/struct.ReadInputStream.html
requires that the input stream is `Send` but the Rust `tar` API
isn't.

This is only 1/3 of the problem; we also need similar APIs
to directly create a symlink, and to stream large objects via
a push-based API.
2021-04-08 14:57:33 +00:00
Dan Nicholson
ff47c57790
Merge pull request #2323 from cgwalters/remove-experimental
build-sys: Remove --enable-experimental-api
2021-04-07 15:31:40 -06:00
Dan Nicholson
d05e7ba2e2 Ensure consistent apidoc .gitignore
If you don't configure with --enable-gtk-doc, these won't be included in
the generated .gitignore. If you later configure with --enable-gtk-doc
in the same checkout, they'll show up as untracked files. Include them
in GITIGNOREFILES to ensure they're always added regardless of configure
options.
2021-04-07 13:37:53 -06:00
Dan Nicholson
36da85ecbe Remove apidoc .gitignore from version control
Like the top level .gitignore, there's no reason to track this if it's
being generated automatically. It only produces spurious diff changes
that you almost certainly don't want.
2021-04-07 13:36:01 -06:00
Colin Walters
a90d59cf2d build-sys: Remove --enable-experimental-api
It was added for the collections bits, but we made that stable.
It's now just cruft and we're very unlikely to reuse the infrastructure
again.

Motivated by a unit test failure when running from a tarball:
https://github.com/ostreedev/ostree/issues/2313
2021-04-07 19:19:05 +00:00
Colin Walters
1b28e6041c sysroot: Add _require_booted_deployment() API
This is a common pattern that is replicated both in our code
and in rpm-ostree a lot.  Let's add a canonical API.
2021-03-17 19:55:56 +00:00
Colin Walters
857587615d Add an API+CLI to inject metadata for bootable OSTree commits
I was doing some rpm-ostree work and I wanted to compare two
OSTree commits to see if the kernel has changed.  I think
this should be a lot more natural.

Add `ostree commit --bootable` which calls into a new generic
library API `ostree_commit_metadata_for_bootable()` that
discovers the kernel version and injects it as an `ostree.linux`
metadata key.  And for extra clarity, add an `ostree.bootable`
key.

It's interesting because the "core" OSTree layer is all about
generic files, but this is adding special APIs around bootable
OSTree commits (as opposed to e.g. flatpak as well as
things like rpm-ostree's pkgcache refs).

Eventually, I'd like to ensure everyone is using this and
hard require this metadata key for the `ostree admin deploy`
flow - mainly to prevent accidents.
2021-03-12 19:01:42 +00:00
Alexander Larsson
8cd796f3f1 Add ostree_repo_gpg_sign_data()
This is similar to ostree_sign_data() but for the old gpg code.
Flatpak will need this to reproduce a signed summary.
2020-10-23 13:55:33 +02:00
Alexander Larsson
c304703e1d deltas: Make ostree_repo_static_delta_reindex() public
It is useful to be able to trigger this without having to regenerate
the summary. For example, if you are not using summaries, or ar generating
the summaries yourself.
2020-10-23 12:30:08 +02:00
Alexander Larsson
8e1f199dd4 deltas: Add ostree_repo_list_static_delta_indexes() function
This lists all the available delta indexes.
2020-10-23 12:30:08 +02:00
Jonathan Lebon
81b13da8e3 lib/deploy: Add support for overlay initrds
In FCOS and RHCOS, the need to configure software in the initramfs has
come up multiple times. Sometimes, using kernel arguments suffices.
Other times, it really must be a configuration file. Rebuilding the
initramfs on the client-side however is a costly operation. Not only
does it add complexity to the update workflow, it also erodes a lot of
the value obtained from using the baked "blessed" initramfs from the
tree itself.

One elegant way to address this is to allow specifying multiple
initramfses. This is supported by most bootloaders (notably GRUB) and
results in each initrd being overlayed on top of each other.

This patch allows libostree clients to leverage this so that they can
avoid regenerating the initramfs entirely. libostree itself is agnostic
as to what kind and how much data overlay initrds contain. It's up to
the clients to enforce such boundaries.

To implement this, we add a new ostree_sysroot_stage_overlay_initrd
which takes a file descriptor and returns a checksum. Then users can
pass these checksums when calling the deploy APIs via the new array
option `overlay_initrds`. We copy these files into `/boot` and add them
to the BLS as another `initrd` entry.
2020-09-30 13:29:32 -04:00
Jonathan Lebon
40fea4c443 lib/deploy: Add deploy/stage APIs with options
And make the `override_kernel_argv` one of those options. This is mostly
a mechanical move here, no functional change otherwise.

Prep for adding a new option.
2020-09-30 13:29:32 -04:00
Jonathan Lebon
f7500bb703 lib/bootconfig: Add support for multiple initrd keys
Prep for actually teaching the rest of the codebase about this.

We keep the primary initrd in the `options` hash table for backwards
compatibility.
2020-09-30 13:29:32 -04:00
Frédéric Danis
fb1faf17d6 lib/deltas: Check signed delta in execute_offline
Add a new function `ostree_repo_static_delta_execute_offline_with_signature`
which takes a signature engine to verify the delta before applying it.
The `ostree_repo_static_delta_execute_offline` is just a wrapper to this
new function, passing a NULL signature engine.
When this function is called without signature engine, but with a sign
delta, it will only fails if `sign-verify-deltas` is set to true in repo
core options.

This commits move signature existence check and delta signature
verification to share common parts between existing APIs and the new
function.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis
02a19b2c96 lib/deltas: Add signature check API for static-delta superblock
This retrieves the signatures and pass the static delta block as an array
of bytes to ostree_sign_data_verify().

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Colin Walters
a9a81f3a29 signing: Change API to create instances directly
This cleans up the verification code; it was weird how
we'd get the list of known names and then try to create
an instance from it (and throw an error if that failed, which
couldn't happen).
2020-05-10 14:18:55 +00:00
Denis Pynkin
ee5d0f350f signapi: expose metadata format and key
Explicitly expose functions for querying the metadata format
and key name used by OstreeSign object:
 - ostree_sign_metadata_format
 - ostree_sign_metadata_key

This allows to use the same metadata format and key name
by 3-rd party applications using signapi.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-04-24 16:34:50 +00:00
Denis Pynkin
b4050b4a34 lib/sign: make ed25519 engine non-public
Remove unneeded public declaration for ed25519 signing engine.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:55 +03:00
Denis Pynkin
4d0e3a66c5 lib/sign: make dummy engine non-public
Remove unneeded public declaration for dummy signing engine.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:55 +03:00
Denis Pynkin
0bdcf14d56 lib/sign: new function for summary file signing
Add function `ostree_sign_summary()` allowing to sign the summary file.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:55 +03:00
Denis Pynkin
908a2cd760 apidoc: add API documentation for signing interface
Add the documentation for all public functions.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
ceaf6d7f54 lib/sign: add ostree_seign_clear_keys function
Add the function for implicit cleanup of all loaded keys.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
bc4488692c lib/sign: public API optimisation
Removed from public `ostree_sign_detached_metadata_append` function.
Renamed `metadata_verify` into `data_verify` to fit to real
functionality.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
95ab57c17e lib/sign-ed25519: cleanup unneeded code
Removed unused code.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
2303202c86 sign: API changes for public keys and CLI keys format
API changes:
- added function `ostree_sign_add_pk()` for multiple public keys using.
- `ostree_sign_set_pk()` now substitutes all previously added keys.
- added function `ostree_sign_load_pk()` allowed to load keys from file.
- `ostree_sign_ed25519_load_pk()` able to load the raw keys list from file.
- use base64 encoded public and private ed25519 keys for CLI and keys file.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
edbbe1c4f2 lib/sign: initial implementation
Added the initial version of signing interface allowing to allowing to
sign and verify commits.
Implemented initial signing modules:
 - dummy -- simple module allowing to sign/verify with ASCII string
 - ed25519 -- module allowing to sign/verify commit with ed25519
   (EdDSA) signature scheme provided by libsodium library.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00