1344 Commits

Author SHA1 Message Date
Colin Walters
c3dff62e1c commit: Try reflinks for local commits by default
I think we originally used to do this, but at some point in a
code refactoring, this optimization got lost.

It's a quite important optimization for the case of writing content
generated by an external system into an ostree repository.
2023-12-04 20:45:08 -05:00
Colin Walters
bab87cbd9d tests: Drop bare-user on tmpfs negative test
As of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2daf18a7884dc03d5164ab9c7dc3f2ea70638469
the restriction is lifted, FINALLY.
2023-12-03 09:32:36 -05:00
Colin Walters
66064d960d tmpfiles: Copy /usr/share/factory/var to /var
This is a pattern we want to encourage.  It's honestly just
way simpler than what rpm-ostree is doing today in auto-synthesizing
individual tmpfiles.d snippets.
2023-11-30 18:11:33 -05:00
Colin Walters
d3de3a0a88 tests: Add a dedicated finalization test
This one covers `admin lock-finalization --unlock`.
2023-11-28 18:45:11 -05:00
Colin Walters
28cc761806 sysroot: Stabilize deployment finalization, add API and CLI
It's about time we do this; deployment finalization locking
is a useful feature.  An absolutely key thing here is that
we've slowly been moving towards the deployments as the primary
"source of truth".

Specifically in bootc for example, we will GC container images
not referenced by a deployment.

This is then neecessary to support a "pull but don't apply automatically" model.

This stabilizes the existing `ostree admin deploy --lock-finalization`
CLI, and adds a new `ostree admin unlock-finalization`.

We still check the old lock file path, but there's a new boolean
value as part of the staged deployment data which is intended
to be the source of truth in the future.  At some point then we
can drop the rpm-ostree lockfile handling.

Closes: https://github.com/ostreedev/ostree/issues/3025
2023-11-27 10:59:56 -05:00
Colin Walters
b6f435fe11 tests: Turn off gpg verification for dev builds
Right now `ostree admin status` errors out in this case, but
`rpm-ostree status` doesn't.  The former behavior is probably
more of a bug, work around it for now.
2023-11-27 10:59:56 -05:00
Eric Curtin
29973774ea
Merge pull request #3091 from cgwalters/fix-soup3
build-sys: Enable libsoup3 by default if installed
2023-11-10 16:39:13 +00:00
Colin Walters
bc62fd5196 unlock: Don't pass options again to overlayfs
There seems to be a tricky regression here with the util-linux
support for the new mount API, plus overlays support for it.

```
[2023-11-09T21:05:30.633Z] Nov 09 21:05:26 qemu0 kola-runext-unlock-transient.sh[2108]: + unshare -m -- /bin/sh -c 'mount -o remount,rw /usr && echo hello from transient unlock >/usr/share/writable-usr-test'
[2023-11-09T21:05:30.633Z] Nov 09 21:05:26 qemu0 kola-runext-unlock-transient.sh[2148]: mount: /usr: mount point not mounted or bad option.
[2023-11-09T21:05:30.633Z] Nov 09 21:05:26 qemu0 kola-runext-unlock-transient.sh[2148]:        dmesg(1) may have more information after failed mount system call.
```

OK this seems related to the new mount API support in util-linux and overlayfs.  From a strace:

```
2095  open_tree(AT_FDCWD, "/usr", OPEN_TREE_CLOEXEC) = 3
2095  mount_setattr(-1, NULL, 0, NULL, 0) = -1 EINVAL (Invalid argument)
...
2095  fspick(3, "", FSPICK_NO_AUTOMOUNT|FSPICK_EMPTY_PATH) = 4
2095  fsconfig(4, FSCONFIG_SET_FLAG, "seclabel", NULL, 0) = 0
2095  fsconfig(4, FSCONFIG_SET_STRING, "lowerdir", "usr", 0) = -1 EINVAL (Invalid argument)
```

I think the core problem here is it's trying to reconfigure the mount with existing options,
but in the new mount namespace we can't see the lowerdir.

Here we really really just want to remount writable.  Telling
util-linux to not pass existing options fixes it.
2023-11-09 20:53:52 -05:00
Colin Walters
c668a8c531 tests: Work around systemd regression for boot id parsing
cc https://github.com/systemd/systemd/issues/29275
2023-11-09 15:10:33 -05:00
Colin Walters
c3cf5b237d undeploy: Parse integer more properly
`atoi` doesn't offer any error checking.

Closes: https://github.com/ostreedev/ostree/issues/3088
2023-11-08 16:15:01 -05:00
Colin Walters
b9e73a3813
Merge pull request #3078 from HuijingHei/karg-delete-array
karg-delete: support multiple times
2023-10-17 09:44:24 -04:00
HuijingHei
003cb99c32 karg-delete: support multiple times
Fix https://github.com/ostreedev/ostree/pull/2612#issuecomment-1764412141
2023-10-17 20:22:08 +08:00
Colin Walters
d788c5b5c9
Merge pull request #3075 from cgwalters/reenable-composefs
tests: Use ext4, re-enable composefs test
2023-10-16 10:24:10 -04:00
Colin Walters
ac69c70750 deploy: Remove lock when re-staging
This closes the biggest foot-gun when doing e.g.
`rpm-ostree rebase` when zincati is running on a FCOS system.

Previously if zincati happened to have staged + locked a deployment,
we'd keep around the lock which is definitely not what is desired.
2023-10-13 13:45:04 -04:00
Colin Walters
833251cb0d tests: Use ext4, re-enable composefs test
Until the XFS fsverity stuff lands.
2023-10-13 12:00:05 -04:00
Colin Walters
265b7f9ecc deploy: Improve error message for nonexistent stateroot
Came up on an internal chat; previously we were only erroring
out when trying to do the SELinux labeling for `/var` which
was really misleading.

Add some other error prefixing while we have the patient open.
2023-10-12 13:22:35 -04:00
Colin Walters
cd0fc9f4f8
Merge pull request #3072 from alexlarsson/fix-whiteout-test
tests: Fix whiteout test
2023-10-12 10:25:40 -04:00
Alexander Larsson
397a11762e tests: Fix whiteout test
This test was always skipped, because the check:

 if touch overlay/baz/.wh.cow &&
    touch overlay/.wh.deeper &&
    touch overlay/baz/another/.wh..wh..opq; then

always fails due to the missing overlay/baz/another directory.
Fix by creating the directory.
2023-10-12 10:18:36 +02:00
Colin Walters
81c08746c6 repo: Add an option to label /usr/etc as /etc
This will be very useful for enabling a "transient /etc" option
because we won't have to do hacks relabling in the initramfs, or
forcing it on just for composefs.
2023-10-11 13:25:08 -04:00
Colin Walters
734ea3ee37 ci: Disable composefs test for now
Need to change the framework to boot with an Ignition config
that switches to ext4 soon.
2023-10-11 09:16:45 -04:00
Owen W. Taylor
3b2fd6e9ff When exporting, use hardlinks for duplicated files
For ostree_repo_export_tree_to_archive(), and 'ostree export', when the
exported tree contains multiple files with the same checksum, write an
archive with hard links.

Without this, importing a tree, then exporting it again breaks
hardlinks.

As an example of savings: this reduces the (compressed) size of the
Fedora Flatpak Runtime image from 1345MiB to 712MiB.

Resolves: #2925
2023-09-29 13:45:29 -04:00
Colin Walters
372cbd7a64 tests: Add an integration test for composefs signatures
Ensure we have some automated test coverage for this.
2023-09-21 07:53:19 -04:00
Colin Walters
e3291ccce9 tests: Rework detection of trivial-httpd
Because it's now at the toplevel.
2023-09-19 16:52:13 -04:00
Jonathan Lebon
fc5aef6fe6 lib/pull: Don't scan commit objects we fetch via deltas
When we're fetching a commit via static delta, we already take care of
fetching the full commit, so there's no need to also scan it using the
regular object workflow.

Closes: #2053
2023-09-08 17:49:25 -04:00
Jonathan Lebon
c8ed1c7a87 lib/pull: Fix miscounting of missing metadata
If e.g. detached metadata or a parent commit is missing, we
don't consider it an error but we erroneously still increment the
`n_fetched_metadata` counter, causing it to be higher than it should.

This will implicitly be tested by a test added in the next patch.
2023-09-08 17:49:25 -04:00
Colin Walters
e93ebd7385
Merge pull request #3018 from ostreedev/dependabot/submodules/composefs-597a766
build(deps): bump composefs from `1aed878` to `597a766`
2023-08-31 12:16:50 -04:00
Eric Curtin
8d838230f1
Merge pull request #3012 from cgwalters/default-early-prune
sysroot: Promote the "early prune" behavior to default
2023-08-31 16:30:26 +01:00
Colin Walters
d5cfbed5bc sysroot: Promote the "early prune" behavior to default
I think we have enough testing for this, let's default it to
on, and change the variable to allow opt-out in case it does
introduce a regression.
2023-08-31 10:25:59 -04:00
dependabot[bot]
0d7b8ebc6b build(deps): bump composefs from 1aed878 to 597a766
Bumps [composefs](https://github.com/containers/composefs) from `1aed878` to `597a766`.
- [Release notes](https://github.com/containers/composefs/releases)
- [Commits](1aed8781d6...597a766095)

---
updated-dependencies:
- dependency-name: composefs
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-30 13:23:13 -04:00
Colin Walters
d36bf35c10 switchroot: Lower config parser to otcore, add unit tests
Part of the continuation of unit testing coverage.
2023-08-25 15:59:34 -04:00
Colin Walters
02d419079d deploy: Add bootloader-naming-2 opt-init
I've verified that this fixes compatibility with GRUB, which
parses the filename:
https://github.com/ostreedev/ostree/issues/2961

However, out of a large degree of conservatism I've made this
an opt-in behavior for now.

My plan is to test it out in the FCOS development streams first.
2023-08-24 09:40:54 -04:00
Colin Walters
20b8cb174c Move prepare-root karg helpers into otcore, add unit tests
Add long overdue unit testing coverage for this, which
at least slightly closes out the android boot CI gap.

Actually, this *copies* the karg parsing code into otcore because
it now uses glib, which we're not yet using in the static
prepare-root.  It's pretty tempting to drop support for the
static prepare root entirely.  But for now we'll live with some
code duplication.
2023-08-23 17:11:10 -04:00
Joseph Marrero Corchado
4451949213
Merge pull request #2997 from cgwalters/test-cleanups-1
tests/destructive: Port more to xshell
2023-08-23 10:38:16 -04:00
Joseph Marrero Corchado
37f247da02
Merge pull request #2999 from cgwalters/add-oscore-units
tests: Add otcore unit tests
2023-08-23 10:37:48 -04:00
Colin Walters
facb9a1a60 tests/inst: Update to latest ostree-ext
(And glib, etc.)
2023-08-22 14:23:56 -04:00
Colin Walters
e3f0c4d49c tests: Add otcore unit tests
This just stubs out the basic infrastructure, to be expanded upon.
2023-08-22 13:04:46 -04:00
Colin Walters
578c87e7ef tests/destructive: Port more to xshell
Just keeping up momentum.
2023-08-22 09:52:43 -04:00
Colin Walters
f4e56b910f admin-deploy: Add --stateroot as alias for --os
To further help deprecate the confusing "osname" terminology.
2023-08-17 16:49:09 -04:00
Colin Walters
9d5ccfefff Add ostree admin stateroot-init as alias for os-init
To further help deprecate the confusing "osname" terminology.
2023-08-17 16:46:02 -04:00
Eric Curtin
2cc6b53199
Merge pull request #2966 from cgwalters/ostree-admin-edit
Add `admin set-default`
2023-08-08 14:24:35 +01:00
Eric Curtin
c57c00569c
Merge pull request #2958 from cgwalters/deploy-loosen-etc-usretc
deploy: Support an empty `/etc` and populated `/usr/etc`
2023-08-05 16:48:28 +01:00
Colin Walters
3cd3251aa1 Add admin set-default
A core underlying primitive in the C library is the ability
to arbitrarily reorder bootloader entries.

Let's expose the basic functionality here with the ability to pick
an arbitrarily deployment for the next boot.

Closes: https://github.com/ostreedev/ostree/issues/2965
2023-08-03 15:09:57 -04:00
Colin Walters
ac42e29d66 os-init: Create a mount namespace
Today on anything using readonly sysroot `os-init` fails, because
we don't create a mount namespace if the `UNLOCKED` flag is specified
because we assume it's a readonly operation.

Since technically this is a mutation, let's just lock the sysroot
and use the tested path.
2023-08-02 14:32:22 -04:00
Colin Walters
0406fd3912 deploy: Support an empty /etc and populated /usr/etc
In preparation for support for a transient `/etc`:
https://github.com/ostreedev/ostree/issues/2868
particularly in combination with composefs.

Basically it's just much more elegant if we can directly mount
an overlayfs on the *empty* `etc` directory, using `usr/etc` as
the lower.

In the composefs case, we'd have to mount the composefs overlayfs
itself writable (and call `mkdir`) *just* so we can make that
empty `etc` directory which is ugly.
2023-07-29 08:47:54 -04:00
Colin Walters
6470429b2b tests/destructive: Turn off global sync()
Let's verify that things work with that off, as they should.

Previously:
cb73129483
"deploy: Add a 5s max timeout on global filesystem sync()"

But we may still have problems even with that, see
https://issues.redhat.com/browse/OCPBUGS-15917
where it might be that even a thread doesn't work because
we're locked in the kernel.
2023-07-28 17:59:28 -04:00
Eric Curtin
250c40a671
Merge pull request #2948 from cgwalters/composefs-more-cleanups
composefs: Use lowerdir in /run
2023-07-25 10:25:29 +01:00
Colin Walters
d7fe9e54ed kernel-args: Move private functions out of public header
External users can't call `_` prefixed APIs, so move them
to a `-private.h`.
2023-07-23 13:22:14 -04:00
Colin Walters
64afbcdeb0 composefs: Use lowerdir in /run
I just noticed that this was another constant string duplicated
between prepare-root.c and libostree-1.so, and I went to make
it a common `#define` in libotcore.la.

But then I thought "it's ugly to have this directory mixed into
the deployment namespace" because in some theoretical world
it could also be in the ostree commit, which would cause weird
behavior.

I think this is transient state that is better in `/run`, so move
it there.
2023-07-22 16:46:17 -04:00
Colin Walters
1a2fac371b tests: Remove dead references to "SEED"
Nothing uses this.
2023-07-18 07:55:14 -04:00
Colin Walters
82d934916b commit: Add --sign-from-file
Passing the private key via a direct command line argument
is just a bad idea because it's highly likely to get logged
or appear in `ps`.
Spotted in review of work for composefs signatures.
2023-07-15 09:50:40 -04:00