Commit Graph

1193 Commits

Author SHA1 Message Date
Colin Walters
c0beedbb48 tests/inst: Drop nix dependency
It has various semver bumps and we were literally just using
it to duplicate what's already in `libc`.
2022-11-22 10:13:42 -05:00
Colin Walters
180a1d8f0a tests/inst: Update to latest ostree-ext 2022-11-22 10:09:20 -05:00
Simon McVittie
29340dba04 Replace calls to g_memdup() with g_memdup2()
g_memdup() is subject to an integer overflow on 64-bit machines if the
object being copied is larger than UINT_MAX bytes. I suspect none of
these objects can actually be that large in practice, but it's easier
to replace all the calls than it is to assess whether we need to
replace them.

A backport in libglnx is used on systems where GLib is older than 2.68.x.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-11-11 17:10:37 -05:00
Colin Walters
e66e0b06ef
Merge pull request #2754 from cgwalters/remove-readdir-rand
Remove readdir-rand
2022-11-10 05:59:36 -05:00
Colin Walters
a6c97e9129
Merge pull request #2759 from oglok/http_500
Enabling retry for HTTP 500 internal server error
2022-11-09 15:34:08 -05:00
Ricardo Noriega
6ce2324e4a Add tests for cases under random 500s
Signed-off-by: Ricardo Noriega <rnoriega@redhat.com>
2022-11-09 11:20:10 +01:00
Colin Walters
23446a0218 Remove readdir-rand
This was only there to reproduce a bug we hit long ago
with bootloader file ordering.  We're extremely unlikely
to reintroduce such a bug, and it's not worth carrying around
this code.
2022-11-01 16:29:17 -04:00
Simon McVittie
3f3a817375 readdir-rand: Copy full size of struct dirent
As noted in readdir(3), in the presence of long filenames it is
possible for a directory entry to be larger than `sizeof (struct dirent)`.
Copy the full length instead.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-10-28 12:21:58 +01:00
Miguel Angel Ajo
e234b630f8 Support overlayfs whiteouts on checkout
Introduces an intermediate format for overlayfs storage, where
.wh-ostree. prefixed files will be converted into char 0:0
whiteout devices used by overlayfs to mark deletions across layers.

The CI scripts now uses a volume for the scratch directories
previously in /var/tmp otherwise we cannot create whiteout
devices into an overlayfs mounted filesystem.

Related-Issue: #2712
2022-09-28 12:26:31 +02:00
Colin Walters
6651b72a7a
Merge pull request #2544 from dbnicholson/finalize-block
finalize-staged: Ensure /boot and /sysroot automounts don't expire
2022-08-30 15:12:32 -04:00
Dan Nicholson
f3db79e7fa finalize-staged: Ensure /boot automount doesn't expire
If `/boot` is an automount, then the unit will be stopped as soon as the
automount expires. That's would defeat the purpose of using systemd to
delay finalizing the deployment until shutdown. This is not uncommon as
`systemd-gpt-auto-generator` will create an automount unit for `/boot`
when it's the EFI System Partition and there's no fstab entry.

To ensure that systemd doesn't stop the service early when the `/boot`
automount expires, introduce a new unit that holds `/boot` open until
it's sent `SIGTERM`. This uses a new `--hold` option for
`finalize-staged` that loads but doesn't lock the sysroot. A separate
unit is used since we want the process to remain active throughout the
finalization run in `ExecStop`. That wouldn't work if it was specified
in `ExecStart` in the same unit since it would be killed before the
`ExecStop` action was run.

Fixes: #2543
2022-08-30 09:16:39 -06:00
Huijing Hei
37aa2ac287 Fix ostree admin kargs edit-in-place assertion when deployments
are pending

This is to support pending deployments instead of rasing assertion.
For example:
```
$ sudo rpm-ostree kargs --append=foo=bar
$ sudo ostree admin kargs edit-in-place --append-if-missing=foobar
```
After reboot we get both `foo=bar foobar`.

Fix https://github.com/ostreedev/ostree/issues/2679
2022-08-29 11:31:32 +08:00
Colin Walters
ad0354ac36 cli/rev-parse: Add --single option
In the current "ostree native container" flow, we're inserting
a commit object into the repo but with no refs.

We have hacks in a few places to find the commit digest via e.g.
`find repo/objects -name *.commit` but that's a horrible hack.
Add `ostree rev-parse --single` which will print the single commit,
and error out if there is not exactly one commit.

Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2022-08-18 11:08:17 -04:00
Jonathan Lebon
84670a0070 tests/kolainst/staged-deploy: parse rpm-ostree status --json instead
Don't parse `rpm-ostree status` output, it's not meant for that. Use
`--json` output instead.

While we're here, fix an obsolete reference to Ansible.

Related: https://github.com/coreos/rpm-ostree/pull/3938
2022-08-15 17:50:11 -04:00
Lukas Kalbertodt
02b162347c
Update to libtest-mimic 0.5.0 2022-08-13 16:41:34 +02:00
Colin Walters
15740d042c Release 2022.5 2022-07-22 15:08:25 -04:00
Huijing Hei
8f24e0826a Add test to verify ostree admin kargs edit-in-place working 2022-07-12 20:56:18 +08:00
Colin Walters
e98988ba17 tests/inst: Port to cap-std
Part of an ongoing effort.
2022-06-24 16:23:05 -04:00
Colin Walters
52d6f4e790 tests/staged-deploy.sh: Hack around cosa systemd unit check
https://github.com/coreos/coreos-assembler/pull/2921 broke this
test which is intentionally causing a systemd unit to fail.

As they say, necessity is the mother of invention.  They don't
say though that need always causes particularly *beautiful* things
to be invented...
2022-06-24 10:18:53 -04:00
Jonathan Lebon
7814d9339e tests/inst/destructive: stop disabling fedora-coreos-pinger
It was removed from FCOS:
https://github.com/coreos/fedora-coreos-tracker/issues/770
2022-06-23 15:50:39 -04:00
Huijing Hei
3bc59a5206 RFE: Add a hidden option to ostree admin kargs edit-in-place to
update all existing deployments in place

Example:
$ sudo ostree admin kargs edit-in-place --append-if-missing=rw
See https://github.com/ostreedev/ostree/issues/2617

This will not add duplicate key, if there is `TESTARG=VAL1` in the
kernel arguments, `--append-if-missing=TESTARG=VAL2` will be ignored.
2022-06-23 22:31:39 +08:00
Simon McVittie
ce428c1f60 test-basic-c: Don't assert that extended attributes are available
Not all filesystems support extended attributes. This test uses
/var/tmp to try to get an extended-attributes-capable filesystem,
but that might not succeed.

Signed-off-by: Simon McVittie <smcv@debian.org>
2022-06-17 14:42:01 +01:00
Colin Walters
e65c8e72c8
Merge pull request #2643 from cgwalters/rust-2021
rust: Switch to 2021 edition, bump MSRV, a few `format!` updates
2022-06-13 09:10:36 -04:00
Colin Walters
93e3784b66 rust: Use inline format! variables in a few places
Since our MSRV now supports it.
2022-06-09 17:51:07 -04:00
Colin Walters
99c122d219 rust: Switch to 2021 edition
No real changes.

```
$ cargo fix --edition
note: Switching to Edition 2021 will enable the use of the version 2 feature resolver in Cargo.
This may cause some dependencies to be built with fewer features enabled than previously.
More information about the resolver changes may be found at https://doc.rust-lang.org/nightly/edition-guide/rust-2021/default-cargo-resolver.html
When building the following dependencies, the given features will no longer be used:

  libc v0.2.126 removed features: extra_traits

The following differences only apply when building with dev-dependencies:

  getrandom v0.2.6 removed features: std
```

which looks OK to me.
2022-06-09 15:55:41 -04:00
Colin Walters
2688d7d261 tests/inst: Bump the version of ostree-ext
In the interest of cross-testing and keeping things up to date.

Hmm, I think we need to set up dependabot here.
2022-06-09 14:52:04 -04:00
Colin Walters
145d91d1c9 lib: Fix symbol versioning inheritance
I messed this up; the last release should inherit from the previous
release (N-1) and not the previous to that (N-2).

I think (hope) this isn't an ABI break...

Just noticed this when I was going to add a new symbol.
2022-06-08 16:44:51 -04:00
Colin Walters
5fa364d399 Release 2022.4 2022-06-06 11:04:44 -04:00
Huijing Hei
0ba3bfe4fd Teach --karg-delete option to ostree admin deploy
See https://github.com/ostreedev/ostree/issues/2586
2022-05-19 22:50:09 +08:00
Colin Walters
891c7dffa7
Merge pull request #2575 from cgwalters/merge-ostree-rs
Merge ostree-rs into this repository
2022-05-09 16:41:08 -04:00
Colin Walters
ee2c31badd tests/inst: Add .gitignore
Need this now that it is it's own workspace.
2022-05-09 14:52:26 -04:00
Colin Walters
558d966420 tests/inst: Fix install rules for ostree-rs merger
`tests/inst` became its own workspace.
2022-05-09 12:57:40 -04:00
Colin Walters
f79b2cea91 Add APIs to get xattrs from disk
I'm aiming to do some more work on the Rust side around `fsck`
like functionality, and this is a useful primitive.  There isn't
a great Rust crate for xattrs, and I think it's better to share this
code.
2022-05-09 12:33:38 -04:00
Colin Walters
252060906b build-sys: Adjust for merge of ostree-rs
Fix up the paths for the crates now that the Rust bindings are in
`rust/`.

We can't today include the test suite because it depends on `ostree-rs-ext`
which would make everything circular.

(Building that now requires a separate `cd tests/inst && cargo build`)
2022-05-06 12:53:57 -04:00
Colin Walters
d3d3e4ea13 Add an ostree-boot-complete.service to propagate staging failures
Quite a while ago we added staged deployments, which solved
a bunch of issues around the `/etc` merge.  However...a persistent
problem since then is that any failures in that process that
happened in the *previous* boot are not very visible.

We ship custom code in `rpm-ostree status` to query the previous
journal.  But that has a few problems - one is that on systems
that have been up a while, that failure message may even get
rotated out.  And second, some systems may not even have a persistent
journal at all.

A general thing we do in e.g. Fedora CoreOS testing is to check
for systemd unit failures.  We do that both in our automated tests,
and we even ship code that displays them on ssh logins.  And beyond
that obviously a lot of other projects do the same; it's easy via
`systemctl --failed`.

So to make failures more visible, change our `ostree-finalize-staged.service`
to have an internal wrapper around the process that "catches" any
errors, and copies the error message into a file in `/boot/ostree`.

Then, a new `ostree-boot-complete.service` looks for this file on
startup and re-emits the error message, and fails.

It also deletes the file.  The rationale is to avoid *continually*
warning.  For example we need to handle the case when an upgrade
process creates a new staged deployment.  Now, we could change the
ostree core code to delete the warning file when that happens instead,
but this is trying to be a conservative change.

This should make failures here much more visible as is.
2022-04-26 13:02:46 -04:00
Colin Walters
b92dbc49f0 ci: Mask zincati for synthetic update
Our CI started falling over because coreos-assembler checks
for units stuck activating as of https://github.com/coreos/coreos-assembler/pull/2810

Really need to centralize the code for this and fix the root
problem, but...not today.

xref https://github.com/coreos/coreos-assembler/pull/2814
2022-04-20 20:23:10 -04:00
Luca BRUNO
7e83726faa
tests: move fixture to fix installed tests
This moves around the packed archive for bare-split-xattrs mode,
in order to fix installed tests.
2022-04-12 09:53:54 +00:00
Jan Tojnar
1898019fac tests/test-cli-extensions: Fix with single-binary coreutils
On systems where `coreutils` are built with `--enable-single-binary=symlinks` like Nix,
`/usr/bin/env` is symlinked to `/usr/bin/coreutils` and uses `argv[0]` to determine which program to run.
Since the `test-cli-extensions.sh` created a new symlink named `ostree-env`,
coreutils would be confused about the utility to choose, so running it would fail:

	ostree-env: unknown program ‘ostree-env’
	Try 'ostree-env --help' for more information.

Fixes: https://github.com/ostreedev/ostree/issues/2553
2022-03-26 10:55:42 +01:00
Jonathan Lebon
73bc62cac3
Merge pull request #2528 from cgwalters/test-crate-update 2022-03-14 15:20:54 -04:00
Colin Walters
afdc84b97b Update to sh-inline 0.2
Syncing up with the latest.
2022-03-14 09:54:48 -04:00
Daniel Kolesa
aca9e8e6a7 glib: bump glib requirement to 2.66 and port to GUri
This removes the old SoupURI copypasta from previous generation
of libsoup and opens up a path for a simple libsoup3 port.
2022-03-12 04:44:18 +01:00
Colin Walters
75ae283f23 Update to rand 0.8
Part of general crate updates.
2022-03-11 15:34:59 -05:00
Colin Walters
fba7efb6da Update to ostree-ext 0.6
Part of general crate updates.
2022-03-11 15:33:02 -05:00
Colin Walters
34d1bcc68a Update to nix 0.23
Part of general crate updates.
2022-03-11 15:32:27 -05:00
Colin Walters
b6d1119f91 tests: Stop using inventory crate
I was reading this thread
https://internals.rust-lang.org/t/from-life-before-main-to-common-life-in-main/16006/30
and that reminded me about this code, which it turns out actually
doesn't compile with my default local cargo config:
```
$ cat ~/.cargo/config
[target.x86_64-unknown-linux-gnu]
rustflags = ["-Ctarget-cpu=native", "-C", "link-arg=-fuse-ld=lld"]

[profile.release]
incremental = true
$ cargo b
...
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/var/srv/walters/src/github/ostreedev/ostree/target/debug/deps/ostree_test-4ca8e730f9dc6ffc.10325uqlhkyr5uol.rcgu.o" "/var/srv/walte"
  = note: ld.lld: error: undefined symbol: __start_linkme_NONDESTRUCTIVE_TESTS
          >>> referenced by 22nn09lfsklfqvyy
          >>>               /var/srv/walters/src/github/ostreedev/ostree/target/debug/deps/ostree_test-4ca8e730f9dc6ffc.22nn09lfsklfqvyy.rcgu.o:(ostree_tes)

```

For now let's just go back to having a static list of functions.
We don't have *too* many of those.
2022-03-11 14:53:29 -05:00
Colin Walters
0d020a7145 tmpfiles: Create /run/ostree
This is referenced by 9645cee4f2/lib/src/globals.rs (L16)
specifically used for the (container image) pull secret in
`/run/ostree/auth.json`.

Let's pre-create the directory so users don't have to.

Motivated by https://github.com/openshift/machine-config-operator/pull/3007#discussion_r824172564
2022-03-11 13:08:23 -05:00
Simon McVittie
92ca2c7794 test-prune: Read to the end of cut(1) output
If we use head(1) to take only the first two lines, then cut(1) and
earlier pipeline entries are killed by SIGPIPE (if they have not already
terminated), and that's flagged as an error under `set -o pipefail`.
Use an equivalent sed command to take exactly the second line, but
without SIGPIPE.

Signed-off-by: Simon McVittie <smcv@debian.org>

Gbp-Pq: Name test-prune-Read-to-the-end-of-cut-1-output.patch
2022-03-07 23:03:33 +00:00
Colin Walters
fbc6d21c2f Release 2022.2 2022-03-03 18:44:30 -05:00
Luca BRUNO
aca5671eb1
tests/basic-bare-split-xattrs: add fixture, check read logic 2022-03-03 11:11:59 +00:00
Luca BRUNO
7e79b82ff8
lib/commit: disallow writing content in 'bare-split-xattrs' mode
This prevents writing content into 'bare-split-xattrs` repository,
while carving some space for experimenting via a temporary
`OSTREE_EXP_WRITE_BARE_SPLIT_XATTRS` environment flag.
2022-03-02 16:45:02 +00:00