Commit Graph

20 Commits

Author SHA1 Message Date
Colin Walters
8821ec6e56 upgrade: Stabilize deployment staging
We're waaay overdue for this, it's been the default
in rpm-ostree for years, and solves several important bugs
around not capturing `/etc` while things are running.

Also, `ostree admin upgrade --stage` (should) become idempotent.

Closes: https://github.com/ostreedev/ostree/issues/2389
2021-09-07 16:12:43 -04:00
Colin Walters
7ff848f101
Merge pull request #2377 from smcv/libtest-core
libtest-core: Add some improvements from bubblewrap
2021-06-21 10:12:40 -04:00
Simon McVittie
14d6e75752 libtest-core: Mention bubblewrap as a user of this file
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-18 10:14:01 +01:00
Simon McVittie
c0157f96b2 libtest-core: Update URL of rpm-ostree
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-18 10:13:38 +01:00
Simon McVittie
d7cdde5d34 libtest-core: On failure, make it clearer what has happened
If we fail as a result of `set -x`, It's often not completely obvious
which command failed or how. Use a trap on ERR to show the command that
failed, and its exit status.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-18 09:49:53 +01:00
Alexander Larsson
c934fc8f99 libtest-core: Add assert_files_equal
[Originally from bubblewrap commits c5c999a7 "tests: test --userns"
and 3e5fe1bf "tests: Better error message if assert_files_equal fails";
separated into this commit by Simon McVittie.]
2021-06-18 09:46:31 +01:00
Colin Walters
73e3ccc401 Use generator to enable ostree-remount.service and ostree-finalize-staged.path
We struggled for a long time with enablement of our "internal units",
trying to follow the philosophy that units should only be enabled
by explicit preset.

See https://bugzilla.redhat.com/show_bug.cgi?id=1451458
and https://github.com/coreos/rpm-ostree/pull/1482
etc.

And I just saw chat (RH internal on a proprietary system sadly) where
someone hit `ostree-remount.service` not being enabled in CentOS8.

Thinking about this more, I realized we've shipped a systemd generator
for a long time and while its only role until now was to generate `var.mount`,
but by using it to force on our internal units, we don't require
people to deal with presets anymore.

Basically we're inverting things so that "if ostree= is on the kernel
cmdline, then enable our units" and not "enable our units, but have
them use ConditionKernelCmdline=ostree to skip".

Drop the weird gyrations we were doing around `ostree-finalize-staged.path`
too; forking `systemctl start` is just asking for bugs.

So after this, hopefully we won't ever again have to think about
distribution presets and our units.
2021-06-16 09:40:28 -04:00
Colin Walters
edf7477ee9 deploy: Warn if we find content in the deployment's /var
This will be ignored, so let's make it very clear
people are doing something wrong.  Motivated by a bug
in a build pipeline that injected `/var/lib/rpm` into an ostree
commit which ended up crashing rpm-ostree because it was an empty db
which it wasn't expecting.

It *also* turns out rpm-ostree is incorrectly dumping content in the
deployment `/var` today, which is another bug.
2021-06-10 07:33:17 -04:00
Colin Walters
6664ee4ed4 ci: Fix staged-delay to work with newer systemd
Yeah, we should stop parsing the text; I need to dig at that
at some point.
2021-05-25 16:28:32 -04:00
Colin Walters
46a0911c6d build-sys: Add toplevel workspace Cargo.toml
rust-analyzer is happier with this because it understands
the project structure out of the box.

We aren't actually again adding a dependency on Rust/cargo in the core,
this is only used to make `cargo build` work out of the box to build
the Rust test code.
2021-04-15 12:33:34 -04:00
Colin Walters
6d0bb10bf0 cci: Update for buildroot changes
See latest in https://github.com/coreos/fedora-coreos-tracker/blob/master/docs/ci-and-builds.md
2021-03-31 15:11:34 -04:00
Colin Walters
975496d241 deploy: Add subbootversion to journal
To help debug an issue we've seen where `/boot` isn't
in sync with the `/ostree/boot` dir, let's log to the journal
what we're doing.
2021-02-28 14:47:58 +00:00
Jonathan Lebon
81b13da8e3 lib/deploy: Add support for overlay initrds
In FCOS and RHCOS, the need to configure software in the initramfs has
come up multiple times. Sometimes, using kernel arguments suffices.
Other times, it really must be a configuration file. Rebuilding the
initramfs on the client-side however is a costly operation. Not only
does it add complexity to the update workflow, it also erodes a lot of
the value obtained from using the baked "blessed" initramfs from the
tree itself.

One elegant way to address this is to allow specifying multiple
initramfses. This is supported by most bootloaders (notably GRUB) and
results in each initrd being overlayed on top of each other.

This patch allows libostree clients to leverage this so that they can
avoid regenerating the initramfs entirely. libostree itself is agnostic
as to what kind and how much data overlay initrds contain. It's up to
the clients to enforce such boundaries.

To implement this, we add a new ostree_sysroot_stage_overlay_initrd
which takes a file descriptor and returns a checksum. Then users can
pass these checksums when calling the deploy APIs via the new array
option `overlay_initrds`. We copy these files into `/boot` and add them
to the BLS as another `initrd` entry.
2020-09-30 13:29:32 -04:00
Colin Walters
1101c02c2a tests/inst: Add destructive test framework
This adds infrastructure to the Rust test suite for destructive
tests, and adds a new `transactionality` test which runs
rpm-ostree in a loop (along with `ostree-finalize-staged`) and
repeatedly uses either `kill -9`, `reboot` and  `reboot -ff`.

The main goal here is to flush out any "logic errors".

So far I've validated that this passes a lot of cycles
using
```
$ kola run --qemu-image=fastbuild-fedora-coreos-ostree-qemu.qcow2 ext.ostree.destructive-rs.transactionality --debug --multiply 8 --parallel 4
```
a number of times.
2020-08-17 14:34:04 +00:00
Colin Walters
f2773c1b55 Add "transient" unlock
I was thinking a bit more recently about the "live" changes
stuff https://github.com/coreos/rpm-ostree/issues/639
(particularly since https://github.com/coreos/rpm-ostree/pull/2060 )
and I realized reading the last debates in that issue that
there's really a much simpler solution; do exactly the same
thing we do for `ostree admin unlock`, except mount it read-only
by default.

Then, anything that wants to modify it does the same thing
libostree does for `/sysroot` and `/boot` as of recently; create
a new mount namespace and do the modifications there.

The advantages of this are numerous.  First, we already have
all of the code, it's basically just plumbing through a new
entry in the state enumeration and passing `MS_RDONLY` into
the `mount()` system call.

"live" changes here also naturally don't persist, unlike what
we are currently doing in rpm-ostree.
2020-08-07 18:57:56 +00:00
Colin Walters
6a5f97c145 tests: Add needs-internet tag for webserver bits
Fixes the tests, see https://github.com/coreos/coreos-assembler/pull/1600
TODO: provide a webserver binary via virtio or so
2020-07-17 01:37:46 +00:00
Colin Walters
5aa22e0b1f tests: Port to Debian autopkgtest reboot API
See https://github.com/coreos/coreos-assembler/pull/1528

I think we can drop the old cosa reboot APIs after this,
though I've already forgotten where else I might have written
tests using it.
2020-06-19 13:04:22 +00:00
Matthew Leeds
d21181653e tests: Check that example symbol isn't released
For the motivation for this see #2132.
2020-06-17 15:28:15 -07:00
Colin Walters
1f637bf341 Add new Rust-based tests
There's a lot going on here.  First, this is intended to run
nicely as part of the new [cosa/kola ext-tests](https://github.com/coreos/coreos-assembler/pull/1252).

With Rust we can get one big static binary that we can upload,
and include a webserver as part of the binary.  This way we don't
need to do the hack of running a container with Python or whatever.

Now, what's even better about Rust for this is that it has macros,
and specifically we are using [commandspec](https://github.com/tcr/commandspec/)
which allows us to "inline" shell script.  I think the macros
could be even better, but this shows how we can intermix
pure Rust code along with using shell safely enough.

We're using my fork of commandspec because the upstream hasn't
merged [a few PRs](https://github.com/tcr/commandspec/pulls?q=is%3Apr+author%3Acgwalters+).

This model is intended to replace *both* some of our
`make check` tests as well.

Oh, and this takes the obvious step of using the Rust OSTree bindings
as part of our tests.  Currently the "commandspec tests" and "API tests"
are separate, but nothing stops us from intermixing them if we wanted.

I haven't yet tried to write destructive tests with this but
I think it will go well.
2020-05-27 21:59:23 +00:00
Colin Walters
718cca8055 tests/kola: Move to tests/kolainst
Follow the precedent set in https://github.com/coreos/rpm-ostree/pull/2106
and rename the directory, to more clearly move away from the
"uninstalled" test model.  Prep for Rust-based tests.
2020-05-27 15:16:50 +00:00