Commit Graph

701 Commits

Author SHA1 Message Date
dependabot[bot]
9a7ffb2b0c build(deps): bump libdnf from 31a8b8b to 930f258
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `31a8b8b` to `930f258`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](31a8b8bcec...930f2582f9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 11:52:49 -04:00
Colin Walters
1541ed4312 tests: Stop running test-lib-introspection.sh
Even more fallout of the buildroot change to stop deriving from
cosa.  The new buildroot doesn't have `pygobject3`.

We could convert this to an installed test but...blah.
Not worth it.
2021-04-07 22:11:56 +00:00
Jonathan Lebon
4624c3bdf8 app/db: Add db diff --advisories
This will be used by at least coreos-assembler to store in `meta.json`
so it's easy to tell new advisories that appeared in a given build.
2021-04-06 17:04:51 -04:00
Jonathan Lebon
136a3ea7e9 app/db: Add db list --advisories
This makes it easy to see the advisories stored in commit metadata.
2021-04-06 17:04:51 -04:00
Colin Walters
fa81456cbf Rewrite bwrap code in Rust
I tried to do this incrementally but it snowballed.
2021-03-29 14:45:35 +00:00
Colin Walters
335415d601 tests: Add infinite loop package to kolainst-build
This way it's easy to get outside of the vmcheck tests.
2021-03-26 16:59:30 +00:00
Jonathan Lebon
0fd82990c7
Merge pull request #2691 from cgwalters/strip-remote-error
clientlib: Strip GDBus remote error before throwing exception
2021-03-23 10:49:45 -04:00
Colin Walters
7ec7e3eb28 clientlib: Strip GDBus remote error before throwing exception
In general our error handling philosophy is "errors are strings".
Previously (before C++) we were relying on calling this API
to strip out the GDBus remote error message.

In the recent refactoring of clientlib to expose the txn APIs
to Rust via C++, we implicitly lost that because the C++
wrapper API throws an exception.

Add a copy of the remote error stripping here.  Eventually
it will likely make sense for it to *only* be here instead
of also in `main.cxx`, but one thing at a time.
2021-03-22 19:30:49 +00:00
Jonathan Lebon
7979e250ee lockfile: Allow locking by source RPM EVR
Right now if we want to lock e.g. systemd, we need to specify every
subpackage of systemd that we use. This is a lot of duplication because
in the majority of cases, what we really mean is "lock at this build of
systemd".

Since RPMs bake in the source RPM they were built from, we can use this
to lock packages more succinctly. See the testcase and #2676 for
examples of how this looks.

Closes: https://github.com/coreos/rpm-ostree/issues/2676
2021-03-22 12:57:49 -04:00
Colin Walters
7f9129d67d compose: Move mutate-os-release string code to Rust
More classic C string manipulation which is much nicer in Rust
2021-03-18 18:05:34 +00:00
Colin Walters
aa64df080d daemon: Fix serialization of unlocked state
It turns out there's a naming clash between `to_string()` here in
Rust *and* introspection is incorrectly associating the method
with `ostree::Deployment` because of the naming prefix.
2021-03-12 18:01:47 -05:00
Colin Walters
f6f6ac5ff3 ci: Add composepost-checks.sh, drop a compose test
The compose tests are expensive; each run involves running
all the `%post` scripts and `dracut` etc.  This is definitely
a source of timeouts in CCI.

Remove `test-boot-location-modules.sh` - it's the default
now and is used by FCOS.  Add dedicated script where we can
test all these things by default after a `cosa build`.

This aims to move the compose tests to only cover bits *not*
in cosa like the non-unified-core path.
2021-03-12 15:53:01 -05:00
Jonathan Lebon
46f556f437
Merge pull request #2658 from cgwalters/live-no-restart
live: Don't output "services restart" for pure package additions
2021-03-12 09:47:45 -05:00
Colin Walters
2187ef0f45 live: Don't output "services restart" for pure package additions
If all we're doing is layering new packages, no need to tell
the admin that things may need restarting.
2021-03-11 22:50:07 +00:00
Colin Walters
445af087d6 tests: Drain vmcheck/test-misc-1 into kola/misc.sh
Where I stalled out before is this file has `pkg-add foo`, but
now that we have the `foo` package pre-built we can move all
this stuff into `misc.sh`.

I dropped the YAML parsing of `--version` because we don't
have python.  This is related to
https://github.com/coreos/coreos-assembler/issues/1645
2021-03-11 16:34:07 -05:00
Jonathan Lebon
2fedc3b0ac lockfile: Allow specifying EVR rather than EVRA
In FCOS, we use "override" lockfiles to pin packages to certain
versions. Right now, we have separate overrides for each base arch we
(eventually want to) support. But that makes maintaining the overrides
cumbersome because of all the duplication.

Let's allow lockfiles to specify only the `evr` of a package, which is
just as good for FCOS, and means that we'll only have to maintain a
single override file for all the architectures.
2021-03-11 00:01:46 +01:00
Colin Walters
1dc7503838 ci/prow: Install in target cosa, also install and run our tests
This is a major downside of reworking and generating new CI
flows, it's super easy to lose testing what you intend to.

Also, we clearly need to figure out a flow where this is shared
across repos, since I don't want to copy-paste this into e.g. ostree too.
That's https://github.com/coreos/fedora-coreos-tracker/issues/263
2021-03-10 16:58:49 +01:00
Colin Walters
3041d648bb install: Add --apply-live
OK I think it's time.  This exposes the `apply-live` functionality
as implicitly stable, but specific to the package install case.

I'd like to add more intelligence to `apply-live` around separating
pure "additions" (as in this case) versus package (file) changes.

The change here doesn't try to do that; the implementation is
incredibly simple, we just have the client chain together the two
distinct transactions.
2021-03-08 20:54:18 +01:00
Colin Walters
485c1862ce live: Disallow replacement/removal by default, add --allow-replacement
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).

The latter is really quite safe, and live layering is one
of the most requested features.
2021-03-08 20:54:18 +01:00
Colin Walters
2a9423ccb8 tests: Port apply-live to kola ext tests
Continuing the momentum to use kola ext tests.

One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
2021-03-08 20:54:18 +01:00
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Colin Walters
6fd9db246a Add basic Rust CLI bindings for DBus
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.

A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.

My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
2021-03-04 09:17:21 -05:00
Colin Walters
8dd5583ae9 Remove cbindgen
We have fully transitioned to cxx-rs!  This drops a lot of now
dead code; only one binding system to think about generating
source code.  For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
2021-03-03 16:51:38 -05:00
Colin Walters
85f22baec7 tests: Add RPMs to installed kola tests, port layering-local
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.

We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over.  Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data.  Add the concept
of multiple repo versions too.

Right now we only generate the one RPM needed for the `layering-local`
test and port it.
2021-03-02 09:47:03 -08:00
Colin Walters
7367aa8469 tests/apply-live: Upgrade+liveapply no-package changes
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction".  I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
2021-03-01 20:40:48 +01:00
Colin Walters
4d7de4fa59 tests/vmcheck: Use .cosa over COSA_DIR
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.

Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
2021-03-01 18:53:56 +01:00
Colin Walters
732e0f232a daemon: Fix crash on upgrade with remote:checksum
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).

The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.

But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.

I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.

Closes: https://github.com/coreos/rpm-ostree/issues/2603
2021-03-01 16:57:51 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
0544d1c92d ci: Drop tests/vmcheck/image.qcow2, use COSA_DIR/.cosa
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default.  We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.

Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
2021-02-23 17:23:26 -05:00
Kelvin Fan
626d021edb app/upgrade: Do not upgrade if updates driver registered
Do not perform an upgrade if detected that an updates driver has
been registered.

Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
2021-02-17 11:43:34 -05:00
Colin Walters
e6c045cada Add an rpmostree-client sub-crate
This is intended to be published to https://crates.io/crates/rpmostree-client
Part of https://github.com/coreos/rpm-ostree/issues/2389

This directly imports the code from
5551c54c6e/tests/inst/src/rpmostree.rs

Once merged and released I'll try converting the ostree test suite
over as well as Zincati.

Internally add a testutils helper to validate it works.
2021-02-16 19:22:26 -05:00
Colin Walters
7bf885ecf7 tests: Fix bodhi link
From review in https://github.com/coreos/rpm-ostree/pull/2585#discussion_r577149232
2021-02-16 18:18:27 -05:00
Colin Walters
c1ff933bc3 tests/layering-non-root-caps: Adapt to new libcap output
I didn't deep dive on this, just observed that the new output
matches what we laid down.  There's ~2.5 years of changes
and ~200 commits between 2.28 and 2.48.
2021-02-16 17:08:27 -05:00
Kelvin Fan
84065c48df tests/common/libvm: Print error if bad tests/vmcheck/image.qcow2
Print error and exit if cannot find `tests/vmcheck/image.qcow2` or
bad symlink causing failure to spawn a VM when calling the
`vm_kola_spawn()` function.
2021-02-11 21:11:37 -05:00
Jonathan Lebon
99486a75e8 Add /usr/lib/rpm/macros.d/macros.rpm-ostree to set %_dbpath to /usr/share/rpm
We trigger a librpm macro file load in many of our paths. Since the
default value shipped by rpm's macro file sets `_dbpath` to
`/var/lib/rpm`, we have to explicitly set that back to `/usr/share/rpm`
in those paths.

This became more problematic recently with libsolv v0.7.17 which fully
keys off of `_dbpath` to find the rpmdb path to load:

04d4d036b2

And it's not technically wrong; we really should make that macro not
lie. This is what this patch does by injecting an RPM macro file in our
composes which sets it to /usr/share/rpm. So then e.g. the `rpm` CLI
doesn't actually need the `/var/lib/rpm` backcompat link anymore, though
there's no harm in leaving it.

In the future, we should be able to drop this once we move all of Fedora
to `/usr/lib/sysimage/rpm` (see
https://github.com/coreos/fedora-coreos-tracker/issues/639).

Closes: #2548
2021-02-09 18:36:35 -05:00
Jonathan Lebon
57edf7e638 tests/compose.sh: Always rebuild supermin appliance
We always want the latest rpm-ostree binaries tested, so we need to
always rerun supermin.

Patch better viewed with whitespace ignored.
2021-02-03 12:22:38 -05:00
Jonathan Lebon
317b920667 extensions: Add support for development extensions
In RHCOS, we ship kernel development-related packages as an extension.
Those aren't really extensions that are meant to be layered onto the
host.  They're meant to be used in a build environment somewhere to
compile kernel modules.

This makes it very different from "OS extensions" in at least two
drastic ways:
1. we don't want to do any depsolving (e.g. we don't want to pull in
   `gcc` or something)
2. some of those packages may be present in the base already, but we
   still want to redownload them

Hesitated putting this functionality in rpm-ostree, but I think in the
end it cuts from the benefit of moving this code to rpm-ostree if we
can't entirely get rid of the Python script it obsoletes. Plus, being
able to use the `match-base-evr` is still really useful for this use
case.

Let's add a new `kind` key to support this. The traditional extensions
are called "OS extensions" and these new extensions are called
"development extensions".

The latter is not yet part of the state checksum, so change detection
doesn't work there. I think that's fine for now though because the
primary use case is the kernel, and there we want to match the base
version. So if the kernel changes, the base would change too. (Though
there's the corner case of adding a new package to the list while at the
same version...)
2021-02-03 12:22:38 -05:00
Kelvin Fan
4dd80a8b9c app/status: Use AutomaticUpdatesDriver if updates driven by external driver
When automatic updates are driven by an external driver like Zincati,
display e.g. `AutomaticUpdatesDriver: Zincati` instead of
`AutomaticUpdates: driven by Zincati`, since the latter might suggest it
is rpm-ostree's own built-in support for automatic updates.
2021-02-03 09:52:43 -05:00
Kelvin Fan
1e79e5abde app/status: Show update driver's state and status
Display update driver's systemd unit's `ActiveState` property.
Also display `StatusText` property if unit is a service unit and
property is available.
2021-02-03 09:52:43 -05:00
Colin Walters
d218f27188 Change main entrypoint to be Rust
We now have bidirectional calling between Rust and C++,
but we are generating two static libraries that we then
link together with a tiny C++ `main.cxx`.
Let's make another huge leap towards oxdiation by
having Rust be the entrypoint.  This way cargo natively
takes care of linking the internal Rust library, and
our C++ internals become the library.

In other words, we've now fully inverted from
"C app with internal Rust library"
to "Rust binary with internal C++ library".

In order to make this work though we have to finally
kill the C unit tests.  But mostly everything covered
there is either being converted to Rust, or covered
elsewhere anyways.

Now as the doc comments in `main.rs` say...this is
a bit awkward because all the CLI code is still in C++.
Porting stuff to use e.g. `structopt` natively would
be a bit of a slog.  For now, we basically rely on
the fact that the Rust-native CLIs are all hidden
commands.

Update submodule: libdnf
2021-02-01 08:17:52 -05:00
Colin Walters
e87a64576c Bridge "next version" API to Rust, use it for unit tests
This demonstrates well the strength of the cxx-rs approach;
we can keep an API in C++ but add unit tests in Rust which
just works much more nicely.

Prep for https://github.com/coreos/rpm-ostree/pull/2502
which wants to drop the C++ unit tests.
2021-01-29 05:29:47 -05:00
Colin Walters
1cef69cf8a lib: Don't link to libdnf
Prep for "Rust-as-main", where I want to build libdnf statically.
And this really completes the "library thinout" story because
now we avoid dragging our *private* `libdnf.so` into the caller's
address space, which can cause potential conflicts if they're
also linking the system one. (Which could easily occur with
something like gnome-software)

All we were using libdnf for (indirectly via libsolv) is comparing
version strings but librpm can already do that for us.
2021-01-28 11:15:44 -05:00
Colin Walters
173db90941 tests: Fix compliation
This makes `make check` go again.
2021-01-27 19:09:33 -05:00
Colin Walters
cc230e25a0 tests/layering-fedorainfra: Update rpm-ostree build
The previous build was GC'd; unfortunately it's very nontrivial
to make this test truly robust over time because FCOS changes;
we might sometimes have an outstanding update, other times might
not etc.

Let's just sanity check the commands; ultimately they're
thin wrappers around just downloading packages so we don't need
deep checks.
2021-01-27 17:12:30 -05:00
Jonathan Lebon
c98c227c0f extensions: Write JSON to output dir
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.

This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
2021-01-27 00:44:42 +01:00
Colin Walters
c3da95a119 Move nevra parsing to Rust, first use of extern "C++"
Until now with cxx-rs we'd been using it effectively as a better
cbindgen - we're exposing Rust code to C++ safely.  This is
the first case of having Rust calling back into C++ using cxx-rs.
2021-01-26 13:47:56 +01:00
Kelvin Fan
34746d0428 deploy: Allow empty string argument if --register-driver
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
2021-01-26 01:32:52 +01:00
Kelvin Fan
e7558c3dc9 app/status: Display update driver info
Read from `/run/rpm-ostree/update-driver.gv` and display the update
driver name (and systemd unit if verbose).
2021-01-24 06:28:08 +01:00