Commit Graph

713 Commits

Author SHA1 Message Date
Kelvin Fan
86cf6ad479 app/clientlib: Don't error if update driver "stopped"
If updates driver is registered but "stopped", ignore driver (i.e.
do not error out during deploy, rebase, ugprade).

We define "stopped" to mean the driver's `ActiveState` is not in
the following states: "active", "activating", "reloading", "failed".
2021-05-07 18:41:45 +00:00
Jonathan Lebon
ba7a72995b treefile: Add new repo-packages field for pinning packages to repos
This addresses the server compose side of
https://github.com/coreos/rpm-ostree/issues/2584.

One tricky bit is handling overrides across included treefiles (or
really, even within a single treefile): as usual, higher-level treefiles
should override lowel-level ones. Rust makes it pretty nice to handle.

For now this just supports a `repo` field, but one could imagine e.g.
`repos` (which takes an array of repoids instead), or e.g.
`exclude-repos`.

The actual core implementation otherwise is pretty straightforward.

This should help a lot in RHCOS where we currently use many `exclude=`
directives in repo files to get it to do what we want.

This is also kind of a requirement for modularity support because as
soon as rpm-ostree becomes modules-aware, modular filtering logic will
break composes which assume rpm-ostree treats modular and non-modular
packages the same.
2021-04-30 09:13:18 -04:00
Jonathan Lebon
9348e44a55
Merge pull request #2782 from dustymabe/dusty-rpm-vercmp
lib/package: Fix rpm version comparison
2021-04-28 17:59:24 -04:00
Dusty Mabe
9e45eed598 lib/package: Fix rpm version comparison
`rpmvercmp()` doesn't properly compare full EVR. It needs to
be given the Version and Release separately in order to do
a comparison. For example `rpmVersionCompare()` first calls
`rpmvercmp()` to compare the Version and then checks the Release
in a second call to `rpmvercmp()`.

35739c2a22/lib/headerutil.c (L434-L440)

Let's just use `rpmverCmp()` instead, which can do the full
EVR comparison.

Fixes: https://github.com/coreos/rpm-ostree/issues/2668
2021-04-28 16:45:50 -04:00
Jonathan Lebon
7b568ebef9 tests/compose: Adapt chrony autovar for newer FCOS 2021-04-27 22:10:36 -04:00
Jonathan Lebon
502088e7d3 tests/compose: Change rpmdb test to check for sqlite
That's what FCOS uses now in the latest bump.
2021-04-27 21:06:50 -04:00
Colin Walters
68392638ea tests/compose: Bump pinned fcos commit
Failing to find a locked package, probably pool GC.
2021-04-27 16:34:12 -04:00
Jonathan Lebon
bb079a8f3c
Merge pull request #2713 from Razaloc/kargs1
kargs: add CLI switches --append-if-missing and --delete-if-present
2021-04-23 10:44:35 -04:00
Rafael G. Ruiz
edd13e35cd kargs: add CLI switches --append-if-missing and --delete-if-present
These new switches will allow users/drivers to more easily make idempotent changes to kernel arguments.

Closes: https://github.com/coreos/rpm-ostree/issues/2709

Signed-off-by: Rafael G. Ruiz <llerrak@hotmail.com>
2021-04-22 17:17:49 +02:00
Luca BRUNO
a1573ed8c7
tests/compose: check symlinks injected into the rootfs
This adds some integration tests, covering symlinks injection logic.
2021-04-21 07:18:23 +00:00
Luca BRUNO
1ec3b7fd55 tests/compose: add more autovar testcases
This adds some testcases in order to cover more codepaths in the
translation logic.
2021-04-19 07:39:58 -04:00
Colin Walters
db227840f1 daemon: Always write staged JSON field
It's an important property, and it's more self-documenting if we
always emit the value rather than only when it's `true`.
2021-04-16 17:05:48 -04:00
dependabot[bot]
9a7ffb2b0c build(deps): bump libdnf from 31a8b8b to 930f258
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `31a8b8b` to `930f258`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](31a8b8bcec...930f2582f9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 11:52:49 -04:00
Colin Walters
1541ed4312 tests: Stop running test-lib-introspection.sh
Even more fallout of the buildroot change to stop deriving from
cosa.  The new buildroot doesn't have `pygobject3`.

We could convert this to an installed test but...blah.
Not worth it.
2021-04-07 22:11:56 +00:00
Jonathan Lebon
4624c3bdf8 app/db: Add db diff --advisories
This will be used by at least coreos-assembler to store in `meta.json`
so it's easy to tell new advisories that appeared in a given build.
2021-04-06 17:04:51 -04:00
Jonathan Lebon
136a3ea7e9 app/db: Add db list --advisories
This makes it easy to see the advisories stored in commit metadata.
2021-04-06 17:04:51 -04:00
Colin Walters
fa81456cbf Rewrite bwrap code in Rust
I tried to do this incrementally but it snowballed.
2021-03-29 14:45:35 +00:00
Colin Walters
335415d601 tests: Add infinite loop package to kolainst-build
This way it's easy to get outside of the vmcheck tests.
2021-03-26 16:59:30 +00:00
Jonathan Lebon
0fd82990c7
Merge pull request #2691 from cgwalters/strip-remote-error
clientlib: Strip GDBus remote error before throwing exception
2021-03-23 10:49:45 -04:00
Colin Walters
7ec7e3eb28 clientlib: Strip GDBus remote error before throwing exception
In general our error handling philosophy is "errors are strings".
Previously (before C++) we were relying on calling this API
to strip out the GDBus remote error message.

In the recent refactoring of clientlib to expose the txn APIs
to Rust via C++, we implicitly lost that because the C++
wrapper API throws an exception.

Add a copy of the remote error stripping here.  Eventually
it will likely make sense for it to *only* be here instead
of also in `main.cxx`, but one thing at a time.
2021-03-22 19:30:49 +00:00
Jonathan Lebon
7979e250ee lockfile: Allow locking by source RPM EVR
Right now if we want to lock e.g. systemd, we need to specify every
subpackage of systemd that we use. This is a lot of duplication because
in the majority of cases, what we really mean is "lock at this build of
systemd".

Since RPMs bake in the source RPM they were built from, we can use this
to lock packages more succinctly. See the testcase and #2676 for
examples of how this looks.

Closes: https://github.com/coreos/rpm-ostree/issues/2676
2021-03-22 12:57:49 -04:00
Colin Walters
7f9129d67d compose: Move mutate-os-release string code to Rust
More classic C string manipulation which is much nicer in Rust
2021-03-18 18:05:34 +00:00
Colin Walters
aa64df080d daemon: Fix serialization of unlocked state
It turns out there's a naming clash between `to_string()` here in
Rust *and* introspection is incorrectly associating the method
with `ostree::Deployment` because of the naming prefix.
2021-03-12 18:01:47 -05:00
Colin Walters
f6f6ac5ff3 ci: Add composepost-checks.sh, drop a compose test
The compose tests are expensive; each run involves running
all the `%post` scripts and `dracut` etc.  This is definitely
a source of timeouts in CCI.

Remove `test-boot-location-modules.sh` - it's the default
now and is used by FCOS.  Add dedicated script where we can
test all these things by default after a `cosa build`.

This aims to move the compose tests to only cover bits *not*
in cosa like the non-unified-core path.
2021-03-12 15:53:01 -05:00
Jonathan Lebon
46f556f437
Merge pull request #2658 from cgwalters/live-no-restart
live: Don't output "services restart" for pure package additions
2021-03-12 09:47:45 -05:00
Colin Walters
2187ef0f45 live: Don't output "services restart" for pure package additions
If all we're doing is layering new packages, no need to tell
the admin that things may need restarting.
2021-03-11 22:50:07 +00:00
Colin Walters
445af087d6 tests: Drain vmcheck/test-misc-1 into kola/misc.sh
Where I stalled out before is this file has `pkg-add foo`, but
now that we have the `foo` package pre-built we can move all
this stuff into `misc.sh`.

I dropped the YAML parsing of `--version` because we don't
have python.  This is related to
https://github.com/coreos/coreos-assembler/issues/1645
2021-03-11 16:34:07 -05:00
Jonathan Lebon
2fedc3b0ac lockfile: Allow specifying EVR rather than EVRA
In FCOS, we use "override" lockfiles to pin packages to certain
versions. Right now, we have separate overrides for each base arch we
(eventually want to) support. But that makes maintaining the overrides
cumbersome because of all the duplication.

Let's allow lockfiles to specify only the `evr` of a package, which is
just as good for FCOS, and means that we'll only have to maintain a
single override file for all the architectures.
2021-03-11 00:01:46 +01:00
Colin Walters
1dc7503838 ci/prow: Install in target cosa, also install and run our tests
This is a major downside of reworking and generating new CI
flows, it's super easy to lose testing what you intend to.

Also, we clearly need to figure out a flow where this is shared
across repos, since I don't want to copy-paste this into e.g. ostree too.
That's https://github.com/coreos/fedora-coreos-tracker/issues/263
2021-03-10 16:58:49 +01:00
Colin Walters
3041d648bb install: Add --apply-live
OK I think it's time.  This exposes the `apply-live` functionality
as implicitly stable, but specific to the package install case.

I'd like to add more intelligence to `apply-live` around separating
pure "additions" (as in this case) versus package (file) changes.

The change here doesn't try to do that; the implementation is
incredibly simple, we just have the client chain together the two
distinct transactions.
2021-03-08 20:54:18 +01:00
Colin Walters
485c1862ce live: Disallow replacement/removal by default, add --allow-replacement
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).

The latter is really quite safe, and live layering is one
of the most requested features.
2021-03-08 20:54:18 +01:00
Colin Walters
2a9423ccb8 tests: Port apply-live to kola ext tests
Continuing the momentum to use kola ext tests.

One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
2021-03-08 20:54:18 +01:00
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Colin Walters
6fd9db246a Add basic Rust CLI bindings for DBus
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.

A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.

My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
2021-03-04 09:17:21 -05:00
Colin Walters
8dd5583ae9 Remove cbindgen
We have fully transitioned to cxx-rs!  This drops a lot of now
dead code; only one binding system to think about generating
source code.  For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
2021-03-03 16:51:38 -05:00
Colin Walters
85f22baec7 tests: Add RPMs to installed kola tests, port layering-local
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.

We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over.  Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data.  Add the concept
of multiple repo versions too.

Right now we only generate the one RPM needed for the `layering-local`
test and port it.
2021-03-02 09:47:03 -08:00
Colin Walters
7367aa8469 tests/apply-live: Upgrade+liveapply no-package changes
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction".  I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
2021-03-01 20:40:48 +01:00
Colin Walters
4d7de4fa59 tests/vmcheck: Use .cosa over COSA_DIR
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.

Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
2021-03-01 18:53:56 +01:00
Colin Walters
732e0f232a daemon: Fix crash on upgrade with remote:checksum
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).

The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.

But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.

I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.

Closes: https://github.com/coreos/rpm-ostree/issues/2603
2021-03-01 16:57:51 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
0544d1c92d ci: Drop tests/vmcheck/image.qcow2, use COSA_DIR/.cosa
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default.  We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.

Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
2021-02-23 17:23:26 -05:00
Kelvin Fan
626d021edb app/upgrade: Do not upgrade if updates driver registered
Do not perform an upgrade if detected that an updates driver has
been registered.

Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
2021-02-17 11:43:34 -05:00
Colin Walters
e6c045cada Add an rpmostree-client sub-crate
This is intended to be published to https://crates.io/crates/rpmostree-client
Part of https://github.com/coreos/rpm-ostree/issues/2389

This directly imports the code from
5551c54c6e/tests/inst/src/rpmostree.rs

Once merged and released I'll try converting the ostree test suite
over as well as Zincati.

Internally add a testutils helper to validate it works.
2021-02-16 19:22:26 -05:00
Colin Walters
7bf885ecf7 tests: Fix bodhi link
From review in https://github.com/coreos/rpm-ostree/pull/2585#discussion_r577149232
2021-02-16 18:18:27 -05:00
Colin Walters
c1ff933bc3 tests/layering-non-root-caps: Adapt to new libcap output
I didn't deep dive on this, just observed that the new output
matches what we laid down.  There's ~2.5 years of changes
and ~200 commits between 2.28 and 2.48.
2021-02-16 17:08:27 -05:00
Kelvin Fan
84065c48df tests/common/libvm: Print error if bad tests/vmcheck/image.qcow2
Print error and exit if cannot find `tests/vmcheck/image.qcow2` or
bad symlink causing failure to spawn a VM when calling the
`vm_kola_spawn()` function.
2021-02-11 21:11:37 -05:00
Jonathan Lebon
99486a75e8 Add /usr/lib/rpm/macros.d/macros.rpm-ostree to set %_dbpath to /usr/share/rpm
We trigger a librpm macro file load in many of our paths. Since the
default value shipped by rpm's macro file sets `_dbpath` to
`/var/lib/rpm`, we have to explicitly set that back to `/usr/share/rpm`
in those paths.

This became more problematic recently with libsolv v0.7.17 which fully
keys off of `_dbpath` to find the rpmdb path to load:

04d4d036b2

And it's not technically wrong; we really should make that macro not
lie. This is what this patch does by injecting an RPM macro file in our
composes which sets it to /usr/share/rpm. So then e.g. the `rpm` CLI
doesn't actually need the `/var/lib/rpm` backcompat link anymore, though
there's no harm in leaving it.

In the future, we should be able to drop this once we move all of Fedora
to `/usr/lib/sysimage/rpm` (see
https://github.com/coreos/fedora-coreos-tracker/issues/639).

Closes: #2548
2021-02-09 18:36:35 -05:00
Jonathan Lebon
57edf7e638 tests/compose.sh: Always rebuild supermin appliance
We always want the latest rpm-ostree binaries tested, so we need to
always rerun supermin.

Patch better viewed with whitespace ignored.
2021-02-03 12:22:38 -05:00