Commit Graph

381 Commits

Author SHA1 Message Date
Colin Walters
562e03f7c1 Remove large chunks of rojig code
The inevitable followup to https://github.com/coreos/rpm-ostree/pull/2278
that I was too cowardly to do at the time.  But it's time to admit
the 2 months or so of work on this was wasted.  We have too much
tech debt and this is a large chunk of C/C++ code that touches everything
in the codebase in a nontrivial way.

Bigger picture, I'm going to work on
https://github.com/coreos/fedora-coreos-tracker/issues/828
which will strongly orient rpm-ostree towards the container world instead.
We'll still obviously keep the rpm package world around, but only
as a secondary layer.  What rojig was trying to do in putting "images"
inside an RPM was conflating layers.  It would have had a lot of
benefits probably if we'd truly pushed it over the edge into completion,
but that didn't happen.  Let's focus on containers instead.

There's still a lot more rojig code to delete but this first patch removes
the bulk of it.  Touching everything that references e.g. `RPMOSTREE_REFSPEC_TYPE_ROJIG`
etc. can come as a 3rd phase.
2021-05-18 17:31:36 -04:00
Jonathan Lebon
195807870a
Merge pull request #2815 from travier/rename
*: rename master branch to main
2021-05-11 12:03:08 -04:00
Jonathan Lebon
5d79fbf375
Merge pull request #2819 from jlebon/pr/skip-branch-validation
app/deploy: Add --skip-branch-check
2021-05-11 09:18:20 -04:00
Jonathan Lebon
7c1072f965 app/deploy: Add --skip-branch-check
In Fedora CoreOS, updates are driven by Zincati and we thus completely
trust the information it gives us. The branch validation rpm-ostree does
is thus not necessary. It's also harmful in the case where the node is
extremely out of date because it may not be able to GPG verify the
commit at the tip of the branch (because the GPG key isn't yet in the
tree).

See: https://github.com/coreos/fedora-coreos-tracker/issues/749
2021-05-07 17:45:09 -04:00
Jonathan Lebon
57250d11ab
Merge pull request #2813 from kelvinfan001/pr/detect-inactive-driver
app/clientlib: Don't error if updates driver inactive
2021-05-07 17:12:42 -04:00
Kelvin Fan
86cf6ad479 app/clientlib: Don't error if update driver "stopped"
If updates driver is registered but "stopped", ignore driver (i.e.
do not error out during deploy, rebase, ugprade).

We define "stopped" to mean the driver's `ActiveState` is not in
the following states: "active", "activating", "reloading", "failed".
2021-05-07 18:41:45 +00:00
Timothée Ravier
95de194f59 *: rename master branch to main 2021-05-07 16:12:45 +02:00
Rafael G. Ruiz
6246b990a9 kargs: Add CLI switch --unchanged-exit-77
The new switch will rise an exit code 77 if kernel args are unchanged

Closes: #1802

Signed-off-by: Rafael G. Ruiz <llerrak@hotmail.com>
2021-05-05 23:19:52 +02:00
Jonathan Lebon
7f19930916 tests/vmcheck/misc-2: Update error message to grep for
For some reason, in f34 the error that libostree gets now when trying to
checkout a file on top of an existing one with different content is no
longer EEXIST but EPERM.

Likely due to a change in the kernel or glibc though some quick googling
doesn't reveal anything.
2021-05-04 15:58:14 -04:00
Luca BRUNO
7ece1846c0
tests/override-kernel: add f34 kernel
This adds support for Fedora 34.
2021-05-04 14:58:12 +00:00
Dusty Mabe
9e45eed598 lib/package: Fix rpm version comparison
`rpmvercmp()` doesn't properly compare full EVR. It needs to
be given the Version and Release separately in order to do
a comparison. For example `rpmVersionCompare()` first calls
`rpmvercmp()` to compare the Version and then checks the Release
in a second call to `rpmvercmp()`.

35739c2a22/lib/headerutil.c (L434-L440)

Let's just use `rpmverCmp()` instead, which can do the full
EVR comparison.

Fixes: https://github.com/coreos/rpm-ostree/issues/2668
2021-04-28 16:45:50 -04:00
Rafael G. Ruiz
edd13e35cd kargs: add CLI switches --append-if-missing and --delete-if-present
These new switches will allow users/drivers to more easily make idempotent changes to kernel arguments.

Closes: https://github.com/coreos/rpm-ostree/issues/2709

Signed-off-by: Rafael G. Ruiz <llerrak@hotmail.com>
2021-04-22 17:17:49 +02:00
dependabot[bot]
9a7ffb2b0c build(deps): bump libdnf from 31a8b8b to 930f258
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `31a8b8b` to `930f258`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](31a8b8bcec...930f2582f9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 11:52:49 -04:00
Colin Walters
fa81456cbf Rewrite bwrap code in Rust
I tried to do this incrementally but it snowballed.
2021-03-29 14:45:35 +00:00
Colin Walters
445af087d6 tests: Drain vmcheck/test-misc-1 into kola/misc.sh
Where I stalled out before is this file has `pkg-add foo`, but
now that we have the `foo` package pre-built we can move all
this stuff into `misc.sh`.

I dropped the YAML parsing of `--version` because we don't
have python.  This is related to
https://github.com/coreos/coreos-assembler/issues/1645
2021-03-11 16:34:07 -05:00
Colin Walters
485c1862ce live: Disallow replacement/removal by default, add --allow-replacement
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).

The latter is really quite safe, and live layering is one
of the most requested features.
2021-03-08 20:54:18 +01:00
Colin Walters
2a9423ccb8 tests: Port apply-live to kola ext tests
Continuing the momentum to use kola ext tests.

One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
2021-03-08 20:54:18 +01:00
Colin Walters
85f22baec7 tests: Add RPMs to installed kola tests, port layering-local
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.

We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over.  Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data.  Add the concept
of multiple repo versions too.

Right now we only generate the one RPM needed for the `layering-local`
test and port it.
2021-03-02 09:47:03 -08:00
Colin Walters
7367aa8469 tests/apply-live: Upgrade+liveapply no-package changes
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction".  I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
2021-03-01 20:40:48 +01:00
Colin Walters
732e0f232a daemon: Fix crash on upgrade with remote:checksum
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).

The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.

But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.

I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.

Closes: https://github.com/coreos/rpm-ostree/issues/2603
2021-03-01 16:57:51 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Kelvin Fan
626d021edb app/upgrade: Do not upgrade if updates driver registered
Do not perform an upgrade if detected that an updates driver has
been registered.

Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
2021-02-17 11:43:34 -05:00
Colin Walters
7bf885ecf7 tests: Fix bodhi link
From review in https://github.com/coreos/rpm-ostree/pull/2585#discussion_r577149232
2021-02-16 18:18:27 -05:00
Colin Walters
c1ff933bc3 tests/layering-non-root-caps: Adapt to new libcap output
I didn't deep dive on this, just observed that the new output
matches what we laid down.  There's ~2.5 years of changes
and ~200 commits between 2.28 and 2.48.
2021-02-16 17:08:27 -05:00
Kelvin Fan
4dd80a8b9c app/status: Use AutomaticUpdatesDriver if updates driven by external driver
When automatic updates are driven by an external driver like Zincati,
display e.g. `AutomaticUpdatesDriver: Zincati` instead of
`AutomaticUpdates: driven by Zincati`, since the latter might suggest it
is rpm-ostree's own built-in support for automatic updates.
2021-02-03 09:52:43 -05:00
Kelvin Fan
1e79e5abde app/status: Show update driver's state and status
Display update driver's systemd unit's `ActiveState` property.
Also display `StatusText` property if unit is a service unit and
property is available.
2021-02-03 09:52:43 -05:00
Kelvin Fan
34746d0428 deploy: Allow empty string argument if --register-driver
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
2021-01-26 01:32:52 +01:00
Kelvin Fan
e7558c3dc9 app/status: Display update driver info
Read from `/run/rpm-ostree/update-driver.gv` and display the update
driver name (and systemd unit if verbose).
2021-01-24 06:28:08 +01:00
Kelvin Fan
7586503ee2 vmcheck/test-misc-2: Fix $cursor variable
Minor cleanup.

Follow up from https://github.com/coreos/rpm-ostree/pull/2461/.
2021-01-20 15:20:41 -05:00
Kelvin Fan
3f9fe06d24 daemon: Record agent's systemd service
In https://github.com/coreos/rpm-ostree/pull/2395/, agent's id is
recorded in the journal. Similarly, record the systemd service that
called the client, as well.

Related to https://github.com/coreos/rpm-ostree/issues/1747.
2021-01-20 10:13:02 -05:00
Colin Walters
2f82733e65 apply-live: Extend /etc test case
- Further nested sub/sub directories
- symbolic links that point to various corner cases like `/`, are
  broken, to parents etc.
2021-01-18 05:51:05 -05:00
Colin Walters
d0c6871d80 apply-live: Avoid clobbering changes in /etc
Gather the current diff of `/etc`, and filter out changes in
the tree which would overwrite it.

There is an OSTree API for diffs but it's a bit awkward, missing
some APIs in the Rust bindings and also `GFile` based unfortunately.
Doing this in Rust is nicer.  The dirdiff code obviously needs
a lot more testing, but I think it's right.
2021-01-15 05:41:53 -05:00
Colin Walters
6c66bf1072 Rename livefs → apply-live in more places
Keep up the renaming momentum by using the new name in more
places.

Prep for further work.
2021-01-12 03:15:49 -05:00
Colin Walters
485dbe8472 apply-live: Print a package diff
The cool thing about this is it emphasizes how "integrated" apply-live
is versus the uncontrolled `rpm-ostree usroverlay`.  We're still
tracking the state of things reliably and can print it.
2020-12-23 16:23:43 +01:00
Colin Walters
9d927e9850 daemon: Record the agent ID in the journal
Related to https://github.com/coreos/rpm-ostree/issues/1747

Basically after this we can have `rpm-ostree status` output
e.g. `Agent: zincati` at least *after* an upgrade has happened.
2020-12-16 13:55:11 +00:00
Colin Walters
d9e9e5cf9b Rename ex livefs to ex apply-live
(Keeping the old name for muscle memory compatibility for now)

I think `apply-live` is a clearer name; it's more imperative
and it may not be obvious (particularly to non-native English speakers)
to parse "livefs" as "live fs".
2020-12-14 11:55:09 -05:00
Colin Walters
cd2307ccc2 tests: Add case for running rpm -q in a %post
On traditional rpm systems this can hang because the outer
process may have an rpmdb lock, and the inner one wants
to acquire a lock.  Here we're sandboxing the `%post` script
and it's targeting a separate temporary filesystem compared to
the booted one (so there's no double locking).  Plus we don't
create the rpmdb in the target until all scripts have run.

Inspired by https://twitter.com/_msw_/status/1335981558717587473
2020-12-07 15:09:44 -05:00
Luca BRUNO
1b00f13d34 tests/override-kernel: support f33 2020-12-07 07:58:18 -05:00
Colin Walters
956d96bd1f livefs: Fix --target, add --reset and print diff
Makes this more useful.
2020-11-17 22:34:44 +01:00
Colin Walters
a76ddf0cef Rewrite livefs
Now always based on an overlayfs:
f2773c1b55
This fixes a whole swath of problems with the previous design,
including the danger in replacing `/usr/lib/ostree-boot` which
broke booting for some people.

Further, we don't need to push a rollback deployment; the livefs
changes are always transient.  So now we store livefs state
in `/run` instead of in the origin file.

Since we're doing a rewrite, it's now in Rust for much more safety.

We also always work in terms of incremental diffs between commits;
the previous huge hammer of swapping `/usr` was way too dangerous.
2020-11-16 19:07:16 +01:00
Colin Walters
5650f376fb Add --enable-rojig, disable by default
We're seeing some CI failures that I think are a bug in rojig.
In the bigger picture...we never actually started using this,
and I think longer term shipping os updates via containers
probably makes more sense.

I put a *lot* of effort into this code and it's pretty cool
so it's hard to just delete it.  And *maybe* someone out there
is using it (but I doubt it).  So rather than just deleting
it entirely let's make it a build-time option.

I verified that it builds at least.
2020-10-22 15:38:50 -04:00
Jonathan Lebon
71992e3d11 core: Use SOLVER_LOCK for locking base packages
For the Fedora CoreOS extensions work, when layering packages, we need
to be able to tell libsolv to pick the packages which will go with the
base packages. IOW, it needs to know that the base packages shouldn't be
uninstalled.

While investigating
https://github.com/coreos/fedora-coreos-tracker/issues/525, I realized
that libsolv does have a flag which allows us to express this:
`SOLVER_LOCK`.

This then allows libsolv to choose the right package for us (if found).
And in the case where it can't find a matching package, libsolv itself
will print exactly what the conflict is, which is more informative than
the "forbidden replacements" error we currently print out.

Update submodule: libdnf
2020-08-28 12:44:46 -04:00
Jonathan Lebon
2c6d24de0a vmcheck/rojig: Don't use ci.centos.org
It doesn't resolve to the right node from inside the CentOS CI:
https://lists.centos.org/pipermail/ci-users/2016-July/000301.html
2020-07-16 15:46:06 -04:00
Jonathan Lebon
f608eb09ed app: Add --lock-finalization switch to other commands
Both `upgrade` and `deploy` already support this. There's no reason why
all the remaining "deployment-creating" commands shouldn't. Prompted by
https://github.com/openshift/machine-config-operator/issues/1897 which
will need this specifically for `rebase`.
2020-07-08 16:02:59 +02:00
Colin Walters
1db9fa000f rebase: Remove requirement for --experimental with local rebases
We've been shipping the MCO code using this for a long time,
it needs to stay in its current form and is not experimental.
2020-05-29 15:14:32 -04:00
Colin Walters
37e7ab2e33 test-override-kernel: Support f32
CI is failing on this now that FCOS has switched.
2020-05-29 12:35:15 -04:00
Colin Walters
6df03da742 tests: Move more code to kola
Continuing the migration.
2020-05-12 00:34:26 +02:00
Colin Walters
22bf449296 tests: Drain more readonly tests into nondestructive/misc
Part of migrating the test suite to kola.
2020-04-30 21:50:41 +02:00