Commit Graph

725 Commits

Author SHA1 Message Date
Colin Walters
562e03f7c1 Remove large chunks of rojig code
The inevitable followup to https://github.com/coreos/rpm-ostree/pull/2278
that I was too cowardly to do at the time.  But it's time to admit
the 2 months or so of work on this was wasted.  We have too much
tech debt and this is a large chunk of C/C++ code that touches everything
in the codebase in a nontrivial way.

Bigger picture, I'm going to work on
https://github.com/coreos/fedora-coreos-tracker/issues/828
which will strongly orient rpm-ostree towards the container world instead.
We'll still obviously keep the rpm package world around, but only
as a secondary layer.  What rojig was trying to do in putting "images"
inside an RPM was conflating layers.  It would have had a lot of
benefits probably if we'd truly pushed it over the edge into completion,
but that didn't happen.  Let's focus on containers instead.

There's still a lot more rojig code to delete but this first patch removes
the bulk of it.  Touching everything that references e.g. `RPMOSTREE_REFSPEC_TYPE_ROJIG`
etc. can come as a 3rd phase.
2021-05-18 17:31:36 -04:00
Colin Walters
54a011df40 Update to ostree-ext v0.1.2, add new ex-container command
This new `rpm-ostree ex-container` CLI is just code copied
from the `ostree-ext-cli container` binary code.  In the future
I may just add the CLI code as a library API too to simplify this.

For now, I don't want to try to add a new Rust CLI as an RPM
package for example.  This exposes it via rpm-ostree, and
in the future rpm-ostree may have some layering on top of this
anyways.
2021-05-18 09:51:35 -04:00
Colin Walters
a6da3c08d4 passwd: Make default perms 0644
This matches the default from the RPM, and we don't want
the file to be writable if an admin adds a user to the `root`
group.  (Which IMO is just a bad idea, but
for historical reasons OpenShift suggests this for
images
https://docs.openshift.com/container-platform/4.7/openshift_images/create-images.html#images-create-guide-openshift_create-images
for example)
2021-05-13 17:49:07 -04:00
Timothée Ravier
aed0f194ad *: rename master branch to main (external repos) 2021-05-11 18:11:30 +02:00
Jonathan Lebon
195807870a
Merge pull request #2815 from travier/rename
*: rename master branch to main
2021-05-11 12:03:08 -04:00
Jonathan Lebon
5d79fbf375
Merge pull request #2819 from jlebon/pr/skip-branch-validation
app/deploy: Add --skip-branch-check
2021-05-11 09:18:20 -04:00
Jonathan Lebon
7c1072f965 app/deploy: Add --skip-branch-check
In Fedora CoreOS, updates are driven by Zincati and we thus completely
trust the information it gives us. The branch validation rpm-ostree does
is thus not necessary. It's also harmful in the case where the node is
extremely out of date because it may not be able to GPG verify the
commit at the tip of the branch (because the GPG key isn't yet in the
tree).

See: https://github.com/coreos/fedora-coreos-tracker/issues/749
2021-05-07 17:45:09 -04:00
Jonathan Lebon
57250d11ab
Merge pull request #2813 from kelvinfan001/pr/detect-inactive-driver
app/clientlib: Don't error if updates driver inactive
2021-05-07 17:12:42 -04:00
Kelvin Fan
86cf6ad479 app/clientlib: Don't error if update driver "stopped"
If updates driver is registered but "stopped", ignore driver (i.e.
do not error out during deploy, rebase, ugprade).

We define "stopped" to mean the driver's `ActiveState` is not in
the following states: "active", "activating", "reloading", "failed".
2021-05-07 18:41:45 +00:00
Timothée Ravier
95de194f59 *: rename master branch to main 2021-05-07 16:12:45 +02:00
Rafael G. Ruiz
6246b990a9 kargs: Add CLI switch --unchanged-exit-77
The new switch will rise an exit code 77 if kernel args are unchanged

Closes: #1802

Signed-off-by: Rafael G. Ruiz <llerrak@hotmail.com>
2021-05-05 23:19:52 +02:00
Jonathan Lebon
7f19930916 tests/vmcheck/misc-2: Update error message to grep for
For some reason, in f34 the error that libostree gets now when trying to
checkout a file on top of an existing one with different content is no
longer EEXIST but EPERM.

Likely due to a change in the kernel or glibc though some quick googling
doesn't reveal anything.
2021-05-04 15:58:14 -04:00
Luca BRUNO
7ece1846c0
tests/override-kernel: add f34 kernel
This adds support for Fedora 34.
2021-05-04 14:58:12 +00:00
Jonathan Lebon
ba7a72995b treefile: Add new repo-packages field for pinning packages to repos
This addresses the server compose side of
https://github.com/coreos/rpm-ostree/issues/2584.

One tricky bit is handling overrides across included treefiles (or
really, even within a single treefile): as usual, higher-level treefiles
should override lowel-level ones. Rust makes it pretty nice to handle.

For now this just supports a `repo` field, but one could imagine e.g.
`repos` (which takes an array of repoids instead), or e.g.
`exclude-repos`.

The actual core implementation otherwise is pretty straightforward.

This should help a lot in RHCOS where we currently use many `exclude=`
directives in repo files to get it to do what we want.

This is also kind of a requirement for modularity support because as
soon as rpm-ostree becomes modules-aware, modular filtering logic will
break composes which assume rpm-ostree treats modular and non-modular
packages the same.
2021-04-30 09:13:18 -04:00
Jonathan Lebon
9348e44a55
Merge pull request #2782 from dustymabe/dusty-rpm-vercmp
lib/package: Fix rpm version comparison
2021-04-28 17:59:24 -04:00
Dusty Mabe
9e45eed598 lib/package: Fix rpm version comparison
`rpmvercmp()` doesn't properly compare full EVR. It needs to
be given the Version and Release separately in order to do
a comparison. For example `rpmVersionCompare()` first calls
`rpmvercmp()` to compare the Version and then checks the Release
in a second call to `rpmvercmp()`.

35739c2a22/lib/headerutil.c (L434-L440)

Let's just use `rpmverCmp()` instead, which can do the full
EVR comparison.

Fixes: https://github.com/coreos/rpm-ostree/issues/2668
2021-04-28 16:45:50 -04:00
Jonathan Lebon
7b568ebef9 tests/compose: Adapt chrony autovar for newer FCOS 2021-04-27 22:10:36 -04:00
Jonathan Lebon
502088e7d3 tests/compose: Change rpmdb test to check for sqlite
That's what FCOS uses now in the latest bump.
2021-04-27 21:06:50 -04:00
Colin Walters
68392638ea tests/compose: Bump pinned fcos commit
Failing to find a locked package, probably pool GC.
2021-04-27 16:34:12 -04:00
Jonathan Lebon
bb079a8f3c
Merge pull request #2713 from Razaloc/kargs1
kargs: add CLI switches --append-if-missing and --delete-if-present
2021-04-23 10:44:35 -04:00
Rafael G. Ruiz
edd13e35cd kargs: add CLI switches --append-if-missing and --delete-if-present
These new switches will allow users/drivers to more easily make idempotent changes to kernel arguments.

Closes: https://github.com/coreos/rpm-ostree/issues/2709

Signed-off-by: Rafael G. Ruiz <llerrak@hotmail.com>
2021-04-22 17:17:49 +02:00
Luca BRUNO
a1573ed8c7
tests/compose: check symlinks injected into the rootfs
This adds some integration tests, covering symlinks injection logic.
2021-04-21 07:18:23 +00:00
Luca BRUNO
1ec3b7fd55 tests/compose: add more autovar testcases
This adds some testcases in order to cover more codepaths in the
translation logic.
2021-04-19 07:39:58 -04:00
Colin Walters
db227840f1 daemon: Always write staged JSON field
It's an important property, and it's more self-documenting if we
always emit the value rather than only when it's `true`.
2021-04-16 17:05:48 -04:00
dependabot[bot]
9a7ffb2b0c build(deps): bump libdnf from 31a8b8b to 930f258
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `31a8b8b` to `930f258`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](31a8b8bcec...930f2582f9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 11:52:49 -04:00
Colin Walters
1541ed4312 tests: Stop running test-lib-introspection.sh
Even more fallout of the buildroot change to stop deriving from
cosa.  The new buildroot doesn't have `pygobject3`.

We could convert this to an installed test but...blah.
Not worth it.
2021-04-07 22:11:56 +00:00
Jonathan Lebon
4624c3bdf8 app/db: Add db diff --advisories
This will be used by at least coreos-assembler to store in `meta.json`
so it's easy to tell new advisories that appeared in a given build.
2021-04-06 17:04:51 -04:00
Jonathan Lebon
136a3ea7e9 app/db: Add db list --advisories
This makes it easy to see the advisories stored in commit metadata.
2021-04-06 17:04:51 -04:00
Colin Walters
fa81456cbf Rewrite bwrap code in Rust
I tried to do this incrementally but it snowballed.
2021-03-29 14:45:35 +00:00
Colin Walters
335415d601 tests: Add infinite loop package to kolainst-build
This way it's easy to get outside of the vmcheck tests.
2021-03-26 16:59:30 +00:00
Jonathan Lebon
0fd82990c7
Merge pull request #2691 from cgwalters/strip-remote-error
clientlib: Strip GDBus remote error before throwing exception
2021-03-23 10:49:45 -04:00
Colin Walters
7ec7e3eb28 clientlib: Strip GDBus remote error before throwing exception
In general our error handling philosophy is "errors are strings".
Previously (before C++) we were relying on calling this API
to strip out the GDBus remote error message.

In the recent refactoring of clientlib to expose the txn APIs
to Rust via C++, we implicitly lost that because the C++
wrapper API throws an exception.

Add a copy of the remote error stripping here.  Eventually
it will likely make sense for it to *only* be here instead
of also in `main.cxx`, but one thing at a time.
2021-03-22 19:30:49 +00:00
Jonathan Lebon
7979e250ee lockfile: Allow locking by source RPM EVR
Right now if we want to lock e.g. systemd, we need to specify every
subpackage of systemd that we use. This is a lot of duplication because
in the majority of cases, what we really mean is "lock at this build of
systemd".

Since RPMs bake in the source RPM they were built from, we can use this
to lock packages more succinctly. See the testcase and #2676 for
examples of how this looks.

Closes: https://github.com/coreos/rpm-ostree/issues/2676
2021-03-22 12:57:49 -04:00
Colin Walters
7f9129d67d compose: Move mutate-os-release string code to Rust
More classic C string manipulation which is much nicer in Rust
2021-03-18 18:05:34 +00:00
Colin Walters
aa64df080d daemon: Fix serialization of unlocked state
It turns out there's a naming clash between `to_string()` here in
Rust *and* introspection is incorrectly associating the method
with `ostree::Deployment` because of the naming prefix.
2021-03-12 18:01:47 -05:00
Colin Walters
f6f6ac5ff3 ci: Add composepost-checks.sh, drop a compose test
The compose tests are expensive; each run involves running
all the `%post` scripts and `dracut` etc.  This is definitely
a source of timeouts in CCI.

Remove `test-boot-location-modules.sh` - it's the default
now and is used by FCOS.  Add dedicated script where we can
test all these things by default after a `cosa build`.

This aims to move the compose tests to only cover bits *not*
in cosa like the non-unified-core path.
2021-03-12 15:53:01 -05:00
Jonathan Lebon
46f556f437
Merge pull request #2658 from cgwalters/live-no-restart
live: Don't output "services restart" for pure package additions
2021-03-12 09:47:45 -05:00
Colin Walters
2187ef0f45 live: Don't output "services restart" for pure package additions
If all we're doing is layering new packages, no need to tell
the admin that things may need restarting.
2021-03-11 22:50:07 +00:00
Colin Walters
445af087d6 tests: Drain vmcheck/test-misc-1 into kola/misc.sh
Where I stalled out before is this file has `pkg-add foo`, but
now that we have the `foo` package pre-built we can move all
this stuff into `misc.sh`.

I dropped the YAML parsing of `--version` because we don't
have python.  This is related to
https://github.com/coreos/coreos-assembler/issues/1645
2021-03-11 16:34:07 -05:00
Jonathan Lebon
2fedc3b0ac lockfile: Allow specifying EVR rather than EVRA
In FCOS, we use "override" lockfiles to pin packages to certain
versions. Right now, we have separate overrides for each base arch we
(eventually want to) support. But that makes maintaining the overrides
cumbersome because of all the duplication.

Let's allow lockfiles to specify only the `evr` of a package, which is
just as good for FCOS, and means that we'll only have to maintain a
single override file for all the architectures.
2021-03-11 00:01:46 +01:00
Colin Walters
1dc7503838 ci/prow: Install in target cosa, also install and run our tests
This is a major downside of reworking and generating new CI
flows, it's super easy to lose testing what you intend to.

Also, we clearly need to figure out a flow where this is shared
across repos, since I don't want to copy-paste this into e.g. ostree too.
That's https://github.com/coreos/fedora-coreos-tracker/issues/263
2021-03-10 16:58:49 +01:00
Colin Walters
3041d648bb install: Add --apply-live
OK I think it's time.  This exposes the `apply-live` functionality
as implicitly stable, but specific to the package install case.

I'd like to add more intelligence to `apply-live` around separating
pure "additions" (as in this case) versus package (file) changes.

The change here doesn't try to do that; the implementation is
incredibly simple, we just have the client chain together the two
distinct transactions.
2021-03-08 20:54:18 +01:00
Colin Walters
485c1862ce live: Disallow replacement/removal by default, add --allow-replacement
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).

The latter is really quite safe, and live layering is one
of the most requested features.
2021-03-08 20:54:18 +01:00
Colin Walters
2a9423ccb8 tests: Port apply-live to kola ext tests
Continuing the momentum to use kola ext tests.

One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
2021-03-08 20:54:18 +01:00
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Colin Walters
6fd9db246a Add basic Rust CLI bindings for DBus
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.

A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.

My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
2021-03-04 09:17:21 -05:00
Colin Walters
8dd5583ae9 Remove cbindgen
We have fully transitioned to cxx-rs!  This drops a lot of now
dead code; only one binding system to think about generating
source code.  For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
2021-03-03 16:51:38 -05:00
Colin Walters
85f22baec7 tests: Add RPMs to installed kola tests, port layering-local
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.

We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over.  Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data.  Add the concept
of multiple repo versions too.

Right now we only generate the one RPM needed for the `layering-local`
test and port it.
2021-03-02 09:47:03 -08:00
Colin Walters
7367aa8469 tests/apply-live: Upgrade+liveapply no-package changes
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction".  I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
2021-03-01 20:40:48 +01:00
Colin Walters
4d7de4fa59 tests/vmcheck: Use .cosa over COSA_DIR
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.

Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
2021-03-01 18:53:56 +01:00