Commit Graph

7 Commits

Author SHA1 Message Date
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Jonathan Lebon
317b920667 extensions: Add support for development extensions
In RHCOS, we ship kernel development-related packages as an extension.
Those aren't really extensions that are meant to be layered onto the
host.  They're meant to be used in a build environment somewhere to
compile kernel modules.

This makes it very different from "OS extensions" in at least two
drastic ways:
1. we don't want to do any depsolving (e.g. we don't want to pull in
   `gcc` or something)
2. some of those packages may be present in the base already, but we
   still want to redownload them

Hesitated putting this functionality in rpm-ostree, but I think in the
end it cuts from the benefit of moving this code to rpm-ostree if we
can't entirely get rid of the Python script it obsoletes. Plus, being
able to use the `match-base-evr` is still really useful for this use
case.

Let's add a new `kind` key to support this. The traditional extensions
are called "OS extensions" and these new extensions are called
"development extensions".

The latter is not yet part of the state checksum, so change detection
doesn't work there. I think that's fine for now though because the
primary use case is the kernel, and there we want to match the base
version. So if the kernel changes, the base would change too. (Though
there's the corner case of adding a new package to the list while at the
same version...)
2021-02-03 12:22:38 -05:00
Jonathan Lebon
c98c227c0f extensions: Write JSON to output dir
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.

This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
2021-01-27 00:44:42 +01:00
Jonathan Lebon
271954a41c app: Add rpm-ostree compose extensions
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.

This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions

The input YAML schema matches the one accepted by that script.

Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
  we've hit in RHCOS (like downloading the wrong `libprotobuf` for
  `usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
  the cosa script uses `reposdir=$dir` which doesn't have the same
  semantics (repo enablement is in that case purely based on the
  `enabled` flag in those repos, which may be different than what the
  rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
  as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
  integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
  the same EVR as the one from a base package: this is helpful in the
  case of extensions which complement a base package, esp. those which
  may not have strong enough reldeps to enforce matching EVRs by
  depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
  extension because IMO it's not at the right level. Instead, we should
  work towards higher-level metadata to represent extensions (see
  https://github.com/openshift/os/issues/409 which is related to this).

Closes: #2055
2021-01-23 17:12:09 +01:00
Colin Walters
c039b72ebd importer: Use /run instead of /var/run
Translate RPM paths under `/var/run` to `/run` automatically; this
quiets down systemd.  Since we end up running `systemd-tmpfiles`
a few times in FCOS reducing spew here is particularly valuable.

The bug is really in the packages here but...we don't have an
agile process for fixing them.

Note that for this fix to take effect, if you have a `cache/pkgcache-repo`
you'll need to remove it.
2020-02-03 07:50:10 -08:00
Jonathan Lebon
9daea46d66 tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.

I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).

(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)

Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)

However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2020-01-08 16:42:54 +01:00
Jonathan Lebon
462a389b3a tests/compose: Move files around
This is mostly cosmetic, though I want the test layout to mirror what we
do for `vmcheck`.
2020-01-08 16:42:54 +01:00