IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The inevitable followup to https://github.com/coreos/rpm-ostree/pull/2278
that I was too cowardly to do at the time. But it's time to admit
the 2 months or so of work on this was wasted. We have too much
tech debt and this is a large chunk of C/C++ code that touches everything
in the codebase in a nontrivial way.
Bigger picture, I'm going to work on
https://github.com/coreos/fedora-coreos-tracker/issues/828
which will strongly orient rpm-ostree towards the container world instead.
We'll still obviously keep the rpm package world around, but only
as a secondary layer. What rojig was trying to do in putting "images"
inside an RPM was conflating layers. It would have had a lot of
benefits probably if we'd truly pushed it over the edge into completion,
but that didn't happen. Let's focus on containers instead.
There's still a lot more rojig code to delete but this first patch removes
the bulk of it. Touching everything that references e.g. `RPMOSTREE_REFSPEC_TYPE_ROJIG`
etc. can come as a 3rd phase.
This addresses the server compose side of
https://github.com/coreos/rpm-ostree/issues/2584.
One tricky bit is handling overrides across included treefiles (or
really, even within a single treefile): as usual, higher-level treefiles
should override lowel-level ones. Rust makes it pretty nice to handle.
For now this just supports a `repo` field, but one could imagine e.g.
`repos` (which takes an array of repoids instead), or e.g.
`exclude-repos`.
The actual core implementation otherwise is pretty straightforward.
This should help a lot in RHCOS where we currently use many `exclude=`
directives in repo files to get it to do what we want.
This is also kind of a requirement for modularity support because as
soon as rpm-ostree becomes modules-aware, modular filtering logic will
break composes which assume rpm-ostree treats modular and non-modular
packages the same.
This section is meant to give a broad overview of this doc.
First introducing the tool in general. Then showing the distinction
between 'compose tree' and the granular approach. Finally transitioning
over to the next section.
This is long overdue. Some of this came up in recent
conversation. Let's keep up some continual background
momentum on documentation, just like CI.
Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default. We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.
Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
In RHCOS, we ship kernel development-related packages as an extension.
Those aren't really extensions that are meant to be layered onto the
host. They're meant to be used in a build environment somewhere to
compile kernel modules.
This makes it very different from "OS extensions" in at least two
drastic ways:
1. we don't want to do any depsolving (e.g. we don't want to pull in
`gcc` or something)
2. some of those packages may be present in the base already, but we
still want to redownload them
Hesitated putting this functionality in rpm-ostree, but I think in the
end it cuts from the benefit of moving this code to rpm-ostree if we
can't entirely get rid of the Python script it obsoletes. Plus, being
able to use the `match-base-evr` is still really useful for this use
case.
Let's add a new `kind` key to support this. The traditional extensions
are called "OS extensions" and these new extensions are called
"development extensions".
The latter is not yet part of the state checksum, so change detection
doesn't work there. I think that's fine for now though because the
primary use case is the kernel, and there we want to match the base
version. So if the kernel changes, the base would change too. (Though
there's the corner case of adding a new package to the list while at the
same version...)
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.
This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions
The input YAML schema matches the one accepted by that script.
Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
we've hit in RHCOS (like downloading the wrong `libprotobuf` for
`usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
the cosa script uses `reposdir=$dir` which doesn't have the same
semantics (repo enablement is in that case purely based on the
`enabled` flag in those repos, which may be different than what the
rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
the same EVR as the one from a base package: this is helpful in the
case of extensions which complement a base package, esp. those which
may not have strong enough reldeps to enforce matching EVRs by
depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
extension because IMO it's not at the right level. Instead, we should
work towards higher-level metadata to represent extensions (see
https://github.com/openshift/os/issues/409 which is related to this).
Closes: #2055
There won't be any support for writing to the bdb backend in f34, so
e.g. pkglayering won't work (and obviously even composes wouldn't work
once the buildroot moves to f34).
Instead of requiring the whole world to add an `rpmdb` key in their
manifests, let's just add a compile flag for it, and tweak the spec file
to use this flag on f34.
Those got moved to GitHub from readthedocs.io, meaning links pointing
to specific pages return a 404, so this fixes that along with
skipping the redirect for the links that just point to the main page.
The design of https://fedoraproject.org/wiki/Changes/Sqlite_Rpmdb
is problematic for us for multiple reasons. The first big reason
is that rpm-ostree is designed for "cross" builds and e.g. today
we use a Fedora-derived container to build RHEL CoreOS images.
However the default database lives inside the `rpm` package which
means that if we e.g. upgrade the coreos-assembler container to F33
it will suddenly try to use sqlite for RHCOS which is obviously broken.
Related to this, rebases from f32 to f33 w/layered packages
are broken: https://bugzilla.redhat.com/show_bug.cgi?id=1876194#c3
With this we can configure things to continue to use bdb for f33
for ostree-based systems, so that by enforcing an upgrade order
f32 → f33 [bdb] → f34 [sqlite] ... the intermediate f33 w/bdb
still understands sqlite and hence rebases will work.
Start adding some pain if `--unified-core` isn't provided
to help flush out anyone relying on it. (And I think today
pungi is not passing it, so e.g. Fedora IoT/Silverblue are impacted)
Prep for merging https://github.com/coreos/rpm-ostree/pull/1793
Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
The lists of pros and cons for each approach in the background section
of the manual use markdown formatting that is rendered as nested
lists. Separate the lists of benefits and drawbacks so that readers of
https://rpm-ostree.readthedocs.io/en/latest/manual/background/ and the
rendered markdown on github can understand which items are pros and
which are cons.
Signed-off-by: Doug Hellmann <dhellmann@redhat.com>
Same motivation as
7392259332
I think we should encourage removing the writable bits from
executables. This has happened to me:
https://thomask.sdf.org/blog/2019/11/09/take-care-editing-bash-scripts.html
And not having the writable bit may help prevent hardlink
corruption with OSTree in some cases.
We can't do this by default, but add a convenient treefile option
for it.
This starts out by just doing this for RPMs, but I'll add
a secondary pass which does it during postprocessing soon too.
We are have been pretty inconsistent about this; I think while it's
true the project is called "OSTree", "RPM-OSTree" is just annoying
to type and looks weird. `rpm-ostree` requires much less
"shift key gymnastics".
Also, the "proper name" style like RPM/OSTree is best when
something can be viewed more "abstractly" - there are multiple
RPM implementations (historically) and OSTree also is a *concept*
in addition to an implementation.
rpm-ostree though is much more of a "concrete" thing so it
makes more sense to use it as a "project name".
In Fedora CoreOS, we have a "coreos-pool" repo from which all packages
in lockfiles are tagged for reproducible builds. This repo is shared
across all streams, including those on f31 and f32.
Thus, it makes no sense for composes to ever pick packages unconstrained
from the pool without being guided by a lockfile. Otherwise, one can
easily end up with e.g. f32 packages in an f31 compose.
Add a new `lockfile-repos` for this which is only used for fetching
lockfile packages and nothing else. For example, this will allow
`cosa fetch --update-lockfile` to Just Work as expected by only fetching
new packages from regular yum repos.
We need to be friendlier to people who are transitioning from
"traditional" yum managed systems. This patchset starts to lay
out the groundwork for supporting "intercepting" binaries that
are in the tree.
For backwards compatibility, this feature is disabled by default,
to enable it, one can add `cliwrap: true` to the manifest.
To start with for example, we wrap `/usr/bin/rpm` and cause it
to drop privileges. This way it can't corrupt anything; we're
not just relying on the read-only bind mount. For example nothing
will accidentally get written to `/var/lib/rpm`.
Now a tricky thing with this one is we *do* want it to write if
we're in an unlocked state.
There are various other examples of binaries we want to intercept,
among them:
- `grubby` -> `rpm-ostree kargs`
- `dracut` -> `rpm-ostree initramfs`
- `yum` -> well...we'll talk about that later
In FCOS we have a kola test that basically does `rpm -q python`.
It's...a bit silly to spawn a whole VM for this. Ensuring that
some specific packages don't get included has come up in a few
cases.
I think FCOS/RHCOS at least will want to blacklist `dnf` for example.
And as noted above, FCOS could blacklist `python`.
One major benefit of doing this inside rpm-ostree is that one
gets the full "libsolv error message experience" when dependency
resolution fails, e.g. blacklisting `glibc` I get:
```
Problem 79: conflicting requests
- package coreos-installer-systemd-0.1.2-1.fc31.x86_64 requires coreos-installer = 0.1.2-1.fc31, but none of the providers can be installed
- package coreos-installer-0.1.2-1.fc31.x86_64 requires rtld(GNU_HASH), but none of the providers can be installed
- package glibc-2.30-10.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-7.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-8.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.i686 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-10.fc31.i686 is filtered out by exclude filtering
```
This allows replacing the `.` in automatic version increments
with whatever one wants (as long as it's a single ASCII character)
right now.
The specific motivation here is for at least RHEL CoreOS to use
`version-suffix: "-"` so that its versions can become valid
semantic versions.
Related: https://github.com/coreos/rpm-ostree/issues/1954
Point at FCOS and not Project Atomic. Add an inline "Why"
section since people will want to know that right away.
(An great thing about Github is the prevalence it gives to `README.md`;
projects should use that as an "elevator pitch")
Drop outdated bits in `background.md`.
Closes: #1895
Approved by: jlebon
