IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
In RHCOS, we ship kernel development-related packages as an extension.
Those aren't really extensions that are meant to be layered onto the
host. They're meant to be used in a build environment somewhere to
compile kernel modules.
This makes it very different from "OS extensions" in at least two
drastic ways:
1. we don't want to do any depsolving (e.g. we don't want to pull in
`gcc` or something)
2. some of those packages may be present in the base already, but we
still want to redownload them
Hesitated putting this functionality in rpm-ostree, but I think in the
end it cuts from the benefit of moving this code to rpm-ostree if we
can't entirely get rid of the Python script it obsoletes. Plus, being
able to use the `match-base-evr` is still really useful for this use
case.
Let's add a new `kind` key to support this. The traditional extensions
are called "OS extensions" and these new extensions are called
"development extensions".
The latter is not yet part of the state checksum, so change detection
doesn't work there. I think that's fine for now though because the
primary use case is the kernel, and there we want to match the base
version. So if the kernel changes, the base would change too. (Though
there's the corner case of adding a new package to the list while at the
same version...)
When automatic updates are driven by an external driver like Zincati,
display e.g. `AutomaticUpdatesDriver: Zincati` instead of
`AutomaticUpdates: driven by Zincati`, since the latter might suggest it
is rpm-ostree's own built-in support for automatic updates.
We now have bidirectional calling between Rust and C++,
but we are generating two static libraries that we then
link together with a tiny C++ `main.cxx`.
Let's make another huge leap towards oxdiation by
having Rust be the entrypoint. This way cargo natively
takes care of linking the internal Rust library, and
our C++ internals become the library.
In other words, we've now fully inverted from
"C app with internal Rust library"
to "Rust binary with internal C++ library".
In order to make this work though we have to finally
kill the C unit tests. But mostly everything covered
there is either being converted to Rust, or covered
elsewhere anyways.
Now as the doc comments in `main.rs` say...this is
a bit awkward because all the CLI code is still in C++.
Porting stuff to use e.g. `structopt` natively would
be a bit of a slog. For now, we basically rely on
the fact that the Rust-native CLIs are all hidden
commands.
Update submodule: libdnf
This demonstrates well the strength of the cxx-rs approach;
we can keep an API in C++ but add unit tests in Rust which
just works much more nicely.
Prep for https://github.com/coreos/rpm-ostree/pull/2502
which wants to drop the C++ unit tests.
Prep for "Rust-as-main", where I want to build libdnf statically.
And this really completes the "library thinout" story because
now we avoid dragging our *private* `libdnf.so` into the caller's
address space, which can cause potential conflicts if they're
also linking the system one. (Which could easily occur with
something like gnome-software)
All we were using libdnf for (indirectly via libsolv) is comparing
version strings but librpm can already do that for us.
The previous build was GC'd; unfortunately it's very nontrivial
to make this test truly robust over time because FCOS changes;
we might sometimes have an outstanding update, other times might
not etc.
Let's just sanity check the commands; ultimately they're
thin wrappers around just downloading packages so we don't need
deep checks.
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.
This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
Until now with cxx-rs we'd been using it effectively as a better
cbindgen - we're exposing Rust code to C++ safely. This is
the first case of having Rust calling back into C++ using cxx-rs.
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.
This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions
The input YAML schema matches the one accepted by that script.
Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
we've hit in RHCOS (like downloading the wrong `libprotobuf` for
`usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
the cosa script uses `reposdir=$dir` which doesn't have the same
semantics (repo enablement is in that case purely based on the
`enabled` flag in those repos, which may be different than what the
rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
the same EVR as the one from a base package: this is helpful in the
case of extensions which complement a base package, esp. those which
may not have strong enough reldeps to enforce matching EVRs by
depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
extension because IMO it's not at the right level. Instead, we should
work towards higher-level metadata to represent extensions (see
https://github.com/openshift/os/issues/409 which is related to this).
Closes: #2055
Our CI isn't running the C unit tests because it goes via RPM,
and while we could potentially add `%check` there...I don't
quite want to do that right now since it also runs the Rust
tests which means we rebuild all the Rust code again in debug
mode etc.
Change the C unit tests to compile in C++ mode, which is
enough for local testing.
Longer term I think the C unit tests will go away in favor
of Rust tests.
Gather the current diff of `/etc`, and filter out changes in
the tree which would overwrite it.
There is an OSTree API for diffs but it's a bit awkward, missing
some APIs in the Rust bindings and also `GFile` based unfortunately.
Doing this in Rust is nicer. The dirdiff code obviously needs
a lot more testing, but I think it's right.
This adds support for e.g.:
```
$ rpm-ostree override replace https://bodhi.fedoraproject.org/updates/FEDORA-2020-2908628031
```
This will find the Koji builds from the listed update, download
all the RPMs (that aren't debuginfo) and pass them for overrides
in the same way we support `override replace http://somewebserver/foo.rpm`
now.
We also support directly linking a Koji build:
```
$ rpm-ostree override replace https://koji.fedoraproject.org/koji/buildinfo?buildID=1625029
```
Bodhi has a modern HTTP+JSON API, and the lack of a Koji equivalent
drove me to create https://github.com/cgwalters/koji-sane-json-api
and we currently depend on an instance set up in the OpenShift CI
cluster.
I hope it shouldn't take long to deploy this in Fedora Infra,
but I don't want to block on it.
Also notably this still downloads *all* the other RPMs even
ones that aren't installed. Handling that truly correctly
would require moving this logic to the daemon and core.
All of this functionality is keyed off a `cfg(feature = "fedora-integration")`
that is detected by a Rust `build.rs` which parses the build environment's
`/etc/os-release` for now.
Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime). It's really ostree's mtime canonicalization
that's unfortunate here.
Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.
(Also remove the `mkdir -p` since it's not necessary since a long time)
Closes: https://github.com/coreos/rpm-ostree/issues/2435
This makes the C++ side a bit uglier because the "variable mapping"
is more Rust-native but we only use it Rust side anyways.
(We can't yet move the basearch bits to rust because it depends on
libdnf, which requires buildsystem unification)
But all the unsafe FFI conversion drops out, as do the duplicated
C unit tests.
The cool thing about this is it emphasizes how "integrated" apply-live
is versus the uncontrolled `rpm-ostree usroverlay`. We're still
tracking the state of things reliably and can print it.
(Keeping the old name for muscle memory compatibility for now)
I think `apply-live` is a clearer name; it's more imperative
and it may not be obvious (particularly to non-native English speakers)
to parse "livefs" as "live fs".
On traditional rpm systems this can hang because the outer
process may have an rpmdb lock, and the inner one wants
to acquire a lock. Here we're sandboxing the `%post` script
and it's targeting a separate temporary filesystem compared to
the booted one (so there's no double locking). Plus we don't
create the rpmdb in the target until all scripts have run.
Inspired by https://twitter.com/_msw_/status/1335981558717587473
I was looking at the output of `ostree admin config-diff`
on a base FCOS boot. It'd be really nice to trim that down
as much as possible, so we can cleanly capture the difference
between user config and system config.
Let's use static enablement rather than presets.
I still think we should do this at some point, but
the experiment with using `GKeyfile` for configuration
is IMO a failure and the variety of data formats
(treefile JSON vs YAML vs origin keyfiles vs container keyfiles)
causes a lot of confusion.
Prep for https://github.com/coreos/rpm-ostree/issues/2326
Now always based on an overlayfs:
f2773c1b55
This fixes a whole swath of problems with the previous design,
including the danger in replacing `/usr/lib/ostree-boot` which
broke booting for some people.
Further, we don't need to push a rollback deployment; the livefs
changes are always transient. So now we store livefs state
in `/run` instead of in the origin file.
Since we're doing a rewrite, it's now in Rust for much more safety.
We also always work in terms of incremental diffs between commits;
the previous huge hammer of swapping `/usr` was way too dangerous.
This effectively reverts commit: c8113bde32
We never ended up using it; instead the `rdcore` bits from
`coreos-installer` have the rootfs reprovisioning logic.
This command allows users to cheaply inject configuration files in the
initramfs stage without having to regenerate the whole initramfs (or
even a new OSTree commit). This will be useful for configuring services
involved in bringing up the root block device.
```
$ echo 'hello world' > /etc/foobar
$ rpm-ostree ex initramfs-etc --track /etc/foobar
Staging deployment... done
Run "systemctl reboot" to start a reboot
$ rpm-ostree status
State: idle
Deployments:
ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
InitramfsEtc: /etc/foobar
● ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
$ reboot
(boot into rd.break)
sh-5.0# cat /etc/foobar
hello world
```
See the libostree side of this at:
https://github.com/ostreedev/ostree/pull/2155
Lots more discussions in:
https://github.com/coreos/fedora-coreos-tracker/issues/94Closes: #1930
We're seeing some CI failures that I think are a bug in rojig.
In the bigger picture...we never actually started using this,
and I think longer term shipping os updates via containers
probably makes more sense.
I put a *lot* of effort into this code and it's pretty cool
so it's hard to just delete it. And *maybe* someone out there
is using it (but I doubt it). So rather than just deleting
it entirely let's make it a build-time option.
I verified that it builds at least.
In f32, ping is no longer privileged since it ships with the sysctl for
`ping_group_range` which allows unpriv pings. Check the file caps of
arping instead, which does still use them.
Now that we've bumped to the latest FCOS commit for compose tests, one
thing that came up was that our compose tests never actually included
FCOS overlays in the compose the way cosa does.
This then cause compose failures because one of the postprocess scripts
expects those files there.
Let's just nuke all postprocess scripts here to work around this. I
initially wanted to import the overlay logic from cosa, but overlays
only work in unified core mode, and sadly we still want some coverage in
non-unified mode until that's fully dropped.
And anyway, we also already do a proper `cosa build` in the vmcheck
branch of CI so it's not like we're losing that coverage.
Down the line though, I think this is a good argument for folding the
overlay dirs into rpm-ostree more natively as discussed here:
https://github.com/coreos/coreos-assembler/pull/639#issuecomment-534713737
The design of https://fedoraproject.org/wiki/Changes/Sqlite_Rpmdb
is problematic for us for multiple reasons. The first big reason
is that rpm-ostree is designed for "cross" builds and e.g. today
we use a Fedora-derived container to build RHEL CoreOS images.
However the default database lives inside the `rpm` package which
means that if we e.g. upgrade the coreos-assembler container to F33
it will suddenly try to use sqlite for RHCOS which is obviously broken.
Related to this, rebases from f32 to f33 w/layered packages
are broken: https://bugzilla.redhat.com/show_bug.cgi?id=1876194#c3
With this we can configure things to continue to use bdb for f33
for ostree-based systems, so that by enforcing an upgrade order
f32 → f33 [bdb] → f34 [sqlite] ... the intermediate f33 w/bdb
still understands sqlite and hence rebases will work.
For the Fedora CoreOS extensions work, when layering packages, we need
to be able to tell libsolv to pick the packages which will go with the
base packages. IOW, it needs to know that the base packages shouldn't be
uninstalled.
While investigating
https://github.com/coreos/fedora-coreos-tracker/issues/525, I realized
that libsolv does have a flag which allows us to express this:
`SOLVER_LOCK`.
This then allows libsolv to choose the right package for us (if found).
And in the case where it can't find a matching package, libsolv itself
will print exactly what the conflict is, which is more informative than
the "forbidden replacements" error we currently print out.
Update submodule: libdnf
We want to test upgrades that actually change files as a general
rule; in some cases we want to test "large" upgrades to validate
performance.
This code generates a "synthetic" upgrade that adds an ELF note
to a percentage of ELF files (randomly selected). By doing
it this way we are only actually testing one version of the code.
Migrated from https://github.com/coreos/coreos-assembler/pull/1635/
using the Rust code from https://github.com/ostreedev/ostree/pull/2127
Fix https://github.com/coreos/rpm-ostree/issues/2068: `remove-from-packages`
deleting files that it shouldn't.
Filter out files that user wants removed at `checkout_package_into_root()`,
instead of at the `handle_remove_files_from_package()` function that does
not check whether files are used by other rpms before removing them.
