IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
In FCOS, we use "override" lockfiles to pin packages to certain
versions. Right now, we have separate overrides for each base arch we
(eventually want to) support. But that makes maintaining the overrides
cumbersome because of all the duplication.
Let's allow lockfiles to specify only the `evr` of a package, which is
just as good for FCOS, and means that we'll only have to maintain a
single override file for all the architectures.
This is a major downside of reworking and generating new CI
flows, it's super easy to lose testing what you intend to.
Also, we clearly need to figure out a flow where this is shared
across repos, since I don't want to copy-paste this into e.g. ostree too.
That's https://github.com/coreos/fedora-coreos-tracker/issues/263
OK I think it's time. This exposes the `apply-live` functionality
as implicitly stable, but specific to the package install case.
I'd like to add more intelligence to `apply-live` around separating
pure "additions" (as in this case) versus package (file) changes.
The change here doesn't try to do that; the implementation is
incredibly simple, we just have the client chain together the two
distinct transactions.
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).
The latter is really quite safe, and live layering is one
of the most requested features.
Continuing the momentum to use kola ext tests.
One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.
A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.
My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
We have fully transitioned to cxx-rs! This drops a lot of now
dead code; only one binding system to think about generating
source code. For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.
We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over. Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data. Add the concept
of multiple repo versions too.
Right now we only generate the one RPM needed for the `layering-local`
test and port it.
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction". I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.
Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).
The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.
But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.
I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.
Closes: https://github.com/coreos/rpm-ostree/issues/2603
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.
The reliability of apply-live depends on the
underlying commits not being garbage collected. Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).
Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.
(Or perhaps what we should add to ostree is like a refs.d model
where a single atomic file can refer to multiple commits)
For now though let's rework the code here to write refs. We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default. We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.
Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
Do not perform an upgrade if detected that an updates driver has
been registered.
Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
I didn't deep dive on this, just observed that the new output
matches what we laid down. There's ~2.5 years of changes
and ~200 commits between 2.28 and 2.48.
Print error and exit if cannot find `tests/vmcheck/image.qcow2` or
bad symlink causing failure to spawn a VM when calling the
`vm_kola_spawn()` function.
We trigger a librpm macro file load in many of our paths. Since the
default value shipped by rpm's macro file sets `_dbpath` to
`/var/lib/rpm`, we have to explicitly set that back to `/usr/share/rpm`
in those paths.
This became more problematic recently with libsolv v0.7.17 which fully
keys off of `_dbpath` to find the rpmdb path to load:
04d4d036b2
And it's not technically wrong; we really should make that macro not
lie. This is what this patch does by injecting an RPM macro file in our
composes which sets it to /usr/share/rpm. So then e.g. the `rpm` CLI
doesn't actually need the `/var/lib/rpm` backcompat link anymore, though
there's no harm in leaving it.
In the future, we should be able to drop this once we move all of Fedora
to `/usr/lib/sysimage/rpm` (see
https://github.com/coreos/fedora-coreos-tracker/issues/639).
Closes: #2548
In RHCOS, we ship kernel development-related packages as an extension.
Those aren't really extensions that are meant to be layered onto the
host. They're meant to be used in a build environment somewhere to
compile kernel modules.
This makes it very different from "OS extensions" in at least two
drastic ways:
1. we don't want to do any depsolving (e.g. we don't want to pull in
`gcc` or something)
2. some of those packages may be present in the base already, but we
still want to redownload them
Hesitated putting this functionality in rpm-ostree, but I think in the
end it cuts from the benefit of moving this code to rpm-ostree if we
can't entirely get rid of the Python script it obsoletes. Plus, being
able to use the `match-base-evr` is still really useful for this use
case.
Let's add a new `kind` key to support this. The traditional extensions
are called "OS extensions" and these new extensions are called
"development extensions".
The latter is not yet part of the state checksum, so change detection
doesn't work there. I think that's fine for now though because the
primary use case is the kernel, and there we want to match the base
version. So if the kernel changes, the base would change too. (Though
there's the corner case of adding a new package to the list while at the
same version...)
When automatic updates are driven by an external driver like Zincati,
display e.g. `AutomaticUpdatesDriver: Zincati` instead of
`AutomaticUpdates: driven by Zincati`, since the latter might suggest it
is rpm-ostree's own built-in support for automatic updates.
We now have bidirectional calling between Rust and C++,
but we are generating two static libraries that we then
link together with a tiny C++ `main.cxx`.
Let's make another huge leap towards oxdiation by
having Rust be the entrypoint. This way cargo natively
takes care of linking the internal Rust library, and
our C++ internals become the library.
In other words, we've now fully inverted from
"C app with internal Rust library"
to "Rust binary with internal C++ library".
In order to make this work though we have to finally
kill the C unit tests. But mostly everything covered
there is either being converted to Rust, or covered
elsewhere anyways.
Now as the doc comments in `main.rs` say...this is
a bit awkward because all the CLI code is still in C++.
Porting stuff to use e.g. `structopt` natively would
be a bit of a slog. For now, we basically rely on
the fact that the Rust-native CLIs are all hidden
commands.
Update submodule: libdnf
This demonstrates well the strength of the cxx-rs approach;
we can keep an API in C++ but add unit tests in Rust which
just works much more nicely.
Prep for https://github.com/coreos/rpm-ostree/pull/2502
which wants to drop the C++ unit tests.
Prep for "Rust-as-main", where I want to build libdnf statically.
And this really completes the "library thinout" story because
now we avoid dragging our *private* `libdnf.so` into the caller's
address space, which can cause potential conflicts if they're
also linking the system one. (Which could easily occur with
something like gnome-software)
All we were using libdnf for (indirectly via libsolv) is comparing
version strings but librpm can already do that for us.
The previous build was GC'd; unfortunately it's very nontrivial
to make this test truly robust over time because FCOS changes;
we might sometimes have an outstanding update, other times might
not etc.
Let's just sanity check the commands; ultimately they're
thin wrappers around just downloading packages so we don't need
deep checks.
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.
This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
Until now with cxx-rs we'd been using it effectively as a better
cbindgen - we're exposing Rust code to C++ safely. This is
the first case of having Rust calling back into C++ using cxx-rs.
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.
This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions
The input YAML schema matches the one accepted by that script.
Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
we've hit in RHCOS (like downloading the wrong `libprotobuf` for
`usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
the cosa script uses `reposdir=$dir` which doesn't have the same
semantics (repo enablement is in that case purely based on the
`enabled` flag in those repos, which may be different than what the
rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
the same EVR as the one from a base package: this is helpful in the
case of extensions which complement a base package, esp. those which
may not have strong enough reldeps to enforce matching EVRs by
depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
extension because IMO it's not at the right level. Instead, we should
work towards higher-level metadata to represent extensions (see
https://github.com/openshift/os/issues/409 which is related to this).
Closes: #2055
Our CI isn't running the C unit tests because it goes via RPM,
and while we could potentially add `%check` there...I don't
quite want to do that right now since it also runs the Rust
tests which means we rebuild all the Rust code again in debug
mode etc.
Change the C unit tests to compile in C++ mode, which is
enough for local testing.
Longer term I think the C unit tests will go away in favor
of Rust tests.
Gather the current diff of `/etc`, and filter out changes in
the tree which would overwrite it.
There is an OSTree API for diffs but it's a bit awkward, missing
some APIs in the Rust bindings and also `GFile` based unfortunately.
Doing this in Rust is nicer. The dirdiff code obviously needs
a lot more testing, but I think it's right.
This adds support for e.g.:
```
$ rpm-ostree override replace https://bodhi.fedoraproject.org/updates/FEDORA-2020-2908628031
```
This will find the Koji builds from the listed update, download
all the RPMs (that aren't debuginfo) and pass them for overrides
in the same way we support `override replace http://somewebserver/foo.rpm`
now.
We also support directly linking a Koji build:
```
$ rpm-ostree override replace https://koji.fedoraproject.org/koji/buildinfo?buildID=1625029
```
Bodhi has a modern HTTP+JSON API, and the lack of a Koji equivalent
drove me to create https://github.com/cgwalters/koji-sane-json-api
and we currently depend on an instance set up in the OpenShift CI
cluster.
I hope it shouldn't take long to deploy this in Fedora Infra,
but I don't want to block on it.
Also notably this still downloads *all* the other RPMs even
ones that aren't installed. Handling that truly correctly
would require moving this logic to the daemon and core.
All of this functionality is keyed off a `cfg(feature = "fedora-integration")`
that is detected by a Rust `build.rs` which parses the build environment's
`/etc/os-release` for now.
Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime). It's really ostree's mtime canonicalization
that's unfortunate here.
Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.
(Also remove the `mkdir -p` since it's not necessary since a long time)
Closes: https://github.com/coreos/rpm-ostree/issues/2435
This makes the C++ side a bit uglier because the "variable mapping"
is more Rust-native but we only use it Rust side anyways.
(We can't yet move the basearch bits to rust because it depends on
libdnf, which requires buildsystem unification)
But all the unsafe FFI conversion drops out, as do the duplicated
C unit tests.
The cool thing about this is it emphasizes how "integrated" apply-live
is versus the uncontrolled `rpm-ostree usroverlay`. We're still
tracking the state of things reliably and can print it.
(Keeping the old name for muscle memory compatibility for now)
I think `apply-live` is a clearer name; it's more imperative
and it may not be obvious (particularly to non-native English speakers)
to parse "livefs" as "live fs".
