IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The previous build was GC'd; unfortunately it's very nontrivial
to make this test truly robust over time because FCOS changes;
we might sometimes have an outstanding update, other times might
not etc.
Let's just sanity check the commands; ultimately they're
thin wrappers around just downloading packages so we don't need
deep checks.
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.
This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
Until now with cxx-rs we'd been using it effectively as a better
cbindgen - we're exposing Rust code to C++ safely. This is
the first case of having Rust calling back into C++ using cxx-rs.
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.
This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions
The input YAML schema matches the one accepted by that script.
Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
we've hit in RHCOS (like downloading the wrong `libprotobuf` for
`usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
the cosa script uses `reposdir=$dir` which doesn't have the same
semantics (repo enablement is in that case purely based on the
`enabled` flag in those repos, which may be different than what the
rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
the same EVR as the one from a base package: this is helpful in the
case of extensions which complement a base package, esp. those which
may not have strong enough reldeps to enforce matching EVRs by
depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
extension because IMO it's not at the right level. Instead, we should
work towards higher-level metadata to represent extensions (see
https://github.com/openshift/os/issues/409 which is related to this).
Closes: #2055
Our CI isn't running the C unit tests because it goes via RPM,
and while we could potentially add `%check` there...I don't
quite want to do that right now since it also runs the Rust
tests which means we rebuild all the Rust code again in debug
mode etc.
Change the C unit tests to compile in C++ mode, which is
enough for local testing.
Longer term I think the C unit tests will go away in favor
of Rust tests.
Gather the current diff of `/etc`, and filter out changes in
the tree which would overwrite it.
There is an OSTree API for diffs but it's a bit awkward, missing
some APIs in the Rust bindings and also `GFile` based unfortunately.
Doing this in Rust is nicer. The dirdiff code obviously needs
a lot more testing, but I think it's right.
This adds support for e.g.:
```
$ rpm-ostree override replace https://bodhi.fedoraproject.org/updates/FEDORA-2020-2908628031
```
This will find the Koji builds from the listed update, download
all the RPMs (that aren't debuginfo) and pass them for overrides
in the same way we support `override replace http://somewebserver/foo.rpm`
now.
We also support directly linking a Koji build:
```
$ rpm-ostree override replace https://koji.fedoraproject.org/koji/buildinfo?buildID=1625029
```
Bodhi has a modern HTTP+JSON API, and the lack of a Koji equivalent
drove me to create https://github.com/cgwalters/koji-sane-json-api
and we currently depend on an instance set up in the OpenShift CI
cluster.
I hope it shouldn't take long to deploy this in Fedora Infra,
but I don't want to block on it.
Also notably this still downloads *all* the other RPMs even
ones that aren't installed. Handling that truly correctly
would require moving this logic to the daemon and core.
All of this functionality is keyed off a `cfg(feature = "fedora-integration")`
that is detected by a Rust `build.rs` which parses the build environment's
`/etc/os-release` for now.
Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime). It's really ostree's mtime canonicalization
that's unfortunate here.
Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.
(Also remove the `mkdir -p` since it's not necessary since a long time)
Closes: https://github.com/coreos/rpm-ostree/issues/2435
This makes the C++ side a bit uglier because the "variable mapping"
is more Rust-native but we only use it Rust side anyways.
(We can't yet move the basearch bits to rust because it depends on
libdnf, which requires buildsystem unification)
But all the unsafe FFI conversion drops out, as do the duplicated
C unit tests.
The cool thing about this is it emphasizes how "integrated" apply-live
is versus the uncontrolled `rpm-ostree usroverlay`. We're still
tracking the state of things reliably and can print it.
(Keeping the old name for muscle memory compatibility for now)
I think `apply-live` is a clearer name; it's more imperative
and it may not be obvious (particularly to non-native English speakers)
to parse "livefs" as "live fs".
On traditional rpm systems this can hang because the outer
process may have an rpmdb lock, and the inner one wants
to acquire a lock. Here we're sandboxing the `%post` script
and it's targeting a separate temporary filesystem compared to
the booted one (so there's no double locking). Plus we don't
create the rpmdb in the target until all scripts have run.
Inspired by https://twitter.com/_msw_/status/1335981558717587473
I was looking at the output of `ostree admin config-diff`
on a base FCOS boot. It'd be really nice to trim that down
as much as possible, so we can cleanly capture the difference
between user config and system config.
Let's use static enablement rather than presets.
I still think we should do this at some point, but
the experiment with using `GKeyfile` for configuration
is IMO a failure and the variety of data formats
(treefile JSON vs YAML vs origin keyfiles vs container keyfiles)
causes a lot of confusion.
Prep for https://github.com/coreos/rpm-ostree/issues/2326
Now always based on an overlayfs:
f2773c1b55
This fixes a whole swath of problems with the previous design,
including the danger in replacing `/usr/lib/ostree-boot` which
broke booting for some people.
Further, we don't need to push a rollback deployment; the livefs
changes are always transient. So now we store livefs state
in `/run` instead of in the origin file.
Since we're doing a rewrite, it's now in Rust for much more safety.
We also always work in terms of incremental diffs between commits;
the previous huge hammer of swapping `/usr` was way too dangerous.
This effectively reverts commit: c8113bde32
We never ended up using it; instead the `rdcore` bits from
`coreos-installer` have the rootfs reprovisioning logic.
This command allows users to cheaply inject configuration files in the
initramfs stage without having to regenerate the whole initramfs (or
even a new OSTree commit). This will be useful for configuring services
involved in bringing up the root block device.
```
$ echo 'hello world' > /etc/foobar
$ rpm-ostree ex initramfs-etc --track /etc/foobar
Staging deployment... done
Run "systemctl reboot" to start a reboot
$ rpm-ostree status
State: idle
Deployments:
ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
InitramfsEtc: /etc/foobar
● ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
$ reboot
(boot into rd.break)
sh-5.0# cat /etc/foobar
hello world
```
See the libostree side of this at:
https://github.com/ostreedev/ostree/pull/2155
Lots more discussions in:
https://github.com/coreos/fedora-coreos-tracker/issues/94Closes: #1930
We're seeing some CI failures that I think are a bug in rojig.
In the bigger picture...we never actually started using this,
and I think longer term shipping os updates via containers
probably makes more sense.
I put a *lot* of effort into this code and it's pretty cool
so it's hard to just delete it. And *maybe* someone out there
is using it (but I doubt it). So rather than just deleting
it entirely let's make it a build-time option.
I verified that it builds at least.
In f32, ping is no longer privileged since it ships with the sysctl for
`ping_group_range` which allows unpriv pings. Check the file caps of
arping instead, which does still use them.
Now that we've bumped to the latest FCOS commit for compose tests, one
thing that came up was that our compose tests never actually included
FCOS overlays in the compose the way cosa does.
This then cause compose failures because one of the postprocess scripts
expects those files there.
Let's just nuke all postprocess scripts here to work around this. I
initially wanted to import the overlay logic from cosa, but overlays
only work in unified core mode, and sadly we still want some coverage in
non-unified mode until that's fully dropped.
And anyway, we also already do a proper `cosa build` in the vmcheck
branch of CI so it's not like we're losing that coverage.
Down the line though, I think this is a good argument for folding the
overlay dirs into rpm-ostree more natively as discussed here:
https://github.com/coreos/coreos-assembler/pull/639#issuecomment-534713737
The design of https://fedoraproject.org/wiki/Changes/Sqlite_Rpmdb
is problematic for us for multiple reasons. The first big reason
is that rpm-ostree is designed for "cross" builds and e.g. today
we use a Fedora-derived container to build RHEL CoreOS images.
However the default database lives inside the `rpm` package which
means that if we e.g. upgrade the coreos-assembler container to F33
it will suddenly try to use sqlite for RHCOS which is obviously broken.
Related to this, rebases from f32 to f33 w/layered packages
are broken: https://bugzilla.redhat.com/show_bug.cgi?id=1876194#c3
With this we can configure things to continue to use bdb for f33
for ostree-based systems, so that by enforcing an upgrade order
f32 → f33 [bdb] → f34 [sqlite] ... the intermediate f33 w/bdb
still understands sqlite and hence rebases will work.
For the Fedora CoreOS extensions work, when layering packages, we need
to be able to tell libsolv to pick the packages which will go with the
base packages. IOW, it needs to know that the base packages shouldn't be
uninstalled.
While investigating
https://github.com/coreos/fedora-coreos-tracker/issues/525, I realized
that libsolv does have a flag which allows us to express this:
`SOLVER_LOCK`.
This then allows libsolv to choose the right package for us (if found).
And in the case where it can't find a matching package, libsolv itself
will print exactly what the conflict is, which is more informative than
the "forbidden replacements" error we currently print out.
Update submodule: libdnf
We want to test upgrades that actually change files as a general
rule; in some cases we want to test "large" upgrades to validate
performance.
This code generates a "synthetic" upgrade that adds an ELF note
to a percentage of ELF files (randomly selected). By doing
it this way we are only actually testing one version of the code.
Migrated from https://github.com/coreos/coreos-assembler/pull/1635/
using the Rust code from https://github.com/ostreedev/ostree/pull/2127
Fix https://github.com/coreos/rpm-ostree/issues/2068: `remove-from-packages`
deleting files that it shouldn't.
Filter out files that user wants removed at `checkout_package_into_root()`,
instead of at the `handle_remove_files_from_package()` function that does
not check whether files are used by other rpms before removing them.
Add a helper function for whitespace_split_packages() so that it now
splits a String by whitespace only if it is not wrapped between single
quotes.
This should allow RHCOS to use syntax like podman > 1.4 in the treefile.
Also add new unit tests and tweak existing compose tests to test this
functionality.
Both `upgrade` and `deploy` already support this. There's no reason why
all the remaining "deployment-creating" commands shouldn't. Prompted by
https://github.com/openshift/machine-config-operator/issues/1897 which
will need this specifically for `rebase`.
The latest crypto-policies package changed recently to dynamically set
the policy at install time so that if FIPS is enabled, the selected
backend is `FIPS`:
9b9c9f7378
This doesn't really make sense for us though since the compose server
configuration should be decoupled from the installroot. (More generally,
this also affects e.g. `yum install --installroot`).
Override the script for now so that we always select the `DEFAULT`
policy. We'll discuss with upstream to see what the right solution is
there.
This also works around the fact that rpm-ostree doesn't yet implement
Lua (#749).
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1847454
Resolves: https://github.com/coreos/fedora-coreos-tracker/issues/540
Now that cosa and FCOS have moved to f32, a bunch of tests are breaking.
Let's make them more resistant to releasever changes.
While we're here though, bump the container image we use on the target
host to f32, and update the systemd example output.
Switch to the "installed" model introduced by:
https://github.com/coreos/coreos-assembler/pull/1441
It's hard to support running tests *both* from the srcdir
and installed; in this case because we have a symlink that needs
to be followed, which kola knows how to do from the srcdir
but not when installed. Let's establish a new convention of
`tests/kolainst`. In our case we follow the symlink manually
for now.
That bit will be cleaned up when we eventually switch entirely
to kola tests.
