IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Now always based on an overlayfs:
f2773c1b55
This fixes a whole swath of problems with the previous design,
including the danger in replacing `/usr/lib/ostree-boot` which
broke booting for some people.
Further, we don't need to push a rollback deployment; the livefs
changes are always transient. So now we store livefs state
in `/run` instead of in the origin file.
Since we're doing a rewrite, it's now in Rust for much more safety.
We also always work in terms of incremental diffs between commits;
the previous huge hammer of swapping `/usr` was way too dangerous.
This effectively reverts commit: c8113bde32
We never ended up using it; instead the `rdcore` bits from
`coreos-installer` have the rootfs reprovisioning logic.
This command allows users to cheaply inject configuration files in the
initramfs stage without having to regenerate the whole initramfs (or
even a new OSTree commit). This will be useful for configuring services
involved in bringing up the root block device.
```
$ echo 'hello world' > /etc/foobar
$ rpm-ostree ex initramfs-etc --track /etc/foobar
Staging deployment... done
Run "systemctl reboot" to start a reboot
$ rpm-ostree status
State: idle
Deployments:
ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
InitramfsEtc: /etc/foobar
● ostree://fedora:fedora/x86_64/coreos/testing-devel
Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
GPGSignature: (unsigned)
$ reboot
(boot into rd.break)
sh-5.0# cat /etc/foobar
hello world
```
See the libostree side of this at:
https://github.com/ostreedev/ostree/pull/2155
Lots more discussions in:
https://github.com/coreos/fedora-coreos-tracker/issues/94Closes: #1930
We're seeing some CI failures that I think are a bug in rojig.
In the bigger picture...we never actually started using this,
and I think longer term shipping os updates via containers
probably makes more sense.
I put a *lot* of effort into this code and it's pretty cool
so it's hard to just delete it. And *maybe* someone out there
is using it (but I doubt it). So rather than just deleting
it entirely let's make it a build-time option.
I verified that it builds at least.
In f32, ping is no longer privileged since it ships with the sysctl for
`ping_group_range` which allows unpriv pings. Check the file caps of
arping instead, which does still use them.
Now that we've bumped to the latest FCOS commit for compose tests, one
thing that came up was that our compose tests never actually included
FCOS overlays in the compose the way cosa does.
This then cause compose failures because one of the postprocess scripts
expects those files there.
Let's just nuke all postprocess scripts here to work around this. I
initially wanted to import the overlay logic from cosa, but overlays
only work in unified core mode, and sadly we still want some coverage in
non-unified mode until that's fully dropped.
And anyway, we also already do a proper `cosa build` in the vmcheck
branch of CI so it's not like we're losing that coverage.
Down the line though, I think this is a good argument for folding the
overlay dirs into rpm-ostree more natively as discussed here:
https://github.com/coreos/coreos-assembler/pull/639#issuecomment-534713737
The design of https://fedoraproject.org/wiki/Changes/Sqlite_Rpmdb
is problematic for us for multiple reasons. The first big reason
is that rpm-ostree is designed for "cross" builds and e.g. today
we use a Fedora-derived container to build RHEL CoreOS images.
However the default database lives inside the `rpm` package which
means that if we e.g. upgrade the coreos-assembler container to F33
it will suddenly try to use sqlite for RHCOS which is obviously broken.
Related to this, rebases from f32 to f33 w/layered packages
are broken: https://bugzilla.redhat.com/show_bug.cgi?id=1876194#c3
With this we can configure things to continue to use bdb for f33
for ostree-based systems, so that by enforcing an upgrade order
f32 → f33 [bdb] → f34 [sqlite] ... the intermediate f33 w/bdb
still understands sqlite and hence rebases will work.
For the Fedora CoreOS extensions work, when layering packages, we need
to be able to tell libsolv to pick the packages which will go with the
base packages. IOW, it needs to know that the base packages shouldn't be
uninstalled.
While investigating
https://github.com/coreos/fedora-coreos-tracker/issues/525, I realized
that libsolv does have a flag which allows us to express this:
`SOLVER_LOCK`.
This then allows libsolv to choose the right package for us (if found).
And in the case where it can't find a matching package, libsolv itself
will print exactly what the conflict is, which is more informative than
the "forbidden replacements" error we currently print out.
Update submodule: libdnf
We want to test upgrades that actually change files as a general
rule; in some cases we want to test "large" upgrades to validate
performance.
This code generates a "synthetic" upgrade that adds an ELF note
to a percentage of ELF files (randomly selected). By doing
it this way we are only actually testing one version of the code.
Migrated from https://github.com/coreos/coreos-assembler/pull/1635/
using the Rust code from https://github.com/ostreedev/ostree/pull/2127
Fix https://github.com/coreos/rpm-ostree/issues/2068: `remove-from-packages`
deleting files that it shouldn't.
Filter out files that user wants removed at `checkout_package_into_root()`,
instead of at the `handle_remove_files_from_package()` function that does
not check whether files are used by other rpms before removing them.
Add a helper function for whitespace_split_packages() so that it now
splits a String by whitespace only if it is not wrapped between single
quotes.
This should allow RHCOS to use syntax like podman > 1.4 in the treefile.
Also add new unit tests and tweak existing compose tests to test this
functionality.
Both `upgrade` and `deploy` already support this. There's no reason why
all the remaining "deployment-creating" commands shouldn't. Prompted by
https://github.com/openshift/machine-config-operator/issues/1897 which
will need this specifically for `rebase`.
The latest crypto-policies package changed recently to dynamically set
the policy at install time so that if FIPS is enabled, the selected
backend is `FIPS`:
9b9c9f7378
This doesn't really make sense for us though since the compose server
configuration should be decoupled from the installroot. (More generally,
this also affects e.g. `yum install --installroot`).
Override the script for now so that we always select the `DEFAULT`
policy. We'll discuss with upstream to see what the right solution is
there.
This also works around the fact that rpm-ostree doesn't yet implement
Lua (#749).
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1847454
Resolves: https://github.com/coreos/fedora-coreos-tracker/issues/540
Now that cosa and FCOS have moved to f32, a bunch of tests are breaking.
Let's make them more resistant to releasever changes.
While we're here though, bump the container image we use on the target
host to f32, and update the systemd example output.
Switch to the "installed" model introduced by:
https://github.com/coreos/coreos-assembler/pull/1441
It's hard to support running tests *both* from the srcdir
and installed; in this case because we have a symlink that needs
to be followed, which kola knows how to do from the srcdir
but not when installed. Let's establish a new convention of
`tests/kolainst`. In our case we follow the symlink manually
for now.
That bit will be cleaned up when we eventually switch entirely
to kola tests.
Same motivation as
7392259332
I think we should encourage removing the writable bits from
executables. This has happened to me:
https://thomask.sdf.org/blog/2019/11/09/take-care-editing-bash-scripts.html
And not having the writable bit may help prevent hardlink
corruption with OSTree in some cases.
We can't do this by default, but add a convenient treefile option
for it.
This starts out by just doing this for RPMs, but I'll add
a secondary pass which does it during postprocessing soon too.
The main goal here is to get `assert_jq()` usable in
kola tests.
This was forked from ostree long ago but we aren't
using most of it. I want to try to move this into kola where
we're just using `tests/common` but this code references
`tests/gpghome` which we weren't using.
Only a few things here reference `SRCDIR` - change those
to fail for now if it's not set, since we're not running
those tests in kola yet. I will eventually try to
clean that up later.
Pre-FCOS we made an effort for automatic updates but nowadays
with Fedora CoreOS we generally expect people to be using zincati.
Until we fix the "agent registration" problem:
https://github.com/coreos/rpm-ostree/issues/1747
Let's not confuse people by printing `AutomaticUpdates: disabled`.
Only print if it's set to a value in non-verbose mode.
In Fedora CoreOS, we have a "coreos-pool" repo from which all packages
in lockfiles are tagged for reproducible builds. This repo is shared
across all streams, including those on f31 and f32.
Thus, it makes no sense for composes to ever pick packages unconstrained
from the pool without being guided by a lockfile. Otherwise, one can
easily end up with e.g. f32 packages in an f31 compose.
Add a new `lockfile-repos` for this which is only used for fetching
lockfile packages and nothing else. For example, this will allow
`cosa fetch --update-lockfile` to Just Work as expected by only fetching
new packages from regular yum repos.
Today, lockfiles only restrict the NEVRA of specifc package names from
which libsolv can pick. But nothing stops libsolv from picking entirely
different packages which still satisfy the manifest requests.
This was mostly a theoretical issue in Fedora CoreOS, but became reality
with the addition of Fedora 32 packages in the pool. libsolv would
happily try to pick e.g. `libcurl-minimal` from f32 instead of sticking
with the f31 `libcurl` from the lockfiles:
https://github.com/coreos/fedora-coreos-streams/issues/75#issuecomment-610734584
(But more generally, see
https://github.com/coreos/fedora-coreos-tracker/issues/454).
Let's add a `--ex-lockfile-strict` mode, which in CI and production
pipeline build contexts will require that (1) *only* locked packages are
considered by libsolv, and (2) *all* locked packages were marked for
install.
One important thing to note here is that we don't short-circuit libsolv
and manually `hy_goal_install` lockfile packages. We want to make sure
the treefile is still canonical. Strict mode simply ensures that the
result agrees with the lockfile.
That said, even in developer contexts, we don't want the
`libcurl-minimal` issue that happened to be triggered. But we still want
to allow flexibility in adding and removing packages to make hacking
easier. I have some follow-up patches which will enable this.
We need to be friendlier to people who are transitioning from
"traditional" yum managed systems. This patchset starts to lay
out the groundwork for supporting "intercepting" binaries that
are in the tree.
For backwards compatibility, this feature is disabled by default,
to enable it, one can add `cliwrap: true` to the manifest.
To start with for example, we wrap `/usr/bin/rpm` and cause it
to drop privileges. This way it can't corrupt anything; we're
not just relying on the read-only bind mount. For example nothing
will accidentally get written to `/var/lib/rpm`.
Now a tricky thing with this one is we *do* want it to write if
we're in an unlocked state.
There are various other examples of binaries we want to intercept,
among them:
- `grubby` -> `rpm-ostree kargs`
- `dracut` -> `rpm-ostree initramfs`
- `yum` -> well...we'll talk about that later
The garbage collection issue should be fixed now, and it's just nicer on
developers' cache to stay on the same commit. And again, it's a nice
sanity-check to know that we're always able to compose an older tree.
That said, we probably should still bump this from time to time.
While we're here, add some comments for making it easier to match `popd`
calls with the original `pushd`.
Start the ball rolling on converting some of our tests into
the coreos-assembler/kola framework:
d940420b78/mantle/kola/README-kola-ext.md
The nondestructive ones are easy.
This way we handle filenames with spaces in `/var` in general,
like `/var/app/foo bar`, but *also* the special `/opt/foo bar`
translation bits.
I saw this bug and thought "oh that'd be easy". But hoo boy
did it take me down a rat's nest. The first thing was verifying
that `systemd-tmpfiles` supports any kind of quotation/escaping; it does.
The next thing was figuring out *exactly* what the syntax for that
is and how it works, as it's obviously not widely used.
Writing tests for this ended up being a painful exercise because
of the multiple levels of shell script, e.g. our `build_rpm` shell
script ends up being inlined into RPM specs, which then interprets
again...and not to mention the usual annoying issues with `ssh`
eating quotes.
Anyways, all that and:
Closes: https://github.com/coreos/rpm-ostree/issues/2029
We need to adapt some of our tests here which assume that `/sysroot` is
writable. However, in FCOS this is no longer the case now that we enable
`sysroot.readonly`.
We only remount rw for the couple of operations that need it so that we
still retain coverage for the ro path everywhere else.
This is the second half of the previous commit. We check if the
canonical dracut args are available in the commit metadata, and prefer
those over using `--rebuild`. The latter is delegated as a backcompat
fallback.
It's the latest, and matches the rest of the host we're running on. But
also, pulling f30 is hitting 503s from the Fedora registry:
https://pagure.io/releng/issue/9282
We're hitting issues with packages getting tagged out of the pool:
https://pagure.io/releng/issue/9281
This in turn means we can't reliably recompose older builds right now,
which breaks our CI. For now at least, let's compose from the latest.
(Note we were already also composing the latest FCOS in the vmcheck
branch.)
Instead of basing our decision to use the local `/etc` on whether we're
using `dracut --rebuild`, base it directly on a boolean parameter.
This is relevant in the client-side when initramfs regeneration is
requested as well as a kernel override. In such cases, we do want to use
the local `/etc`, but we'd skip that path because we didn't also use
`dracut --rebuild`.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1806588
This fixes a longstanding spew of error messages from the initramfs
because we don't have nss-altfiles set up there. Rather than
trying to do it, just do the dance of re-synthesizing `/etc/passwd`
as it traditionally looks around running dracut, the same as we
do for scripts during core layering.
Yes, this is all a mess and hopefully I'll get to sysusers soon...
In FCOS we have a kola test that basically does `rpm -q python`.
It's...a bit silly to spawn a whole VM for this. Ensuring that
some specific packages don't get included has come up in a few
cases.
I think FCOS/RHCOS at least will want to blacklist `dnf` for example.
And as noted above, FCOS could blacklist `python`.
One major benefit of doing this inside rpm-ostree is that one
gets the full "libsolv error message experience" when dependency
resolution fails, e.g. blacklisting `glibc` I get:
```
Problem 79: conflicting requests
- package coreos-installer-systemd-0.1.2-1.fc31.x86_64 requires coreos-installer = 0.1.2-1.fc31, but none of the providers can be installed
- package coreos-installer-0.1.2-1.fc31.x86_64 requires rtld(GNU_HASH), but none of the providers can be installed
- package glibc-2.30-10.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-7.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-8.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.i686 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-10.fc31.i686 is filtered out by exclude filtering
```
Translate RPM paths under `/var/run` to `/run` automatically; this
quiets down systemd. Since we end up running `systemd-tmpfiles`
a few times in FCOS reducing spew here is particularly valuable.
The bug is really in the packages here but...we don't have an
agile process for fixing them.
Note that for this fix to take effect, if you have a `cache/pkgcache-repo`
you'll need to remove it.
We were basing whether to print the `Upgraded`/`Downgraded` heading on
the iteration count rather than the actual first iteration where a valid
upgrade/downgrade was found. And because of how we print our diff, this
confusingly can make it look like downgrades are part of the same
upgrade section.
Closes: #1821
