IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Continuing the momentum to use kola ext tests.
One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
We have fully transitioned to cxx-rs! This drops a lot of now
dead code; only one binding system to think about generating
source code. For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.
We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over. Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data. Add the concept
of multiple repo versions too.
Right now we only generate the one RPM needed for the `layering-local`
test and port it.
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.
Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default. We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.
Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
Print error and exit if cannot find `tests/vmcheck/image.qcow2` or
bad symlink causing failure to spawn a VM when calling the
`vm_kola_spawn()` function.
Our CI isn't running the C unit tests because it goes via RPM,
and while we could potentially add `%check` there...I don't
quite want to do that right now since it also runs the Rust
tests which means we rebuild all the Rust code again in debug
mode etc.
Change the C unit tests to compile in C++ mode, which is
enough for local testing.
Longer term I think the C unit tests will go away in favor
of Rust tests.
Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime). It's really ostree's mtime canonicalization
that's unfortunate here.
Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.
(Also remove the `mkdir -p` since it's not necessary since a long time)
Closes: https://github.com/coreos/rpm-ostree/issues/2435
We want to test upgrades that actually change files as a general
rule; in some cases we want to test "large" upgrades to validate
performance.
This code generates a "synthetic" upgrade that adds an ELF note
to a percentage of ELF files (randomly selected). By doing
it this way we are only actually testing one version of the code.
Migrated from https://github.com/coreos/coreos-assembler/pull/1635/
using the Rust code from https://github.com/ostreedev/ostree/pull/2127
Now that cosa and FCOS have moved to f32, a bunch of tests are breaking.
Let's make them more resistant to releasever changes.
While we're here though, bump the container image we use on the target
host to f32, and update the systemd example output.
The main goal here is to get `assert_jq()` usable in
kola tests.
This was forked from ostree long ago but we aren't
using most of it. I want to try to move this into kola where
we're just using `tests/common` but this code references
`tests/gpghome` which we weren't using.
Only a few things here reference `SRCDIR` - change those
to fail for now if it's not set, since we're not running
those tests in kola yet. I will eventually try to
clean that up later.
Pre-FCOS we made an effort for automatic updates but nowadays
with Fedora CoreOS we generally expect people to be using zincati.
Until we fix the "agent registration" problem:
https://github.com/coreos/rpm-ostree/issues/1747
Let's not confuse people by printing `AutomaticUpdates: disabled`.
Only print if it's set to a value in non-verbose mode.
We need to adapt some of our tests here which assume that `/sysroot` is
writable. However, in FCOS this is no longer the case now that we enable
`sysroot.readonly`.
We only remount rw for the couple of operations that need it so that we
still retain coverage for the ro path everywhere else.
It's the latest, and matches the rest of the host we're running on. But
also, pulling f30 is hitting 503s from the Fedora registry:
https://pagure.io/releng/issue/9282
In FCOS we have a kola test that basically does `rpm -q python`.
It's...a bit silly to spawn a whole VM for this. Ensuring that
some specific packages don't get included has come up in a few
cases.
I think FCOS/RHCOS at least will want to blacklist `dnf` for example.
And as noted above, FCOS could blacklist `python`.
One major benefit of doing this inside rpm-ostree is that one
gets the full "libsolv error message experience" when dependency
resolution fails, e.g. blacklisting `glibc` I get:
```
Problem 79: conflicting requests
- package coreos-installer-systemd-0.1.2-1.fc31.x86_64 requires coreos-installer = 0.1.2-1.fc31, but none of the providers can be installed
- package coreos-installer-0.1.2-1.fc31.x86_64 requires rtld(GNU_HASH), but none of the providers can be installed
- package glibc-2.30-10.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-7.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-8.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.i686 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-10.fc31.i686 is filtered out by exclude filtering
```
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
There's a lot going on here, but essentially:
1. We change the `vmcheck` model so that it always operates on an
immutable base image. It takes that image and dynamically launches a
separate VM for each test using `kola spawn`. This means we can drop
a lot of hacks around re-using the same VMs.
2. Following from 1., `vmoverlay` now takes as input a base image,
overlays the built rpm-ostree bits, then creates a new base image. Of
course, we don't have to do this in CI, because we build FCOS with
the freshly built RPMs (so it uses `SKIP_VMOVERLAY=1`). `vmoverlay`
then will be more for the developer case where one doesn't want to
iterate via `cosa build` to test rpm-ostree changes. I say "will"
because the functionality doesn't exist yet; I'd like to enhance
`cosa dev-overlay` to do this. (Note `vmsync` should still works just
as before too.)
3. `vmcheck` can be run without building the tree first, as
`tests/vmcheck.sh`. The `make vmcheck` target still exists though for
finger compatibility and better meshing with `vmoverlay` in the
developer case.
What's really nice about using kola spawn is that it takes care of a lot
of things for us, such as the qemu command, journal and console
gathering, and SSH.
Similarly to the compose testsuites, we're using parallel here to run
multiple vmcheck tests at once. (On developer laptops, we cap
parallelism at `$(nproc) - 1`).
This is a hack to allow using `inject-pkglist` without having to build
the tree first.
Higher-level, I think we can split this back out again if we have a
`-tests` subpackage where we ship the vmcheck testsuite.
Drop the use of Ansible everywhere. In the few cases where we really
Python, just spawn a container instead.
This is required to be able to hack on Fedora CoreOS.
Closes: #1850
Approved by: jlebon
In the app, rebuild the exact command-line that the client used and pass
that to the daemon to be used as the transaction title. Especially in
transactions like `UpdateDeployment()`, we can avoid reverse-engineering
what the original command used was.
This will be used by the upcoming history feature to record the
command-line used in the journal.
Closes: #1824
Approved by: rfairley
This bumps the requirement on the controlling host to Python 3 only.
It also bumps the requirement on the target host to Python 3 as well
since FCOS doesn't ship Python 2 right now.
Though we'll need to eventually drop all Python usage anyway, but at
least let's get tests passing on FCOS first. (See related previous
patch).
Closes: #1828
Approved by: cgwalters
In Fedora 29, and Fedora 30 Silverblue, I have come across the
following error when executing `make vmsync` from my build container
(also on Fedora 29 and Fedora 30 images respectively):
```
...
Failed to connect to new control master
...
Control socket connect(/var/tmp/ssh-vmcheck-1556768111752693879.sock): Connection refused
Failed to connect to new control master
...
```
Previously this worked with Fedora 28 as the host.
After changing the socket to be in /dev/shm, the SSH connection to
the `vmcheck` VM is successful and the sources sync over.
The cause of this seems to be a problem with overlayfs and unix
sockets: https://github.com/moby/moby/issues/12080
Since overlayfs is the default graph driver in Fedora now, work
around this by switching the socket to be in /dev/shm.
Closes: #1827
Approved by: jlebon
One question I often have when looking at the output of `status -a`:
```
AvailableUpdate:
Version: 29.20181202.0 (2018-12-02T08:37:50Z)
Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
SecAdvisories: FEDORA-2018-042156f164 Unknown net-snmp-libs-1:5.8-3.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-core-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-extra-4.19.5-300.fc29.x86_64
FEDORA-2018-f467c36c2b Moderate git-core-2.19.2-1.fc29.x86_64
Diff: 67 upgraded, 1 removed, 16 added
```
is "How serious and relevant are these advisories to me? How soon should
I reboot?". For the packages that I'm most familiar with, e.g. `kernel`
and `git-core`, I usually look up the advisory and check why it was
marked as a security update, mentioned CVEs, and how those affect me.
The updateinfo metadata includes a wealth of information that could be
useful here. In Fedora, CVEs treated by the security response team
result in RHBZs, which end up attached to the advisories and thus make
it into that metadata.
This patch tries to reduce friction in answering some of those questions
above by checking for those CVEs and printing a short description in the
output of `status -a`. Example:
```
AvailableUpdate:
Version: 29.20181202.0 (2018-12-02T08:37:50Z)
Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
SecAdvisories: FEDORA-2018-042156f164 Unknown net-snmp-libs-1:5.8-3.fc29.x86_64
CVE-2018-18065 CVE-2018-18066 net-snmp: various flaws [fedora-all]
https://bugzilla.redhat.com/show_bug.cgi?id=1637573
FEDORA-2018-87ba0312c2 Moderate kernel-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-core-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-4.19.5-300.fc29.x86_64
FEDORA-2018-87ba0312c2 Moderate kernel-modules-extra-4.19.5-300.fc29.x86_64
CVE-2018-16862 kernel: cleancache: Infoleak of deleted files after reuse of old inodes
https://bugzilla.redhat.com/show_bug.cgi?id=1649017
CVE-2018-19407 kernel: kvm: NULL pointer dereference in vcpu_scan_ioapic in arch/x86/kvm/x86.c
https://bugzilla.redhat.com/show_bug.cgi?id=1652656
FEDORA-2018-f467c36c2b Moderate git-core-2.19.2-1.fc29.x86_64
CVE-2018-19486 git: Improper handling of PATH allows for commands to executed from current directory
https://bugzilla.redhat.com/show_bug.cgi?id=1653143
Diff: 67 upgraded, 1 removed, 16 added
```
Including the CVE name and RHBZ link also makes it easier to look for
more details if desired.
Closes: #1695
Approved by: rfairley
This is prep for running inside (unprivileged) Kube containers
as they exist today: https://github.com/projectatomic/rpm-ostree/issues/1329
Sadly FUSE today uses a suid binary that ends up wanting CAP_SYS_ADMIN.
I think there's some work on FUSE-in-containers but I'm not sure of
the current status.
What rofiles-fuse here is doing here is protecting is the hardlinked
repo imports. But if `--cachedir` isn't specified, that repository
gets thrown away anyways. So there's no real value to using FUSE
here.
Also since nothing is cached, disable the devino cache.
We also make use of --force-copy-zerosized that just landed
in libostree: https://github.com/ostreedev/ostree/pull/1752
Down the line ideally we gain the capability to detect if either
unprivileged overlayfs/FUSE are available. Then if `--cachedir`
is specified we can make things work.
Closes: #1591
Approved by: jlebon
Our trick of using layered commits as base commit updates doesn't jive
well with the new opportunistic reuse of the base rpmdb. The issue is
that a layered commit includes the rpmdb of *its* base commit at the
`/usr/lib/sysimage` location. So to convert it into a proper base commit
means that the layered rpmdb should move there.
Closes: #1502
Approved by: cgwalters
We had two `libcomposetest.sh` which I always found confusing.
Fix the naming of the one that's shared with `ex-container`
to be more obvious.
Closes: #1543
Approved by: jlebon
Since `/tmp` might be on tmpfs, so we'd lose it on reboot. But we have
tests that need it to persist across reboots.
Closes: #1531
Approved by: miabbott
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units. For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.
(I briefly looked at implementing the `Trigger` line from `systemctl status`
but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)
Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.
Closes: #1432
Approved by: jlebon
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system. It's tempting to do a `reset` before each one
and reboot but this makes things work.
There's some additional assertions here as I went through and
was debugging.
Prep for making staging the default.
Closes: #1438
Approved by: jlebon
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.
I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.
This was all quite straightforward, just plumbing through lots
of layers.
Closes: #1321
Approved by: jlebon
Now we stop running rpm-ostreed as non-root, which is going to be
a maintenance pain going forward. If we do introduce non-VM based
tests I think we should look to doing in-container testing.
Closes: #1339
Approved by: cgwalters
The unit tests run an rpm-ostree daemon as non-root, which worked
surprisingly well for quite a while. But it started failing when
working on a patch which adds caching that writes to `/var`.
Since we have the vmcheck system now, let's switch over to that.
This PR moves the random "basic" tests we'd accumulated like
one to verify `StateRoot` is only in `status --verbose`, but not
the tests for the `rebase` command etc.
Closes: #1336
Approved by: jlebon
When juggling multiple test VMs for different purposes, it's useful to
be able to easily e.g. `make vmcheck` a specific one by overriding the
ssh-config file to use, rather than editing the latter each time.
Closes: #1324
Approved by: cgwalters
Because otherwise, there's no way to see the output of the script.
Also, turn off `gather_facts` since in the majority of cases, we don't
need it, so let's avoid the overhead. We can make this an opt-in flag
later on if needed.
Closes: #1304
Approved by: cgwalters
This adds a shell primitive to make it easy to execute a playbook
task list.
The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.
Closes: #1297
Approved by: jlebon
We would error out when trying to start the transient httpd service if
it already exists, e.g. from a previous test.
Depending on how we exit, the `vm_stop_httpd` trap for the previous test
might not have been able to kick in. I think this happens when we exit
using `fatal`, which just does an `exit 1`. It's not strictly an error,
so doesn't trip the `ERR` handler.
Let's just go the extra mile and explicitly delete transient services if
they already exist.
Closes: #1284
Approved by: cgwalters
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.
This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.
Closes: #1249
Approved by: cgwalters
