IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Add a helper function for whitespace_split_packages() so that it now
splits a String by whitespace only if it is not wrapped between single
quotes.
This should allow RHCOS to use syntax like podman > 1.4 in the treefile.
Also add new unit tests and tweak existing compose tests to test this
functionality.
Now that cosa and FCOS have moved to f32, a bunch of tests are breaking.
Let's make them more resistant to releasever changes.
While we're here though, bump the container image we use on the target
host to f32, and update the systemd example output.
Same motivation as
7392259332
I think we should encourage removing the writable bits from
executables. This has happened to me:
https://thomask.sdf.org/blog/2019/11/09/take-care-editing-bash-scripts.html
And not having the writable bit may help prevent hardlink
corruption with OSTree in some cases.
We can't do this by default, but add a convenient treefile option
for it.
This starts out by just doing this for RPMs, but I'll add
a secondary pass which does it during postprocessing soon too.
In Fedora CoreOS, we have a "coreos-pool" repo from which all packages
in lockfiles are tagged for reproducible builds. This repo is shared
across all streams, including those on f31 and f32.
Thus, it makes no sense for composes to ever pick packages unconstrained
from the pool without being guided by a lockfile. Otherwise, one can
easily end up with e.g. f32 packages in an f31 compose.
Add a new `lockfile-repos` for this which is only used for fetching
lockfile packages and nothing else. For example, this will allow
`cosa fetch --update-lockfile` to Just Work as expected by only fetching
new packages from regular yum repos.
Today, lockfiles only restrict the NEVRA of specifc package names from
which libsolv can pick. But nothing stops libsolv from picking entirely
different packages which still satisfy the manifest requests.
This was mostly a theoretical issue in Fedora CoreOS, but became reality
with the addition of Fedora 32 packages in the pool. libsolv would
happily try to pick e.g. `libcurl-minimal` from f32 instead of sticking
with the f31 `libcurl` from the lockfiles:
https://github.com/coreos/fedora-coreos-streams/issues/75#issuecomment-610734584
(But more generally, see
https://github.com/coreos/fedora-coreos-tracker/issues/454).
Let's add a `--ex-lockfile-strict` mode, which in CI and production
pipeline build contexts will require that (1) *only* locked packages are
considered by libsolv, and (2) *all* locked packages were marked for
install.
One important thing to note here is that we don't short-circuit libsolv
and manually `hy_goal_install` lockfile packages. We want to make sure
the treefile is still canonical. Strict mode simply ensures that the
result agrees with the lockfile.
That said, even in developer contexts, we don't want the
`libcurl-minimal` issue that happened to be triggered. But we still want
to allow flexibility in adding and removing packages to make hacking
easier. I have some follow-up patches which will enable this.
In FCOS we have a kola test that basically does `rpm -q python`.
It's...a bit silly to spawn a whole VM for this. Ensuring that
some specific packages don't get included has come up in a few
cases.
I think FCOS/RHCOS at least will want to blacklist `dnf` for example.
And as noted above, FCOS could blacklist `python`.
One major benefit of doing this inside rpm-ostree is that one
gets the full "libsolv error message experience" when dependency
resolution fails, e.g. blacklisting `glibc` I get:
```
Problem 79: conflicting requests
- package coreos-installer-systemd-0.1.2-1.fc31.x86_64 requires coreos-installer = 0.1.2-1.fc31, but none of the providers can be installed
- package coreos-installer-0.1.2-1.fc31.x86_64 requires rtld(GNU_HASH), but none of the providers can be installed
- package glibc-2.30-10.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-7.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-8.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.i686 is filtered out by exclude filtering
- package glibc-2.30-5.fc31.x86_64 is filtered out by exclude filtering
- package glibc-2.30-10.fc31.i686 is filtered out by exclude filtering
```
Translate RPM paths under `/var/run` to `/run` automatically; this
quiets down systemd. Since we end up running `systemd-tmpfiles`
a few times in FCOS reducing spew here is particularly valuable.
The bug is really in the packages here but...we don't have an
agile process for fixing them.
Note that for this fix to take effect, if you have a `cache/pkgcache-repo`
you'll need to remove it.
This is a follow-up hack to #1797 to force libdnf to let us use modular
packages as if they were regular packages until we actually support
modules correctly (#1435).
A repo marked as a modular hotfix means that libdnf doesn't try to
filter out modular RPMs from the repo as it usually does.
Resolves: https://pagure.io/releng/failed-composes/issue/717
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
Prep for making `--unified-core` the only path. It turns
out our compose testsuite has a lot of hardcoded ideas about
how the two paths work. The rojig tests in particular need
cached RPMs, so we can't just rely on caching the pkgcache repo.
Add a `--download-only-rpms` that always returns RPMs, and doesn't
import into the pkgcache repo.
Closes: #1798
Approved by: jlebon
We had two `libcomposetest.sh` which I always found confusing.
Fix the naming of the one that's shared with `ex-container`
to be more obvious.
Closes: #1543
Approved by: jlebon
The `f28-compose` test keeps timing out. Some time recently, I/O
performance of the internal OpenStack instance used for testing has
degraded. I have a ticket open to investigate the regression though
haven't had any luck so far.
Let's just take the easy way out and split the test into two testsuites.
This is obviously hacky, and sad, and unfortunate. But the PRs must keep
flowing until we finally wean off of OpenStack.
Closes: #1498
Approved by: cgwalters
Make logging work the same as it does for the vmcheck-STI work
(at some point I'll try to unify the 3 parallel+script implementions
we have). This fixes the problem that when the test times out,
the filename won't have `.txt` and S3 won't have the right MIME type.
Closes: #1479
Approved by: jlebon
- Actually use separate `${test_tmpdir}` for test setup (closes a race)
- Merge stdout/stderr (more readable)
- Ensure logs are renamed to `.txt` even on failure
- Use `--progress` for some feedback
- Use `-j +1` so that even on unicore machines we get at least 2
jobs (and in general NCPUS+1)
Closes: #1188
Approved by: jlebon
Over a year later, the "opening the host rpmdb" bug is fixed,
so we can do composes in parallel ∥, hooray!
I'm dusting this off since we were running into CI (PAPR) timeouts
when I was adding more to the compose tests.
Closes: #545
Approved by: jlebon
When we added the `--ex-unified-core` option our caching story got
very messy because the non-unified core caches RPMs, but unified
does ostree repo caching.
For jigdo, we want the RPMs. Fix this by mirroring the RPMs using
`--download-only` and pointing the tests consistently at that.
Closes: #1122
Approved by: jlebon
We have some unit-style tests that run `ex container`, but
they aren't "real"; they don't use scripts for example. Let's
add tests for this similar to `tests/compose`.
We use a 26 base, but the target repos need to be 27
to pick up the fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1478172
Add some bits to share infra between `tests/compose` and `tests/ex-container`;
basically handling the rpmmd repos. I tweaked things to be more streamlined
there between the `.papr.yml` and the test script.
Right now this is just one test for `bash`, but lays some of the infrastructure
for doing more. One thing that we need to do to improve more here is to better
cache RPMs, a bit like the compose tests do.
Closes: #1024
Approved by: jlebon
We can be a bit less wasteful here by merging the check and vmcheck
suites into a single suite. The check suite today takes a negligible
amount of time to run, so we're not gaining much by parallelizing them.
It's more of a sanity check at this point before we start vmcheck.
Also start running vmcheck on CentOS 7. We adapt the ci scripts to
accomodate both Fedora and CentOS target machines.
This commit also switches to Fedora 26 as the primary test base.
Closes: #871
Approved by: cgwalters
During provisioning, PAPR injects a fedora.repo pointing at a much
better & faster mirror than dl.fp.o. Let's use that to make the compose
test less flaky. Hoping to make these sorts of optimizations more
discoverable in upstream PAPR.
Closes: #799
Approved by: cgwalters
Being able to just reuse metadata is especially helpful when trying to
debug things lower down that path, as well as cuts about 2 mins from the
compose test.
Closes: #561
Approved by: cgwalters
During a deploy operation, we would fetch commit objects from the remote
to resolve the version string. If gpg-verify was turned on, we would
fail to pull them if some of the commits were not signed. This is
because we pulled them in batches. We partially address this by only
fetching the HEAD commit on the first pass. This allows `upgrade`
operations to work just as well as `deploy` operations.
Though there is still an issue if we have to traverse farther back than
when signed commits become unsigned (unless they happen to fall on a
batch boundary). We leave that unsolved for now, since that would likely
require a more complex solution and it's not clear whether it's a real
world issue (signers can just retroactively sign commits).
Copy the gpghome from ostree so that we can test GPG-related cases in
our suite.
Closes: #527Closes: #557
Approved by: cgwalters
Add a few more tests to exercise some of the treefile options. We do
need to also expand test-basic.sh itself to sanity-check the structure
of a normal ostree compose. That's up next on the list.
Closes: #548
Approved by: cgwalters
Our current compose tests only use a synthetic `empty.rpm`, but
this really limits usefulness.
Let's make a test suite that requires an internet connection and
downloads Fedora RPMs and does "real" tree composes.
See the updated `tests/README.md` for more information.
This is still a WIP.
Closes: #531
Approved by: jlebon
- Move all the uninstalled, non-destructive tests to check/
- Add a README
- Move utilities to utils/
- Move common files to common/
Closes: #304
Approved by: cgwalters
This works around a potential issue with libsolv if we go down the
rpmostree_get_pkglist_for_root() path. Though rpm has been using the
/usr/share/rpm location (since the RpmOstreeContext set the _dbpath
macro), the /var/lib/rpm directory will still exist, but be empty.
libsolv gets confused because it sees the /var/lib/rpm dir and doesn't
even try the /usr/share/rpm location, and eventually dies when it tries
to load the data.
So we set the symlink now. This is also what we do on boot anyway for
compatibility reasons using tmpfiles.
This also means we don't have to do the /var/lib/rpm --> /usr/share/rpm
transition during the rootfs postprocess (but we still have to clean up
db and lock files).
Also get rid of the unused pkglist variable.
NB: I used the GFile & gs APIs to mesh with the surrounding code.
Closes: #290
Approved by: cgwalters
This comes from the tradition of yum repo files. It's significantly
easier for rpm-ostree users building multiple architectures if some
core templating for the treefiles is built in.
Otherwise, everyone needs to learn about a new wrapper tool for
rpm-ostree, and said tool would need to re-do the same "basearch"
evaluation that is already occuring inside libhif.
This commit also paves the way for introducing `${releasever}`
substitution.
NOTE: This depends on pending changes to libhif git.
Closes: #274
Approved by: jlebon
I was tracking another regression where we seem to have lost
`/usr/etc` contents which manifested as `Labeling with... (null)`
which was clearly wrong.
Now this change actually impacts the test suite - we now (again IMO
correctly) error out if `selinux: true`. The `no-selinux-tag` test
no longer makes sense, so delete it.
We do need more "real" tests that use selinux on and off.
Pull request: #243
Approved by: jlebon
A local yum repository, which contains a dummy .rpm package adding
support for programs needed by the post-process phase, is created as
part of the test.
Since no libc is present, the empty.c program (which is a no-op)
directly invokes the interrupt 0x80 to exit.
The test works only on x86_64, it is skipped on other architectures.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
