Commit Graph

2854 Commits

Author SHA1 Message Date
Jonathan Lebon
020752bfe8 tests/compose: Stop freezing on specific FCOS commit
We're hitting issues with packages getting tagged out of the pool:
https://pagure.io/releng/issue/9281

This in turn means we can't reliably recompose older builds right now,
which breaks our CI. For now at least, let's compose from the latest.
(Note we were already also composing the latest FCOS in the vmcheck
branch.)
2020-02-27 04:50:59 +01:00
Jonathan Lebon
1675058768 initramfs: Fix using local /etc when also replacing kernel
Instead of basing our decision to use the local `/etc` on whether we're
using `dracut --rebuild`, base it directly on a boolean parameter.

This is relevant in the client-side when initramfs regeneration is
requested as well as a kernel override. In such cases, we do want to use
the local `/etc`, but we'd skip that path because we didn't also use
`dracut --rebuild`.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1806588
2020-02-27 04:50:59 +01:00
Jonathan Lebon
14701be788 status: Quote initramfs args if necessary
E.g. if it contains spaces as is the case when one does
`rpm-ostree initramfs --arg=-I --arg='/file1 /file2'`.
2020-02-27 04:50:59 +01:00
Jonathan Lebon
2ae1c04ef8 libpriv/kernel: Fix minor comment typo 2020-02-27 04:50:59 +01:00
Jonathan Lebon
d69a6c82b9 core: Simplify check for kernel package name 2020-02-27 04:50:59 +01:00
Jonathan Lebon
21bf9ec96c app/main: Tell libdnf not to look for dnf.conf upfront
We don't support `/etc/dnf/dnf.conf`, so tell libdnf to not look for it.
This squashes warnings from libdnf (which turn into hard errors when
hacking locally with `G_DEBUG=fatal-warnings`).

As mentioned in the comment, the reason for putting this in `main.c` is
that it controls a global variable, which is used in a few places in
libdnf. So rather than duplicating this across callsites, just set it
upfront. And yeah... we should improve that API.
2020-02-27 04:50:59 +01:00
Jonathan Lebon
69e3af4331 ci/installdeps: Only builddep from spec file
Doing builddep once based on the baked config and then once more from
the spec file can cause issues sometimes. For example, right now the
latest rpm-ostree release uses libmodulemd1, but we want to rebase to
libmodulemd (2.0). And `dnf` will get confused trying to move from one
to the other.

Really, we don't need to builddep from the last release at all, so just
drop that and rely only on the spec file.

Adapt `pkg_install_builddeps` to allow no args to mean only installing
the basic buildroot stuff like `dnf builddep` and `@buildsys-build`.
2020-02-25 16:48:15 +01:00
Jonathan Lebon
4b15c59b77 ci: Move cargo test into ci/msrv.sh
We need `cargo` in our `PATH` and we already do the `PATH=...` dance in
`ci/msrv.sh`. This only worked before because we were inadvertedly
re-installing cargo when calling `ci/build.sh`, which was fixed in the
previous commit.
2020-02-25 16:48:15 +01:00
Jonathan Lebon
2290999251 ci/msrv.sh: Use SKIP_INSTALLDEPS=1 before building
We've already manually installed dependencies higher up. This saves us
some time, but also we don't want the script to e.g. re-install cargo.
(This also works as a short term hack we need to adapt to libdnf moving
to `libmodulemd-2.0` due to `ci/installdeps.sh` not being entirely
idempotent).
2020-02-25 16:48:15 +01:00
Jonathan Lebon
7501b39cbf spec: Bump libmodulemd version requirement
This matches the latest bump in libdnf:
763ccc3e33
2020-02-25 16:48:15 +01:00
dependabot-preview[bot]
c9751ae64d build(deps): bump libdnf from 3fe7935 to 159a4bc
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `3fe7935` to `159a4bc`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](3fe793530e...159a4bce12)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-25 16:48:15 +01:00
dependabot-preview[bot]
547b37465b build(deps): bump libc from 0.2.66 to 0.2.67 in /rust
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.66 to 0.2.67.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.66...0.2.67)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-21 17:00:33 -05:00
dependabot-preview[bot]
265e4500b0 build(deps): bump serde_json from 1.0.47 to 1.0.48 in /rust
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.47 to 1.0.48.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.47...v1.0.48)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-15 07:45:56 +01:00
Colin Walters
b797a42f3e core,kernel: Set up /etc/passwd for dracut
This fixes a longstanding spew of error messages from the initramfs
because we don't have nss-altfiles set up there.  Rather than
trying to do it, just do the dance of re-synthesizing `/etc/passwd`
as it traditionally looks around running dracut, the same as we
do for scripts during core layering.

Yes, this is all a mess and hopefully I'll get to sysusers soon...
2020-02-13 23:26:04 +01:00
dependabot-preview[bot]
9c4e7590b2 build(deps): bump libdnf from 249eda5 to 3fe7935
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `249eda5` to `3fe7935`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](249eda5cae...3fe793530e)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-11 17:00:12 +01:00
dependabot-preview[bot]
4b779befac build(deps): bump serde_json from 1.0.46 to 1.0.47 in /rust
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.46 to 1.0.47.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.46...v1.0.47)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-10 15:53:44 +01:00
dependabot-preview[bot]
dd835e861f build(deps): bump libdnf from 4a7ab08 to 249eda5
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `4a7ab08` to `249eda5`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](4a7ab081ec...249eda5cae)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-10 04:36:00 +01:00
Jonathan Lebon
35818cefd5 Release 2020.1 2020-02-05 23:18:45 +01:00
Colin Walters
5a0d3356ef treefile: Add exclude-packages
In FCOS we have a kola test that basically does `rpm -q python`.
It's...a bit silly to spawn a whole VM for this.  Ensuring that
some specific packages don't get included has come up in a few
cases.

I think FCOS/RHCOS at least will want to blacklist `dnf` for example.
And as noted above, FCOS could blacklist `python`.

One major benefit of doing this inside rpm-ostree is that one
gets the full "libsolv error message experience" when dependency
resolution fails, e.g. blacklisting `glibc` I get:

```
 Problem 79: conflicting requests
  - package coreos-installer-systemd-0.1.2-1.fc31.x86_64 requires coreos-installer = 0.1.2-1.fc31, but none of the providers can be installed
  - package coreos-installer-0.1.2-1.fc31.x86_64 requires rtld(GNU_HASH), but none of the providers can be installed
  - package glibc-2.30-10.fc31.x86_64 is filtered out by exclude filtering
  - package glibc-2.30-7.fc31.x86_64 is filtered out by exclude filtering
  - package glibc-2.30-8.fc31.x86_64 is filtered out by exclude filtering
  - package glibc-2.30-5.fc31.i686 is filtered out by exclude filtering
  - package glibc-2.30-5.fc31.x86_64 is filtered out by exclude filtering
  - package glibc-2.30-10.fc31.i686 is filtered out by exclude filtering
```
2020-02-05 21:02:06 +01:00
dependabot-preview[bot]
70c385633c build(deps): bump libdnf from 367cf8a to 4a7ab08
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `367cf8a` to `4a7ab08`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](367cf8aa97...4a7ab081ec)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-04 13:00:29 -08:00
dependabot-preview[bot]
6b80c95368 build(deps): bump nix from 0.16.1 to 0.17.0 in /rust
Bumps [nix](https://github.com/nix-rust/nix) from 0.16.1 to 0.17.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits/v0.17.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-04 06:24:55 -08:00
Colin Walters
c039b72ebd importer: Use /run instead of /var/run
Translate RPM paths under `/var/run` to `/run` automatically; this
quiets down systemd.  Since we end up running `systemd-tmpfiles`
a few times in FCOS reducing spew here is particularly valuable.

The bug is really in the packages here but...we don't have an
agile process for fixing them.

Note that for this fix to take effect, if you have a `cache/pkgcache-repo`
you'll need to remove it.
2020-02-03 07:50:10 -08:00
dependabot-preview[bot]
1afa96e414 build(deps): bump serde_json from 1.0.45 to 1.0.46 in /rust
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.45 to 1.0.46.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.45...v1.0.46)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-03 07:04:33 -08:00
dependabot-preview[bot]
06f139eb5d build(deps): bump structopt from 0.3.8 to 0.3.9 in /rust
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.8 to 0.3.9.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.8...v0.3.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-03 06:35:53 -08:00
dependabot-preview[bot]
470cd197e2 build(deps): bump indicatif from 0.13.0 to 0.14.0 in /rust
Bumps [indicatif](https://github.com/mitsuhiko/indicatif) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/mitsuhiko/indicatif/releases)
- [Commits](https://github.com/mitsuhiko/indicatif/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-31 06:17:12 -08:00
Colin Walters
17d3d7b313 treefile: Use ref_from_raw_ptr
This was missed in the conversion; just noticed while
working on the code around here.
2020-01-30 20:02:54 -08:00
dependabot-preview[bot]
0c734db457 build(deps): bump structopt from 0.2.18 to 0.3.8 in /rust
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.2.18 to 0.3.8.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.2.18...v0.3.8)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-29 08:15:28 -08:00
dependabot-preview[bot]
839b96ce59 build(deps): bump clap from 2.32.0 to 2.33.0 in /rust
Bumps [clap](https://github.com/clap-rs/clap) from 2.32.0 to 2.33.0.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-28 18:42:50 -08:00
dependabot-preview[bot]
8db7821c30 build(deps): bump nix from 0.15.0 to 0.16.1 in /rust
Bumps [nix](https://github.com/nix-rust/nix) from 0.15.0 to 0.16.1.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-28 18:42:41 -08:00
dependabot-preview[bot]
de21edf4e5 build(deps): bump libdnf from cc36cb7 to 367cf8a
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `cc36cb7` to `367cf8a`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](cc36cb7492...367cf8aa97)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-28 18:22:29 -08:00
dependabot-preview[bot]
e537de8321 build(deps): bump indicatif from 0.11.0 to 0.13.0 in /rust
Bumps [indicatif](https://github.com/mitsuhiko/indicatif) from 0.11.0 to 0.13.0.
- [Release notes](https://github.com/mitsuhiko/indicatif/releases)
- [Commits](https://github.com/mitsuhiko/indicatif/compare/0.11.0...0.13.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-28 13:45:16 -08:00
Jonathan Lebon
a40014f5a7 ci: Allow submodule bumps from Dependabot
Tweak the logic in `ci-commitmessage-submodules.sh` to allow Dependabot
bumps.
2020-01-28 13:14:47 -08:00
dependabot-preview[bot]
30c2444f19 build(deps): bump libglnx from 470af87 to 5f3d352
Bumps libglnx from `470af87` to `5f3d352`.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-28 13:14:47 -08:00
Jonathan Lebon
7ad4d58bbc ci: Bump compose tests timeout to 45m
Still trying to find the sweet spot on this. I think it may also depend
on how fast/busy the node we get allocated to is.
2020-01-28 11:13:47 -08:00
Jonathan Lebon
2dadc7b03c rust: cargo update
We haven't done one of these in a while.
2020-01-28 11:13:47 -08:00
Jonathan Lebon
c09f5412a5 libpriv/util: Fix human diff printing for upgrades/downgrades
We were basing whether to print the `Upgraded`/`Downgraded` heading on
the iteration count rather than the actual first iteration where a valid
upgrade/downgrade was found. And because of how we print our diff, this
confusingly can make it look like downgrades are part of the same
upgrade section.

Closes: #1821
2020-01-21 18:36:53 +01:00
Jonathan Lebon
2966afe44e core: Mark all repos as "modular hotfixes"
This is a follow-up hack to #1797 to force libdnf to let us use modular
packages as if they were regular packages until we actually support
modules correctly (#1435).

A repo marked as a modular hotfix means that libdnf doesn't try to
filter out modular RPMs from the repo as it usually does.

Resolves: https://pagure.io/releng/failed-composes/issue/717
2020-01-09 15:31:02 +01:00
Jonathan Lebon
654ab64409 ci: Re-org stages and parallelize tests
Build FCOS and run vmcheck in the same container, since it's only used
for that anyway right now. The main advantage is that we save time
provisioning another container and not having to stash and unstash the
FCOS image.

Also, since the compose tests don't actually need to wait for the FCOS
image, start running them in parallel with the FCOS + vmcheck branch.
2020-01-08 16:42:54 +01:00
Jonathan Lebon
9daea46d66 tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.

I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).

(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)

Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)

However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2020-01-08 16:42:54 +01:00
Jonathan Lebon
462a389b3a tests/compose: Move files around
This is mostly cosmetic, though I want the test layout to mirror what we
do for `vmcheck`.
2020-01-08 16:42:54 +01:00
Jonathan Lebon
c59b9de3d4 ci: Run Rust unit tests
We definitely want this too.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
13f554ea5c ci: Bump MSRV to 1.39.0
This is in el8 now.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
37a6a3ab69 Don't require an OSTree repo for --print-only
This makes it easier to flatten a treefile. To do this, we make having a
workdir optional on the Rust side, and on the C side, just move
`--print-only` handling up front.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
932e8eec2a app/compose: Skip bwrap self-test if --download-only[-rpms]
We don't need any privs for just downloading the RPMs.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
c5058c662a app/tree: Drop "YAML" from error message
This helper is used for both JSON and YAML treefiles now.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
1d8537bb03 app/rojig: Explicitly check a rojig section was provided
Let's make that explicit to make it easier to debug. (This works because
right now, we auto-create the spec file on the Rust side at parse time.)
2019-12-20 21:16:24 +01:00
Jonathan Lebon
c7361af2f6 app: Print help messages on stderr
So that we keep `stdout` clean. This is important for `--print-only`.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
28d3fe428f app/compose: Support multiple --add-metadata-from-json
No reason we can't do this. I initially was going to make use of this
for tests, but ended up not needing it.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
0e84776519 libpriv/rojig: Fix unref'ing using wrong function
We were trying to unref a `GHashTable` using `g_ptr_array_unref`.
Something something... rewrite it in Rust.
2019-12-20 21:16:24 +01:00
Jonathan Lebon
7d36ea7ebc rust: Wrap parent directory handling for Path
The parent of a `Path` of `"foobar"` is actually `""`, because
`parent()` wants to return a slice of its buffer:
https://github.com/rust-lang/rust/issues/36861

Just do a trivial wrapper around it to correctly return `"."` instead.

Hit this when trying to pass a treefile in the working directory.
2019-12-20 21:16:24 +01:00