Commit Graph

2826 Commits

Author SHA1 Message Date
Jonathan Lebon
11af9c41d3 composeutil: Return NULL instead of FALSE
Even though C happily accepts 0 for NULL, this is ambiguous with our
coding style. E.g. here we're actually returning a pointer, not a
gboolean. So let's be less misleading.
2020-04-29 23:40:49 +02:00
Jonathan Lebon
016ad6b424 compose: Allow specifying lockfile-repos only
For Fedora CoreOS production streams, we want to *only* source from
lockfile repos. But right now, rpm-ostree doesn't allow not specifying
`repos`. Relax this restriction and just check that at least one of the
two was provided.
2020-04-29 23:40:49 +02:00
dependabot-preview[bot]
2d65b86922 build(deps): bump structopt from 0.3.13 to 0.3.14 in /rust
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.13 to 0.3.14.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.13...v0.3.14)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-27 20:07:54 +02:00
Colin Walters
a238ca9c76 status: Don't output AutomaticUpdates: disabled by default
Pre-FCOS we made an effort for automatic updates but nowadays
with Fedora CoreOS we generally expect people to be using zincati.

Until we fix the "agent registration" problem:
https://github.com/coreos/rpm-ostree/issues/1747
Let's not confuse people by printing `AutomaticUpdates: disabled`.

Only print if it's set to a value in non-verbose mode.
2020-04-22 16:45:05 +02:00
dependabot-preview[bot]
a70da06b71 build(deps): bump libdnf from 5339635 to 2750d81
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `5339635` to `2750d81`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](533963525a...2750d81ca8)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-22 08:30:54 -04:00
Colin Walters
c94639f546 ci: Explicitly fetch before build
See https://github.com/coreos/coreos-assembler/pull/1379
2020-04-21 14:02:16 -04:00
Jonathan Lebon
682aad249b app/compose: Fix help string for --ex-lockfile-strict
Minor follow-up to #1858. Make the help string here more helpful and
accurate.
2020-04-20 17:41:09 -04:00
dependabot-preview[bot]
dbb776f17d build(deps): bump libc from 0.2.68 to 0.2.69 in /rust/libdnf-sys
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.68 to 0.2.69.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.68...0.2.69)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-20 11:29:45 -04:00
dependabot-preview[bot]
53f8faa3b0 build(deps): bump libc from 0.2.68 to 0.2.69 in /rust
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.68 to 0.2.69.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.68...0.2.69)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-20 10:37:15 -04:00
Jonathan Lebon
3ec5e2878d manifest: Add lockfile-repos field
In Fedora CoreOS, we have a "coreos-pool" repo from which all packages
in lockfiles are tagged for reproducible builds. This repo is shared
across all streams, including those on f31 and f32.

Thus, it makes no sense for composes to ever pick packages unconstrained
from the pool without being guided by a lockfile. Otherwise, one can
easily end up with e.g. f32 packages in an f31 compose.

Add a new `lockfile-repos` for this which is only used for fetching
lockfile packages and nothing else. For example, this will allow
`cosa fetch --update-lockfile` to Just Work as expected by only fetching
new packages from regular yum repos.
2020-04-19 09:17:17 -04:00
Colin Walters
0d57ab9117 ci: Actually run kola tests
Noticed in https://github.com/coreos/rpm-ostree/pull/2052#issuecomment-613694719
2020-04-18 13:52:34 -04:00
Jonathan Lebon
53456730bf compose: Add --ex-lockfile-strict
Today, lockfiles only restrict the NEVRA of specifc package names from
which libsolv can pick. But nothing stops libsolv from picking entirely
different packages which still satisfy the manifest requests.

This was mostly a theoretical issue in Fedora CoreOS, but became reality
with the addition of Fedora 32 packages in the pool. libsolv would
happily try to pick e.g. `libcurl-minimal` from f32 instead of sticking
with the f31 `libcurl` from the lockfiles:

https://github.com/coreos/fedora-coreos-streams/issues/75#issuecomment-610734584

(But more generally, see
https://github.com/coreos/fedora-coreos-tracker/issues/454).

Let's add a `--ex-lockfile-strict` mode, which in CI and production
pipeline build contexts will require that (1) *only* locked packages are
considered by libsolv, and (2) *all* locked packages were marked for
install.

One important thing to note here is that we don't short-circuit libsolv
and manually `hy_goal_install` lockfile packages. We want to make sure
the treefile is still canonical. Strict mode simply ensures that the
result agrees with the lockfile.

That said, even in developer contexts, we don't want the
`libcurl-minimal` issue that happened to be triggered. But we still want
to allow flexibility in adding and removing packages to make hacking
easier. I have some follow-up patches which will enable this.
2020-04-17 15:48:40 -04:00
Colin Walters
e41a8ab26f Add support for wrapping binaries (rpm, dracut, grubby)
We need to be friendlier to people who are transitioning from
"traditional" yum managed systems.  This patchset starts to lay
out the groundwork for supporting "intercepting" binaries that
are in the tree.

For backwards compatibility, this feature is disabled by default,
to enable it, one can add `cliwrap: true` to the manifest.

To start with for example, we wrap `/usr/bin/rpm` and cause it
to drop privileges.  This way it can't corrupt anything; we're
not just relying on the read-only bind mount.  For example nothing
will accidentally get written to `/var/lib/rpm`.

Now a tricky thing with this one is we *do* want it to write if
we're in an unlocked state.

There are various other examples of binaries we want to intercept,
among them:

 - `grubby` -> `rpm-ostree kargs`
 - `dracut` -> `rpm-ostree initramfs`
 - `yum` -> well...we'll talk about that later
2020-04-15 16:22:57 +02:00
Jonathan Lebon
357c527320 core: Use hy_query_run_set() for excludes
Instead of manually recreating the packageset ourselves.
2020-04-15 15:18:16 +02:00
Jonathan Lebon
af86fcd649 tests/compose: Go back to freezing FCOS commit
The garbage collection issue should be fixed now, and it's just nicer on
developers' cache to stay on the same commit. And again, it's a nice
sanity-check to know that we're always able to compose an older tree.
That said, we probably should still bump this from time to time.

While we're here, add some comments for making it easier to match `popd`
calls with the original `pushd`.
2020-04-15 15:18:16 +02:00
Jonathan Lebon
bca19d74e8 tests/compose: Don't use lockfiles by default
Otherwise, it muddles testing in `test-lockfile.sh` where we want to be
in full control of all the lockfiles fed to `rpm-ostree compose tree`.
2020-04-15 15:18:16 +02:00
Jonathan Lebon
e6986d85cd core: Factor out functions to enable/disable repos
Prep for future patch.
2020-04-15 15:18:16 +02:00
Jonathan Lebon
67662f10d0 app/compose: Rename lockfile variables
Minor cosmetic change; rename the variables so they match the name of
the options they represent.
2020-04-15 15:18:16 +02:00
dependabot-preview[bot]
d1a42777fe build(deps): bump structopt from 0.3.12 to 0.3.13 in /rust
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.12 to 0.3.13.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-14 17:41:08 +02:00
dependabot-preview[bot]
9c9458aed1 build(deps): bump libdnf from a5e5f5a to 5339635
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `a5e5f5a` to `5339635`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](a5e5f5aad5...533963525a)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-14 17:13:59 +02:00
Colin Walters
770856d018 tests: Start converting some bits into kola ext framework
Start the ball rolling on converting some of our tests into
the coreos-assembler/kola framework:
d940420b78/mantle/kola/README-kola-ext.md

The nondestructive ones are easy.
2020-04-09 23:07:45 +02:00
dependabot-preview[bot]
6a0570baa9 build(deps): bump libdnf from 466752c to a5e5f5a
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `466752c` to `a5e5f5a`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](466752c71b...a5e5f5aad5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-09 16:16:14 +02:00
Colin Walters
b54b50c6a4 core: Add error prefixing when we fail to fetch an rpm-md repo
Someone in Freenode #silverblue hit
https://discussion.fedoraproject.org/t/rpm-ostree-giving-error/772
again, the problem would have been slightly more obvious
with this I hope.

(`Read only repository` could also be an OSTree one e.g.)
2020-04-09 10:51:38 +02:00
Colin Walters
8a172a2e05 rust: rustfmt(*) and (re)add a CI check for it
We haven't been consistent about doing this; I personally
think rustfmt is a big aggressive with the line wrapping
but eh, consistency is more important.

And heh so I tried to `git push --set-upstream cgwalters` and
that failed because there was an already extant `rustfmt`
branch from a while ago...looking at that code it got lost
in the CI refactoring - we're not running `build-check.sh`
at the moment.

Move the rustfmt bits into `codestyle.sh` which is closer
to where it should be anyways.
2020-04-08 02:52:30 +02:00
dependabot-preview[bot]
ea17ec9df1 build(deps): bump serde_json from 1.0.50 to 1.0.51 in /rust
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.50 to 1.0.51.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.50...v1.0.51)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-07 19:45:12 +02:00
Jonathan Lebon
3a6044a44b rust: move libdnf-sys module to its own crate
This is just a cleaner arrangement to make the separation more explicit.
It also matches what most other wrapper crates do.

One advantage of this is that we can tell cbindgen directly that we
don't want it to ever export symbols from `libdnf-sys`.

Related discussions in:
https://github.com/coreos/rpm-ostree/pull/2047
2020-04-07 19:01:02 +02:00
Colin Walters
66816eeabe rust/libdnf_sys: Use uninstantiable type rather than wrapper struct
When we ran rustfmt, it converted our bare `extern` blocks to
`extern "C"` which has a different meaning apparently.

This caused cbindgen to try to interpret the structs, and it barfed
on the newtype void wrappers.

Looking at libgit2-rs, it seems to use these "uninstantiable types"
instead.

Prep for using `rustfmt`.
2020-04-07 17:32:36 +02:00
dependabot-preview[bot]
3e91d5a38e build(deps): bump libdnf from 8330eea to 466752c
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `8330eea` to `466752c`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](8330eea698...466752c71b)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-07 00:55:08 +02:00
dependabot-preview[bot]
86f808d49c build(deps): bump envsubst from 0.1.1 to 0.2.0 in /rust
Bumps [envsubst](https://github.com/lucab/envsubst-rs) from 0.1.1 to 0.2.0.
- [Release notes](https://github.com/lucab/envsubst-rs/releases)
- [Commits](https://github.com/lucab/envsubst-rs/compare/0.1.1...v0.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-06 20:21:54 +02:00
dependabot-preview[bot]
3bc6f4496b build(deps): bump serde from 1.0.105 to 1.0.106 in /rust
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.105 to 1.0.106.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.105...v1.0.106)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-06 18:03:30 +02:00
dependabot-preview[bot]
9cfa01db97 build(deps): bump serde_derive from 1.0.105 to 1.0.106 in /rust
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.105 to 1.0.106.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.105...v1.0.106)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-06 16:52:15 +02:00
dependabot-preview[bot]
b6789026ed build(deps): bump libdnf from 141491e to 8330eea
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `141491e` to `8330eea`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](141491eed6...8330eea698)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-02 21:36:28 +02:00
dependabot-preview[bot]
a97c8977aa build(deps): bump libdnf from 98ad20a to 141491e
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `98ad20a` to `141491e`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](98ad20ad33...141491eed6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-01 15:53:15 +02:00
Colin Walters
bf8f295122 rust: Fully remove failure crate
I previously ran out of steam in the switch and wanted to
get the PR out for feedback before continuing, but it turns
out I basically stopped 2 meters from the finish line.  Completing
the switch from `failure` → `anyhow` was quite easy.
2020-04-01 14:53:08 +02:00
Colin Walters
430f92c382 rust: Start porting from failure to anyhow
The failure crate isn't actively developed anymore.  The
main benefit of `anyhow` is it uses the standard error type.
More info:
https://docs.rs/crate/anyhow/1.0.28

Start the porting process.

Note that the `envsubst` crate has a public dependency on
`failure`, so we need to start mapping its errors.
2020-04-01 10:34:28 +02:00
dependabot-preview[bot]
2b0e08ac9c build(deps): bump serde_json from 1.0.48 to 1.0.50 in /rust
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.48 to 1.0.50.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.48...v1.0.50)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-31 17:17:44 +02:00
dependabot-preview[bot]
1124a982c8 build(deps): bump libdnf from 1796b8c to 98ad20a
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `1796b8c` to `98ad20a`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](1796b8cba4...98ad20ad33)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-30 15:36:55 +02:00
dependabot-preview[bot]
5cb92511ed build(deps): bump libdnf from f68a9d8 to 1796b8c
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `f68a9d8` to `1796b8c`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](f68a9d8d2e...1796b8cba4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-26 20:38:51 +01:00
dependabot-preview[bot]
469e6e963a build(deps): bump libdnf from c606657 to f68a9d8
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `c606657` to `f68a9d8`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](c606657d46...f68a9d8d2e)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-23 20:15:33 +01:00
Colin Walters
659fb0b23a importer: Quote filenames with spaces for tmpfiles.d
This way we handle filenames with spaces in `/var` in general,
like `/var/app/foo bar`, but *also* the special `/opt/foo bar`
translation bits.

I saw this bug and thought "oh that'd be easy".  But hoo boy
did it take me down a rat's nest.  The first thing was verifying
that `systemd-tmpfiles` supports any kind of quotation/escaping; it does.
The next thing was figuring out *exactly* what the syntax for that
is and how it works, as it's obviously not widely used.

Writing tests for this ended up being a painful exercise because
of the multiple levels of shell script, e.g. our `build_rpm` shell
script ends up being inlined into RPM specs, which then interprets
again...and not to mention the usual annoying issues with `ssh`
eating quotes.

Anyways, all that and:
Closes: https://github.com/coreos/rpm-ostree/issues/2029
2020-03-23 19:16:36 +01:00
dependabot-preview[bot]
466f66fcd4 build(deps): bump serde from 1.0.104 to 1.0.105 in /rust
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.104 to 1.0.105.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.104...v1.0.105)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-21 22:37:42 +01:00
dependabot-preview[bot]
a4b480f4ec build(deps): bump serde_derive from 1.0.104 to 1.0.105 in /rust
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.104 to 1.0.105.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.104...v1.0.105)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-20 15:06:04 +01:00
Colin Walters
59a804467d core: Add error prefixing in import path
Seeing this in the FCOS pipeline:
```
Downloading from 'fedora-coreos-pool'... done
error: cannot open Packages database in /proc/self/fd/21/usr/share/rpm
Importing packages... done
error: Can't stat fd 38
```

The first error is librpm...which, is somehow not fatal?  It
also appears to be swallowing the underlying real error.

For the second had to Google search it but the main hit for `Can't stat fd` is
in libarchive which led me to this code, which is probably right.
But let's be sure by adding some error prefixing.
2020-03-20 14:28:34 +01:00
dependabot-preview[bot]
9827cc6569 build(deps): bump libglnx from 5f3d352 to bf851fb
Bumps libglnx from `5f3d352` to `bf851fb`.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-19 22:40:39 +01:00
dependabot-preview[bot]
5c5bda4bab build(deps): bump libc from 0.2.67 to 0.2.68 in /rust
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.67 to 0.2.68.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.67...0.2.68)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-19 21:33:58 +01:00
Colin Walters
cf4853e548 syscore: Add some error prefixing
I was getting a dreaded not-quite-specific `syscore cleanup: No such file or directory`
error when hacking on the ostree tests.  I am pretty sure it's the history
code, but let's just do the usual thing and spread the error-prefixing love
in the whole area.
2020-03-19 20:59:59 +01:00
Colin Walters
1b893c1c3a history: Avoid erroring if no history database to clean
The ostree test suite was creating deployments manually
(skipping the rpm-ostree upgrader layer which would write history)
and then calling `rpm-ostree cleanup` which tried to open the
history dir and failed.

Just return early if there's no history directory when we're
asked to clean up.
2020-03-19 20:59:51 +01:00
Jonathan Lebon
e2a8aea11b app/dbus: Suggest journalctl -xe if bus owner changes
In the large majority of cases, the `"Bus owner changed"` error is due
to something going wrong with the daemon rather than D-Bus itself. Let's
give a hint to check the journal so that users can investigate and e.g.
just paste the journal output as part of the initial issue report.
2020-03-19 20:59:41 +01:00
dependabot-preview[bot]
caccd1edfb build(deps): bump libdnf from 608490d to c606657
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `608490d` to `c606657`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](608490d775...c606657d46)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-19 19:25:28 +01:00
dependabot-preview[bot]
7957422d2b build(deps): bump structopt from 0.3.11 to 0.3.12 in /rust
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.11 to 0.3.12.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.11...v0.3.12)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-19 18:31:29 +01:00