Commit Graph

3441 Commits

Author SHA1 Message Date
Colin Walters
0f325b3490 Makefile.bindings: Also generate pure rust/cxx.h
This is necessary in order to use `rust::` in our header files
that are also used by bindings.
2021-01-27 19:36:34 -05:00
Colin Walters
56fde36e8d ci: Re-enable make check
This seems to work for me; we want to run the C unit tests.
2021-01-27 19:09:33 -05:00
Colin Walters
173db90941 tests: Fix compliation
This makes `make check` go again.
2021-01-27 19:09:33 -05:00
Kelvin Fan
1c3695c8e2 daemon/rpmostreed-daemon: Look up sd unit if caller not registered
Registration through `RegisterClient` is not mandatory today; for
example, Zincati does not register itself currently.
Look up systemd unit of caller if it is not already registered.
2021-01-27 18:19:50 -05:00
Colin Walters
cc230e25a0 tests/layering-fedorainfra: Update rpm-ostree build
The previous build was GC'd; unfortunately it's very nontrivial
to make this test truly robust over time because FCOS changes;
we might sometimes have an outstanding update, other times might
not etc.

Let's just sanity check the commands; ultimately they're
thin wrappers around just downloading packages so we don't need
deep checks.
2021-01-27 17:12:30 -05:00
Jonathan Lebon
c98c227c0f extensions: Write JSON to output dir
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.

This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
2021-01-27 00:44:42 +01:00
Colin Walters
ea81a1ee6e compose: Remove unused treefile_rs parameter
Prep for further cleanup.
2021-01-26 22:25:43 +01:00
dependabot[bot]
55ea74a5bd build(deps): bump libdnf from 1ebad38 to e92a15a
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `1ebad38` to `e92a15a`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](1ebad384f6...e92a15a5da)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-26 19:20:46 +01:00
Colin Walters
80dc572514 rust: Add some allow(dead_code)
One is only used in the unit tests right now, the other we'll
likely use soon.
2021-01-26 18:28:58 +01:00
Colin Walters
3f9a81b2c1 libmain: Avoid use of goto out
Prep for reworking our binary entrypoint to be Rust and not C++.

We need to split up main into sub-pieces; but before we do
that let's avoid `goto out` and rework into declare-and-initialize
style which cleans things up here.
2021-01-26 17:01:12 +01:00
Colin Walters
c3da95a119 Move nevra parsing to Rust, first use of extern "C++"
Until now with cxx-rs we'd been using it effectively as a better
cbindgen - we're exposing Rust code to C++ safely.  This is
the first case of having Rust calling back into C++ using cxx-rs.
2021-01-26 13:47:56 +01:00
Colin Walters
4ce5f42d12 rust: Link to our C/C++ dependencies and internal library
This allows us to fully use cxx-rs with `extern "C++"`.  Now
we do call back into the C/C++ today, but it only works outside
of cargo/Rust's knowledge.  Most notably, it means we can't
use our C code in `cargo test`.  And that's a problem
for moving some C/C++ code to Rust, because we want to port
the unit tests too.

For now, re-declare our dependencies and part of the build
system inside the Cargo build.  However, this is also
an important step towards using Cargo as our *sole* build
system.

We don't add build dependencies too often, so the short
term duplication should be OK.

However, a major unfortunate side effect of this is that
we now need to serialize the build process; almost all the
C/C++ comes first (`librpmostreeinternals.la`) and then
the Rust build, then we finally generate the executable
with both.

The only way out of this really is to move more of the
C/C++ build into Cargo, and we probably want to refactor
into internal crates.
2021-01-26 13:47:56 +01:00
Colin Walters
632cecdca6 cxxrs: Work around failure with -Werror=missing-declarations
See https://github.com/dtolnay/cxx/issues/590
2021-01-26 13:47:56 +01:00
dependabot[bot]
e540ca10be build(deps): bump rand from 0.8.2 to 0.8.3
Bumps [rand](https://github.com/rust-random/rand) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-26 12:53:04 +01:00
dependabot[bot]
a04ba7c22e build(deps): bump serde from 1.0.122 to 1.0.123
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.122 to 1.0.123.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.122...v1.0.123)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-26 12:52:57 +01:00
Luca BRUNO
867ba1eba8 rust/passwd: minor cleanup, switch to copy_file_at
This updates file copying to use the newer `copy_file_at` from
openat-ext.
2021-01-26 12:14:59 +01:00
Colin Walters
86ce9ea1f5 ci: Make msrv test do full build + unit tests
I think we did this at some point, but then stopped.
Prep for https://github.com/coreos/rpm-ostree/pull/2413
because we'll need a full build of the C++ side too in order
to `cargo test`.
2021-01-26 10:31:57 +01:00
Kelvin Fan
34746d0428 deploy: Allow empty string argument if --register-driver
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
2021-01-26 01:32:52 +01:00
Jonathan Lebon
bc5a788366 core: Set _dbpath back to /usr/share/rpm after writing rpmdb
We temporarily set the rpmdb path to be an absolute path pointing under
the tmprootfs when writing the rpmdb. This throws off libsolv 0.7.17,
which learned to give the `_dbpath` macro precedence on where the rpmdb
is located:

04d4d036b2

So then the rpmdb sanity-check we do when exiting
`rpmostree_context_assemble()` breaks because it can't find the expected
packages.

Because RPM macros are in global state, there's no elegant way of
setting it just for the rpmdb write operation (short of forking), so
just fix this by setting `_dbpath` back to the correct value after we're
done writing the rpmdb.

Closes: https://github.com/coreos/fedora-coreos-tracker/issues/723
2021-01-25 23:20:53 +01:00
Jonathan Lebon
49318cd950 rust: Fix clippy warnings
Mostly straightforward stuff. It taught me about the `matches!` macro,
which looks really useful.

Wanted to turn this on in CI, but there's still a bunch of clippy
warnings coming from the `cxx.rs` stuff and some of our unsafe blocks.
For example, it wants the `files` arg in `initramfs_overlay_generate` to
be `&[String]` instead of `&Vec<String>` but that would break cxx.rs (it
looks like cxx.rs does support slices, but it would require creating one
from the vector we have to create anyway).
2021-01-25 20:06:53 +01:00
Luca BRUNO
15a32c12d6 rust/passwd: finish moving group and passwd parsers
This drops the remaining C compatibility hops, moving group and
passwd parsing logic fully into Rust, under a `nameservice`
module.
2021-01-25 19:43:54 +01:00
Luca BRUNO
a0e6427bb6 testutils: port to new 'rand' library API 2021-01-25 18:51:53 +01:00
dependabot[bot]
f8ff6075c7 build(deps): bump rand from 0.7.3 to 0.8.2
Bumps [rand](https://github.com/rust-random/rand) from 0.7.3 to 0.8.2.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.7.3...0.8.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 18:51:53 +01:00
dependabot[bot]
966ad4e1ca build(deps): bump cxx from 1.0.20 to 1.0.28
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.20 to 1.0.28.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.20...1.0.28)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 17:29:54 +01:00
Luca BRUNO
85640d44f6 ci: fix install-extra logic
This fixes `install-extra-builddeps.sh` helper, by letting cargo
detect whether the target binary is already present in the
environment with the expected version.
This is in order to avoid mismatches in generated code when the
library version is bumped, and stale binaries are present on
the system.
2021-01-25 17:05:53 +01:00
dependabot[bot]
2e7192466a build(deps): bump rust-ini from 0.16.0 to 0.16.1
Bumps [rust-ini](https://github.com/zonyitoo/rust-ini) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/zonyitoo/rust-ini/releases)
- [Commits](https://github.com/zonyitoo/rust-ini/compare/v0.16.0...v0.16.1)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 16:27:52 +01:00
dependabot[bot]
bd930b7e46 build(deps): bump tempfile from 3.1.0 to 3.2.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 09:19:54 -05:00
dependabot[bot]
2fa928ccca build(deps): bump systemd from 0.8.1 to 0.8.2
Bumps [systemd](https://github.com/jmesmon/rust-systemd) from 0.8.1 to 0.8.2.
- [Release notes](https://github.com/jmesmon/rust-systemd/releases)
- [Commits](https://github.com/jmesmon/rust-systemd/compare/systemd-0.8.1...systemd-0.8.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 13:38:18 +01:00
dependabot[bot]
8a87121a4d build(deps): bump serde_yaml from 0.8.14 to 0.8.15
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.14 to 0.8.15.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.14...0.8.15)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 13:15:13 +01:00
dependabot[bot]
5a1f8f411e build(deps): bump libc from 0.2.81 to 0.2.82
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.81 to 0.2.82.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.81...0.2.82)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 12:02:15 +01:00
dependabot[bot]
7492f44db2 build(deps): bump anyhow from 1.0.35 to 1.0.38
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.35 to 1.0.38.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.35...1.0.38)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 11:22:16 +01:00
dependabot[bot]
b4b13bbc53 build(deps): bump serde from 1.0.120 to 1.0.122
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.120 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.120...v1.0.122)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 10:17:12 +01:00
Kelvin Fan
e7558c3dc9 app/status: Display update driver info
Read from `/run/rpm-ostree/update-driver.gv` and display the update
driver name (and systemd unit if verbose).
2021-01-24 06:28:08 +01:00
Kelvin Fan
a63476d90f Add register-driver option to UpdateDeployment D-Bus API
Record the calling agent's systemd unit and serialize it into a
g-variant file at `/run/rpmostree/update-driver.gv`, along with the
human-readable name of the update driver provided as a string
argument.

Also add the companion `--register-driver` option to the `deploy`
CLI argument.

Closes https://github.com/coreos/rpm-ostree/issues/1747.
2021-01-24 06:28:08 +01:00
Jonathan Lebon
271954a41c app: Add rpm-ostree compose extensions
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.

This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions

The input YAML schema matches the one accepted by that script.

Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
  we've hit in RHCOS (like downloading the wrong `libprotobuf` for
  `usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
  the cosa script uses `reposdir=$dir` which doesn't have the same
  semantics (repo enablement is in that case purely based on the
  `enabled` flag in those repos, which may be different than what the
  rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
  as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
  integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
  the same EVR as the one from a base package: this is helpful in the
  case of extensions which complement a base package, esp. those which
  may not have strong enough reldeps to enforce matching EVRs by
  depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
  extension because IMO it's not at the right level. Instead, we should
  work towards higher-level metadata to represent extensions (see
  https://github.com/openshift/os/issues/409 which is related to this).

Closes: #2055
2021-01-23 17:12:09 +01:00
Jonathan Lebon
40af45814c Revert "core: Allow overriding downloaded RPMs target dir"
This reverts commit 6a3e3d807d.

This isn't as useful in the implementation of a `rpm-ostree compose
extensions` because it doesn't account for locally cached repos where no
downloading happens.

Instead, just let libdnf download the packages to the default location
if it's a remote package and we'll just copy it over to the output dir.
2021-01-23 17:12:09 +01:00
Jonathan Lebon
14d51a6a9e .editorconfig: Adjust for CXX files 2021-01-23 17:12:09 +01:00
Kelvin Fan
9198e50c47 docs/HACKING: Update kola ext test instructions 2021-01-22 21:05:22 +01:00
Colin Walters
71496c0958 rust: Add a bit more info on cxx, move up StringMapping
Docs are good, and move up `StringMapping` since multiple things
are likely to use it.
2021-01-22 03:00:21 -05:00
Colin Walters
a4487578a7 Remove some uses of goto out
All of these cases are actually fine, but in general we
can't use `goto out` since we started using C++ exceptions
which will skip that control flow.
2021-01-21 21:01:45 -05:00
Jonathan Lebon
ee34a2d57f docs/HACKING: drop reference to make vmcheck HOSTS=...
This is no longer supported.

Keep and rework the bit about `make vmsync`.

Also drop the bit about FAHC and CAHC.
2021-01-21 18:55:22 -05:00
Kelvin Fan
f8efa6808d HACKING: Update vmcheck instructions
Following c7a9c3b1dd,
it is no longer necessary to provision your own VM.
2021-01-21 17:16:47 -05:00
Colin Walters
01e6c56415 rust/fedora_integration: Support export RPMOSTREE_KOJI_JSON_API_HOST=...
The service where this is hosted was intended to be temporary; support
overriding it so if it goes down in the future people can at
least use a systemd unit file override to change it easily.
2021-01-21 12:46:32 -05:00
Jonathan Lebon
91b48be098 ci: Set RPM_BUILD_NCPUS when building RPMs
Otherwise, it defaults to `_SC_NPROCESSORS_ONLN` (via `%make_build` ->
`%_smp_mflags` -> `%_smp_build_ncpus` -> `%{getncpus}` ->
48c0f28834/rpmio/macro.c (L583)).
And that's going to be wrong in Kubernetes because we're constrained via
cgroups.

The `%_smp_build_ncpus` macro allows overriding this logic via
`RPM_BUILD_NCPUS`.

See: https://github.com/coreos/coreos-ci/issues/23
See: https://github.com/coreos/coreos-assembler/pull/632
See: https://github.com/coreos/coreos-assembler/pull/1287
2021-01-21 11:57:13 -05:00
dependabot[bot]
9138739c49 build(deps): bump serde from 1.0.118 to 1.0.120
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.118 to 1.0.120.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.118...v1.0.120)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-21 08:03:12 -05:00
dependabot[bot]
a974d95288 build(deps): bump serde_json from 1.0.60 to 1.0.61
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.60 to 1.0.61.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.60...v1.0.61)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-21 06:06:56 -05:00
Colin Walters
a9a4e022e6 rust: Port history to cxx-rs
Just keeping up momentum on the porting, we're getting closer
to being able to drop cbindgen entirely.
2021-01-20 17:22:29 -05:00
Kelvin Fan
7586503ee2 vmcheck/test-misc-2: Fix $cursor variable
Minor cleanup.

Follow up from https://github.com/coreos/rpm-ostree/pull/2461/.
2021-01-20 15:20:41 -05:00
Colin Walters
402912e030 Make make check work again
Our CI isn't running the C unit tests because it goes via RPM,
and while we could potentially add `%check` there...I don't
quite want to do that right now since it also runs the Rust
tests which means we rebuild all the Rust code again in debug
mode etc.

Change the C unit tests to compile in C++ mode, which is
enough for local testing.

Longer term I think the C unit tests will go away in favor
of Rust tests.
2021-01-20 13:10:14 -05:00
dependabot[bot]
70cfb765f6 build(deps): bump libdnf from 45981d5 to 1ebad38
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `45981d5` to `1ebad38`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](45981d5f53...1ebad384f6)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-20 11:32:41 -05:00