Commit Graph

3420 Commits

Author SHA1 Message Date
Timothée Ravier
6cfdb1f585 countme: Add DNF Count Me support
Add support for the DNF Count Me feature [1,2,3] as a standalone
rpm-ostree subcommand called weekly via a systemd timer.

[1] https://fedoraproject.org/wiki/Changes/DNF_Better_Counting
[2] https://dnf.readthedocs.io/en/latest/conf_ref.html?highlight=countme#options-for-both-main-and-repo
[3] https://github.com/rpm-software-management/ci-dnf-stack/blob/master/dnf-behave-tests/features/countme.feature
2021-01-13 14:53:50 -05:00
Colin Walters
0e9941495e Add a Rust helper to create a sealed memfd, use in shlib backend
Prep for other code using memfds.
2021-01-12 22:07:18 -05:00
Colin Walters
241cce1ec4 testutils: Port to cxx-rs
Another easy port like cliwrap.
2021-01-12 19:35:59 -05:00
Luca BRUNO
bdf8269dfa libpriv/passwd: move passwd database to Rust
This moves to Rust the in-memory structure holding passwd entries
(users and groups).
2021-01-12 18:50:30 -05:00
Colin Walters
803e4db50c journal: Add filters for _UID=0
This strengthens our journal filtering to ensure that we won't
read entries from e.g. a malicious process logging with a
syslog identifier `ostree`.
2021-01-12 17:13:00 -05:00
Colin Walters
6c66bf1072 Rename livefs → apply-live in more places
Keep up the renaming momentum by using the new name in more
places.

Prep for further work.
2021-01-12 03:15:49 -05:00
Colin Walters
2247e7af62 configure: Update contact email
atomic-devel@ is dead.
2021-01-11 15:10:07 -05:00
Colin Walters
5b19195fe8 With vendored source, use cargo build --offline
This way we're sure it will build in e.g. Koji.  Right now
it's annoying to test that locally; one needs to explicitly
create a no-network container to do so strictly.  But
cargo has a convenient `--offline` flag, and nothing else
in our build stack should touch the network.
2021-01-11 15:10:07 -05:00
Colin Walters
9f590fa155 Also print Rust-side features/cfg in rpm-ostree --version
Let's make it obvious whether the `fedora-infrastructure` feature
is compiled in.
2021-01-11 13:03:04 -05:00
Colin Walters
29d051e895 Add fedora-integration: Support override replace https://bodhi/...
This adds support for e.g.:

```
$ rpm-ostree override replace https://bodhi.fedoraproject.org/updates/FEDORA-2020-2908628031
```

This will find the Koji builds from the listed update, download
all the RPMs (that aren't debuginfo) and pass them for overrides
in the same way we support `override replace http://somewebserver/foo.rpm`
now.

We also support directly linking a Koji build:
```
$ rpm-ostree override replace https://koji.fedoraproject.org/koji/buildinfo?buildID=1625029
```

Bodhi has a modern HTTP+JSON API, and the lack of a Koji equivalent
drove me to create https://github.com/cgwalters/koji-sane-json-api
and we currently depend on an instance set up in the OpenShift CI
cluster.

I hope it shouldn't take long to deploy this in Fedora Infra,
but I don't want to block on it.

Also notably this still downloads *all* the other RPMs even
ones that aren't installed.  Handling that truly correctly
would require moving this logic to the daemon and core.

All of this functionality is keyed off a `cfg(feature = "fedora-integration")`
that is detected by a Rust `build.rs` which parses the build environment's
`/etc/os-release` for now.
2021-01-11 13:03:04 -05:00
dependabot[bot]
1f408bd396 build(deps): bump libdnf from 2ee2a47 to 45981d5
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `2ee2a47` to `45981d5`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](2ee2a47e19...45981d5f53)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-11 03:31:29 -05:00
Colin Walters
9d080fc1fa Port Rust journal bits to cxx-rs
This is cleaner if we move the "catch and print errors" into
the Rust side.  We lose the "print errors as red" part but
eh we can readd that later Rust side.
2021-01-09 11:41:41 -05:00
Colin Walters
2b9b83d9ce tests: Fix rsync of base rpmdb with sqlite
Apparently small rpmdb changes can cause the size to stay the
same due to preallocation, and rsync defaults to skipping files
based on (name, size, mtime).  It's really ostree's mtime canonicalization
that's unfortunate here.

Anyways, we obviously don't care about performance here so use
`-I` to disable that rsync check.

(Also remove the `mkdir -p` since it's not necessary since a long time)

Closes: https://github.com/coreos/rpm-ostree/issues/2435
2021-01-09 10:13:20 -05:00
Colin Walters
4b233daca7 Port composepost to cxx-rs
This one was easy.
2021-01-07 20:50:43 -05:00
Colin Walters
52eaa6b6b6 Port cliwrap FFI to cxx-rs
The example of how the `cliwrap_entrypoint()` function
can just be directly bound with this is a great
example of the cleanup.
2021-01-07 17:05:41 -05:00
Colin Walters
92a1fa5bc3 Complete move to cxx-rs for utils
This makes the C++ side a bit uglier because the "variable mapping"
is more Rust-native but we only use it Rust side anyways.
(We can't yet move the basearch bits to rust because it depends on
 libdnf, which requires buildsystem unification)

But all the unsafe FFI conversion drops out, as do the duplicated
C unit tests.
2021-01-07 11:46:52 -05:00
Colin Walters
cba87eaf87 Move some client argument handling to Rust
Prep for the [fedora-integration PR](https://github.com/coreos/rpm-ostree/pull/2420).

This also generalizes the "fetch URL to tempfile" code into
supporting multiple at once - it's much more efficient to do
it that way because we can reuse a TCP connection to servers,
parsed certificates etc.
2021-01-06 12:40:06 -05:00
Colin Walters
60d7a89aaa util: Remove #ifdef __cplusplus
Now that the whole core codebase is "compile in C++ mode",
remove the ifdef and move the C++-specific `#include <>`s
up so it's more obviously supported to add new
C++ code (if necessary).
2021-01-06 04:05:39 -05:00
Colin Walters
fd88e8aba7 Port initramfs Rust code to cxx-rs
Also adds support for passing through `GCancellable`.
2021-01-05 19:59:24 -05:00
Colin Walters
14cafd81e9 lib: Cleanup #include <>s
We previously severed the dependency between the shared library
and our internal code, but the library still included internal
headers even though they weren't used.

This broke my attempt to natively use C++ in some headers.

Drop the unused headers and clean up the `#include`s.
2021-01-05 18:13:19 -05:00
Colin Walters
62464ed438 Use cxx-rs for live-apply
(I'm going to start calling it `live-apply` instead of "livefs")

On one hand, this drops the unsafe bindgen glue.  On the other
hand it demonstrates a notable current ergonomic shortfall of
cxx-rs in that it doesn't support `Option<T>`, so we represent
optional strings by converting them to empty strings.

(Relatedly I discovered in C++ there is `std::optional<>` but it's in C++17.
 I assume we can depend on that but let's for now match libdnf's
 usage of `-std=c++14` because we know that works everywhere)
2021-01-05 10:26:00 -05:00
Colin Walters
29c78c420e Bump to cxx-rs 1.0.20
This gives us `c_str()`.
2021-01-05 10:26:00 -05:00
Colin Walters
ec1e248b11 Add support for some GObject bridging to cxx-rs
cxx-rs has support for bridging types, but it's more awkward
for us because those types are defined in other crates, so
we need to do a newtype dance.  Further cxx-rs doesn't currently
support automatically generating wrappers, so add a custom
`gobj_wrap()` for now.
2021-01-05 10:26:00 -05:00
Colin Walters
85f4ce448f ci: Don't run autotools twice
I started writing a comment about why we run autotools twice,
then decided that was *much* uglier than extracting the binding
rules to a separate `Makefile`.  But I forgot to go back
and remove the first part, so do that now and fix up the comment.
2021-01-04 15:36:22 -05:00
Colin Walters
b5ad234b59 Add a partial reimplementation of origin parsing in Rust
Let's land the half-completed Rust code so we can iterate on
and improve it while other patches land that might conflict
with work on the C++ side.
2021-01-04 13:54:38 +01:00
Colin Walters
15811eca4f Makefile.bindings: Use content-based change detection
We've had a problem for a long time that e.g. `rpmostree-rust.h`
didn't have dependencies on the relevant Rust sources, so changing
it required `rm -f rpmostree-rust && make` which is very confusing.

Improve things here for both it and cxx.rs so that we use
content-based change detection (this is a default in more modern
build systems).  This way e.g. `touch Cargo.toml && make` won't
implicitly result in rebuilding all of the C/C++ side.
2021-01-04 13:17:35 +01:00
Colin Walters
08c414f897 Rework bindgen/cxx.rs usage and CI build
cxx.rs (aka cxxbridge) and cbindgen are
both generating source code.  Since the last release
we've introduced the former, and we need to ensure
that the generated cxx.rs source ends up in release tarballs
the same way as the cbindgen code.

Rationalize and clean up the binding infrastructure.
Drop support for the vendored cbindgen which we
weren't actually using:
Closes: https://github.com/coreos/rpm-ostree/issues/2392

Move the cxx-rs and cbindgen bits into the same place,
and update our CoreOS CI build to use a separate `Makefile.bindings`
that just generates the code, so our CI still "works like"
a main Koji RPM build.
2021-01-04 13:17:35 +01:00
Colin Walters
562acedbaa Move "ignored script list" to Rust, drop gperf
Rust has a nice crate for doing perfect hashing.  Move that
code into Rust and drop the dependency on `gperf`.  This also
helps move away from Autotools.
2020-12-24 16:37:04 +01:00
Jonathan Lebon
c96ad53dcf ci: Add ~/.cargo/bin to PATH in Prow path as well
This matches what we do in CoreOS CI.
2020-12-23 22:52:40 +01:00
Jonathan Lebon
8461c053c1 ci/install-extra-builddeps: Drop --root=/usr arg
AFAICT, we just need `cxxbridge` to be in the `PATH` of the building
user. Let's avoid targeting privileged paths so devs can just run this
script directly without `sudo`.
2020-12-23 22:52:40 +01:00
Colin Walters
6579ab791b Use cxx-rs instead of gresources
The way gresources work using a constructor function started
failing when I was refactoring the build system, and I couldn't
figure it out.  It's just easier to use Rust for this which
has nice toolchain-integrated functionality for this.
2020-12-23 17:45:29 +01:00
Colin Walters
8ab6a1f4f0 Use cxx-rs for utils.rs download_to_fd
A step towards converting all of utils.
2020-12-23 17:45:29 +01:00
Colin Walters
9565c19ef0 Use cxx-rs for core.rs
This is much better than bindgen because it's fully safe.  It's
much more ergonomic too:

 - Invoke Rust methods-on-structs just like C++ methods-on-structs
 - Rust `Result<>` is translated automatically to exceptions

See https://cxx.rs/context.html for more.
2020-12-23 17:45:29 +01:00
Colin Walters
9f19ed2ac8 ci: Introduce install-extra-builddeps.sh
We need to cleanly split off "test dependencies" that we
install inside the cosa pod from builds (where we won't
have `cargo`) from the build time where we use the cosa
buildroot image.

Prep for using https://cxx.rs
2020-12-23 17:45:29 +01:00
Colin Walters
9b13ce3906 buildsys: Make one big rpmostreeinternals.la, enable unit tests again
In the previous buildsytem rework we disabled the unit tests because
of linking problems.  Now I realized that a simple solution is
to continue to build one big object, just make it an internal
static library and have a tiny "stub main" that delegates to an entrypoint.

That's basically what the C unit tests are - an alternative `main()`
with some extra code.
2020-12-23 17:45:29 +01:00
Colin Walters
485dbe8472 apply-live: Print a package diff
The cool thing about this is it emphasizes how "integrated" apply-live
is versus the uncontrolled `rpm-ostree usroverlay`.  We're still
tracking the state of things reliably and can print it.
2020-12-23 16:23:43 +01:00
Colin Walters
4c3d3fcb45 build-sys: Drop autoconf bits to find dbus service directory
We set it to the same value in both cases.  There's no reason
to require it at build time anyways.

Part of thinning out the Autotools side in prep for using Cargo.
2020-12-22 17:14:19 -05:00
Colin Walters
352d9f816f build-sys: Remove RHEL7 autotools backport
We aren't going to be shipping major new versions to RHEL7
ever again, so drop this compat definition.

Drop this as part of thinning out our Autotools build system
in preparation for moving to `build.rs`.
2020-12-22 17:14:19 -05:00
Colin Walters
70e7cd9bbb build-sys: Remove --with-bubblewrap
This was only needed for RHEL7; RHEL8 and above ships bwrap with
the OS.

Drop this as part of thinning out our Autotools build system
in preparation for moving to `build.rs`.
2020-12-22 17:14:19 -05:00
Colin Walters
de749fb2a5 build-sys: Remove --enable-installed-tests
This has never been used here; it was added
in f93cb5bcc5
and then we went in a different direction for testing.
2020-12-22 17:14:19 -05:00
Colin Walters
509ea0dbe1 Port rpmostreed-os-experimental to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
1d610604a2 Port rpmostree-package-variants to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
a3aa9c29d2 Port rpmostreed-errors to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
f0df041b95 Port rpmostreed-sysroot to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
57a08735d5 Port rpmostreed-utils to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
b00e87facc Port rpmostreed-daemon to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
d8263d2c38 Port rpmostree-builtin-compose to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
642c8b1453 Port rpmostree-builtin-kargs to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
9a91eecbb0 Port rpmostree-polkit-agent to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00
Colin Walters
665bb07994 Port rpmostree-override-builtins to C++
Prep for using https://cxx.rs/
2020-12-22 16:36:13 -05:00