Commit Graph

391 Commits

Author SHA1 Message Date
Colin Walters
01b59f20ba daemon: Move a bit more deployment-variant code to Rust
These are "base ostree" things we can move into the Rust
function.
2021-03-08 19:58:18 +01:00
dependabot[bot]
fbe01b403f build(deps): bump serde from 1.0.123 to 1.0.124
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.123 to 1.0.124.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.123...v1.0.124)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-08 11:06:40 -05:00
Colin Walters
3d7ac1d637 rust/client: derive(Debug, Clone)
Zincati wants this and it's a friendly thing to do.
2021-03-08 09:58:56 -05:00
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Colin Walters
8f77970683 client: Add API to fetch base commit metadata
Also desired by Zincati.
2021-03-05 18:20:03 -05:00
Colin Walters
5b647af14a rust/client: Add Deployment/get_base_commit() API
Zincati wants this.
2021-03-05 18:20:03 -05:00
Colin Walters
14c17cfe52 rust/client: Add methods to find/require booted deployment
This is a common need.
2021-03-05 18:20:03 -05:00
Colin Walters
1662f246fd rust/client: Add a CliClient with agent ID, require for status
In prep for adding more methods, require the caller to identify
themselves.

For now this is `CliClient` - one could imagine in the future
we actually do direct DBus, but there's a whole other world
of stuff there.
2021-03-05 18:20:03 -05:00
Colin Walters
3354ca9d30 rust/client: Extend with more metadata for zincati
Add more metadata that zincati needs, like `base-commit-meta`
which includes the `fedora-coreos.stream` key and the cosa basearch,
etc.

Also `Derive(Debug)` since it's used in a cache struct that also
derives debug, and that's a friendly thing to do in general.
2021-03-05 05:47:01 -05:00
Colin Walters
0bae05d4c0 apply-live: Fix --target option
It is an option, not a positional.  (Not covered in CI it turns out)
2021-03-04 16:21:51 -05:00
Colin Walters
13804d84a1 Add more client Rust bindings, port apply-live builtin
This adds sufficient infrastructure to fully port the
`rpmostree-builtin-applylive.cxx` client code to Rust.
We just keep a stub entrypoint for now until we port
the rest of `rpmostree-builtin-ex.cxx`, at which point
a lot of C++ files go away.

The "finish" bits move from the daemon-oriented `live.rs`
into a new `rust/src/builtins` directory.  I'd like
to try to more cleanly split up the Rust sources along
core(shared)/client/daemon directories in the future.
2021-03-04 09:17:21 -05:00
Colin Walters
6fd9db246a Add basic Rust CLI bindings for DBus
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.

A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.

My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
2021-03-04 09:17:21 -05:00
Colin Walters
60e605b34e Introduce CxxGObjectArray, use in lockfile code
cxx-rs only supports a few basic types in `Vec<T>`/`CxxVector<T>`
and we need to pass an array of GObjects in a few cases.
Add a wrapper class hack instead of using `u64` so we at least
have some basic safety here and have a convenient place to
grep for later when we want to improve this.
2021-03-02 16:25:32 -08:00
Jonathan Lebon
5c1911445c Finish moving lockfile to cxx.rs
This moves the `ror_lockfile_write` to cxx.rs, which brings us closer to
getting rid of cbindgen now.

There's one massive hack this uses, which is that we pass an array of
pointers to `DnfPackage` and `DnfRepo` objects as u64. We'll want to
circle back and fix that up once either cxx.rs supports natively arrays
of pointers, or we just come up with our own wrapper type for it.

But for now at least, this unblocks the cbindgen transition and hacking
on the lockfile code.
2021-03-02 16:25:32 -08:00
Luca BRUNO
6b13f2596c rust/treefile: add support for check-passwd/groups data entries
This adds treefile support for both `check-passwd` and `check-groups`
entries with "type: data".
2021-03-02 08:08:05 -08:00
Colin Walters
9b3612b3ff cxxrsutil: Generalize cxxrs_bind! macro
This generalizes our binding macro, so we can trivially e.g.
add `Variant` into the entry of bound glib types.
2021-03-02 01:14:30 -08:00
dependabot[bot]
6dd2df19bc build(deps): bump serde_json from 1.0.62 to 1.0.64
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.62 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.62...v1.0.64)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-01 17:46:49 +01:00
Luca BRUNO
de7d20e43b libpriv/passwd: move UID/GID checker to Rust
This ports to Rust the logic for checking whether a directory tree
contains content owned by a given UID/GID.
2021-02-26 00:34:56 +01:00
Luca BRUNO
dbe2d50cf1 treefile: add getters for check-passwd and check-groups
This add default-aware getters for `check-passwd` and `check-groups`.
2021-02-25 19:24:12 +01:00
dependabot[bot]
380ebc5ada build(deps): bump cxx-build from 1.0.31 to 1.0.32
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.31...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-24 23:27:25 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
ce20267b2d ostree_utils: Add a workaround for optional ref
Until we have https://github.com/ostreedev/ostree/pull/2282
in an ostree release and an updated `ostree` crate.
2021-02-23 21:51:22 -05:00
Colin Walters
d8230bfb6d daemon: Move some "deployment variant" generation to Rust
More prep for https://github.com/coreos/rpm-ostree/pull/2388

This was actually also my first time really trying out the
latest gtk-rs `glib::Variant` API, which is one of the major
things we need to use to progress oxidation more.
2021-02-23 13:56:26 -05:00
Luca BRUNO
9b94e85e5b libpriv/passwd: move compose preparation to Rust
This moves passwd/group compose preparation logic to Rust,
dropping all the remaining minor helpers related to JSON parsing,
file stream creation, and entries deduplication.
2021-02-22 11:44:13 -05:00
Luca BRUNO
a1ecc3d583 rust: add missing SPDX-License-Identifier tag
This adds a missing `SPDX-License-Identifier` tag, fixing CI.
2021-02-22 09:07:33 -05:00
Colin Walters
49c55089d2 lockfile: Be robust to non-seconds-since-epoch repodata
We should really have safety comments around our use of `.unwrap()`.
This Google kubernetes rpm-md repo appears to use seconds-since-epoch.
We could try to detect that, probably best done in libdnf.

I think it'd be better to get them to match the informal schema
for this actually though instead.

For now let's just log an error and not crash.

Closes: https://github.com/coreos/rpm-ostree/issues/2600
2021-02-19 15:56:29 -05:00
Colin Walters
f882d6ddff rust: Add SPDX-License-Identifier and validate it in ci/codestyle.sh
It turns out we accidentally added GPL'd code into the Rust
side, which wasn't intentional on my part and I think it's since
been copied around.

Honestly I think half of the problem is the gigantic
"blah blah blah GNU General blah blah" just makes people's eyes
glaze over.  In contrast the `SPDX-License-Identifier` is short
and obvious.

So let's validate that in CI.

This follows a similar change in ostree:
https://github.com/ostreedev/ostree/pull/1439

If we merge this I'll do the C/C++ side too after that.
2021-02-19 15:56:23 -05:00
Colin Walters
27f9734300 apply-live: Delete code to check for live commit post-txn
Per discussion in
https://github.com/coreos/rpm-ostree/pull/2581#discussion_r578621458
it is inherently racy and we'll do a followup fix to write a
ref.
2021-02-19 09:08:22 -05:00
Colin Walters
246f02fd47 apply-live: Move client-side finish to Rust
Now that we have an RPM diff+printing binding, we can
move the client side postprocessing for `apply-live`
to Rust.
2021-02-19 09:08:22 -05:00
Colin Walters
04e0c4e01a Fix progress API to have "output message" separate from task
A lot of our output is outside of a "task"; the Rust binding
incorrectly made it a method on `Progress`.  This is really
just a `println!()` that is backed by our dispatch system.
2021-02-19 09:08:22 -05:00
Colin Walters
1d242ddb59 Add a C++ rpmdb-diff API wrapping the C one, bind in Rust
I'd like to compute diffs in apply-live to differentiate
between "pure layering" versus modifications/removals.
2021-02-19 09:08:22 -05:00
Colin Walters
0c9f7ab7eb Add daemon.rs with one helper function, use it from C++
Trying to port to use origin Rust code, I think actually
it gets simpler if we move more to Rust to start.
2021-02-18 10:22:39 -05:00
dependabot[bot]
2306c919ec build(deps): bump system-deps from 2.0.3 to 3.0.0
Bumps [system-deps](https://github.com/gdesmott/system-deps) from 2.0.3 to 3.0.0.
- [Release notes](https://github.com/gdesmott/system-deps/releases)
- [Commits](https://github.com/gdesmott/system-deps/compare/v2.0.3...v3.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-18 08:06:41 -05:00
Colin Walters
8a62ad9f3d Add gobj_rewrap() API to pass glib-rs objects back to C++
When we started using cxxrs, most of the glib-rs objects like
`OstreeRepo`/`OstreeSysroot` were owned by C++ and passed
down into Rust.  That motivated the addition of the special
bridging infrastructure to re-create a glib-rs wrapper
type from what cxxrs wants (a `Pin<&mut T>`).

But now that we're adding more in Rust, we have the need
to pass these objects back into C++.  In fact this will
hopefully soon because the default case as more of the
binary entrypoint becomes Rust.

Add another trait with a method `gobj_rewrap()` that converts
in the other direction.  This implementation took me a number
of tries before I finally settled on simply using `mem::transmute()`.
There are a *lot* of caveats listed on the docs for that function,
but I think it really is what we want here.  See the link for pending work
on a Rust RFC to enable safe transmutes for some cases, and I believe
that would cover this use case:
https://internals.rust-lang.org/t/pre-rfc-v2-safe-transmute/11431

I've verified this works in a separate patch, but this commit
also adds a simple test case - this goes all the way from:
   Rust glib-rs `ostree::Repo` (holding strong ref)
   -> Rust `Pin<&mut ostree_sys::OstreeRepo>`
   -> (internal cxx-rs C bridge)
   -> C++ `OstreeRepo&` reference
   -> C `OstreeRepo*` pointer
Which is quite the dance if you think about it!
2021-02-18 06:29:41 -05:00
Colin Walters
8d9e113e8f rust/countme: Fix clippy lint by refactoring serialization
clippy complains about `to_*` not taking `&self` - I think
we can simplify this more by using our `write_file_with()` API
that we're using in other places.  It was explicitly designed
with serde in mind.  It's just more efficient and nicer to
serialize to a `BufWriter` instead of to a string, then
writing the string.
2021-02-18 05:27:41 -05:00
Colin Walters
9854a3aa8e rust/extensions: Fix two clippy lints
These are easy fixes.
2021-02-18 05:27:41 -05:00
Colin Walters
c027b638db rust: Quiet clippy Vec<Foo> arg
cxx.rs only supports `Vec<String>` and not `&[String]` right now.
2021-02-18 05:27:41 -05:00
Colin Walters
35fcf3eac0 rust: Fix some misc clippy lints
Nothing important here really, but clippy is useful so
let's try to keep it quiet.
2021-02-18 05:27:41 -05:00
dependabot[bot]
f343146efd build(deps): bump cxx from 1.0.30 to 1.0.32
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.30 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.30...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-17 17:42:33 -05:00
Colin Walters
8a1b2f3739 libdnf-sys: Include gio.h before libdnf.h
See b77f710cfb

Alternative fix to rpm-software-management/libdnf#1139
aka https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1935

This way libdnf's `extern "C"` over the glib headers doesn't apply
because we already processed that header.
2021-02-17 14:02:34 -05:00
Colin Walters
e6c045cada Add an rpmostree-client sub-crate
This is intended to be published to https://crates.io/crates/rpmostree-client
Part of https://github.com/coreos/rpm-ostree/issues/2389

This directly imports the code from
5551c54c6e/tests/inst/src/rpmostree.rs

Once merged and released I'll try converting the ostree test suite
over as well as Zincati.

Internally add a testutils helper to validate it works.
2021-02-16 19:22:26 -05:00
Colin Walters
764de41cc6 Switch to using cxx-rs for treefile
This is one half of https://github.com/coreos/rpm-ostree/issues/2544
which aims to drop our use of `cbindgen`.
2021-02-11 15:44:38 -05:00
Colin Walters
094be6c469 rust: Add module doc headers
Specifically motivated by adding some docs in `treefile.rs` around
how to add a field, but I decided to just do a pass and
document everything at least a little.

View with e.g. `cargo doc --document-private-items`.
2021-02-11 14:16:38 -05:00
Jonathan Lebon
49fe650124 lockfile: Move ror_lockfile_read to cxx.rs
Pretty straightforward. Haven't moved `ror_lockfile_write` yet because
that's trickier to do and I'm still figuring out the most elegant way to
do this within cxx.rs' constraints.
2021-02-10 13:39:38 -05:00
Jonathan Lebon
b640892f04 libdnf-sys: Drop C API, replace with cxx.rs bridge
Right now, we're using libdnf APIs from Rust via hand-crafted `extern C`
interfaces, which is extra dangerous because there is no signature
checking that happens at compile-time.

Until either we can automate libdnf bindings or use its C++ API directly
via cxx.rs, let's do some basic wrapping in C++ ourselves and use libdnf
through that API only instead. That gives us a lot more confidence and
makes the libdnf API feel more natural to use in Rust.
2021-02-10 13:39:38 -05:00
Jonathan Lebon
99486a75e8 Add /usr/lib/rpm/macros.d/macros.rpm-ostree to set %_dbpath to /usr/share/rpm
We trigger a librpm macro file load in many of our paths. Since the
default value shipped by rpm's macro file sets `_dbpath` to
`/var/lib/rpm`, we have to explicitly set that back to `/usr/share/rpm`
in those paths.

This became more problematic recently with libsolv v0.7.17 which fully
keys off of `_dbpath` to find the rpmdb path to load:

04d4d036b2

And it's not technically wrong; we really should make that macro not
lie. This is what this patch does by injecting an RPM macro file in our
composes which sets it to /usr/share/rpm. So then e.g. the `rpm` CLI
doesn't actually need the `/var/lib/rpm` backcompat link anymore, though
there's no harm in leaving it.

In the future, we should be able to drop this once we move all of Fedora
to `/usr/lib/sysimage/rpm` (see
https://github.com/coreos/fedora-coreos-tracker/issues/639).

Closes: #2548
2021-02-09 18:36:35 -05:00
Colin Walters
2f6b5a654d Bind output core into Rust, use in apply-live
Originally the Rust apply-live code was exposed from Rust to C
via bindgen.  But when working on that, I hit the problem
that our output infrastructure was C...and the "reverse direction"
binding stuff was just ugly.

This PR again IMO shows the value of the investment in cxx-rs
because we can now seamlessly call back from the Rust side
into a "C++-ish" progress API, which the C++ side is updated
to use.

The level of indirection here is obviously pretty silly
because the main thing on the C++ output side is basically
a function dispatcher, but...I didn't want to try to rework
that into Rust fully yet.  (But, the moment we do this
whole area will get a *lot* cleaner)

Anyways, in the end this makes it easy for the apply-live
code to output progress to the user which was sorely
needed.
2021-02-09 04:43:29 -05:00
Colin Walters
c9e9269770 Rename internal Rust progress to console_
Our output system is very confusing in that we bridge over
DBus in some cases and not others.  In preparation for allowing
Rust code to call into the C++ progress system which contains
that delegation layer, rename the Rust progress to `console_`
to clearly show that it should only be invoked by code that
knows it's writing to a tty.
2021-02-09 04:43:29 -05:00
Colin Walters
47c60eb6ce libdnf: Various buildsys fixes
WITH_SWDB: Removed in 99309fbe04
WITH_GIR Removed in e2f2862bed

Also, most importantly: don't always reconfigure libdnf

This is a questionable default for the cargo `cmake` crate.
Building in Koji is failing I think due to timestamp issues
causing cmake to run twice.
2021-02-04 17:57:33 -05:00
Colin Walters
588541c60d Move libdnf build over to Cargo
This is now further migration towards Cargo/Rust possible
because we switched our main binary.  We've had an internal
`libdnf-sys` crate for a while, but now it can take over
the build of the underlying library too (like many `-sys`
crates support).

This itself is just an incremental step towards migrating
the main rpm-ostree build system to e.g. cmake too (or
perhaps directly with the `cc` crate, not sure yet) and
driving it via `cargo` too.
2021-02-04 10:59:20 -05:00