Commit Graph

2004 Commits

Author SHA1 Message Date
Luca BRUNO
de7d20e43b libpriv/passwd: move UID/GID checker to Rust
This ports to Rust the logic for checking whether a directory tree
contains content owned by a given UID/GID.
2021-02-26 00:34:56 +01:00
Timothée Ravier
101d531f3b Install rpm-ostree sysusers config and countme units
Install the systemd unit, timer and sysusers configuration for Count Me
support. We do not enable or pull as a dependency those units by default
as this is a decision that should be taken at the distribution level and
needs support on the infrastructure side.

To enable those units in a disctribution package, you can add the
following symlink:
$ ln -snf /usr/lib/systemd/system/rpm-ostree-countme.timer /usr/lib/systemd/system/rpm-ostreed.service.wants/

or add the following config snippet to the rpm-ostreed.service unit:
Wants=rpm-ostree-countme.timer
2021-02-25 16:58:14 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Kelvin Fan
e10df33e88 app/dbus-helpers: Factor out updates driver-related functions
Following https://github.com/coreos/rpm-ostree/pull/2566, we would
like to give other commands (deploy and rebase) the same treatment.

Factor out functions to retrieve systemd unit docs and a function
that detects updates drivers so they can be reused elsewhere.
2021-02-25 06:44:53 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
d8230bfb6d daemon: Move some "deployment variant" generation to Rust
More prep for https://github.com/coreos/rpm-ostree/pull/2388

This was actually also my first time really trying out the
latest gtk-rs `glib::Variant` API, which is one of the major
things we need to use to progress oxidation more.
2021-02-23 13:56:26 -05:00
Luca BRUNO
7283aef8af libpriv/passwd: remove unused functions
This drops some functions that used to be part of C unit-tests but
are currently dead code.
2021-02-22 13:24:41 -05:00
Luca BRUNO
9b94e85e5b libpriv/passwd: move compose preparation to Rust
This moves passwd/group compose preparation logic to Rust,
dropping all the remaining minor helpers related to JSON parsing,
file stream creation, and entries deduplication.
2021-02-22 11:44:13 -05:00
Colin Walters
246f02fd47 apply-live: Move client-side finish to Rust
Now that we have an RPM diff+printing binding, we can
move the client side postprocessing for `apply-live`
to Rust.
2021-02-19 09:08:22 -05:00
Colin Walters
04e0c4e01a Fix progress API to have "output message" separate from task
A lot of our output is outside of a "task"; the Rust binding
incorrectly made it a method on `Progress`.  This is really
just a `println!()` that is backed by our dispatch system.
2021-02-19 09:08:22 -05:00
Colin Walters
1d242ddb59 Add a C++ rpmdb-diff API wrapping the C one, bind in Rust
I'd like to compute diffs in apply-live to differentiate
between "pure layering" versus modifications/removals.
2021-02-19 09:08:22 -05:00
Colin Walters
29a2304fe2 daemon: Rename transaction_livefs → transaction_apply_live
Continuing the renaming.
2021-02-19 09:08:22 -05:00
Colin Walters
ffe72d28ec app: Rename livefs.cxx → apply-live.cxx
Continuing the renaming.
2021-02-19 09:08:22 -05:00
Timothée Ravier
790e709360 countme: Persist last timer trigger to handle system suspends
Set `Persistent=yes` in the timer unit to make sure that systems that
are not running 24/7 gets counted when they wake up if they missed the
previous timer run / counting window.

This is mostly for Fedora Silverblue and IoT like systems as Fedora
CoreOS systems usually run 24/7.

https://www.freedesktop.org/software/systemd/man/systemd.timer.html#Persistent=
2021-02-18 15:14:34 -05:00
Colin Walters
0c9f7ab7eb Add daemon.rs with one helper function, use it from C++
Trying to port to use origin Rust code, I think actually
it gets simpler if we move more to Rust to start.
2021-02-18 10:22:39 -05:00
Colin Walters
3910e77ae8 daemon: Refactor deployment variant generation code
Changes to this ad-hoc `a{sv}` code have been scattered
around over time.  Move the "core" bits to the top of
the function, then have the more complex stuff that depends
on the origin (and refspec/layering status) later.

Prep for further cleanups.
2021-02-18 10:22:39 -05:00
Colin Walters
8a62ad9f3d Add gobj_rewrap() API to pass glib-rs objects back to C++
When we started using cxxrs, most of the glib-rs objects like
`OstreeRepo`/`OstreeSysroot` were owned by C++ and passed
down into Rust.  That motivated the addition of the special
bridging infrastructure to re-create a glib-rs wrapper
type from what cxxrs wants (a `Pin<&mut T>`).

But now that we're adding more in Rust, we have the need
to pass these objects back into C++.  In fact this will
hopefully soon because the default case as more of the
binary entrypoint becomes Rust.

Add another trait with a method `gobj_rewrap()` that converts
in the other direction.  This implementation took me a number
of tries before I finally settled on simply using `mem::transmute()`.
There are a *lot* of caveats listed on the docs for that function,
but I think it really is what we want here.  See the link for pending work
on a Rust RFC to enable safe transmutes for some cases, and I believe
that would cover this use case:
https://internals.rust-lang.org/t/pre-rfc-v2-safe-transmute/11431

I've verified this works in a separate patch, but this commit
also adds a simple test case - this goes all the way from:
   Rust glib-rs `ostree::Repo` (holding strong ref)
   -> Rust `Pin<&mut ostree_sys::OstreeRepo>`
   -> (internal cxx-rs C bridge)
   -> C++ `OstreeRepo&` reference
   -> C `OstreeRepo*` pointer
Which is quite the dance if you think about it!
2021-02-18 06:29:41 -05:00
Colin Walters
f8dc856aa9 compose: Add missing #include <optional>
This only fails on newer gcc/libstdc++ apparently in current
Fedora rawhide: https://kojipkgs.fedoraproject.org//work/tasks/4372/62164372/build.log
2021-02-17 14:02:34 -05:00
Kelvin Fan
626d021edb app/upgrade: Do not upgrade if updates driver registered
Do not perform an upgrade if detected that an updates driver has
been registered.

Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
2021-02-17 11:43:34 -05:00
Kelvin Fan
261b9cbb87 app/status: factor out get_sd_unit_objpath() function
Factor out a `get_sd_unit_objpath()` function into `libbuiltin`
so we can reuse the code in `builtin_upgrade`.
2021-02-17 11:43:34 -05:00
Colin Walters
118178cef1 tree-wide: Fix C++ range iteration to use references
This is actually a perfect example of the tradeoffs involved
in our team of C/Rust programmers trying to use C++ so we
can use cxx-rs =)

It turns out that "for (auto foo : bar)" is really yet another
one of those C++ sharp edges just waiting to cut.  That
version does a copy of each value; one almost always wants to use
"for (auto & foo : bar)" which like Rust's `.iter()`.

In most of our code we were OK making copies, they were just
a performance hit, but in another place I was relying on the
"side effect" of `.c_str()` but that meant our value got
destroyed.
2021-02-12 12:50:02 -05:00
Colin Walters
764de41cc6 Switch to using cxx-rs for treefile
This is one half of https://github.com/coreos/rpm-ostree/issues/2544
which aims to drop our use of `cbindgen`.
2021-02-11 15:44:38 -05:00
Colin Walters
20178bb25a tree-wide: Include GLib headers before libdnf
Alternative fix to https://github.com/rpm-software-management/libdnf/pull/1139
aka https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1935

This way libdnf's `extern "C"` over the glib headers doesn't apply
because we already processed that header.
2021-02-10 17:05:42 -05:00
Jonathan Lebon
49fe650124 lockfile: Move ror_lockfile_read to cxx.rs
Pretty straightforward. Haven't moved `ror_lockfile_write` yet because
that's trickier to do and I'm still figuring out the most elegant way to
do this within cxx.rs' constraints.
2021-02-10 13:39:38 -05:00
Jonathan Lebon
56f95cc5c1 extensions: Don't try to chown RPM packages
We don't need it and it won't work in the unprivileged path where we're
running this in a cosa supermin (of course, this is all a bit silly
because we don't actually need privileges to begin with for this, but
there's a lot of momentum in sticking with that workflow).

Update submodule: libglnx
2021-02-10 10:20:38 -05:00
Colin Walters
6886e44425 scripts: Bind /usr/share/empty over /usr/share/rpm
Now that we inject the `%_dbpath /usr/share/rpm` macro, `rpm -q`
will start using it.  But in RPM script invocation, we don't
want them to see any RPM database at all - trying to query it
should be a clean failure.
2021-02-09 18:36:35 -05:00
Jonathan Lebon
667fdc9ff4 libpriv/rpm-util: Use /usr/share/rpm for base rpmdb query
Follow-up to previous commit: we had another path where we made a
temporary rootfs and symlinked `/var/lib/rpm` to the base rpmdb. That of
course broke now that we inject a macro to point the rpmdb to
`/usr/share/rpm`.

Rework this to use `/usr/share/rpm` since that's our canonical location
for now, but also add the compat symlinks so that this logic should keep
working even on trees without the injected macro yet.
2021-02-09 18:36:35 -05:00
Jonathan Lebon
60215ae865 libpriv/rpm-util: Add /usr/lib/sysimage/rpm symlink in rpmdb checkout
We don't technically need this yet, but it mirrors how it's set up in
our composes so that if there's code that wants to use the new location
too, it'll just work.
2021-02-09 18:36:35 -05:00
Jonathan Lebon
aa1a0f8719 lib/package: Fix comment about rpmdb checkout
We do fallback here.
2021-02-09 18:36:35 -05:00
Jonathan Lebon
99486a75e8 Add /usr/lib/rpm/macros.d/macros.rpm-ostree to set %_dbpath to /usr/share/rpm
We trigger a librpm macro file load in many of our paths. Since the
default value shipped by rpm's macro file sets `_dbpath` to
`/var/lib/rpm`, we have to explicitly set that back to `/usr/share/rpm`
in those paths.

This became more problematic recently with libsolv v0.7.17 which fully
keys off of `_dbpath` to find the rpmdb path to load:

04d4d036b2

And it's not technically wrong; we really should make that macro not
lie. This is what this patch does by injecting an RPM macro file in our
composes which sets it to /usr/share/rpm. So then e.g. the `rpm` CLI
doesn't actually need the `/var/lib/rpm` backcompat link anymore, though
there's no harm in leaving it.

In the future, we should be able to drop this once we move all of Fedora
to `/usr/lib/sysimage/rpm` (see
https://github.com/coreos/fedora-coreos-tracker/issues/639).

Closes: #2548
2021-02-09 18:36:35 -05:00
Luca BRUNO
8a2f281143 libpriv/postprocess: get rid of goto statements
This removes all goto statements in the postprocess module,
replacing them with an exception catcher instead.
2021-02-09 16:43:32 +01:00
Colin Walters
2f6b5a654d Bind output core into Rust, use in apply-live
Originally the Rust apply-live code was exposed from Rust to C
via bindgen.  But when working on that, I hit the problem
that our output infrastructure was C...and the "reverse direction"
binding stuff was just ugly.

This PR again IMO shows the value of the investment in cxx-rs
because we can now seamlessly call back from the Rust side
into a "C++-ish" progress API, which the C++ side is updated
to use.

The level of indirection here is obviously pretty silly
because the main thing on the C++ output side is basically
a function dispatcher, but...I didn't want to try to rework
that into Rust fully yet.  (But, the moment we do this
whole area will get a *lot* cleaner)

Anyways, in the end this makes it easy for the apply-live
code to output progress to the user which was sorely
needed.
2021-02-09 04:43:29 -05:00
Colin Walters
c9e9269770 Rename internal Rust progress to console_
Our output system is very confusing in that we bridge over
DBus in some cases and not others.  In preparation for allowing
Rust code to call into the C++ progress system which contains
that delegation layer, rename the Rust progress to `console_`
to clearly show that it should only be invoked by code that
knows it's writing to a tty.
2021-02-09 04:43:29 -05:00
Colin Walters
a8fe6cf32b transaction: Fix missing return in error case
Hooray for compiler warnings.
2021-02-08 14:38:31 -05:00
Colin Walters
dba43e201e tree-wide: Fix clang -Wgnu-designator/-Wunused-variable
clang warns about this:
https://clang.llvm.org/docs/DiagnosticsReference.html#wgnu-designator
Also fix unused variables and hard error on them.
2021-02-08 14:38:31 -05:00
Colin Walters
8ddaf0bbd6 Make failure to find packages fatal, add more error prefixing
To help debug https://bugzilla.redhat.com/show_bug.cgi?id=1925584
2021-02-08 11:20:50 -05:00
Jonathan Lebon
d22d241754 core: Don't allow noent when resolving pkgcache rev
If we get here, it's that we expect the pkgcache to be there. So don't
allow ENOENT (we weren't even checking for the ENOENT case here, which
shows that this was the intent).

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1925584
2021-02-08 11:20:50 -05:00
Colin Walters
5cefee81ff tree-wide: Squash some clang-analyzer found unused variables
We weren't using the return values, so just drop them.
2021-02-05 12:39:07 -05:00
Colin Walters
4d2a6e6de0 tree-wide: Pacify some clang-analyzer "Dead nested assignment"
It doesn't understand the "$x_owned" pattern (which is really
much like Rust's `std::borrow::Cow`).
2021-02-05 12:39:07 -05:00
Colin Walters
18c29fa5d6 libmain: Refactor to fix analyzer warning
Previously this function was in `goto out` style
and so we had this awkward "track return value in pointer"
thing.  But `clang-analyzer` correctly points out that
we don't need this anymore because we never read the value
initially stored.
2021-02-05 12:39:07 -05:00
Colin Walters
6a594dbe6b util: Annotate our "throw" wrappers as [[ noreturn ]]
This way the compiler and `clang-analyzer` understand and won't
issue an error if we are missing a `return` after an unconditional
`throw_gerror()`.
2021-02-05 12:39:07 -05:00
Colin Walters
a36f9716c6 tree-wide: Fix some "Dead assignment" from clang-analyzer
The `have_rpmdb` one was a leftover looks like.  In the `disabled_all_repos`
case it was clearly there for symmetry, but eh; it seems somewhat
unlikely that we add a *3rd* case there.  Also while we're
here change it to C++ `bool` so tools like analyzers know it really
is a boolean.
2021-02-05 12:39:07 -05:00
Colin Walters
7b5b35210b tree-wide: Fix some spurious "Dead assignment" from clang-analyzer
This fixes some spurious warnings from clang-analyzer (aka `scan-build`) around
"Dead assignment".  Unfortunately the analyzer doesn't understand
the side effects of `__attribute__((cleanup))` here.

More info on the `(void)` pattern: https://clang-analyzer.llvm.org/faq.html#dead_store
2021-02-05 12:39:07 -05:00
Colin Walters
588541c60d Move libdnf build over to Cargo
This is now further migration towards Cargo/Rust possible
because we switched our main binary.  We've had an internal
`libdnf-sys` crate for a while, but now it can take over
the build of the underlying library too (like many `-sys`
crates support).

This itself is just an incremental step towards migrating
the main rpm-ostree build system to e.g. cmake too (or
perhaps directly with the `cc` crate, not sure yet) and
driving it via `cargo` too.
2021-02-04 10:59:20 -05:00
Luca BRUNO
9c3864b97e libpriv/passwd: move entries deduplication logic to Rust
This moves `group` and `passwd` merging/deduplication to Rust.
2021-02-03 15:59:38 -05:00
Jonathan Lebon
317b920667 extensions: Add support for development extensions
In RHCOS, we ship kernel development-related packages as an extension.
Those aren't really extensions that are meant to be layered onto the
host.  They're meant to be used in a build environment somewhere to
compile kernel modules.

This makes it very different from "OS extensions" in at least two
drastic ways:
1. we don't want to do any depsolving (e.g. we don't want to pull in
   `gcc` or something)
2. some of those packages may be present in the base already, but we
   still want to redownload them

Hesitated putting this functionality in rpm-ostree, but I think in the
end it cuts from the benefit of moving this code to rpm-ostree if we
can't entirely get rid of the Python script it obsoletes. Plus, being
able to use the `match-base-evr` is still really useful for this use
case.

Let's add a new `kind` key to support this. The traditional extensions
are called "OS extensions" and these new extensions are called
"development extensions".

The latter is not yet part of the state checksum, so change detection
doesn't work there. I think that's fine for now though because the
primary use case is the kernel, and there we want to match the base
version. So if the kernel changes, the base would change too. (Though
there's the corner case of adding a new package to the list while at the
same version...)
2021-02-03 12:22:38 -05:00
Jonathan Lebon
f4bee1352d app/compose: Add comment about pkgcache
Gives a bit more info about how the extensions path is different from
the base treecompose.
2021-02-03 12:22:38 -05:00
Jonathan Lebon
f7090358be extensions: Support enabling additional repos
We want to be able to enable more repos than those in the treefile when
downloading extensions. In RHCOS for example, the `kernel-rt` packages
come from a separate repo.

But also, once we support "development" extensions, we want to support
the case where devel packages come from another repo.
2021-02-03 12:22:38 -05:00
Jonathan Lebon
bbc72cbf6b core: Fix handling of local packages when downloading
In the core context, this is redundant with `sort_packages` because it
won't put local packages in the `pkgs_to_download` array anyway, but we
want this check even if we call `rpmostree_download_packages` directly
and pass some packages which may be local.
2021-02-03 12:22:38 -05:00