Commit Graph

3438 Commits

Author SHA1 Message Date
Colin Walters
2b294d68bd client: Remove peer_pid value from option parsing
Instead of passing the pid back up the stack and using a cleanup
function on it to invoke `kill()`, use `PR_SET_PDEATHSIG` which
has the kernel take care of this for us.

(In practice we don't actually use this peer functionality anymore
 because all of the client/daemon code kind of requires being run under systemd
 on a real system now)

This shrinks the API surface and is much less repetitive in
the codebase.

Prep for moving more of the CLI code to Rust.
2021-03-01 18:53:50 +01:00
dependabot[bot]
6dd2df19bc build(deps): bump serde_json from 1.0.62 to 1.0.64
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.62 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.62...v1.0.64)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-01 17:46:49 +01:00
Colin Walters
732e0f232a daemon: Fix crash on upgrade with remote:checksum
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).

The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.

But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.

I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.

Closes: https://github.com/coreos/rpm-ostree/issues/2603
2021-03-01 16:57:51 +01:00
Colin Walters
10ad264ecd daemon: Heap allocate GVariantDict to pass to Rust
I was looking at a different problem and noticed a spam of
```
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_ref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_unref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_ref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_unref: assertion 'is_valid_heap_dict (dict)' failed
```

in the journal.  It turns out that the Rust bindings try to call
`g_variant_ref()`, but that's (understandably) not allowed on
a stack allocated instance.

Since we heap allocate a *ton* here, let's just heap allocate
the dict too.

Now, we also along the way stopped using the bits to inject
```
[Service]
Environment=G_DEBUG=fatal-warnings
```
in our CI.  I'll look at that as a followup.
2021-02-26 16:54:26 +01:00
Luca BRUNO
de7d20e43b libpriv/passwd: move UID/GID checker to Rust
This ports to Rust the logic for checking whether a directory tree
contains content owned by a given UID/GID.
2021-02-26 00:34:56 +01:00
Luca BRUNO
dbe2d50cf1 treefile: add getters for check-passwd and check-groups
This add default-aware getters for `check-passwd` and `check-groups`.
2021-02-25 19:24:12 +01:00
Timothée Ravier
101d531f3b Install rpm-ostree sysusers config and countme units
Install the systemd unit, timer and sysusers configuration for Count Me
support. We do not enable or pull as a dependency those units by default
as this is a decision that should be taken at the distribution level and
needs support on the infrastructure side.

To enable those units in a disctribution package, you can add the
following symlink:
$ ln -snf /usr/lib/systemd/system/rpm-ostree-countme.timer /usr/lib/systemd/system/rpm-ostreed.service.wants/

or add the following config snippet to the rpm-ostreed.service unit:
Wants=rpm-ostree-countme.timer
2021-02-25 16:58:14 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Kelvin Fan
e10df33e88 app/dbus-helpers: Factor out updates driver-related functions
Following https://github.com/coreos/rpm-ostree/pull/2566, we would
like to give other commands (deploy and rebase) the same treatment.

Factor out functions to retrieve systemd unit docs and a function
that detects updates drivers so they can be reused elsewhere.
2021-02-25 06:44:53 +01:00
dependabot[bot]
380ebc5ada build(deps): bump cxx-build from 1.0.31 to 1.0.32
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.31...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-24 23:27:25 +01:00
Colin Walters
927e02100f docs/architecture-core.md: New file
This is long overdue.  Some of this came up in recent
conversation.  Let's keep up some continual background
momentum on documentation, just like CI.

Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2021-02-24 22:01:23 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
ce20267b2d ostree_utils: Add a workaround for optional ref
Until we have https://github.com/ostreedev/ostree/pull/2282
in an ostree release and an updated `ostree` crate.
2021-02-23 21:51:22 -05:00
Colin Walters
0544d1c92d ci: Drop tests/vmcheck/image.qcow2, use COSA_DIR/.cosa
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default.  We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.

Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
2021-02-23 17:23:26 -05:00
Colin Walters
d8230bfb6d daemon: Move some "deployment variant" generation to Rust
More prep for https://github.com/coreos/rpm-ostree/pull/2388

This was actually also my first time really trying out the
latest gtk-rs `glib::Variant` API, which is one of the major
things we need to use to progress oxidation more.
2021-02-23 13:56:26 -05:00
Colin Walters
6cf0f87412 docs/HACKING.md: Various updates
In particular describe the cxx tools trap.
2021-02-23 12:40:27 -05:00
dependabot[bot]
20feab6d2a build(deps): bump serde_yaml from 0.8.16 to 0.8.17
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.16 to 0.8.17.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.16...0.8.17)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-23 11:07:28 -05:00
dependabot[bot]
170095bd60 build(deps): bump nix from 0.19.1 to 0.20.0
Bumps [nix](https://github.com/nix-rust/nix) from 0.19.1 to 0.20.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-22 15:33:41 -05:00
Luca BRUNO
7283aef8af libpriv/passwd: remove unused functions
This drops some functions that used to be part of C unit-tests but
are currently dead code.
2021-02-22 13:24:41 -05:00
Luca BRUNO
9b94e85e5b libpriv/passwd: move compose preparation to Rust
This moves passwd/group compose preparation logic to Rust,
dropping all the remaining minor helpers related to JSON parsing,
file stream creation, and entries deduplication.
2021-02-22 11:44:13 -05:00
Luca BRUNO
a1ecc3d583 rust: add missing SPDX-License-Identifier tag
This adds a missing `SPDX-License-Identifier` tag, fixing CI.
2021-02-22 09:07:33 -05:00
Colin Walters
49c55089d2 lockfile: Be robust to non-seconds-since-epoch repodata
We should really have safety comments around our use of `.unwrap()`.
This Google kubernetes rpm-md repo appears to use seconds-since-epoch.
We could try to detect that, probably best done in libdnf.

I think it'd be better to get them to match the informal schema
for this actually though instead.

For now let's just log an error and not crash.

Closes: https://github.com/coreos/rpm-ostree/issues/2600
2021-02-19 15:56:29 -05:00
Colin Walters
f882d6ddff rust: Add SPDX-License-Identifier and validate it in ci/codestyle.sh
It turns out we accidentally added GPL'd code into the Rust
side, which wasn't intentional on my part and I think it's since
been copied around.

Honestly I think half of the problem is the gigantic
"blah blah blah GNU General blah blah" just makes people's eyes
glaze over.  In contrast the `SPDX-License-Identifier` is short
and obvious.

So let's validate that in CI.

This follows a similar change in ostree:
https://github.com/ostreedev/ostree/pull/1439

If we merge this I'll do the C/C++ side too after that.
2021-02-19 15:56:23 -05:00
Colin Walters
a62ace8d5b codestyle: Remove tabdamage.txt in OK case
Avoid leaving cruft in the srcdir.
2021-02-19 15:56:23 -05:00
dependabot[bot]
1421dcbc14 build(deps): bump openat-ext from 0.1.11 to 0.1.12
Bumps [openat-ext](https://github.com/cgwalters/openat-ext) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/cgwalters/openat-ext/releases)
- [Commits](https://github.com/cgwalters/openat-ext/compare/v0.1.11...v0.1.12)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-19 15:05:23 -05:00
Colin Walters
27f9734300 apply-live: Delete code to check for live commit post-txn
Per discussion in
https://github.com/coreos/rpm-ostree/pull/2581#discussion_r578621458
it is inherently racy and we'll do a followup fix to write a
ref.
2021-02-19 09:08:22 -05:00
Colin Walters
246f02fd47 apply-live: Move client-side finish to Rust
Now that we have an RPM diff+printing binding, we can
move the client side postprocessing for `apply-live`
to Rust.
2021-02-19 09:08:22 -05:00
Colin Walters
04e0c4e01a Fix progress API to have "output message" separate from task
A lot of our output is outside of a "task"; the Rust binding
incorrectly made it a method on `Progress`.  This is really
just a `println!()` that is backed by our dispatch system.
2021-02-19 09:08:22 -05:00
Colin Walters
1d242ddb59 Add a C++ rpmdb-diff API wrapping the C one, bind in Rust
I'd like to compute diffs in apply-live to differentiate
between "pure layering" versus modifications/removals.
2021-02-19 09:08:22 -05:00
Colin Walters
29a2304fe2 daemon: Rename transaction_livefs → transaction_apply_live
Continuing the renaming.
2021-02-19 09:08:22 -05:00
Colin Walters
ffe72d28ec app: Rename livefs.cxx → apply-live.cxx
Continuing the renaming.
2021-02-19 09:08:22 -05:00
Timothée Ravier
790e709360 countme: Persist last timer trigger to handle system suspends
Set `Persistent=yes` in the timer unit to make sure that systems that
are not running 24/7 gets counted when they wake up if they missed the
previous timer run / counting window.

This is mostly for Fedora Silverblue and IoT like systems as Fedora
CoreOS systems usually run 24/7.

https://www.freedesktop.org/software/systemd/man/systemd.timer.html#Persistent=
2021-02-18 15:14:34 -05:00
Colin Walters
0c9f7ab7eb Add daemon.rs with one helper function, use it from C++
Trying to port to use origin Rust code, I think actually
it gets simpler if we move more to Rust to start.
2021-02-18 10:22:39 -05:00
Colin Walters
3910e77ae8 daemon: Refactor deployment variant generation code
Changes to this ad-hoc `a{sv}` code have been scattered
around over time.  Move the "core" bits to the top of
the function, then have the more complex stuff that depends
on the origin (and refspec/layering status) later.

Prep for further cleanups.
2021-02-18 10:22:39 -05:00
dependabot[bot]
2306c919ec build(deps): bump system-deps from 2.0.3 to 3.0.0
Bumps [system-deps](https://github.com/gdesmott/system-deps) from 2.0.3 to 3.0.0.
- [Release notes](https://github.com/gdesmott/system-deps/releases)
- [Commits](https://github.com/gdesmott/system-deps/compare/v2.0.3...v3.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-18 08:06:41 -05:00
Colin Walters
9eb217bd89 ci: Add Prow e2e testing
Our "CoreOS CI Jenkins" flow is great, but the reliability/debuggabilty
of the system isn't the best.  This is an experient to *add* another CI
flow that can be run in Prow too.

I'd like to at least have a basic e2e (compose tree, run client tests)
flow that can operate in Prow so that if CoreOS CI Jenkins is down or
flaky, we at least have some confidence in our commits.

Longer term I think we can figure out how to better share CI flows
across these:
https://github.com/coreos/fedora-coreos-tracker/issues/263#issuecomment-772903759
2021-02-18 06:52:42 -05:00
Colin Walters
8a62ad9f3d Add gobj_rewrap() API to pass glib-rs objects back to C++
When we started using cxxrs, most of the glib-rs objects like
`OstreeRepo`/`OstreeSysroot` were owned by C++ and passed
down into Rust.  That motivated the addition of the special
bridging infrastructure to re-create a glib-rs wrapper
type from what cxxrs wants (a `Pin<&mut T>`).

But now that we're adding more in Rust, we have the need
to pass these objects back into C++.  In fact this will
hopefully soon because the default case as more of the
binary entrypoint becomes Rust.

Add another trait with a method `gobj_rewrap()` that converts
in the other direction.  This implementation took me a number
of tries before I finally settled on simply using `mem::transmute()`.
There are a *lot* of caveats listed on the docs for that function,
but I think it really is what we want here.  See the link for pending work
on a Rust RFC to enable safe transmutes for some cases, and I believe
that would cover this use case:
https://internals.rust-lang.org/t/pre-rfc-v2-safe-transmute/11431

I've verified this works in a separate patch, but this commit
also adds a simple test case - this goes all the way from:
   Rust glib-rs `ostree::Repo` (holding strong ref)
   -> Rust `Pin<&mut ostree_sys::OstreeRepo>`
   -> (internal cxx-rs C bridge)
   -> C++ `OstreeRepo&` reference
   -> C `OstreeRepo*` pointer
Which is quite the dance if you think about it!
2021-02-18 06:29:41 -05:00
Colin Walters
8d9e113e8f rust/countme: Fix clippy lint by refactoring serialization
clippy complains about `to_*` not taking `&self` - I think
we can simplify this more by using our `write_file_with()` API
that we're using in other places.  It was explicitly designed
with serde in mind.  It's just more efficient and nicer to
serialize to a `BufWriter` instead of to a string, then
writing the string.
2021-02-18 05:27:41 -05:00
Colin Walters
9854a3aa8e rust/extensions: Fix two clippy lints
These are easy fixes.
2021-02-18 05:27:41 -05:00
Colin Walters
c027b638db rust: Quiet clippy Vec<Foo> arg
cxx.rs only supports `Vec<String>` and not `&[String]` right now.
2021-02-18 05:27:41 -05:00
Colin Walters
35fcf3eac0 rust: Fix some misc clippy lints
Nothing important here really, but clippy is useful so
let's try to keep it quiet.
2021-02-18 05:27:41 -05:00
dependabot[bot]
5f3b692e60 build(deps): bump libc from 0.2.84 to 0.2.86
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.84 to 0.2.86.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.84...0.2.86)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-18 05:13:41 -05:00
dependabot[bot]
3e7ad771fb build(deps): bump memfd from 0.3.0 to 0.4.0
Bumps [memfd](https://github.com/lucab/memfd-rs) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/lucab/memfd-rs/releases)
- [Commits](https://github.com/lucab/memfd-rs/compare/v0.3.0...v0.4.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-17 18:42:32 -05:00
dependabot[bot]
f343146efd build(deps): bump cxx from 1.0.30 to 1.0.32
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.30 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.30...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-17 17:42:33 -05:00
Colin Walters
047f931746 Release 2021.2 2021-02-17 15:17:33 -05:00
Colin Walters
8a1b2f3739 libdnf-sys: Include gio.h before libdnf.h
See b77f710cfb

Alternative fix to rpm-software-management/libdnf#1139
aka https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1935

This way libdnf's `extern "C"` over the glib headers doesn't apply
because we already processed that header.
2021-02-17 14:02:34 -05:00
Colin Walters
f8dc856aa9 compose: Add missing #include <optional>
This only fails on newer gcc/libstdc++ apparently in current
Fedora rawhide: https://kojipkgs.fedoraproject.org//work/tasks/4372/62164372/build.log
2021-02-17 14:02:34 -05:00
Kelvin Fan
626d021edb app/upgrade: Do not upgrade if updates driver registered
Do not perform an upgrade if detected that an updates driver has
been registered.

Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
2021-02-17 11:43:34 -05:00
Kelvin Fan
261b9cbb87 app/status: factor out get_sd_unit_objpath() function
Factor out a `get_sd_unit_objpath()` function into `libbuiltin`
so we can reuse the code in `builtin_upgrade`.
2021-02-17 11:43:34 -05:00