Commit Graph

3469 Commits

Author SHA1 Message Date
Colin Walters
b0eccadb22 live: Pass options GVariant down
I have no idea why I didn't do this originally, it's much
cleaner than parsing the options in C.
2021-03-08 20:54:18 +01:00
Colin Walters
01b59f20ba daemon: Move a bit more deployment-variant code to Rust
These are "base ostree" things we can move into the Rust
function.
2021-03-08 19:58:18 +01:00
dependabot[bot]
fbe01b403f build(deps): bump serde from 1.0.123 to 1.0.124
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.123 to 1.0.124.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.123...v1.0.124)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-08 11:06:40 -05:00
Colin Walters
3d7ac1d637 rust/client: derive(Debug, Clone)
Zincati wants this and it's a friendly thing to do.
2021-03-08 09:58:56 -05:00
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Jonathan Lebon
0f8dd0344d app/override: Hint at rpm-ostree override reset
It's always nice when apps provide useful hints about other commands you
may be interested in.

For instance, if they've done `rpm-ostree override replace/remove`,
let's be helpful and tell users that they can use `rpm-ostree override
reset` to unpin packages.
2021-03-05 19:04:06 -05:00
Colin Walters
8f77970683 client: Add API to fetch base commit metadata
Also desired by Zincati.
2021-03-05 18:20:03 -05:00
Colin Walters
5b647af14a rust/client: Add Deployment/get_base_commit() API
Zincati wants this.
2021-03-05 18:20:03 -05:00
Colin Walters
14c17cfe52 rust/client: Add methods to find/require booted deployment
This is a common need.
2021-03-05 18:20:03 -05:00
Colin Walters
1662f246fd rust/client: Add a CliClient with agent ID, require for status
In prep for adding more methods, require the caller to identify
themselves.

For now this is `CliClient` - one could imagine in the future
we actually do direct DBus, but there's a whole other world
of stuff there.
2021-03-05 18:20:03 -05:00
dependabot[bot]
b571819e15 build(deps): bump openat-ext from 0.1.12 to 0.1.13
Bumps [openat-ext](https://github.com/cgwalters/openat-ext) from 0.1.12 to 0.1.13.
- [Release notes](https://github.com/cgwalters/openat-ext/releases)
- [Commits](https://github.com/cgwalters/openat-ext/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-05 14:44:26 -05:00
Colin Walters
3354ca9d30 rust/client: Extend with more metadata for zincati
Add more metadata that zincati needs, like `base-commit-meta`
which includes the `fedora-coreos.stream` key and the cosa basearch,
etc.

Also `Derive(Debug)` since it's used in a cache struct that also
derives debug, and that's a friendly thing to do in general.
2021-03-05 05:47:01 -05:00
Colin Walters
6e589ff438 design/package-layering.md: Remove
This is long obsolete, we have better stuff in `docs/` now.
2021-03-04 16:21:51 -05:00
Colin Walters
d2510fdbe2 man: Add docs for apply-live
Let's try to keep up rigor around keeping the man pages up to date.
2021-03-04 16:21:51 -05:00
Colin Walters
0bae05d4c0 apply-live: Fix --target option
It is an option, not a positional.  (Not covered in CI it turns out)
2021-03-04 16:21:51 -05:00
Colin Walters
13804d84a1 Add more client Rust bindings, port apply-live builtin
This adds sufficient infrastructure to fully port the
`rpmostree-builtin-applylive.cxx` client code to Rust.
We just keep a stub entrypoint for now until we port
the rest of `rpmostree-builtin-ex.cxx`, at which point
a lot of C++ files go away.

The "finish" bits move from the daemon-oriented `live.rs`
into a new `rust/src/builtins` directory.  I'd like
to try to more cleanly split up the Rust sources along
core(shared)/client/daemon directories in the future.
2021-03-04 09:17:21 -05:00
Colin Walters
6fd9db246a Add basic Rust CLI bindings for DBus
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.

A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.

My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
2021-03-04 09:17:21 -05:00
Colin Walters
283824c6cb app: Factor out a helper function to register as a client
Prep for exposing just this functionality via cxxrs so we
can more easily write CLI entrypoints in Rust.
2021-03-04 09:17:21 -05:00
Colin Walters
e21914f677 app: Rename dbus-helpers to clientlib
Originally it was dbus helpers, but it grew into "shared code for client",
so let's name the file better.
2021-03-04 09:17:21 -05:00
Colin Walters
8dd5583ae9 Remove cbindgen
We have fully transitioned to cxx-rs!  This drops a lot of now
dead code; only one binding system to think about generating
source code.  For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
2021-03-03 16:51:38 -05:00
Colin Walters
60e605b34e Introduce CxxGObjectArray, use in lockfile code
cxx-rs only supports a few basic types in `Vec<T>`/`CxxVector<T>`
and we need to pass an array of GObjects in a few cases.
Add a wrapper class hack instead of using `u64` so we at least
have some basic safety here and have a convenient place to
grep for later when we want to improve this.
2021-03-02 16:25:32 -08:00
Jonathan Lebon
5c1911445c Finish moving lockfile to cxx.rs
This moves the `ror_lockfile_write` to cxx.rs, which brings us closer to
getting rid of cbindgen now.

There's one massive hack this uses, which is that we pass an array of
pointers to `DnfPackage` and `DnfRepo` objects as u64. We'll want to
circle back and fix that up once either cxx.rs supports natively arrays
of pointers, or we just come up with our own wrapper type for it.

But for now at least, this unblocks the cbindgen transition and hacking
on the lockfile code.
2021-03-02 16:25:32 -08:00
Colin Walters
85f22baec7 tests: Add RPMs to installed kola tests, port layering-local
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.

We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over.  Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data.  Add the concept
of multiple repo versions too.

Right now we only generate the one RPM needed for the `layering-local`
test and port it.
2021-03-02 09:47:03 -08:00
Luca BRUNO
6b13f2596c rust/treefile: add support for check-passwd/groups data entries
This adds treefile support for both `check-passwd` and `check-groups`
entries with "type: data".
2021-03-02 08:08:05 -08:00
Luca BRUNO
a7e333925e cargo: add maplit 2021-03-02 08:08:05 -08:00
Luca BRUNO
95446c8187 cargo: sort dependencies 2021-03-02 08:08:05 -08:00
Colin Walters
9b3612b3ff cxxrsutil: Generalize cxxrs_bind! macro
This generalizes our binding macro, so we can trivially e.g.
add `Variant` into the entry of bound glib types.
2021-03-02 01:14:30 -08:00
Colin Walters
7367aa8469 tests/apply-live: Upgrade+liveapply no-package changes
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction".  I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
2021-03-01 20:40:48 +01:00
dependabot[bot]
348384a914 build(deps): bump cbindgen from 0.16.0 to 0.18.0
Bumps [cbindgen](https://github.com/eqrion/cbindgen) from 0.16.0 to 0.18.0.
- [Release notes](https://github.com/eqrion/cbindgen/releases)
- [Changelog](https://github.com/eqrion/cbindgen/blob/master/CHANGES)
- [Commits](https://github.com/eqrion/cbindgen/compare/v0.16.0...v0.18.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-01 19:53:56 +01:00
Colin Walters
b0780f0efd ex: Drop erroneous REQUIRES_ROOT flag on apply-live, initramfs-etc
These commands use DBus so fall under the usual polkit usage,
there's no reason to hard require root.  I think this was a copy-pasta.
2021-03-01 19:53:50 +01:00
Colin Walters
4d7de4fa59 tests/vmcheck: Use .cosa over COSA_DIR
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.

Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
2021-03-01 18:53:56 +01:00
Colin Walters
2b294d68bd client: Remove peer_pid value from option parsing
Instead of passing the pid back up the stack and using a cleanup
function on it to invoke `kill()`, use `PR_SET_PDEATHSIG` which
has the kernel take care of this for us.

(In practice we don't actually use this peer functionality anymore
 because all of the client/daemon code kind of requires being run under systemd
 on a real system now)

This shrinks the API surface and is much less repetitive in
the codebase.

Prep for moving more of the CLI code to Rust.
2021-03-01 18:53:50 +01:00
dependabot[bot]
6dd2df19bc build(deps): bump serde_json from 1.0.62 to 1.0.64
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.62 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.62...v1.0.64)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-01 17:46:49 +01:00
Colin Walters
732e0f232a daemon: Fix crash on upgrade with remote:checksum
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).

The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.

But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.

I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.

Closes: https://github.com/coreos/rpm-ostree/issues/2603
2021-03-01 16:57:51 +01:00
Colin Walters
10ad264ecd daemon: Heap allocate GVariantDict to pass to Rust
I was looking at a different problem and noticed a spam of
```
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_ref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_unref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_ref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_unref: assertion 'is_valid_heap_dict (dict)' failed
```

in the journal.  It turns out that the Rust bindings try to call
`g_variant_ref()`, but that's (understandably) not allowed on
a stack allocated instance.

Since we heap allocate a *ton* here, let's just heap allocate
the dict too.

Now, we also along the way stopped using the bits to inject
```
[Service]
Environment=G_DEBUG=fatal-warnings
```
in our CI.  I'll look at that as a followup.
2021-02-26 16:54:26 +01:00
Luca BRUNO
de7d20e43b libpriv/passwd: move UID/GID checker to Rust
This ports to Rust the logic for checking whether a directory tree
contains content owned by a given UID/GID.
2021-02-26 00:34:56 +01:00
Luca BRUNO
dbe2d50cf1 treefile: add getters for check-passwd and check-groups
This add default-aware getters for `check-passwd` and `check-groups`.
2021-02-25 19:24:12 +01:00
Timothée Ravier
101d531f3b Install rpm-ostree sysusers config and countme units
Install the systemd unit, timer and sysusers configuration for Count Me
support. We do not enable or pull as a dependency those units by default
as this is a decision that should be taken at the distribution level and
needs support on the infrastructure side.

To enable those units in a disctribution package, you can add the
following symlink:
$ ln -snf /usr/lib/systemd/system/rpm-ostree-countme.timer /usr/lib/systemd/system/rpm-ostreed.service.wants/

or add the following config snippet to the rpm-ostreed.service unit:
Wants=rpm-ostree-countme.timer
2021-02-25 16:58:14 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Kelvin Fan
e10df33e88 app/dbus-helpers: Factor out updates driver-related functions
Following https://github.com/coreos/rpm-ostree/pull/2566, we would
like to give other commands (deploy and rebase) the same treatment.

Factor out functions to retrieve systemd unit docs and a function
that detects updates drivers so they can be reused elsewhere.
2021-02-25 06:44:53 +01:00
dependabot[bot]
380ebc5ada build(deps): bump cxx-build from 1.0.31 to 1.0.32
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.31...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-24 23:27:25 +01:00
Colin Walters
927e02100f docs/architecture-core.md: New file
This is long overdue.  Some of this came up in recent
conversation.  Let's keep up some continual background
momentum on documentation, just like CI.

Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2021-02-24 22:01:23 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
ce20267b2d ostree_utils: Add a workaround for optional ref
Until we have https://github.com/ostreedev/ostree/pull/2282
in an ostree release and an updated `ostree` crate.
2021-02-23 21:51:22 -05:00
Colin Walters
0544d1c92d ci: Drop tests/vmcheck/image.qcow2, use COSA_DIR/.cosa
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default.  We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.

Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
2021-02-23 17:23:26 -05:00
Colin Walters
d8230bfb6d daemon: Move some "deployment variant" generation to Rust
More prep for https://github.com/coreos/rpm-ostree/pull/2388

This was actually also my first time really trying out the
latest gtk-rs `glib::Variant` API, which is one of the major
things we need to use to progress oxidation more.
2021-02-23 13:56:26 -05:00
Colin Walters
6cf0f87412 docs/HACKING.md: Various updates
In particular describe the cxx tools trap.
2021-02-23 12:40:27 -05:00
dependabot[bot]
20feab6d2a build(deps): bump serde_yaml from 0.8.16 to 0.8.17
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.16 to 0.8.17.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.16...0.8.17)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-23 11:07:28 -05:00
dependabot[bot]
170095bd60 build(deps): bump nix from 0.19.1 to 0.20.0
Bumps [nix](https://github.com/nix-rust/nix) from 0.19.1 to 0.20.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-22 15:33:41 -05:00