Commit Graph

3476 Commits

Author SHA1 Message Date
Colin Walters
1dc7503838 ci/prow: Install in target cosa, also install and run our tests
This is a major downside of reworking and generating new CI
flows, it's super easy to lose testing what you intend to.

Also, we clearly need to figure out a flow where this is shared
across repos, since I don't want to copy-paste this into e.g. ostree too.
That's https://github.com/coreos/fedora-coreos-tracker/issues/263
2021-03-10 16:58:49 +01:00
dependabot[bot]
b02987a22e build(deps): bump libc from 0.2.86 to 0.2.88
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.86 to 0.2.88.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.86...0.2.88)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-08 22:07:49 +01:00
Colin Walters
3041d648bb install: Add --apply-live
OK I think it's time.  This exposes the `apply-live` functionality
as implicitly stable, but specific to the package install case.

I'd like to add more intelligence to `apply-live` around separating
pure "additions" (as in this case) versus package (file) changes.

The change here doesn't try to do that; the implementation is
incredibly simple, we just have the client chain together the two
distinct transactions.
2021-03-08 20:54:18 +01:00
Colin Walters
485c1862ce live: Disallow replacement/removal by default, add --allow-replacement
There's a huge difference between live updates that change
existing things, versus simply adding new packages (files).

The latter is really quite safe, and live layering is one
of the most requested features.
2021-03-08 20:54:18 +01:00
Colin Walters
2a9423ccb8 tests: Port apply-live to kola ext tests
Continuing the momentum to use kola ext tests.

One obvious benefit of this as the porting continues
is that we can share our built test RPMs across
different tests, e.g. we can have a `testdaemon` package
instead of a `test-livefs-service` package.
2021-03-08 20:54:18 +01:00
Colin Walters
0ec09e6155 live: Hold a single sysroot object
Rather than creating one only when `--reset` is used, hold
one open across the whole operation.
2021-03-08 20:54:18 +01:00
Colin Walters
03fd76d615 live: Use a shared constant for option
In the name of avoiding typos etc.
2021-03-08 20:54:18 +01:00
Colin Walters
b0eccadb22 live: Pass options GVariant down
I have no idea why I didn't do this originally, it's much
cleaner than parsing the options in C.
2021-03-08 20:54:18 +01:00
Colin Walters
01b59f20ba daemon: Move a bit more deployment-variant code to Rust
These are "base ostree" things we can move into the Rust
function.
2021-03-08 19:58:18 +01:00
dependabot[bot]
fbe01b403f build(deps): bump serde from 1.0.123 to 1.0.124
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.123 to 1.0.124.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.123...v1.0.124)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-08 11:06:40 -05:00
Colin Walters
3d7ac1d637 rust/client: derive(Debug, Clone)
Zincati wants this and it's a friendly thing to do.
2021-03-08 09:58:56 -05:00
Jonathan Lebon
175de36732 compose/extensions: Handle no --base-rev
In this case, let's default to the tip of the tree ref.

Closes: #2633
2021-03-07 19:46:52 -05:00
Jonathan Lebon
0f8dd0344d app/override: Hint at rpm-ostree override reset
It's always nice when apps provide useful hints about other commands you
may be interested in.

For instance, if they've done `rpm-ostree override replace/remove`,
let's be helpful and tell users that they can use `rpm-ostree override
reset` to unpin packages.
2021-03-05 19:04:06 -05:00
Colin Walters
8f77970683 client: Add API to fetch base commit metadata
Also desired by Zincati.
2021-03-05 18:20:03 -05:00
Colin Walters
5b647af14a rust/client: Add Deployment/get_base_commit() API
Zincati wants this.
2021-03-05 18:20:03 -05:00
Colin Walters
14c17cfe52 rust/client: Add methods to find/require booted deployment
This is a common need.
2021-03-05 18:20:03 -05:00
Colin Walters
1662f246fd rust/client: Add a CliClient with agent ID, require for status
In prep for adding more methods, require the caller to identify
themselves.

For now this is `CliClient` - one could imagine in the future
we actually do direct DBus, but there's a whole other world
of stuff there.
2021-03-05 18:20:03 -05:00
dependabot[bot]
b571819e15 build(deps): bump openat-ext from 0.1.12 to 0.1.13
Bumps [openat-ext](https://github.com/cgwalters/openat-ext) from 0.1.12 to 0.1.13.
- [Release notes](https://github.com/cgwalters/openat-ext/releases)
- [Commits](https://github.com/cgwalters/openat-ext/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-05 14:44:26 -05:00
Colin Walters
3354ca9d30 rust/client: Extend with more metadata for zincati
Add more metadata that zincati needs, like `base-commit-meta`
which includes the `fedora-coreos.stream` key and the cosa basearch,
etc.

Also `Derive(Debug)` since it's used in a cache struct that also
derives debug, and that's a friendly thing to do in general.
2021-03-05 05:47:01 -05:00
Colin Walters
6e589ff438 design/package-layering.md: Remove
This is long obsolete, we have better stuff in `docs/` now.
2021-03-04 16:21:51 -05:00
Colin Walters
d2510fdbe2 man: Add docs for apply-live
Let's try to keep up rigor around keeping the man pages up to date.
2021-03-04 16:21:51 -05:00
Colin Walters
0bae05d4c0 apply-live: Fix --target option
It is an option, not a positional.  (Not covered in CI it turns out)
2021-03-04 16:21:51 -05:00
Colin Walters
13804d84a1 Add more client Rust bindings, port apply-live builtin
This adds sufficient infrastructure to fully port the
`rpmostree-builtin-applylive.cxx` client code to Rust.
We just keep a stub entrypoint for now until we port
the rest of `rpmostree-builtin-ex.cxx`, at which point
a lot of C++ files go away.

The "finish" bits move from the daemon-oriented `live.rs`
into a new `rust/src/builtins` directory.  I'd like
to try to more cleanly split up the Rust sources along
core(shared)/client/daemon directories in the future.
2021-03-04 09:17:21 -05:00
Colin Walters
6fd9db246a Add basic Rust CLI bindings for DBus
This stubs out sufficient infrastructure for us to register
as a client and call the Moo API.

A glaring problem here is the lack of extensive `glib::Variant`
bindings; that's covered in the next gtk-rs release.

My real goal was to try porting the `rpmostree-builtin-apply-live.cxx`
code entirely to Rust, but there's more to do to expose the
transaction helper APIs we have.
2021-03-04 09:17:21 -05:00
Colin Walters
283824c6cb app: Factor out a helper function to register as a client
Prep for exposing just this functionality via cxxrs so we
can more easily write CLI entrypoints in Rust.
2021-03-04 09:17:21 -05:00
Colin Walters
e21914f677 app: Rename dbus-helpers to clientlib
Originally it was dbus helpers, but it grew into "shared code for client",
so let's name the file better.
2021-03-04 09:17:21 -05:00
Colin Walters
8dd5583ae9 Remove cbindgen
We have fully transitioned to cxx-rs!  This drops a lot of now
dead code; only one binding system to think about generating
source code.  For example, a notable advantage of cxx-rs
is it doesn't scan the whole source code, so running `make`
doesn't spew errors from cbindgen not understanding bits.
2021-03-03 16:51:38 -05:00
Colin Walters
60e605b34e Introduce CxxGObjectArray, use in lockfile code
cxx-rs only supports a few basic types in `Vec<T>`/`CxxVector<T>`
and we need to pass an array of GObjects in a few cases.
Add a wrapper class hack instead of using `u64` so we at least
have some basic safety here and have a convenient place to
grep for later when we want to improve this.
2021-03-02 16:25:32 -08:00
Jonathan Lebon
5c1911445c Finish moving lockfile to cxx.rs
This moves the `ror_lockfile_write` to cxx.rs, which brings us closer to
getting rid of cbindgen now.

There's one massive hack this uses, which is that we pass an array of
pointers to `DnfPackage` and `DnfRepo` objects as u64. We'll want to
circle back and fix that up once either cxx.rs supports natively arrays
of pointers, or we just come up with our own wrapper type for it.

But for now at least, this unblocks the cbindgen transition and hacking
on the lockfile code.
2021-03-02 16:25:32 -08:00
Colin Walters
85f22baec7 tests: Add RPMs to installed kola tests, port layering-local
I'd like to get to the point where we drop the `vmcheck.sh`/`libvm.sh` stuff.
Instead we use kola directly, and write our tests in a way that they
default to run on the target, not on the host because it's *much*
more natural to type e.g. `rpm-ostree upgrade` instead of `vm_rpmostree upgrade`.

We'd done a bit of porting, but a blocker was that a lot of our
tests dynamically generate RPMs and send them over.  Instead,
let's generate the RPMs ahead of time in a "build" step, then
they all get passed at once via kola ext data.  Add the concept
of multiple repo versions too.

Right now we only generate the one RPM needed for the `layering-local`
test and port it.
2021-03-02 09:47:03 -08:00
Luca BRUNO
6b13f2596c rust/treefile: add support for check-passwd/groups data entries
This adds treefile support for both `check-passwd` and `check-groups`
entries with "type: data".
2021-03-02 08:08:05 -08:00
Luca BRUNO
a7e333925e cargo: add maplit 2021-03-02 08:08:05 -08:00
Luca BRUNO
95446c8187 cargo: sort dependencies 2021-03-02 08:08:05 -08:00
Colin Walters
9b3612b3ff cxxrsutil: Generalize cxxrs_bind! macro
This generalizes our binding macro, so we can trivially e.g.
add `Variant` into the entry of bound glib types.
2021-03-02 01:14:30 -08:00
Colin Walters
7367aa8469 tests/apply-live: Upgrade+liveapply no-package changes
Came up on `#fedora-iot` channel, some people are hitting
"No packages in transaction".  I believe we have a bug,
but I didn't hit it with at least this simple test case.
It may be related to layering while doing this too, going to
test that next.
2021-03-01 20:40:48 +01:00
dependabot[bot]
348384a914 build(deps): bump cbindgen from 0.16.0 to 0.18.0
Bumps [cbindgen](https://github.com/eqrion/cbindgen) from 0.16.0 to 0.18.0.
- [Release notes](https://github.com/eqrion/cbindgen/releases)
- [Changelog](https://github.com/eqrion/cbindgen/blob/master/CHANGES)
- [Commits](https://github.com/eqrion/cbindgen/compare/v0.16.0...v0.18.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-01 19:53:56 +01:00
Colin Walters
b0780f0efd ex: Drop erroneous REQUIRES_ROOT flag on apply-live, initramfs-etc
These commands use DBus so fall under the usual polkit usage,
there's no reason to hard require root.  I think this was a copy-pasta.
2021-03-01 19:53:50 +01:00
Colin Walters
4d7de4fa59 tests/vmcheck: Use .cosa over COSA_DIR
I spent longer than I'd care to admit being confused why my
changes from `cosa build-fast` weren't being picked up.
We need to honor `.cosa` first because the expected case
is you have both set in the `build-fast` case.

Will look at fixing `kola spawn` to handle all this too; the
problem is we haven't taught kola/cosa about `COSA_DIR`.
2021-03-01 18:53:56 +01:00
Colin Walters
2b294d68bd client: Remove peer_pid value from option parsing
Instead of passing the pid back up the stack and using a cleanup
function on it to invoke `kill()`, use `PR_SET_PDEATHSIG` which
has the kernel take care of this for us.

(In practice we don't actually use this peer functionality anymore
 because all of the client/daemon code kind of requires being run under systemd
 on a real system now)

This shrinks the API surface and is much less repetitive in
the codebase.

Prep for moving more of the CLI code to Rust.
2021-03-01 18:53:50 +01:00
dependabot[bot]
6dd2df19bc build(deps): bump serde_json from 1.0.62 to 1.0.64
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.62 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.62...v1.0.64)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-01 17:46:49 +01:00
Colin Walters
732e0f232a daemon: Fix crash on upgrade with remote:checksum
The refspec code really needs to be cleaned up and oxidized (and
unit tested more).

The original intention is that if you're pinned to a commit, we say
"No upgrade available" but we were crashing if one (understandably)
rebased to the combination of `remote:checksum` instead of just `checksum`.

But, we can't change the classifier to call this `CHECKSUM` and
output an error, because that just conflicts with us accepting
the syntax `rpm-ostree rebase :<checksum>`.

I'm actually coming around to the idea that this `remote:checksum` syntax means
"no upgrade available" is correct, whereas pinning to just `checksum`
is more of an error when you try to upgrade.

Closes: https://github.com/coreos/rpm-ostree/issues/2603
2021-03-01 16:57:51 +01:00
Colin Walters
10ad264ecd daemon: Heap allocate GVariantDict to pass to Rust
I was looking at a different problem and noticed a spam of
```
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_ref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_unref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_ref: assertion 'is_valid_heap_dict (dict)' failed
Feb 24 23:06:37 cosa-devsh rpm-ostree[1099]: g_variant_dict_unref: assertion 'is_valid_heap_dict (dict)' failed
```

in the journal.  It turns out that the Rust bindings try to call
`g_variant_ref()`, but that's (understandably) not allowed on
a stack allocated instance.

Since we heap allocate a *ton* here, let's just heap allocate
the dict too.

Now, we also along the way stopped using the bits to inject
```
[Service]
Environment=G_DEBUG=fatal-warnings
```
in our CI.  I'll look at that as a followup.
2021-02-26 16:54:26 +01:00
Luca BRUNO
de7d20e43b libpriv/passwd: move UID/GID checker to Rust
This ports to Rust the logic for checking whether a directory tree
contains content owned by a given UID/GID.
2021-02-26 00:34:56 +01:00
Luca BRUNO
dbe2d50cf1 treefile: add getters for check-passwd and check-groups
This add default-aware getters for `check-passwd` and `check-groups`.
2021-02-25 19:24:12 +01:00
Timothée Ravier
101d531f3b Install rpm-ostree sysusers config and countme units
Install the systemd unit, timer and sysusers configuration for Count Me
support. We do not enable or pull as a dependency those units by default
as this is a decision that should be taken at the distribution level and
needs support on the infrastructure side.

To enable those units in a disctribution package, you can add the
following symlink:
$ ln -snf /usr/lib/systemd/system/rpm-ostree-countme.timer /usr/lib/systemd/system/rpm-ostreed.service.wants/

or add the following config snippet to the rpm-ostreed.service unit:
Wants=rpm-ostree-countme.timer
2021-02-25 16:58:14 +01:00
Kelvin Fan
1c826e993b app/dbus-helpers: Don't error out if caller is updates driver
If the systemd unit associated with the client's PID is the updates
driver's unit, don't require the --bypass-driver option for operations
like upgrade, deploy, and rebase.
This is useful for updates drivers that shell out to rpm-ostree's
binary (e.g. Zincati, currently).
Also refactor some helper functions to make them more general and
reusable.
2021-02-25 06:44:53 +01:00
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Kelvin Fan
e10df33e88 app/dbus-helpers: Factor out updates driver-related functions
Following https://github.com/coreos/rpm-ostree/pull/2566, we would
like to give other commands (deploy and rebase) the same treatment.

Factor out functions to retrieve systemd unit docs and a function
that detects updates drivers so they can be reused elsewhere.
2021-02-25 06:44:53 +01:00
dependabot[bot]
380ebc5ada build(deps): bump cxx-build from 1.0.31 to 1.0.32
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.31...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-24 23:27:25 +01:00
Colin Walters
927e02100f docs/architecture-core.md: New file
This is long overdue.  Some of this came up in recent
conversation.  Let's keep up some continual background
momentum on documentation, just like CI.

Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2021-02-24 22:01:23 +01:00