Commit Graph

3430 Commits

Author SHA1 Message Date
Kelvin Fan
705b22df28 app/{deploy,rebase}: Do not deploy/rebase if updates driver registered
Follow up to https://github.com/coreos/rpm-ostree/pull/2566.
Error out if users try to manually do a deploy/rebase if an updates
driver is registered. Provide `--bypass-driver` option to proceed
anyway.
2021-02-25 06:44:53 +01:00
Kelvin Fan
e10df33e88 app/dbus-helpers: Factor out updates driver-related functions
Following https://github.com/coreos/rpm-ostree/pull/2566, we would
like to give other commands (deploy and rebase) the same treatment.

Factor out functions to retrieve systemd unit docs and a function
that detects updates drivers so they can be reused elsewhere.
2021-02-25 06:44:53 +01:00
dependabot[bot]
380ebc5ada build(deps): bump cxx-build from 1.0.31 to 1.0.32
Bumps [cxx-build](https://github.com/dtolnay/cxx) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.31...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-24 23:27:25 +01:00
Colin Walters
927e02100f docs/architecture-core.md: New file
This is long overdue.  Some of this came up in recent
conversation.  Let's keep up some continual background
momentum on documentation, just like CI.

Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2021-02-24 22:01:23 +01:00
Colin Walters
5a79ca9035 apply-live: Rework to use refs to store state
Came out of discussion in https://github.com/coreos/rpm-ostree/pull/2581
around some racy code for checking for the live commit object.

The reliability of apply-live depends on the
underlying commits not being garbage collected.  Our diff logic
is in terms of ostree commits, not the physical filesystem (this
allows us to make various optimizations too).

Ultimately I think we should drive some of the live-apply
logic into libostree itself; we can more easily have an atomic
state file instead of the two split refs.

(Or perhaps what we should add to ostree is like a refs.d model
 where a single atomic file can refer to multiple commits)

For now though let's rework the code here to write refs.  We
retain the file in `/run` as just a "stamp file" that signals
that a deployment has had `apply-live` run.
2021-02-23 21:51:22 -05:00
Colin Walters
ce20267b2d ostree_utils: Add a workaround for optional ref
Until we have https://github.com/ostreedev/ostree/pull/2282
in an ostree release and an updated `ostree` crate.
2021-02-23 21:51:22 -05:00
Colin Walters
0544d1c92d ci: Drop tests/vmcheck/image.qcow2, use COSA_DIR/.cosa
Now that `cosa build-fast` writes to `.cosa`, teach our
test suite to pick that up by default.  We don't anymore
support non-CoreOS (i.e. non-Ignition) hosts for our test
suite, so making this more CoreOS specific is fine.

Then use the "standard" COSA_DIR as a way to find the target
cosa dir in the e2e CI.
2021-02-23 17:23:26 -05:00
Colin Walters
d8230bfb6d daemon: Move some "deployment variant" generation to Rust
More prep for https://github.com/coreos/rpm-ostree/pull/2388

This was actually also my first time really trying out the
latest gtk-rs `glib::Variant` API, which is one of the major
things we need to use to progress oxidation more.
2021-02-23 13:56:26 -05:00
Colin Walters
6cf0f87412 docs/HACKING.md: Various updates
In particular describe the cxx tools trap.
2021-02-23 12:40:27 -05:00
dependabot[bot]
20feab6d2a build(deps): bump serde_yaml from 0.8.16 to 0.8.17
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.16 to 0.8.17.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.16...0.8.17)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-23 11:07:28 -05:00
dependabot[bot]
170095bd60 build(deps): bump nix from 0.19.1 to 0.20.0
Bumps [nix](https://github.com/nix-rust/nix) from 0.19.1 to 0.20.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-22 15:33:41 -05:00
Luca BRUNO
7283aef8af libpriv/passwd: remove unused functions
This drops some functions that used to be part of C unit-tests but
are currently dead code.
2021-02-22 13:24:41 -05:00
Luca BRUNO
9b94e85e5b libpriv/passwd: move compose preparation to Rust
This moves passwd/group compose preparation logic to Rust,
dropping all the remaining minor helpers related to JSON parsing,
file stream creation, and entries deduplication.
2021-02-22 11:44:13 -05:00
Luca BRUNO
a1ecc3d583 rust: add missing SPDX-License-Identifier tag
This adds a missing `SPDX-License-Identifier` tag, fixing CI.
2021-02-22 09:07:33 -05:00
Colin Walters
49c55089d2 lockfile: Be robust to non-seconds-since-epoch repodata
We should really have safety comments around our use of `.unwrap()`.
This Google kubernetes rpm-md repo appears to use seconds-since-epoch.
We could try to detect that, probably best done in libdnf.

I think it'd be better to get them to match the informal schema
for this actually though instead.

For now let's just log an error and not crash.

Closes: https://github.com/coreos/rpm-ostree/issues/2600
2021-02-19 15:56:29 -05:00
Colin Walters
f882d6ddff rust: Add SPDX-License-Identifier and validate it in ci/codestyle.sh
It turns out we accidentally added GPL'd code into the Rust
side, which wasn't intentional on my part and I think it's since
been copied around.

Honestly I think half of the problem is the gigantic
"blah blah blah GNU General blah blah" just makes people's eyes
glaze over.  In contrast the `SPDX-License-Identifier` is short
and obvious.

So let's validate that in CI.

This follows a similar change in ostree:
https://github.com/ostreedev/ostree/pull/1439

If we merge this I'll do the C/C++ side too after that.
2021-02-19 15:56:23 -05:00
Colin Walters
a62ace8d5b codestyle: Remove tabdamage.txt in OK case
Avoid leaving cruft in the srcdir.
2021-02-19 15:56:23 -05:00
dependabot[bot]
1421dcbc14 build(deps): bump openat-ext from 0.1.11 to 0.1.12
Bumps [openat-ext](https://github.com/cgwalters/openat-ext) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/cgwalters/openat-ext/releases)
- [Commits](https://github.com/cgwalters/openat-ext/compare/v0.1.11...v0.1.12)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-19 15:05:23 -05:00
Colin Walters
27f9734300 apply-live: Delete code to check for live commit post-txn
Per discussion in
https://github.com/coreos/rpm-ostree/pull/2581#discussion_r578621458
it is inherently racy and we'll do a followup fix to write a
ref.
2021-02-19 09:08:22 -05:00
Colin Walters
246f02fd47 apply-live: Move client-side finish to Rust
Now that we have an RPM diff+printing binding, we can
move the client side postprocessing for `apply-live`
to Rust.
2021-02-19 09:08:22 -05:00
Colin Walters
04e0c4e01a Fix progress API to have "output message" separate from task
A lot of our output is outside of a "task"; the Rust binding
incorrectly made it a method on `Progress`.  This is really
just a `println!()` that is backed by our dispatch system.
2021-02-19 09:08:22 -05:00
Colin Walters
1d242ddb59 Add a C++ rpmdb-diff API wrapping the C one, bind in Rust
I'd like to compute diffs in apply-live to differentiate
between "pure layering" versus modifications/removals.
2021-02-19 09:08:22 -05:00
Colin Walters
29a2304fe2 daemon: Rename transaction_livefs → transaction_apply_live
Continuing the renaming.
2021-02-19 09:08:22 -05:00
Colin Walters
ffe72d28ec app: Rename livefs.cxx → apply-live.cxx
Continuing the renaming.
2021-02-19 09:08:22 -05:00
Timothée Ravier
790e709360 countme: Persist last timer trigger to handle system suspends
Set `Persistent=yes` in the timer unit to make sure that systems that
are not running 24/7 gets counted when they wake up if they missed the
previous timer run / counting window.

This is mostly for Fedora Silverblue and IoT like systems as Fedora
CoreOS systems usually run 24/7.

https://www.freedesktop.org/software/systemd/man/systemd.timer.html#Persistent=
2021-02-18 15:14:34 -05:00
Colin Walters
0c9f7ab7eb Add daemon.rs with one helper function, use it from C++
Trying to port to use origin Rust code, I think actually
it gets simpler if we move more to Rust to start.
2021-02-18 10:22:39 -05:00
Colin Walters
3910e77ae8 daemon: Refactor deployment variant generation code
Changes to this ad-hoc `a{sv}` code have been scattered
around over time.  Move the "core" bits to the top of
the function, then have the more complex stuff that depends
on the origin (and refspec/layering status) later.

Prep for further cleanups.
2021-02-18 10:22:39 -05:00
dependabot[bot]
2306c919ec build(deps): bump system-deps from 2.0.3 to 3.0.0
Bumps [system-deps](https://github.com/gdesmott/system-deps) from 2.0.3 to 3.0.0.
- [Release notes](https://github.com/gdesmott/system-deps/releases)
- [Commits](https://github.com/gdesmott/system-deps/compare/v2.0.3...v3.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-18 08:06:41 -05:00
Colin Walters
9eb217bd89 ci: Add Prow e2e testing
Our "CoreOS CI Jenkins" flow is great, but the reliability/debuggabilty
of the system isn't the best.  This is an experient to *add* another CI
flow that can be run in Prow too.

I'd like to at least have a basic e2e (compose tree, run client tests)
flow that can operate in Prow so that if CoreOS CI Jenkins is down or
flaky, we at least have some confidence in our commits.

Longer term I think we can figure out how to better share CI flows
across these:
https://github.com/coreos/fedora-coreos-tracker/issues/263#issuecomment-772903759
2021-02-18 06:52:42 -05:00
Colin Walters
8a62ad9f3d Add gobj_rewrap() API to pass glib-rs objects back to C++
When we started using cxxrs, most of the glib-rs objects like
`OstreeRepo`/`OstreeSysroot` were owned by C++ and passed
down into Rust.  That motivated the addition of the special
bridging infrastructure to re-create a glib-rs wrapper
type from what cxxrs wants (a `Pin<&mut T>`).

But now that we're adding more in Rust, we have the need
to pass these objects back into C++.  In fact this will
hopefully soon because the default case as more of the
binary entrypoint becomes Rust.

Add another trait with a method `gobj_rewrap()` that converts
in the other direction.  This implementation took me a number
of tries before I finally settled on simply using `mem::transmute()`.
There are a *lot* of caveats listed on the docs for that function,
but I think it really is what we want here.  See the link for pending work
on a Rust RFC to enable safe transmutes for some cases, and I believe
that would cover this use case:
https://internals.rust-lang.org/t/pre-rfc-v2-safe-transmute/11431

I've verified this works in a separate patch, but this commit
also adds a simple test case - this goes all the way from:
   Rust glib-rs `ostree::Repo` (holding strong ref)
   -> Rust `Pin<&mut ostree_sys::OstreeRepo>`
   -> (internal cxx-rs C bridge)
   -> C++ `OstreeRepo&` reference
   -> C `OstreeRepo*` pointer
Which is quite the dance if you think about it!
2021-02-18 06:29:41 -05:00
Colin Walters
8d9e113e8f rust/countme: Fix clippy lint by refactoring serialization
clippy complains about `to_*` not taking `&self` - I think
we can simplify this more by using our `write_file_with()` API
that we're using in other places.  It was explicitly designed
with serde in mind.  It's just more efficient and nicer to
serialize to a `BufWriter` instead of to a string, then
writing the string.
2021-02-18 05:27:41 -05:00
Colin Walters
9854a3aa8e rust/extensions: Fix two clippy lints
These are easy fixes.
2021-02-18 05:27:41 -05:00
Colin Walters
c027b638db rust: Quiet clippy Vec<Foo> arg
cxx.rs only supports `Vec<String>` and not `&[String]` right now.
2021-02-18 05:27:41 -05:00
Colin Walters
35fcf3eac0 rust: Fix some misc clippy lints
Nothing important here really, but clippy is useful so
let's try to keep it quiet.
2021-02-18 05:27:41 -05:00
dependabot[bot]
5f3b692e60 build(deps): bump libc from 0.2.84 to 0.2.86
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.84 to 0.2.86.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.84...0.2.86)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-18 05:13:41 -05:00
dependabot[bot]
3e7ad771fb build(deps): bump memfd from 0.3.0 to 0.4.0
Bumps [memfd](https://github.com/lucab/memfd-rs) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/lucab/memfd-rs/releases)
- [Commits](https://github.com/lucab/memfd-rs/compare/v0.3.0...v0.4.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-17 18:42:32 -05:00
dependabot[bot]
f343146efd build(deps): bump cxx from 1.0.30 to 1.0.32
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.30 to 1.0.32.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.30...1.0.32)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-17 17:42:33 -05:00
Colin Walters
047f931746 Release 2021.2 2021-02-17 15:17:33 -05:00
Colin Walters
8a1b2f3739 libdnf-sys: Include gio.h before libdnf.h
See b77f710cfb

Alternative fix to rpm-software-management/libdnf#1139
aka https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1935

This way libdnf's `extern "C"` over the glib headers doesn't apply
because we already processed that header.
2021-02-17 14:02:34 -05:00
Colin Walters
f8dc856aa9 compose: Add missing #include <optional>
This only fails on newer gcc/libstdc++ apparently in current
Fedora rawhide: https://kojipkgs.fedoraproject.org//work/tasks/4372/62164372/build.log
2021-02-17 14:02:34 -05:00
Kelvin Fan
626d021edb app/upgrade: Do not upgrade if updates driver registered
Do not perform an upgrade if detected that an updates driver has
been registered.

Add --bypass-driver option to force an upgrade regardless of whether an
updates driver has been registered.
2021-02-17 11:43:34 -05:00
Kelvin Fan
261b9cbb87 app/status: factor out get_sd_unit_objpath() function
Factor out a `get_sd_unit_objpath()` function into `libbuiltin`
so we can reuse the code in `builtin_upgrade`.
2021-02-17 11:43:34 -05:00
Colin Walters
e6c045cada Add an rpmostree-client sub-crate
This is intended to be published to https://crates.io/crates/rpmostree-client
Part of https://github.com/coreos/rpm-ostree/issues/2389

This directly imports the code from
5551c54c6e/tests/inst/src/rpmostree.rs

Once merged and released I'll try converting the ostree test suite
over as well as Zincati.

Internally add a testutils helper to validate it works.
2021-02-16 19:22:26 -05:00
Colin Walters
7bf885ecf7 tests: Fix bodhi link
From review in https://github.com/coreos/rpm-ostree/pull/2585#discussion_r577149232
2021-02-16 18:18:27 -05:00
Colin Walters
c48e8bfad3 ci: Rework build/test dependency install
Now that `ci/installdeps.sh` gracefully exits if run as non-root,
we can fold the cargo bits into the our build scripts and avoid
invoking both of them.

However, now we need to split test deps to separate file because
we won't have `cargo` in the main cosa pod.  This also fixes a FIXME.

Steal the `grep` invocation from cosa and make it a declarative
text file so we can have comments per package etc.
2021-02-16 18:18:27 -05:00
Colin Walters
d7b75f8459 build-sys: Ignore .cosa
This dir was added recently in coreos-assembler.
2021-02-16 18:18:27 -05:00
Colin Walters
836315084e ci: Add time prefixing before most commands
We don't have timestamps set up right now, and including
timing information is easy and useful to debug CI speed.
2021-02-16 18:18:27 -05:00
Colin Walters
d5227781de ci: Drop: distro-sync, continuous repo
Doing the distro-sync costs ~3 minutes per execution, and it
happens multiple times.  Let's just ensure our images are up
to date instead.

Also drop the continuous repo (for now) - we added this
to test bleeding edge ostree, but I think we need to reintroduce
"git master" builds as whole separate CI flow (layered container)
instead.
2021-02-16 18:18:27 -05:00
Colin Walters
c1ff933bc3 tests/layering-non-root-caps: Adapt to new libcap output
I didn't deep dive on this, just observed that the new output
matches what we laid down.  There's ~2.5 years of changes
and ~200 commits between 2.28 and 2.48.
2021-02-16 17:08:27 -05:00
Colin Walters
04839e200f Misc doc tweaks/cleanups
- Link to osbuild blog
- Drop CentOS 7 stuff
- clarify goals a bit
2021-02-12 16:45:46 -05:00