Commit Graph

3516 Commits

Author SHA1 Message Date
Colin Walters
9248668d35 Also export PATH here
Prow job is running this one directly.
We really need to clean this up...
2021-02-03 12:00:08 -05:00
Colin Walters
9b2e78ed05 ci: Add a commit validation entrypoint 2021-02-03 12:00:08 -05:00
Kelvin Fan
4dd80a8b9c app/status: Use AutomaticUpdatesDriver if updates driven by external driver
When automatic updates are driven by an external driver like Zincati,
display e.g. `AutomaticUpdatesDriver: Zincati` instead of
`AutomaticUpdates: driven by Zincati`, since the latter might suggest it
is rpm-ostree's own built-in support for automatic updates.
2021-02-03 09:52:43 -05:00
Kelvin Fan
1e79e5abde app/status: Show update driver's state and status
Display update driver's systemd unit's `ActiveState` property.
Also display `StatusText` property if unit is a service unit and
property is available.
2021-02-03 09:52:43 -05:00
Colin Walters
d616f73539 scripts: Fix a stack use-after-free
I think this changed in a recent refactoring; basically since
we're passing this stack-allocated value to the child spawn
function we need to keep it alive.  This of course would
have been caught by Rust...

```
==672376==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7ffc290d9440 at pc 0x55c88c318946 bp 0x7ffc290d8b10 sp 0x7ffc290d8b08
    #0 0x55c88c318945 in script_child_setup src/libpriv/rpmostree-scripts.cxx:272
    #1 0x7f92089da902  (/lib64/libglib-2.0.so.0+0x9f902)
    #2 0x7f92089de20f  (/lib64/libglib-2.0.so.0+0xa320f)
    #3 0x7f92089de52e  (/lib64/libglib-2.0.so.0+0xa352e)
    #4 0x7f92089def02 in g_spawn_async_with_pipes (/lib64/libglib-2.0.so.0+0xa3f02)
    #5 0x7f9208b7445f  (/lib64/libgio-2.0.so.0+0xab45f)
    #6 0x7f9208b736d8 in g_subprocess_launcher_spawnv (/lib64/libgio-2.0.so.0+0xaa6d8)
    #7 0x55c88c3831b9 in rpmostree_bwrap_execute src/libpriv/rpmostree-bwrap.cxx:504
    #8 0x55c88c3836df in rpmostree_bwrap_run_captured src/libpriv/rpmostree-bwrap.cxx:450
    #9 0x55c88c31b5f1 in rpmostree_run_script_in_bwrap_container src/libpriv/rpmostree-scripts.cxx:469
    #10 0x55c88c31ca9d in impl_run_rpm_script src/libpriv/rpmostree-scripts.cxx:588
    #11 0x55c88c31d22b in run_script src/libpriv/rpmostree-scripts.cxx:637
    #12 0x55c88c31d22b in rpmostree_script_run_sync src/libpriv/rpmostree-scripts.cxx:778
    #13 0x55c88c2ef830 in run_script_sync src/libpriv/rpmostree-core.cxx:3661
    #14 0x55c88c30afa6 in rpmostree_context_assemble src/libpriv/rpmostree-core.cxx:4422
    #15 0x55c88c34a9af in install_packages src/app/rpmostree-compose-builtin-tree.cxx:451
    #16 0x55c88c34c174 in impl_install_tree src/app/rpmostree-compose-builtin-tree.cxx:925
    #17 0x55c88c350f84 in rpmostree_compose_builtin_tree src/app/rpmostree-compose-builtin-tree.cxx:1421
    #18 0x55c88c276ec8 in rpmostree_handle_subcommand src/app/libmain.cxx:405
    #19 0x55c88c27827c in rpmostree_main_inner src/app/libmain.cxx:521
    #20 0x55c88c27827c in rpmostreecxx::rpmostree_main(rust::cxxbridge1::Slice<rust::cxxbridge1::Str const>) src/app/libmain.cxx:546
    #21 0x55c88c271c25 in operator() /var/srv/walters/src/github/coreos/rpm-ostree/rpmostree-cxxrs.cxx:1257
    #22 0x55c88c271c25 in trycatch<rpmostreecxx::rpmostreecxx$cxxbridge1$rpmostree_main(rust::cxxbridge1::Slice<const rust::cxxbridge1::Str>)::<lambda()>, rpmostreecxx::rpmostreecxx$cxxbridge1$rpmostree_main(rust::cxxbridge1::Slice<const rust::cxxbridge1::Str>)::<lambda(char const*)> > /var/srv/walters/src/github/coreos/rpm-ostree/rpmostree-cxxrs.cxx:997
    #23 0x55c88c271c25 in rpmostreecxx$cxxbridge1$rpmostree_main /var/srv/walters/src/github/coreos/rpm-ostree/rpmostree-cxxrs.cxx:1255
    #24 0x55c88c0468f7 in rpmostree_rust::ffi::rpmostree_main::hfedda48c684245ce rust/src/lib.rs:25
    #25 0x55c88c0468f7 in rpm_ostree::inner_main::hf078b99ca4b270aa rust/src/main.rs:9
    #26 0x55c88c0468f7 in rpm_ostree::main::hc0ca527cfaa3f556 rust/src/main.rs:28
    #27 0x55c88c046b22 in core::ops::function::FnOnce::call_once::h8567110dac55274e /var/home/walters/.rustup/toolchains/1.48-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
    #28 0x55c88c046b22 in std::sys_common::backtrace::__rust_begin_short_backtrace::h1c67f2f52d05cfa0 /var/home/walters/.rustup/toolchains/1.48-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:137
    #29 0x55c88c045fd7 in main (/usr/bin/rpm-ostree+0xc9fd7)
    #30 0x7f92076091e1 in __libc_start_main (/lib64/libc.so.6+0x281e1)
    #31 0x55c88c045b9d in _start (/usr/bin/rpm-ostree+0xc9b9d)

Address 0x7ffc290d9440 is located in stack of thread T0 at offset 272 in frame
    #0 0x55c88c31a1af in rpmostree_run_script_in_bwrap_container src/libpriv/rpmostree-scripts.cxx:349
```
2021-02-03 04:30:23 -05:00
Colin Walters
058178680f ci: Skip installing deps if non-root
Assume in the pet/dev container case the user already has
deps.  And for Prow we're using the buildroot container which
should have them.
2021-02-02 19:34:22 -05:00
Colin Walters
57978fe099 ci/build-check: Don't make install
We may be running unprivileged (e.g. Prow), and `make install` needs to
be a uid 0 task.
2021-02-02 19:34:22 -05:00
Colin Walters
61a50e3d0e build-sys: Rebuild on C++ changes
Not running the code you think you are is an evil trap.
Fixes fallout from b122579222
2021-02-02 05:38:15 -05:00
Colin Walters
ded61a472f build-sys: Move some linkage purely to Rust
Now that we are generating solely a Rust binary, we can
have the canonical list of things to link on the Rust side.
2021-02-02 04:13:14 -05:00
dependabot[bot]
5b91ac6a3c build(deps): bump cxx from 1.0.28 to 1.0.29
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.28 to 1.0.29.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.28...1.0.29)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-01 12:20:49 -05:00
Colin Walters
d218f27188 Change main entrypoint to be Rust
We now have bidirectional calling between Rust and C++,
but we are generating two static libraries that we then
link together with a tiny C++ `main.cxx`.
Let's make another huge leap towards oxdiation by
having Rust be the entrypoint.  This way cargo natively
takes care of linking the internal Rust library, and
our C++ internals become the library.

In other words, we've now fully inverted from
"C app with internal Rust library"
to "Rust binary with internal C++ library".

In order to make this work though we have to finally
kill the C unit tests.  But mostly everything covered
there is either being converted to Rust, or covered
elsewhere anyways.

Now as the doc comments in `main.rs` say...this is
a bit awkward because all the CLI code is still in C++.
Porting stuff to use e.g. `structopt` natively would
be a bit of a slog.  For now, we basically rely on
the fact that the Rust-native CLIs are all hidden
commands.

Update submodule: libdnf
2021-02-01 08:17:52 -05:00
Colin Walters
e3375626d5 ci: Drop custom msrv checking
The way this tries to replace the system Rust is hacky and
actually I realized belatedly I may have broken it recently; basically
`installdeps.sh` re-adds the system one, and it's hard to be sure
with our current buildsystem we're using the newer one from `$PATH`.

What we really want to do here is use a CentOS8 buildroot,
which will automatically enforce this in a better way along
with solving other problems.  But right now we've broken
that because libdnf requires a too-new libmodulemd.

So let's just rely on the Fedora rust for now.
2021-02-01 04:54:52 -05:00
Colin Walters
14f75f94ef ci: Split clang into separate script, run it in CoreOS CI
Let's do a build with clang as a cleanly separate context
instead of serially; and also do it unconditionally.  This
is prep for turning on more `-Werror` flow in both cases,
and also using clang `scan-build` in CI.
2021-02-01 04:54:52 -05:00
Colin Walters
4b1120d57e ci: Drop testuser creation
Nothing is using this; our unit tests don't change uids and
most of our testing is in VMs.

Dropping this makes it easier to run the scripts outside of CI.
2021-02-01 04:54:52 -05:00
Colin Walters
83c7c90b83 tree-wide: Convert g_return.*if_fail to g_assert
See https://github.com/rpm-software-management/libdnf/pull/1127

Basically IMO the "return if fail" pattern is a carry-over
from C GTK+ apps where the idea is it's better to try to stumble
forward than have the app crash for the user.

In our case though, IMO we absolutely should just crash
if our assertions trip, because we're maintaining the user's
root filesystem.
2021-02-01 04:02:52 -05:00
dependabot[bot]
d9cd07f92b build(deps): bump libc from 0.2.82 to 0.2.84
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.82 to 0.2.84.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.82...0.2.84)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-01 03:37:52 -05:00
Colin Walters
ba9a17d861 build: Use -std=c++17
Notably this includes [std::optional](https://en.cppreference.com/w/cpp/utility/optional)
which I'd like to use.

The dnf-5-devel branch uses this too, and C++-17 is now well
supported in current GCC, which is available in RHEL8 DTS too.
2021-01-29 14:28:21 -05:00
Luca BRUNO
140357d549 rust/passwd: minor cleanup, use write_file_with_sync 2021-01-29 12:41:20 -05:00
Luca BRUNO
536bfc62eb libpriv/passwd-util: move migration logic to Rust
This moves passwd/group splitting logic to Rust, also decoupling
the two implementations in order to reduce overall complexity.
2021-01-29 08:15:48 -05:00
Colin Walters
e87a64576c Bridge "next version" API to Rust, use it for unit tests
This demonstrates well the strength of the cxx-rs approach;
we can keep an API in C++ but add unit tests in Rust which
just works much more nicely.

Prep for https://github.com/coreos/rpm-ostree/pull/2502
which wants to drop the C++ unit tests.
2021-01-29 05:29:47 -05:00
Colin Walters
7ab1936814 cliwrap: Fix indentation
We need to trim the starting whitespace, otherwise
it ends up in the script.
2021-01-28 21:26:47 -05:00
Colin Walters
bbaab4c1ff scripts: Remove, not used
The correct place for this would be...something like ostree-releng-tools
or coreos-assembler.  Or perhaps in the future a Rust ostree-ext-tools repository.
2021-01-28 15:58:24 -05:00
Colin Walters
47f6d9850e vagrant: Remove this
The default dev workflow is coreos-assembler based.
2021-01-28 15:58:24 -05:00
Colin Walters
1cef69cf8a lib: Don't link to libdnf
Prep for "Rust-as-main", where I want to build libdnf statically.
And this really completes the "library thinout" story because
now we avoid dragging our *private* `libdnf.so` into the caller's
address space, which can cause potential conflicts if they're
also linking the system one. (Which could easily occur with
something like gnome-software)

All we were using libdnf for (indirectly via libsolv) is comparing
version strings but librpm can already do that for us.
2021-01-28 11:15:44 -05:00
Colin Walters
e88a736e55 lockfile: Note that comment is JSON
Otherwise rustc tries compile and run it as a doctest.
2021-01-27 19:36:34 -05:00
Colin Walters
0f325b3490 Makefile.bindings: Also generate pure rust/cxx.h
This is necessary in order to use `rust::` in our header files
that are also used by bindings.
2021-01-27 19:36:34 -05:00
Colin Walters
56fde36e8d ci: Re-enable make check
This seems to work for me; we want to run the C unit tests.
2021-01-27 19:09:33 -05:00
Colin Walters
173db90941 tests: Fix compliation
This makes `make check` go again.
2021-01-27 19:09:33 -05:00
Kelvin Fan
1c3695c8e2 daemon/rpmostreed-daemon: Look up sd unit if caller not registered
Registration through `RegisterClient` is not mandatory today; for
example, Zincati does not register itself currently.
Look up systemd unit of caller if it is not already registered.
2021-01-27 18:19:50 -05:00
Colin Walters
cc230e25a0 tests/layering-fedorainfra: Update rpm-ostree build
The previous build was GC'd; unfortunately it's very nontrivial
to make this test truly robust over time because FCOS changes;
we might sometimes have an outstanding update, other times might
not etc.

Let's just sanity check the commands; ultimately they're
thin wrappers around just downloading packages so we don't need
deep checks.
2021-01-27 17:12:30 -05:00
Jonathan Lebon
c98c227c0f extensions: Write JSON to output dir
Let's include the final extensions file in JSON format as part of the
output directory. A key difference from the input file (apart from YAML
vs JSON) is that this is post-filtering, so any extensions which were
removed because the architecture does not match are not present.

This JSON file will be used by cosa and the MCO. See discussions in:
https://github.com/openshift/os/issues/409
2021-01-27 00:44:42 +01:00
Colin Walters
ea81a1ee6e compose: Remove unused treefile_rs parameter
Prep for further cleanup.
2021-01-26 22:25:43 +01:00
dependabot[bot]
55ea74a5bd build(deps): bump libdnf from 1ebad38 to e92a15a
Bumps [libdnf](https://github.com/rpm-software-management/libdnf) from `1ebad38` to `e92a15a`.
- [Release notes](https://github.com/rpm-software-management/libdnf/releases)
- [Commits](1ebad384f6...e92a15a5da)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-26 19:20:46 +01:00
Colin Walters
80dc572514 rust: Add some allow(dead_code)
One is only used in the unit tests right now, the other we'll
likely use soon.
2021-01-26 18:28:58 +01:00
Colin Walters
3f9a81b2c1 libmain: Avoid use of goto out
Prep for reworking our binary entrypoint to be Rust and not C++.

We need to split up main into sub-pieces; but before we do
that let's avoid `goto out` and rework into declare-and-initialize
style which cleans things up here.
2021-01-26 17:01:12 +01:00
Colin Walters
c3da95a119 Move nevra parsing to Rust, first use of extern "C++"
Until now with cxx-rs we'd been using it effectively as a better
cbindgen - we're exposing Rust code to C++ safely.  This is
the first case of having Rust calling back into C++ using cxx-rs.
2021-01-26 13:47:56 +01:00
Colin Walters
4ce5f42d12 rust: Link to our C/C++ dependencies and internal library
This allows us to fully use cxx-rs with `extern "C++"`.  Now
we do call back into the C/C++ today, but it only works outside
of cargo/Rust's knowledge.  Most notably, it means we can't
use our C code in `cargo test`.  And that's a problem
for moving some C/C++ code to Rust, because we want to port
the unit tests too.

For now, re-declare our dependencies and part of the build
system inside the Cargo build.  However, this is also
an important step towards using Cargo as our *sole* build
system.

We don't add build dependencies too often, so the short
term duplication should be OK.

However, a major unfortunate side effect of this is that
we now need to serialize the build process; almost all the
C/C++ comes first (`librpmostreeinternals.la`) and then
the Rust build, then we finally generate the executable
with both.

The only way out of this really is to move more of the
C/C++ build into Cargo, and we probably want to refactor
into internal crates.
2021-01-26 13:47:56 +01:00
Colin Walters
632cecdca6 cxxrs: Work around failure with -Werror=missing-declarations
See https://github.com/dtolnay/cxx/issues/590
2021-01-26 13:47:56 +01:00
dependabot[bot]
e540ca10be build(deps): bump rand from 0.8.2 to 0.8.3
Bumps [rand](https://github.com/rust-random/rand) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-26 12:53:04 +01:00
dependabot[bot]
a04ba7c22e build(deps): bump serde from 1.0.122 to 1.0.123
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.122 to 1.0.123.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.122...v1.0.123)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-26 12:52:57 +01:00
Luca BRUNO
867ba1eba8 rust/passwd: minor cleanup, switch to copy_file_at
This updates file copying to use the newer `copy_file_at` from
openat-ext.
2021-01-26 12:14:59 +01:00
Colin Walters
86ce9ea1f5 ci: Make msrv test do full build + unit tests
I think we did this at some point, but then stopped.
Prep for https://github.com/coreos/rpm-ostree/pull/2413
because we'll need a full build of the C++ side too in order
to `cargo test`.
2021-01-26 10:31:57 +01:00
Kelvin Fan
34746d0428 deploy: Allow empty string argument if --register-driver
It is sometimes useful to only register an update driver without
actually deploying anything. If the argument for `deploy` is an
empty string, only register driver and then no-op.
2021-01-26 01:32:52 +01:00
Jonathan Lebon
bc5a788366 core: Set _dbpath back to /usr/share/rpm after writing rpmdb
We temporarily set the rpmdb path to be an absolute path pointing under
the tmprootfs when writing the rpmdb. This throws off libsolv 0.7.17,
which learned to give the `_dbpath` macro precedence on where the rpmdb
is located:

04d4d036b2

So then the rpmdb sanity-check we do when exiting
`rpmostree_context_assemble()` breaks because it can't find the expected
packages.

Because RPM macros are in global state, there's no elegant way of
setting it just for the rpmdb write operation (short of forking), so
just fix this by setting `_dbpath` back to the correct value after we're
done writing the rpmdb.

Closes: https://github.com/coreos/fedora-coreos-tracker/issues/723
2021-01-25 23:20:53 +01:00
Jonathan Lebon
49318cd950 rust: Fix clippy warnings
Mostly straightforward stuff. It taught me about the `matches!` macro,
which looks really useful.

Wanted to turn this on in CI, but there's still a bunch of clippy
warnings coming from the `cxx.rs` stuff and some of our unsafe blocks.
For example, it wants the `files` arg in `initramfs_overlay_generate` to
be `&[String]` instead of `&Vec<String>` but that would break cxx.rs (it
looks like cxx.rs does support slices, but it would require creating one
from the vector we have to create anyway).
2021-01-25 20:06:53 +01:00
Luca BRUNO
15a32c12d6 rust/passwd: finish moving group and passwd parsers
This drops the remaining C compatibility hops, moving group and
passwd parsing logic fully into Rust, under a `nameservice`
module.
2021-01-25 19:43:54 +01:00
Luca BRUNO
a0e6427bb6 testutils: port to new 'rand' library API 2021-01-25 18:51:53 +01:00
dependabot[bot]
f8ff6075c7 build(deps): bump rand from 0.7.3 to 0.8.2
Bumps [rand](https://github.com/rust-random/rand) from 0.7.3 to 0.8.2.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/0.7.3...0.8.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 18:51:53 +01:00
dependabot[bot]
966ad4e1ca build(deps): bump cxx from 1.0.20 to 1.0.28
Bumps [cxx](https://github.com/dtolnay/cxx) from 1.0.20 to 1.0.28.
- [Release notes](https://github.com/dtolnay/cxx/releases)
- [Commits](https://github.com/dtolnay/cxx/compare/1.0.20...1.0.28)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-25 17:29:54 +01:00
Luca BRUNO
85640d44f6 ci: fix install-extra logic
This fixes `install-extra-builddeps.sh` helper, by letting cargo
detect whether the target binary is already present in the
environment with the expected version.
This is in order to avoid mismatches in generated code when the
library version is bumped, and stale binaries are present on
the system.
2021-01-25 17:05:53 +01:00