Commit Graph

145 Commits

Author SHA1 Message Date
Colin Walters
5820fd2136 Port rpmostree-builtin-rollback to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
7c1b6df951 Port rpmostree-builtin-upgrade to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
8f3c50aacf Port rpmostree-builtin-livefs to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
86efc2035a Port rpmostree-builtin-refresh-md to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
8560e0ba3e Port rpmostree-builtin-usroverlay to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
626e82fb3a Port rpmostree-builtin-override to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
6c47792bd0 Port rpmostree-builtin-reset to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Colin Walters
89b9020e31 Port rpmostree-pkg-builtins to C++
Prep for using https://cxx.rs/
2020-12-22 09:24:34 -05:00
Jonathan Lebon
df9c60ea62 Makefile-rpm-ostree: fix path to cbindgen.toml
Minor regression from #2365.
2020-12-21 20:31:30 -05:00
Jonathan Lebon
db775f43ec Default to sqlite rpmdb backend on f34
There won't be any support for writing to the bdb backend in f34, so
e.g. pkglayering won't work (and obviously even composes wouldn't work
once the buildroot moves to f34).

Instead of requiring the whole world to add an `rpmdb` key in their
manifests, let's just add a compile flag for it, and tweak the spec file
to use this flag on f34.
2020-12-21 19:01:48 -05:00
Colin Walters
aeb3dc2c07 Port rpmostree-libbuiltin to C++
Prep for using https://cxx.rs/
2020-12-21 08:58:14 +00:00
Colin Walters
9ad21b205a Port compose-builtin-rojig to C++
Prep for using https://cxx.rs/
2020-12-21 08:58:14 +00:00
Colin Walters
b62b9850c2 Port rojig2commit to C++
Prep for using https://cxx.rs/
2020-12-21 08:58:14 +00:00
Colin Walters
29dbeee72f More C++ porting, round 6
Port rojig-build and rpm-util to C++.

Prep for using https://cxx.rs/
2020-12-21 08:58:14 +00:00
Colin Walters
f35afb197d buildsys: Unify build of binary and internal shared library
This completes the task of unifying the build of all of our C/C++
into a single unit, which avoids code duplication and will allow
us to more easily use LTO in the future.
2020-12-20 15:23:53 +00:00
Colin Walters
ec5bdca9e5 buildsys: Compile daemon code directly with binary
A long time ago we de-duplicated the daemon and binary
into a single executable, but left the daemon code
building as an internal static library.

Let's take the next step and compile the sources directly as part
of the executable build.  For example, we can then de-duplicate
the `CFLAGS/CXXFLAGS`.  And in the future this will help us
turn on LTO.
2020-12-20 15:23:53 +00:00
Colin Walters
1de524f668 shlib: Make basearch APIs call /usr/bin/rpm-ostree shlib-backend
This is a potential path to fix
https://github.com/coreos/rpm-ostree/issues/2391

Basically our shared library and executable duplicate too much
code (particularly Rust).  Since most of the shlib APIs aren't
performance sensitive, let's have them fork off the binary
via a hidden CLI entrypoint and parse the output.
2020-12-20 15:23:53 +00:00
Colin Walters
b78de32d21 Port rpmostree-composeutil to C++
Prep for using https://cxx.rs
2020-12-19 12:40:43 +00:00
Colin Walters
2ec303b244 main: Convert to C++
Also here, catch C++ exceptions and convert to GError.
2020-12-11 15:17:00 -05:00
Colin Walters
c718f34d43 Fix and improve support for rpmostree-rust.h being pre-generated
First, we now need the `vendor/` directory at the toplevel because
that's where `Cargo.toml` is.

Now this triggers another bug introduced in the build system
with how we're handling the `rpmostree-rust.h` header.
We ended up vendoring a pre-generated one in the tarball
mainly because RHEL doesn't include cbindgen.
Now probably in the future I'd like to fix that.

But let's clean this up - the tarball generation process copies
the file into `rpmostree-rust-prebuilt.h`, and build machinery
detects that and entirely skips looking for or trying to build
our internal cbindgen.
2020-12-11 11:54:34 -05:00
Colin Walters
f1488e52f0 Move the main Rust infra (i.e. Cargo.toml) to the toplevel
I think we should have done this as soon as it was clear that
Rust was sticking and not just an optional thing.

Reasons to make this change now:
 - More clear that Rust is going to be the majority of code in the future
 - `cargo build` and `cargo test` in a fresh git clone Just Work
 - Paves the way for using `cargo` to build C/C++ instead of Automake
2020-12-09 17:42:35 -05:00
Colin Walters
90f4427bbf Move bindgen into tooling/, add dependabot
Prep for adding cxxbridge-cmd which we need to vendor too.
2020-12-08 16:32:24 -05:00
Colin Walters
6722496bb1 app: Port dbus-helpers to C++
Prep for using https://cxx.rs/
2020-12-08 09:57:08 -05:00
Colin Walters
bee9c0f605 Remove ex-container
I still think we should do this at some point, but
the experiment with using `GKeyfile` for configuration
is IMO a failure and the variety of data formats
(treefile JSON vs YAML vs origin keyfiles vs container keyfiles)
causes a lot of confusion.

Prep for https://github.com/coreos/rpm-ostree/issues/2326
2020-12-01 23:51:45 +01:00
Colin Walters
887d362906 Convert some code to compile in C++ mode
This is part of investigating using https://cxx.rs/
In order to make this really work, we need to convert some of our C
code to C++ so we can include cxx.rs-generated code.

This starts by converting just two files as a starting point.
I did the minimal porting; I didn't try to actually rewrite them
to resemble modern C++, just "C in C++ mode".
2020-12-01 22:07:49 +01:00
Colin Walters
7ed1c0b1e8 Remove coreos-rootfs command
This effectively reverts commit: c8113bde32
We never ended up using it; instead the `rdcore` bits from
`coreos-installer` have the rootfs reprovisioning logic.
2020-11-12 18:59:49 +01:00
Jonathan Lebon
4da10dc11e Add new ex initramfs-etc command
This command allows users to cheaply inject configuration files in the
initramfs stage without having to regenerate the whole initramfs (or
even a new OSTree commit). This will be useful for configuring services
involved in bringing up the root block device.

```
$ echo 'hello world' > /etc/foobar
$ rpm-ostree ex initramfs-etc --track /etc/foobar
Staging deployment... done
Run "systemctl reboot" to start a reboot
$ rpm-ostree status
State: idle
Deployments:
  ostree://fedora:fedora/x86_64/coreos/testing-devel
                   Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
                    Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
              GPGSignature: (unsigned)
              InitramfsEtc: /etc/foobar

● ostree://fedora:fedora/x86_64/coreos/testing-devel
                   Version: 32.20200716.dev.1 (2020-07-16T02:47:29Z)
                    Commit: 9a817d75bef81b955179be6e602d1e6ae350645b6323231a62ba2ee6e5b9644b
              GPGSignature: (unsigned)
$ reboot
(boot into rd.break)
sh-5.0# cat /etc/foobar
hello world
```

See the libostree side of this at:
https://github.com/ostreedev/ostree/pull/2155

Lots more discussions in:
https://github.com/coreos/fedora-coreos-tracker/issues/94

Closes: #1930
2020-10-30 00:48:54 +01:00
Colin Walters
5650f376fb Add --enable-rojig, disable by default
We're seeing some CI failures that I think are a bug in rojig.
In the bigger picture...we never actually started using this,
and I think longer term shipping os updates via containers
probably makes more sense.

I put a *lot* of effort into this code and it's pretty cool
so it's hard to just delete it.  And *maybe* someone out there
is using it (but I doubt it).  So rather than just deleting
it entirely let's make it a build-time option.

I verified that it builds at least.
2020-10-22 15:38:50 -04:00
Colin Walters
0de332bd84 build-sys: Remove --enable-new-name
We didn't ship this for a long time and...well, naming is hard, I guess
it's not going to happen soon.
2020-05-14 13:18:00 -07:00
Colin Walters
e41a8ab26f Add support for wrapping binaries (rpm, dracut, grubby)
We need to be friendlier to people who are transitioning from
"traditional" yum managed systems.  This patchset starts to lay
out the groundwork for supporting "intercepting" binaries that
are in the tree.

For backwards compatibility, this feature is disabled by default,
to enable it, one can add `cliwrap: true` to the manifest.

To start with for example, we wrap `/usr/bin/rpm` and cause it
to drop privileges.  This way it can't corrupt anything; we're
not just relying on the read-only bind mount.  For example nothing
will accidentally get written to `/var/lib/rpm`.

Now a tricky thing with this one is we *do* want it to write if
we're in an unlocked state.

There are various other examples of binaries we want to intercept,
among them:

 - `grubby` -> `rpm-ostree kargs`
 - `dracut` -> `rpm-ostree initramfs`
 - `yum` -> well...we'll talk about that later
2020-04-15 16:22:57 +02:00
Jonathan Lebon
571bd3a558 tests: Add hidden testutils subcommand
This is a hack to allow using `inject-pkglist` without having to build
the tree first.

Higher-level, I think we can split this back out again if we have a
`-tests` subpackage where we ship the vmcheck testsuite.
2019-12-13 19:18:30 +01:00
Colin Walters
c8113bde32 Add hidden coreos-rootfs seal command
All this does is put the immutable bit on the target directory.
The intention is to replace this bit to start:
8b205bfbb9/src/create_disk.sh (L229)

However, the real goal here is to add code in this file
to handle redeploying the rootfs for Fedora CoreOS which
combines OSTree+Ignition:
https://github.com/coreos/fedora-coreos-tracker/issues/94

Basically doing this in proper Rust is going to be a lot
nicer than shell script in dracut modules.  Among other
details, coreutils `mv` doesn't seem to do the right thing
for SELinux labels when policy isn't loaded.
2019-10-04 08:03:03 -07:00
Jonathan Lebon
83a267409d Add new D-Bus APIs for deployment finalization
Teach `UpdateDeployment` to make use of libostree's staging lock and
then add a `FinalizeDeployment` API to perform the final unlock &
reboot.

I also added a hidden CLI to make testing this easier, but also because
it's likely the FCOS-agent-yet-to-be-named will just end up using the
CLI to keep it simple.

Closes: #1748

Closes: #1814
Approved by: lucab
2019-04-30 08:36:28 +00:00
Colin Walters
ccc244879c build-sys: Always enable compose tooling
There's no reason to separate it anymore.

Closes: #1794
Approved by: jlebon
2019-03-25 13:46:29 +00:00
Colin Walters
387da3a420 ci: Verify rustfmt
Split out the command into a separate `Makefile` that doesn't
use Automake so we can invoke it early.

Closes: #1674
Approved by: dustymabe
2018-11-21 21:16:03 +00:00
Colin Walters
11ae25792c build-sys: Add bindgen bits to .gitignore
Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
dcc78cc2f1 build-sys: Fix uid check when we don't have /usr/bin/cbindgen
Make this match the other case.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
bd4d0d8e0a build-sys: With external cbindgen, drop Rust lib dependency
Perhaps an unexpected side benefit of slow compilation processes
is that one has an opportunity to reflect and ponder.

I realized during exactly such a moment that since we moved
`cbindgen` out of our library build, there's no need to wait
for the library to be built before we can start building the C
code.

This is a notable local quality-of-life development improvement.

Closes: #1665
Approved by: jlebon
2018-11-05 19:59:58 +00:00
Colin Walters
88ffdc0ae2 Add compose rojig command
This currently requires a `--i-know-this-is-experimental` flag;
I know it'd be a bit more consistent to have it under `ex`, but
what feels weird about that is *most* of the `ex` commands people
use are client side.  This is where we want it to ultimately end
up.

We've landed a lot of prep patches, but I know there's still
a notable amount of code duplication with `compose tree`.  What's
left is about ~700 lines but it's mostly not hard/complex code
anymore.

In the future, I'd like to extract more of the compose code
to a `rust/src/compose.rs` or so, but I think this is sustainable
fow now.

My high level goal is to get this into coreos-assembler and stand
up a Silverblue build that uses it.

Closes: #1512
Approved by: jlebon
2018-10-31 17:57:45 +00:00
Colin Walters
fe5adaf7ab build-sys: Fix make rustfmt, and run it
When we added the bindgen config this broke as it's not a `.rs`.
Let's keep the source formatted canonically.

Closes: #1635
Approved by: jlebon
2018-10-23 19:05:06 +00:00
Colin Walters
d24f21343f build-sys: Just error out if building with multiple uids
The `--frozen` stuff ends up being annoying when switching
branches.  What we're really trying to protect against here
is the `sudo make install` problem, so let's test for that
more directly by verifying the uids.

(The previous code was also totally broken as it used `$` where
`$$` should have been in multiple places)

Closes: #1601
Approved by: cgwalters
2018-10-16 17:41:09 +00:00
Colin Walters
5d57025e16 build-sys: Add a rust-test target
I often am editing just the Rust code, and want the fast iteration
feedback of a `cargo test` - don't want to pay the cost of full
optimization (particularly LTO) for the release build, and I
*just* want to run the Rust tests.

Basically if you're editing our Rust code a lot, this target is
your friend.

Closes: #1585
Approved by: jlebon
2018-10-02 13:14:08 +00:00
Colin Walters
c96f93f0ad build-sys: Propagate Automake verbose mode to Rust/cargo
This is analogous to commit c62058e548
which propagated `V=1` into `cmake`.  Except if the build *isn't*
verbose (for local development), let's not force `--verbose` on
for Rust.

Closes: #1583
Approved by: jlebon
2018-09-26 17:07:19 +00:00
Colin Walters
f50f9e8d7e Split cbindgen to separate build, support external version
The problem is building bindgen as part of our single run
locks serde to way old versions, and I want to use newer versions.

Since Fedora will now again ship a `cbindgen` package, let's
also support using it if we find it, saving ourselves
the cost of building it.

For distros that don't ship it (e.g. CentOS) for CI purposes
we build it.  For downstream builds that are offline, rather
than vendor the cbindgen sources like we do with our main Rust,
let's just vendor the `rpmostree-rust.h` file as was suggested
in https://bugzilla.redhat.com/show_bug.cgi?id=1608670

Closes: https://github.com/projectatomic/rpm-ostree/issues/1557

Closes: #1573
Approved by: jlebon
2018-09-25 20:29:21 +00:00
Colin Walters
ca6f32371a compose: Split off composeutil.[ch]
Prep for adding a new toplevel rojig command.  It's also just
cleaner to avoid cluttering up the main compose logic with
distractions like the `_legacy_prep_dev()` bits.

Closes: #1564
Approved by: jlebon
2018-09-20 12:35:24 +00:00
Colin Walters
d17381265e build-sys: Rework Rust --frozen logic
For local development, I want to be able to e.g. update `Cargo.toml`
or switch branches and not have to `rm -rf target`.

Let's tweak the logic here so we only pass `--frozen` if the ownership
of the dir is different.

Probably an even better fix would be to just error out, but
this is a conservative tweak.

Closes: #1563
Approved by: jlebon
2018-09-17 15:51:53 +00:00
Colin Walters
36bf3c9164 build: Regenerate rpmostree-rust.h if cbindgen.toml changes
So when I edit it I get the changes.  Figuring this out
took me a little while, we were missing two dependencies.

Closes: #1559
Approved by: jlebon
2018-09-13 21:49:06 +00:00
Colin Walters
d2b0e42bfc Use cbindgen
If we're going to scale out our oxidation, let's follow
the path of Firefox (and other projects) further and use
cbindgen: https://github.com/eqrion/cbindgen

It's actually nice that `cbindgen` is packaged today in Fedora,
but I doubt it is elsewhere; we may end up needing to push
that forward, or just vendor it via a `build.rs` script and Cargo.

I chose to rename things to `ROR`/`ror_` since it's shorter.  I
am tempted a bit to rename our internal functions to just `ro_` to
or so.

Closes: #1516
Approved by: jlebon
2018-09-06 13:34:32 +00:00
Colin Walters
884427d948 core: Disable --as-needed for -ldl -lm linkage
https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking
broke our build, since Rust doesn't yet have a way to express
the fact that the static library has dynamic dependencies.
(AIUI this is actually something libtool can handle with `.la` but eh)

Closes: #1522
Approved by: cgwalters
2018-08-28 21:08:15 +00:00
Jonathan Lebon
04c0678fa6 app: Add support for passing URLs to RPMs
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).

One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).

This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.

Closes: #1508
Approved by: cgwalters
2018-08-23 11:16:15 +00:00