Commit Graph

31 Commits

Author SHA1 Message Date
Jonathan Lebon
2589cd1f92 rust/lockfile: Add more metadata to generated lockfiles
E.g. the generation timestamp, repos that were enabled, and their
generation timestamps.

This is just generally useful, though I'd like to make use specifically
of the new `metadata.generated` key in FCOS to drive versioning:

https://github.com/coreos/fedora-coreos-releng-automation/pull/50
2019-11-12 16:17:03 +01:00
Colin Walters
c8113bde32 Add hidden coreos-rootfs seal command
All this does is put the immutable bit on the target directory.
The intention is to replace this bit to start:
8b205bfbb9/src/create_disk.sh (L229)

However, the real goal here is to add code in this file
to handle redeploying the rootfs for Fedora CoreOS which
combines OSTree+Ignition:
https://github.com/coreos/fedora-coreos-tracker/issues/94

Basically doing this in proper Rust is going to be a lot
nicer than shell script in dracut modules.  Among other
details, coreutils `mv` doesn't seem to do the right thing
for SELinux labels when policy isn't loaded.
2019-10-04 08:03:03 -07:00
Colin Walters
3d6b33b6d4 rust: Add nix as a dependency
I have multiple outstanding PRs that use it and they conflict on
this, so let's just upstream it now.

It helps avoid some use of `unsafe` talking to libc directly.
2019-08-30 10:29:23 -04:00
Colin Walters
acdd04dac6 Move openat_utils to its own crate
I wanted to use this in another project, and I don't think
we're the only ones.  I extracted the code, added better docs
and tests, and published to crates.io:

https://crates.io/crates/openat-ext
2019-08-21 16:44:04 -04:00
Rafael Fonseca
c94bd08b02 treefile: allow ${releasever} in more keys
Besides allowing ${releasever}, only do the substitution as the final
pass after merging the treefiles for all the keys (currently ${basearch}
and ${releasever}) instead of doing it per parse. This way we have the
expected semantics where one could do:

```
include: "fedora-coreos.yaml"
releasever: "42"
```

and have that releasever used.

Fixes #1809

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>

Closes: #1848
Approved by: cgwalters
2019-06-18 18:19:42 +00:00
Jonathan Lebon
60e71dfc66 rust: Update to 2018 edition
This mostly involves adjusting our path uses to be anchored, e.g. at
`self::` or `crate::`. I took the opportunity to also tweak `use`
ordering so that we consistently list modules in external crates first
before our own.

Closes: #1812
Approved by: lucab
2019-04-15 13:03:21 +00:00
Jonathan Lebon
749facba86 rust: Bump dependencies
Now that our minimum Rust toolchain version was bumped, we can bump all
our deps to the latest and drop the frozen `rand` version that was done
for #1777.

Closes: #1810
Approved by: jlebon
2019-04-05 19:28:39 +00:00
Jonathan Lebon
e49f625b61 rust: Freeze rand crate to v0.6.1
The recent dep bump caused rand to jump to v0.6.5, which pulled in
`fuchsia-cprng`. Sadly, the cargo version we're frozen on chokes on it
because its manifest makes use of the `edition` keyword, which wasn't
stabilized yet.

Let's just freeze the problematic crate for now. We should be able to
bump our Rust toolchain soon-ish.

(Note this commit also does some other minor crate bumps as a result of
regenerating the lockfile to prune away `fuchsia-cprng`.)

Closes: #1777

Closes: #1778
Approved by: cgwalters
2019-03-07 21:57:08 +00:00
Jonathan Lebon
82594b5052 rust: Bump dependencies
Notably, the latest indicatif compiles fine again. This is now the
latest of everything, except the glib stack:

```
$ cargo outdated -R
Name      Project  Compat  Latest  Kind    Platform
----      -------  ------  ------  ----    --------
gio-sys   0.7.0    ---     0.8.0   Normal  ---
glib      0.6.1    ---     0.7.1   Normal  ---
glib-sys  0.7.0    ---     0.8.0   Normal  ---
```

because it requires a newer rustc than we're currently locked at.

Closes: #1770
Approved by: cgwalters
2019-03-01 15:51:07 +00:00
Colin Walters
579f017fc2 rust: Add clap
A few of my outstanding Rust PRs use this (multiprocess, sysusers).
Let's add it now.

Closes: #1764
Approved by: jlebon
2019-02-25 20:56:44 +00:00
Colin Walters
89cdeb5131 Add rayon dependency, do postprocessing in parallel
Obviously doing these trivial functions in parallel doesn't matter,
but I wanted to dip my toes into Rayon (and Rust parallelism in
general).

`try_for_each()` is just...staggeringly simple and beautiful.

Closes: #1727
Approved by: jlebon
2019-01-07 18:50:49 +00:00
Jonathan Lebon
fa64ebfa72 rust: Bump dependencies
This leaves only indicatif outdated (see link in diff).

Closes: #1720
Approved by: cgwalters
2018-12-19 21:25:13 +00:00
Jonathan Lebon
f6de8f3dfc rust: Drop backtrace lock
We can build against the latest now that we bumped our minimum Rust
version.

Closes: #1716
Approved by: cgwalters
2018-12-17 22:06:49 +00:00
Jonathan Lebon
24d401072a Cargo.toml: Fix backtrace crate to compatibile version
The latest backtrace release isn't compatible with 1.26.2. A patch for
this is underway in:

https://github.com/alexcrichton/backtrace-rs/pull/137

Though for now, let's just restrict the version since we should be
bumping our minimum rustc requirement soon-ish anyway.

Closes: #1713
Approved by: cgwalters
2018-12-13 16:34:56 +00:00
Colin Walters
e0a976386a rust: Use failure crate for errors
In a lot of places we're abusing `io::Error(io::ErrorKind::InvalidInput)`
which is both verbose and inaccurate really.  Maybe in some
places we should be defining custom errors, but eh.

I like the `failure` crate.  Use it in just `utils.rs` for now.
Tweak our error handling FFI wrappers to accept `Display` since
all we do is convert the error to a string.

Closes: #1675
Approved by: lucab
2018-11-21 22:03:53 +00:00
Colin Walters
1dee43319c Use indicatif for progress
This turned out to be messier than I thought, because of two primary
factors; the biggest mess here of course is the indirection
through the DBus API.

The other problem is that previously we passed the string to render
each time, and with current indicatif that'd trigger a rerender.
Since (usually) don't change the "prefix string", rework the API.

Change the "percent/n_items" bits to use autocleanups as well, and
to take the prefix string as an initial argument.

Since the state expands to multiple components, also change the
API to use the `0-initialized` pattern rather than trying to
return an aggregate.

We also gain a "sub message" which we use to display e.g.
package names as we're doing checkouts.  Note this ends up
at the end, since otherwise everything else jumps around.

Closes: #1661
Approved by: rfairley
2018-11-09 13:59:43 +00:00
Colin Walters
ef587b5c5b build-sys: Turn Rust LTO off by default, add --enable-lto flag
For us, this is primarily right now a size issue.  See:
https://internals.rust-lang.org/t/rust-staticlibs-and-optimizing-for-size/5746

For more information, there are these two issues:
https://github.com/rust-lang/cargo/issues/4349
https://bugzilla.mozilla.org/show_bug.cgi?id=1386371

The basic issue here is that a build with LTO off (and a trivial
change to add a `println!` takes 14s here, and with it on takes 38s.
However, with LTO off the stripped size of `librpmostree_rust.a` is
`6M`, with LTO on it's `1.1M`.

I named this `--enable-lto` as I'd like to investigate doing this
for the C code too.

Closes: #1664
Approved by: jlebon
2018-11-05 18:41:09 +00:00
Jonathan Lebon
a87486e18e rust: Drop crates-io patch and use 0.4.0
The latest release of the crate includes the features and fixes we need.

Closes: #1639
Approved by: cgwalters
2018-10-25 19:01:39 +00:00
Jonathan Lebon
f6c3616148 status: Detect if staging failed in previous boot
Sample output:

```
$ rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

(Though open to tweaking it).

I also played with directly invoking `journalctl` for the user, but that
can get really spammy with e.g. `os-prober` output and such.

I wrote this in Rust using journal API wrappers because I also plan to
implement the `history` command in Rust and will also enhance that new
`journal` module there for that.

Requires: https://github.com/ostreedev/ostree/pull/1750
Requires: https://github.com/jmesmon/rust-systemd/pull/54
(Though I've pointed the manifest at my branch for now for CI).

Closes: #1567

Closes: #1601
Approved by: cgwalters
2018-10-16 17:41:09 +00:00
Colin Walters
7595bce228 rust/treefile: Use the c_utf8 crate
The advantage of this over CStr is that Rust knows it's UTF-8
too.  I also tweaked our path code to use String, and only
view it as a `Path`.  This avoids having to `unwrap()` later
back to a `str`.

Closes: #1588
Approved by: jlebon
2018-09-28 16:21:18 +00:00
Colin Walters
8df07a3b1e rust/treefile: Add support for parsing JSON too
Prep for moving more of our parsing into Rust.  The main
thing here is that for JSON, we need to continue to ignore
unknown fields.  It took me a little while but I eventually
figured out that using `#[serde(flatten)]` works for this.
Seriously: serde is freaking amazingly awesome.

Closes: #1580
Approved by: jlebon
2018-09-26 16:40:40 +00:00
Colin Walters
f50f9e8d7e Split cbindgen to separate build, support external version
The problem is building bindgen as part of our single run
locks serde to way old versions, and I want to use newer versions.

Since Fedora will now again ship a `cbindgen` package, let's
also support using it if we find it, saving ourselves
the cost of building it.

For distros that don't ship it (e.g. CentOS) for CI purposes
we build it.  For downstream builds that are offline, rather
than vendor the cbindgen sources like we do with our main Rust,
let's just vendor the `rpmostree-rust.h` file as was suggested
in https://bugzilla.redhat.com/show_bug.cgi?id=1608670

Closes: https://github.com/projectatomic/rpm-ostree/issues/1557

Closes: #1573
Approved by: jlebon
2018-09-25 20:29:21 +00:00
Colin Walters
23698a8aa3 rust: Use debug=true for release builds
Because our primary delivery vechicle uses RPM which
supports separate debuginfo, there's no reason not to build release
builds with debuginfo, so that one can use gdb on them just
like we do with C.

Closes: #1550
Approved by: jlebon
2018-09-11 16:28:15 +00:00
Colin Walters
d2b0e42bfc Use cbindgen
If we're going to scale out our oxidation, let's follow
the path of Firefox (and other projects) further and use
cbindgen: https://github.com/eqrion/cbindgen

It's actually nice that `cbindgen` is packaged today in Fedora,
but I doubt it is elsewhere; we may end up needing to push
that forward, or just vendor it via a `build.rs` script and Cargo.

I chose to rename things to `ROR`/`ror_` since it's shorter.  I
am tempted a bit to rename our internal functions to just `ro_` to
or so.

Closes: #1516
Approved by: jlebon
2018-09-06 13:34:32 +00:00
Jonathan Lebon
04c0678fa6 app: Add support for passing URLs to RPMs
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).

One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).

This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.

Closes: #1508
Approved by: cgwalters
2018-08-23 11:16:15 +00:00
Colin Walters
344aee1d76 rust: Add support for inline rojig spec files
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it.  Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).

Prep for stabilizing rojig.

I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated.  I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.

Closes: #1484
Approved by: jlebon
2018-08-03 16:54:47 +00:00
Colin Walters
16be1a0bad rust: Rework treefile to be an object
In a later patch I'm going to add more API; basically rather
than doing the JSON parsing from C, we can add APIs to directly
access the treefile object.  This also demonstrates how we
can do more extensive APIs, in particular implement an "object"
in Rust.

The ownership across the FFI boundary becomes nicer here too,
we don't need to do a dance with the fd.

For writing this I found
http://jakegoulding.com/rust-ffi-omnibus/objects/
quite useful, as well as
https://github.com/rust-lang/regex/blob/master/regex-capi/src/rure.rs

Closes: #1474
Approved by: jlebon
2018-07-30 18:54:48 +00:00
Colin Walters
e731f59007 rust: Split out treefile into its own module
Further prep for adding more code here.  The `lib.rs` then is the
collection of glue functions; perhaps in some ideal future it could
be generated even.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
2894d61fbf build-sys: Make Rust library naming/entrypoint more generic
Prep for adding more functionality beyond treefile bits.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
702daa2335 rust: Fix automake warnings, add unit tests to glibutils
Automake was warning about duplicate `clean-local` definitions, let's
do the nonrecursive automake dance.

And while I'm here, let's add some Rust unit tests that actually run
on `make check` too, since the whole unit testing bits of Rust are
awesome.

(I also tweaked the propagate bits to use the nicer `is_null()` method)

Closes: #1389
Approved by: jlebon
2018-06-05 19:53:41 +00:00
Jonathan Lebon
479406e6a5 Add support for YAML treefiles
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.

This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.

If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.

[1] https://github.com/cgwalters/coreos-assembler

Closes: #1377
Approved by: cgwalters
2018-06-05 13:08:33 +00:00