IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Use `pub(crate)` on the ffiutils functions since it's just for us. Drop
unnecessary `pub` fields on `TreefileExternals`.
Closes: #1812
Approved by: lucab
This mostly involves adjusting our path uses to be anchored, e.g. at
`self::` or `crate::`. I took the opportunity to also tweak `use`
ordering so that we consistently list modules in external crates first
before our own.
Closes: #1812
Approved by: lucab
Now that we no longer support el7, let's peg our minimum `rustc` version
to the latest Rust module available in el8 instead. That's 1.31.0 right
now.
And drop use of deprecated `trim_left_matches()` (it was deprecated in
1.33.0, which is currently used in the other testsuites that pull
`rustc` from the Fedora repos; we might want to eventually just use the
`RUST_MIN_VERSION` in all the testsuites to not get caught in the middle
like this again).
Closes: #1787
Approved by: cgwalters
And this (for now at least) completes the epic journey of the
"where's the kernel"? With this it's found solely in
`/usr/lib/modules/$kver`.
There are a few reasons to do this; most prominent is that
it avoids duplicating the content as the locations may have
different SELinux labels.
Closes: #1773
Approved by: jlebon
I'd like to add a new `boot-location: modules`. In prep
for that, let's remove the legacy `both` which drops into
`/boot`.
The libostree support for handling `/usr/lib/ostree-boot` has
existed for over 4 years:
```
commit 37a059925f6b96d30190b65bee6bdde0ae1c6915
Commit: Colin Walters <walters@verbum.org>
CommitDate: Sun Nov 30 23:14:05 2014 -0500
deploy: Ensure that we can deploy using only /usr/lib/ostree-boot
```
I think we assume now that no one is now making *new* treecomposes and needs
a newer rpm-ostree and that they expect people to be able to use as an
upgrade target from a libostree that predates that.
Closes: #1773
Approved by: jlebon
Now that we support YAML, it's a gotcha/eyesore that some of our options
use underscores rather than dashes. Let's be nice and switch those few
options over, while of course still supporting the previous name.
Co-authored-by: Colin Walters <walters@verbum.org>
Closes: #1749
Approved by: cgwalters
Instead of the `Strict` and `Permissive` variants of
`TreeComposeConfig`, just collapse all the excess fields in a new
`extra` member and check for any keys there in the strict YAML path.
This will also allow us to drop the hardcoded list of architectures in
the next patch.
Closes: #1749
Approved by: cgwalters
Add a `basearch` key to the manifest. This can be used at compose time
to assert the architecture the compose is running on. Though my
motivation is for the common case where it gets omitted from the input
manifest and gets automatically added by rpm-ostree into
`/usr/share/rpm-ostree/treefile.json` for introspection on the client.
(The crucial part here is that the treefile created by rpm-ostree
remains deserializable into a `TreeComposeConfig`).
Closes: https://github.com/coreos/fedora-coreos-tracker/issues/154Closes: #1766
Approved by: cgwalters
We're really using this variable to substitute `${basearch}` and find
basearch-specific packages. Let's rename the variable to make that more
obvious.
Closes: #1766
Approved by: cgwalters
Manually patch `file_contexts.subs_dist` so that `/home` is equivalent
to `/var/home`. This is required now that the generated homedirs rules
use `/var/home`. Otherwise, `matchpathcon` for example will return wrong
results.
This patch also includes the *removal* of `/var/home -> /home` so that
we're not dependent on this selinux-policy patch making it at the same
time as downstream:
https://src.fedoraproject.org/rpms/selinux-policy/pull-request/14
(See the conversation there for more information.)
Closes: #1754
Approved by: cgwalters
This uses the `Context` feature of the failure crate to make error
messages more useful when we fail to open a file. The difference with
`map_err` is that one can still obtain the underlying error from the
context if need be. Though surprisingly, the normal `Display` for a
`Context` doesn't include the original error, so we essentially have to
do a prefix here (see [1]).
Before:
```
error: Failed to load YAML treefile: No such file or directory (os error 2)
```
After:
```
error: Failed to load YAML treefile: Can't open file "treecompose-post.sh": No such file or directory (os error 2)
```
[1] https://github.com/rust-lang-nursery/failure/issues/182Closes: #1735
Approved by: cgwalters
Obviously doing these trivial functions in parallel doesn't matter,
but I wanted to dip my toes into Rayon (and Rust parallelism in
general).
`try_for_each()` is just...staggeringly simple and beautiful.
Closes: #1727
Approved by: jlebon
This was omitted since in practice we aren't actually testing it,
the container path is mostly via `ex container` which uses keyfiles.
Closes: #1701Closes: #1702
Approved by: jlebon
Avoids passing an allocated buffer from Rust to C; there's
controversy in the PR I sent to rust-lang around defining this as
supported.
Closes: #1691
Approved by: jlebon
Since use of the `failure` crate has been a success, let's use
it a bit more. The big thing to convert left is `treefile.rs` which
does need a custom error so we can stop abusing `io::ErrorKind::InvalidInput`.
Closes: #1690
Approved by: jlebon
Create a new `_new_` naming convention, and extend the FFI
documentation to describe the new state as well as background assumptions.
Closes: #1688
Approved by: jlebon
Change our existing "view as [u8] API", and also add one
that does a view as `OsStr`. The motivation for the latter
is I noticed ithat `OsStr::from_bytes()` *doesn't* copy,
or rather it just copies the pointer value. Rust's lifetime
inference ensures that the returned lifetime matches the input array.
I think the previous code in `treefile.rs` was confused about this.
Closes: #1688
Approved by: jlebon
I was going to add another usage of this function, and I think the
gerror stuff is unnecessary - if we are handed a bad file descriptor
(or a fd pointing to a regular file) that's something where we should
just abort.
While we're here, I'd like to codify expected usage in the function
names here. If you like this I'll e.g. also change `str_from_nullable`
to `ffi_view_nullable_str`.
Closes: #1685
Approved by: jlebon
In a lot of places we're abusing `io::Error(io::ErrorKind::InvalidInput)`
which is both verbose and inaccurate really. Maybe in some
places we should be defining custom errors, but eh.
I like the `failure` crate. Use it in just `utils.rs` for now.
Tweak our error handling FFI wrappers to accept `Display` since
all we do is convert the error to a string.
Closes: #1675
Approved by: lucab
This turned out to be messier than I thought, because of two primary
factors; the biggest mess here of course is the indirection
through the DBus API.
The other problem is that previously we passed the string to render
each time, and with current indicatif that'd trigger a rerender.
Since (usually) don't change the "prefix string", rework the API.
Change the "percent/n_items" bits to use autocleanups as well, and
to take the prefix string as an initial argument.
Since the state expands to multiple components, also change the
API to use the `0-initialized` pattern rather than trying to
return an aggregate.
We also gain a "sub message" which we use to display e.g.
package names as we're doing checkouts. Note this ends up
at the end, since otherwise everything else jumps around.
Closes: #1661
Approved by: rfairley
External tools often want to parse the ref; for example coreos-assembler
currently does so. Let's ensure `${basearch}` is expanded with
`--print-only` so they can parse that JSON to get the expanded version
reliably.
Implementation note: this is the first Rust code which exposes a
"GLib-like" C API, notably with GHashTable, so we're making more use
of the glib-rs bindings.
Closes: #1653Closes: #1655
Approved by: jlebon
This currently requires a `--i-know-this-is-experimental` flag;
I know it'd be a bit more consistent to have it under `ex`, but
what feels weird about that is *most* of the `ex` commands people
use are client side. This is where we want it to ultimately end
up.
We've landed a lot of prep patches, but I know there's still
a notable amount of code duplication with `compose tree`. What's
left is about ~700 lines but it's mostly not hard/complex code
anymore.
In the future, I'd like to extract more of the compose code
to a `rust/src/compose.rs` or so, but I think this is sustainable
fow now.
My high level goal is to get this into coreos-assembler and stand
up a Silverblue build that uses it.
Closes: #1512
Approved by: jlebon
The `lib.rs` file was starting to accumulate, and I want to do this
before adding more. I just made this up, but an `ffi` submodule
in each file seems to work well. It isolates the FFI consumption
there still (so e.g. if we want pure Rust unit tests, we don't
need to deal with FFI).
Closes: #1646
Approved by: jlebon
While serde gives us type checking, it of course doesn't understand
semantics beyond that. One example is checking the compatibility of
`add-files` entries with the OSTree model. This is something we can do
upfront early on to avoid surprises for users.
Also tweak the docs to reflect this new check.
Related: #1642Closes: #1643
Approved by: cgwalters
I need to do some hunting for a "ffi helper" crate. I kind of
understand why these things aren't in the std library.
Anyways this is easy for now, prep for adding more types.
Closes: #1641
Approved by: jlebon
On the plus side, when submitting a patch to Github, no one knows
how long it took you to figure out...
Anyways so this reduces redundancy. I double-checked the list.
I was inspired to pick this back up after seeing a Rust code
snippet somewhere noting that macros defined inside a function
can capture variables, which simplifies this even more.
Closes: #1631
Approved by: jlebon
Supporting ancient systemd is painful, though given that it may end up
in RHCOS at least, it seems worth the effort.
Basically, the big changes here are:
- avoid using `_SYSTEMD_UNIT` since it might be missing in some entries
- also grep for the el7 version of the systemd msg when a service fails
- use `_TRANSPORT=stdout` for the error msg in case of ordering issues
Closes: #1601
Approved by: cgwalters
Try to tease out a bit more info from the journal by looking at the
systemd message when the service transitions to the dead state or even
looking at the OSTree output itself.
Example outputs:
```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
error: opendir(ostree/deploy/fedora-atomic/deploy/887c95887a3047a60372016a0d84536530755b60df3cca33c819f7606e220adf.0): No such file or directory
check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```
```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
ostree-finalize-staged.service: Failed with result 'timeout'.
check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```
Closes: #1601
Approved by: cgwalters
Sample output:
```
$ rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```
(Though open to tweaking it).
I also played with directly invoking `journalctl` for the user, but that
can get really spammy with e.g. `os-prober` output and such.
I wrote this in Rust using journal API wrappers because I also plan to
implement the `history` command in Rust and will also enhance that new
`journal` module there for that.
Requires: https://github.com/ostreedev/ostree/pull/1750
Requires: https://github.com/jmesmon/rust-systemd/pull/54
(Though I've pointed the manifest at my branch for now for CI).
Closes: #1567Closes: #1601
Approved by: cgwalters
This way the fds are always known to be at the start. For e.g.
`add-files` we need to both checksum them and then copy them
later.
Closes: #1600
Approved by: jlebon
Now that we have `CUtf8`, let's just store the serialized JSON
as a string, owned by the Rust side. This way we can drop the
`serialized_treefile` buffer we were passing around and simplify
various bits of code. Most notably, we only serialize the JSON
once (via Serde) rather than also doing it again in the C side.
Closes: #1600
Approved by: jlebon
This follows up to https://github.com/projectatomic/rpm-ostree/pull/1576
AKA commit 2e567840ca - we now process
treefile inheritance in Rust code. Previously for elements which
reference external files (`postprocess-script` and `add-files`)
we'd hardcoded things to only look in the first context dir.
Now we open file descriptors in the Rust side for these "externals"
as we're parsing, and load them C side. Hence we'll correctly handle
a `postprocess-script` from an included config.
Other advantages are that the include handling was ugly un-typesafe C code
with no unit tests, now it's memory safe Rust with unit tests.
The downside here is I ended up spelling out the list of fields
again - there's probably a way to unify this via macros but
for now I think this is OK.
Closes: #1574
Approved by: jlebon
The advantage of this over CStr is that Rust knows it's UTF-8
too. I also tweaked our path code to use String, and only
view it as a `Path`. This avoids having to `unwrap()` later
back to a `str`.
Closes: #1588
Approved by: jlebon
It's really an array of pairs. Tripped over this while working
on the pure-Rust treefile parsing on the `misc-tweaks` compose test.
Closes: #1581
Approved by: jlebon
Prep for moving more of our parsing into Rust. The main
thing here is that for JSON, we need to continue to ignore
unknown fields. It took me a little while but I eventually
figured out that using `#[serde(flatten)]` works for this.
Seriously: serde is freaking amazingly awesome.
Closes: #1580
Approved by: jlebon
Apparently I was just totally confused into thinking `OsStr`
was `NUL` terminated. I don't know where I got that idea
but a quick inspection of the Rust source code makes this
clear. I may submit a PR to the docs.
So we have two choices: allocate, or internally use a `NUL`-terminated
type. Let's just use `Box<CStr>` internally. Note I chose that
over `CString` as the latter is theoretically mutable, and when
passing data back over to C we don't have ownership/mutability
guarantees.
Closes: #1563
Approved by: jlebon
If we're going to scale out our oxidation, let's follow
the path of Firefox (and other projects) further and use
cbindgen: https://github.com/eqrion/cbindgen
It's actually nice that `cbindgen` is packaged today in Fedora,
but I doubt it is elsewhere; we may end up needing to push
that forward, or just vendor it via a `build.rs` script and Cargo.
I chose to rename things to `ROR`/`ror_` since it's shorter. I
am tempted a bit to rename our internal functions to just `ro_` to
or so.
Closes: #1516
Approved by: jlebon
I'm trying to have a more opinionated model where custom builds
use inheritance, and currently one can only have a single
`postprocess-script`.
Further, in YAML it's very convenient to use inline vs external
data.
Closes: #1527
Approved by: jlebon
It turns out we basically have to slap an `Option<T>` around
everything, (in particular `bool` etc.) we need to be able
to distinguish in (I believe) all the cases between
"value unspecified" and "value provided".
Concretely it didn't work to try to set `machineid-compat: false`
in an included yaml treefile becuase it was just defaulted to `true`
by the toplevel.
Down the line we should move all of the parsing into Rust
and have two different `struct` types for "YAML we load" versus
"verified treefile".
Closes: https://github.com/projectatomic/rpm-ostree/issues/1524Closes: #1525
Approved by: lucab
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).
One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).
This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.
Closes: #1508
Approved by: cgwalters
This is for: https://github.com/projectatomic/rpm-ostree/issues/718
But I'm not going to close that issue as this only does the server
side, and I think we should support it client side too.
Since I wrote that issue, we ended up skipping the `dnf_transaction_depsolve()`
API, and hence we don't need to block on a libdnf change. So
this was quite simple.
Closes: #1513
Approved by: jlebon
It currently works to convert *toplevel* configs to YAML, but
if one wants to make use of `include:`, today you'd have to specify
dummy `ref` entries on the includes.
Further, for rojig-only mode, one doesn't need `ref`.
Closes: #1511
Approved by: jlebon
Otherwise we risk rejecting perfectly valid treefiles. E.g.
fedora-atomic only defines packages in the `-base` file. Let's just move
the check to after having processed all the includes, right where we
collate packages from all the various entries.
The FAHC treecompose is hitting this right now.
Closes: #1500
Approved by: cgwalters
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it. Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).
Prep for stabilizing rojig.
I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated. I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.
Closes: #1484
Approved by: jlebon
In a later patch I'm going to add more API; basically rather
than doing the JSON parsing from C, we can add APIs to directly
access the treefile object. This also demonstrates how we
can do more extensive APIs, in particular implement an "object"
in Rust.
The ownership across the FFI boundary becomes nicer here too,
we don't need to do a dance with the fd.
For writing this I found
http://jakegoulding.com/rust-ffi-omnibus/objects/
quite useful, as well as
https://github.com/rust-lang/regex/blob/master/regex-capi/src/rure.rsCloses: #1474
Approved by: jlebon
I was going to add a new API that returns a pointer, and that just
wasn't going to work with the trait AFAICS; we'd need to be generic
across many different types. Let's use plain functions; it's slightly
less elegant but is clear and works.
Closes: #1474
Approved by: jlebon
Follow up to: https://github.com/projectatomic/rpm-ostree/pull/1459
We now honor arch-specific packages in YAML, and reject unknown
architectures. I looked a little bit at how to avoid having hardcoded
arch lists, but it doesn't seem worth it right now.
Closes: #1468
Approved by: jlebon
Let's not make the same mistake we did with JSON where typoing a
field means it's silently ignored. This actually caught a bug
in a YAML usage we had:
```
error: Failed to load YAML treefile: unknown field `install_langs`, expected one of ... `install-langs` ...
```
Yes, this is a compatibility break with the feature we just announced
but...I seriously doubt anyone (that isn't known to me) has converted
yet, and if they are excited enough to start using a two-week-old feature
they can adjust.
Closes: #1459
Approved by: cgwalters
We weren't checking for an error from `dup()`, and further it shouldn't
be necessary. This is the best I could come up with, though it's
obviously not pretty.
Closes: #1444
Approved by: jlebon
It is actually really nice that there's One Canonical Style, even
if I sometimes don't like some details of what rustfmt does.
Closes: #1444
Approved by: jlebon
Further prep for adding more code here. The `lib.rs` then is the
collection of glue functions; perhaps in some ideal future it could
be generated even.
Closes: #1444
Approved by: jlebon
We actually want systemd's `ConditionFirstBoot` to fire. The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.
However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`. The correct thing
to do from the start is to use presets. We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.
See: https://github.com/dustymabe/bootengine/pull/11Closes: #1425
Approved by: jlebon
I was surprised when I fired up GDB and found out that my
`etc-group-members` modifications weren't being obeyed. It turns out
serde was looking for the wrong field name. I did a quick sanity check
and found a few others.
Closes: #1417
Approved by: cgwalters
Automake was warning about duplicate `clean-local` definitions, let's
do the nonrecursive automake dance.
And while I'm here, let's add some Rust unit tests that actually run
on `make check` too, since the whole unit testing bits of Rust are
awesome.
(I also tweaked the propagate bits to use the nicer `is_null()` method)
Closes: #1389
Approved by: jlebon
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.
This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.
If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.
[1] https://github.com/cgwalters/coreos-assemblerCloses: #1377
Approved by: cgwalters
