Commit Graph

2279 Commits

Author SHA1 Message Date
Micah Abbott
bce966a981 vmcheck/misc-1: skip the overlay check when needed
It's possible to run the `vmcheck` tests against an existing host that
has `rpm-ostree` already present.  We don't overlay the built binaries
in this situation, so we should not check for the presence of the
overlay in the commit meta.

Closes: #1555
Approved by: cgwalters
2018-09-12 14:48:02 +00:00
Jonathan Lebon
807f21788e Hard require staging
This removes the logic around supporting opting out of the staging
feature. We don't want to support multiple configurations here, and at
this point, staging should be considered stable.

Closes: #1546
Approved by: cgwalters
2018-09-11 20:55:48 +00:00
Colin Walters
0d2b819e19 compose: Fix double-parsing of first included treefile
This is [brown paper bag bug](http://www.catb.org/jargon/html/B/brown-paper-bag-bug.html)
for sure.  But it's existed forever.  Basically our recursion
when parsing `include:` ended up parsing the first included treefile
twice.

I noticed this with the new `postprocess` element - luckily
my script was idempotent.  Historically we've gotten away with
this probably because the main thing is the `packages` array
and duplicates don't matter there.

Closes: #1552
Approved by: jlebon
2018-09-11 18:53:35 +00:00
Colin Walters
66d127ddd7 compose: Update to decl-and-init coding style
No functional changes, prep for more work.

Closes: #1552
Approved by: jlebon
2018-09-11 18:53:35 +00:00
Colin Walters
23698a8aa3 rust: Use debug=true for release builds
Because our primary delivery vechicle uses RPM which
supports separate debuginfo, there's no reason not to build release
builds with debuginfo, so that one can use gdb on them just
like we do with C.

Closes: #1550
Approved by: jlebon
2018-09-11 16:28:15 +00:00
Colin Walters
08ad9d49c8 ci: Stop installing python-{sphinx,devel}
We should no longer need it. Just noticed this while I was in the file recently.

Closes: #1551
Approved by: jlebon
2018-09-11 15:35:32 +00:00
Jonathan Lebon
57c4f8acff packaging: Don't vendor bundled libcurl
The `curl-sys` crate includes with it a bundled copy of libcurl which is
used if `pkgconfig` doesn't find libcurl configuration files. In our
case, we always want to use the system libcurl. So filter it out. This
also drops our *compressed* tarball by 2.5M.

One tricky bit is that cargo crates include a checksum JSON that's read
by `cargo build` later on to validate the crate. So we need to do some
JSON surgery.

What made me look into this was that Koji builds were failing due to the
`%configure` macro including hardening bits that sub out e.g. all
`config.sub` and `ltmain.sh` files which then caused the checksum to
fail validation. This completely sidesteps that issue.

Closes: #1554
Approved by: cgwalters
2018-09-11 14:44:44 +00:00
Jonathan Lebon
7467952c96 packaging: Use standard strict mode header
The script already turned on `-e` later on. Make it conform to the
standard strict mode with the rest of the options right at the
beginning.

Closes: #1554
Approved by: cgwalters
2018-09-11 14:44:44 +00:00
Jonathan Lebon
772f29282e Release 2018.8
Closes: #1547
Approved by: cgwalters
2018-09-10 17:59:21 +00:00
Colin Walters
5927d85192 tests: Rename one libcomposetest.sh
We had two `libcomposetest.sh` which I always found confusing.
Fix the naming of the one that's shared with `ex-container`
to be more obvious.

Closes: #1543
Approved by: jlebon
2018-09-10 17:06:10 +00:00
Jonathan Lebon
8976e502fc daemon: Make inactive requests a hard error
Flip the switch and make `rpm-ostree install glibc` a hard error if
`--allow-inactive` is not provided.

Closes: #1545
Approved by: cgwalters
2018-09-10 16:17:14 +00:00
Colin Walters
fe87ad4f29 tests: Drop conditionals for Rust enablement
We now hard require it, so change these tests to do so as
well.

Closes: #1544
Approved by: jlebon
2018-09-10 13:19:51 +00:00
Jonathan Lebon
651a9e01f4 core: Accumulate all missing pkgs into a single error
Accumulate the list of requested packages that were not found and error
out with the full list rather than failing early. This fixes a small UX
papercut in certain situations.

Closes: #1540

Closes: #1541
Approved by: cgwalters
2018-09-08 23:32:13 +00:00
Jonathan Lebon
f6c0788b4e ci: Use FAH updates branch for newer librpm
Right now, we're compiling against librpm 4.14.2, but running against
4.14.1, so we're hitting missing symbol issues from #1469. Just use the
updates ref for now until the next stable FAH update.

Closes: #1538
Approved by: cgwalters
2018-09-08 21:51:22 +00:00
Colin Walters
af5c74f9ea app/main: Give an immediate error if no /run/ostree-booted
Let's be more graceful here for people who `yum install rpm-ostree`
and expect magical[1] things to happen.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1537

[1] Understandably if one doesn't know lots of background

Closes: #1538
Approved by: cgwalters
2018-09-08 21:51:22 +00:00
Colin Walters
6f6dab1092 compose: Do import with --unified-core and --download-only
No reason to defer it to build time.

Closes: #1535
Approved by: jlebon
2018-09-07 20:24:11 +00:00
Colin Walters
326a9e0391 importer: Don't generate var-tmpfiles.d for rpm itself
This caused `/var/lib/rpm` to be a directory in my Fedora CoreOS
builds.

Closes: #1533
Approved by: jlebon
2018-09-07 19:41:48 +00:00
Colin Walters
6f6728b992 compose: Add --write-composejson-to
For higher level tools driving rpm-ostree, the command line arguments
are mostly OK as inputs, but the addition of `--write-commitid-to`
shows that we really want structured data that these tools can parse.
JSON is a good enough interchange format, let's use that.

Most notably this does a pkgdiff in the JSON which several higher
level tools do too.

Closes: #1529
Approved by: jlebon
2018-09-07 18:52:54 +00:00
Colin Walters
d2aee36f8f compose: Don't print message about --force-nocache with --dry-run
I'm splitting up coreos-assembler to have separate fetch/build
stages, with the first using `--download-only` and the second
using `--cache-only`, and the message from `fetch` is basically
wrong.

Closes: #1534
Approved by: jlebon
2018-09-07 18:09:31 +00:00
Colin Walters
b9e3bfd7b3 importer: Don't generate var-tmpfiles.d for rpm itself
This caused `/var/lib/rpm` to be a directory in my Fedora CoreOS
builds.

Closes: #1532
Approved by: sinnykumari
2018-09-07 15:10:10 +00:00
Colin Walters
1ee2a62343 scripts: Workaround Lua in fedora-release-coreos too
Just like the other ones from commit: 716b60b7d7

Yes, we probably should just patch the package but then
we'd have to wait for all of the koji/bodhi pain.

Closes: #1532
Approved by: sinnykumari
2018-09-07 15:10:10 +00:00
Jonathan Lebon
86f2b67d2b vmcheck: Store temporary files in /var/tmp instead
Since `/tmp` might be on tmpfs, so we'd lose it on reboot. But we have
tests that need it to persist across reboots.

Closes: #1531
Approved by: miabbott
2018-09-07 14:20:01 +00:00
Micah Abbott
c9b4a8195e tests/vmcheck: handle missing /etc/yum.repos.d
It's possible to run the `vmcheck` tests against an existing VM which
may not have any existing `/etc/yum.repos.d` directory.  Since the
tests are providing their own repo files, we should be able to run
them in this uncommon configuration.

Closes: #1530
Approved by: jlebon
2018-09-06 20:50:25 +00:00
Colin Walters
961482f088 Add inputhash Provides to rojig RPMs
Prep for rojig-only compose support.  We want to be able
to quickly detect whether or not there were changes in the
input, just like we do when targeting an OSTree repository.

Closes: #1528
Approved by: jlebon
2018-09-06 14:33:48 +00:00
Colin Walters
0a82e39052 compose: Fix workdir tmp dfd reference
If a `--workdir` was explicitly provided, we would pass `-1`
down into the Rust libs.

I found this while working on rojig-only which is reworking
the workdir bits.

Closes: #1528
Approved by: jlebon
2018-09-06 14:33:48 +00:00
Colin Walters
d2b0e42bfc Use cbindgen
If we're going to scale out our oxidation, let's follow
the path of Firefox (and other projects) further and use
cbindgen: https://github.com/eqrion/cbindgen

It's actually nice that `cbindgen` is packaged today in Fedora,
but I doubt it is elsewhere; we may end up needing to push
that forward, or just vendor it via a `build.rs` script and Cargo.

I chose to rename things to `ROR`/`ror_` since it's shorter.  I
am tempted a bit to rename our internal functions to just `ro_` to
or so.

Closes: #1516
Approved by: jlebon
2018-09-06 13:34:32 +00:00
Colin Walters
279e7c4f1b treefile: Support inline postprocess element (for inheritance)
I'm trying to have a more opinionated model where custom builds
use inheritance, and currently one can only have a single
`postprocess-script`.

Further, in YAML it's very convenient to use inline vs external
data.

Closes: #1527
Approved by: jlebon
2018-09-05 15:27:07 +00:00
Ruixin Bao
9920094cfa tests: add test for sorting + conversion of sysuser
As title, added tests to check both conversion + sorting for sysuser
entries, and verifies its correctness.

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
6facef9733 passwd-util: adds sorting logic for sysuser_entry
Per tile, added logic for sysusers_sorting. The comparison
for each sysuer_ent is like the following:
1: g > u > m for sysent_type and this takes precedence
2: if type matches, we compare the name for sysent using strcmp
3: If both type and name matches, we want to error out because it indicates a duplicate entry

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
d7dc6fe33c passwd-util: add sysuser_ent to char function
Adds function that interprets sysuser struct into usable string
content, which can later be used as content for `sysusers.d` files

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
4a942acdce passwd-util: adds conversion of group entries to sysusers
Similar to the passwd conversion, this commit
implements conversion of group entries into sysuser struct entries

Tests are also included in the commit

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
b4d301836d test: add unit test for passwd to sysuser conversion
Add initial implementation for unit test to test
converting passwd entries -> sysusers

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
69cd195c3a passwd-util: add conversion for passwd entries to sysusers
Adds initial implementation of converting passwd entries(read from
/etc/passwd file) into sysuser_ent structs.

The converted sysuser_ent struct will then get tracked inside
a pointer array.

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Ruixin Bao
e752b5aec1 passwd-util: add entries for conv_passwd_ent
This commit adds dir, geco and shell information to passwd_ent so
that they can be used later in sysusers section.

This also adds the sysuser_ent struct to track information of the
converted sysuser entries

For more information, please refer to https://github.com/projectatomic/rpm-ostree/issues/49

Closes: #1519
Approved by: cgwalters
2018-08-30 17:37:27 +00:00
Colin Walters
3d60a31aaa Fix include: with machineid-compat and a few other keys
It turns out we basically have to slap an `Option<T>` around
everything, (in particular `bool` etc.) we need to be able
to distinguish in (I believe) all the cases between
"value unspecified" and "value provided".

Concretely it didn't work to try to set `machineid-compat: false`
in an included yaml treefile becuase it was just defaulted to `true`
by the toplevel.

Down the line we should move all of the parsing into Rust
and have two different `struct` types for "YAML we load" versus
"verified treefile".

Closes: https://github.com/projectatomic/rpm-ostree/issues/1524

Closes: #1525
Approved by: lucab
2018-08-29 12:59:34 +00:00
Colin Walters
1984848b8e Update libdnf
To pull in the `RPMPROB_VERIFY` fix and the linking fix.

Update submodule: libdnf

Closes: #1521
Approved by: dustymabe
2018-08-29 02:16:21 +00:00
Colin Walters
884427d948 core: Disable --as-needed for -ldl -lm linkage
https://fedoraproject.org/wiki/Changes/RemoveExcessiveLinking
broke our build, since Rust doesn't yet have a way to express
the fact that the static library has dynamic dependencies.
(AIUI this is actually something libtool can handle with `.la` but eh)

Closes: #1522
Approved by: cgwalters
2018-08-28 21:08:15 +00:00
Colin Walters
c62058e548 build-sys: Propagate verbosity into libdnf
I'm trying to debug a libdnf build issue with the rpm-injected
`export LDFLAGS='-Wl,-z,relro ...`
and it's very helpful to have `--disable-silent-rules` or `make V=1`
do the right thing automatically for CMake too.

Closes: #1520
Approved by: peterbaouoft
2018-08-27 19:04:10 +00:00
Jonathan Lebon
53a98445e2 build: Tweak rpm version check for HAVE_NEW_RPM_VERIFY
It seems like `4.14.2-rc1` orders later than `4.14.2`, which is causing
issues in f28 builds.

Also print the version of librpm at configure time.

Closes: #1515
Approved by: cgwalters
2018-08-27 18:19:59 +00:00
Jonathan Lebon
04c0678fa6 app: Add support for passing URLs to RPMs
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).

One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).

This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.

Closes: #1508
Approved by: cgwalters
2018-08-23 11:16:15 +00:00
Colin Walters
40be3fb1cf Add recommends to treefile (default on)
This is for: https://github.com/projectatomic/rpm-ostree/issues/718
But I'm not going to close that issue as this only does the server
side, and I think we should support it client side too.

Since I wrote that issue, we ended up skipping the `dnf_transaction_depsolve()`
API, and hence we don't need to block on a libdnf change.  So
this was quite simple.

Closes: #1513
Approved by: jlebon
2018-08-23 04:24:07 +00:00
Robert Fairley
81889a0a35 libpriv/rpmostree-util: Handle NULL passed to rpm_str_ptrarray_contains
This treats strs=NULL as an empty array and returns FALSE on the basis
that a string (str) is not contained in an empty array. If str == NULL,
then a run-time assertion will fail.

Previously, a segmentation fault occured when NULL was dereferenced,
expecting NULL to be handled by rpm_str_ptrarray_contains.

fixes #1494

Closes: #1507
Approved by: jlebon
2018-08-23 02:15:25 +00:00
Jonathan Lebon
8c301401af build: Fix building rust in debug mode
When building in `debug` mode, `RUST_DEBUG` was still turned off because
`rust_debug_release` was set to `yes`, not `debug`.

Fix this by tweaking how `--enable-rust-debug` works: when it's *not*
provided, we default to the `$CFLAGS` detection logic. Otherwise, it
overrides it.

Closes: #1514
Approved by: cgwalters
2018-08-23 01:35:57 +00:00
Colin Walters
6a274b831d build-sys: Hard require Rust
As something that manages your base operating system, we care
about reliability, predictability, as well as performance and
low-level access to native operating system facilities.  The
C programming language is great for the latter two, but fails
at providing a truly memory-safe environment.  Rust is fairly
unique in providing a language that doesn't carry a runtime,
so we can gradually "oxidize" and convert our C code without
imposing additional overhead.  It's also got a lot of modern
design niceties, like not having a null pointer.

Let's pull the trigger here and hard require Rust.  It's the
programming language I personally want to be primarily writing in for
years to come.

This is also in line with a recent trend of reducing our
experimental/optional matrix.

Closes: #1509
Approved by: jlebon
2018-08-21 14:49:26 +00:00
Colin Walters
2032da9b21 treefile: Make ref optional in YAML
It currently works to convert *toplevel* configs to YAML, but
if one wants to make use of `include:`, today you'd have to specify
dummy `ref` entries on the includes.

Further, for rojig-only mode, one doesn't need `ref`.

Closes: #1511
Approved by: jlebon
2018-08-21 01:04:18 +00:00
Colin Walters
5126b2774e Stabilize stage automatic update policy
Now that staging is the default, there's no reason for this
policy not to be stable too.

Closes: #1510
Approved by: jlebon
2018-08-21 01:01:45 +00:00
Colin Walters
b6d07487d5 Turn staged deployments on by default
We've put a lot of work into staged deployments, it's time
to pull the trigger and turn them on by default.  This is
a key step for enabling `stage` mode automatic updates by
default in e.g. Fedora CoreOS/Silverblue.

We add a new `--disable-staged` build-time option to flip
things back.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
44b39a7340 livefs: Require deployment staging
Staging fixes the `/etc` bug for livefs.  There's actually more
we could do here around taking advantage of staging for livefs;
for example, I think once the livefs is complete, we could just delete
the staged deployment.  And then we don't need to render on the next
boot the live status, etc.

Anyways, all that can come in the future.  This is prep for
enabling staging by default.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
e1b62d0f7d tests/vmcheck: Display human-readable status on jq failure
So it's easier to debug.

This inlines the helper into the only function that uses it.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
2278b60a15 tests: Remove dead assert_status_jq
Nothing calls it today; looks like it was last used in
283b915ecf

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00