1598 Commits

Author SHA1 Message Date
Jonathan Lebon
eae3a40f3b libpriv/core: Always call rpmtsSetRootDir()
Call `rpmtsSetRootDir()` even in the `JUSTDB` run. This brings us in
line with libdnf and the rpm CLI itself, which both unconditionally call
`rpmtsSetRootDir()` regardless of whether we're targeting the system
root or a subdir.

This doesn't cause a `chroot()` to happen since librpm only does this
if the target root dir is not `/`.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1613517

Closes: #1492
Approved by: cgwalters
2018-08-08 18:29:45 +00:00
Jonathan Lebon
e94f8c9b5f compose: Fix mutate-os-release handling
I noticed that the latest Fedora Atomic Host 28 and Silverblue did not
have an `OSTREE_VERSION` line in `/etc/os-release` even though both
specified `mutate-os-release` in their manifests. This turned out to be
due to the fact that `/usr/lib/os-release` is now a symlink to a
variant-specific file (e.g. `os-release-atomichost`), so we would
fallback to mutating `/usr/lib/os.release.d/os-release-fedora` instead.

Fix this by just taking the nuclear option of running `realpath` in the
rootfs directly. This is more maintainable than trying to keep up with
changes in variants/naming/etc. There's related discussions to this in
the original [PR](https://github.com/projectatomic/rpm-ostree/pull/410)
which introduced the feature re. resolving symlinks within the rootfs.

Closes: #1481
Approved by: jlebon
2018-08-06 01:32:49 +00:00
Jonathan Lebon
5219df0875 libpriv/bwrap: Add bwrap_run_captured()
The `GSubprocess` API has a great `g_subprocess_communicate` function
that makes it easy to capture stdout and/or stderr. Let's expose that.

Closes: #1481
Approved by: jlebon
2018-08-06 01:32:49 +00:00
Colin Walters
588a0327db postprocess: Error if units with machineid-compat: false
Rather than silently ignoring it.

In theory...we could write to /usr/lib/systemd instead of `/etc`
but eh...I feel like what we really want to do is make it convenient
to write a preset file from the YAML.

(We could have an `add-files` content that takes values literally
 which would be nice in YAML and suck in JSON)

A general thread running through this is that for people making
*derivatives* of a CoreOS-like system, having to create their
own `exampleos-release` package is an annoying hurdle.

Anyways for now we're fixing the bug that we were silently ignoring
it.

Closes: #1488
Approved by: jlebon
2018-08-06 00:33:56 +00:00
Colin Walters
344aee1d76 rust: Add support for inline rojig spec files
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it.  Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).

Prep for stabilizing rojig.

I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated.  I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.

Closes: #1484
Approved by: jlebon
2018-08-03 16:54:47 +00:00
Jonathan Lebon
a6e4994bf6 app/compose: Fix g_propagate_error ownership
We weren't actually transferring ownership of the error, which meant
that the error message was freed before we could even print it to the
user.

Closes: #1486
Approved by: cgwalters
2018-08-01 23:27:12 +00:00
Colin Walters
dcbd2d91ec compose: Stop calling fchdir in --unified-core mode
Changing the process' working directory is evil in potentially
multi-threaded code, and at this point we really should be using
`openat()`/fd-relative bits most everywhere.  But let's be
conservative and only stop doing it in `--unified-core` mode.

Closes: #1485
Approved by: jlebon
2018-08-01 22:22:47 +00:00
Colin Walters
23badcd288 compose: Write a preset file to enable ostree-remount.service
I've lost count now of how many times people have hit variants
of https://github.com/projectatomic/centos-release-atomic-host-devel/pull/6
Let's just bake it in.

Closes: #1482
Approved by: jlebon
2018-08-01 20:11:38 +00:00
Colin Walters
da27b94b29 core: Use new rpmtsSetVfyLevel() API for writing rpmdb
Newer librpm defaults to doing a full payload checksum, which we can't
do at this point (writing the db) because we imported the RPMs into
ostree commits, saving just the header in metadata - we don't have the
exact original content to provide again.

Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1607223

Closes: #1469
Approved by: jlebon
2018-07-31 19:22:30 +00:00
Colin Walters
3446caa190 importer: Prefix error with package name
I think we have issues with librepo not handling being forcibly
interrupted; I saw an lzma error from what I think was a partial
download.

Let's add an error prefix here so this is easier to debug.  I
think we used to have a prefix but lot it when we moved to async.

Closes: #1477
Approved by: jlebon
2018-07-31 15:18:45 +00:00
Jonathan Lebon
67860c5ece app/pkg-builtins: Add --unchanged-exit-77
This is analogous to `upgrade --upgrade-unchanged-exit-77`, but for
`install`/`uninstall`. This way, one can determine whether the command
truly had an effect on the default deployment or not.

Since this works by comparing against the previous default deployment,
this works correctly even if we already had a pending deployment at the
beginning of the transaction.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
d35fbb665e app/pkg-builtins: Add --idempotent
Add a new `install/uninstall --idempotent` option to make it easier to
interact with the CLI through scripts. E.g. one doesn't have to check
first if a request has already been installed/uninstalled.

Closes: #1467

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
a9e1de0276 daemon/dbus: Document a few missing newer options
Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
fcb061b19a app/status: Add --pending-exit-77 switch
This makes it easier for scripts to determine whether there is a pending
deployment instead of using `--json/--jsonpath`.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Colin Walters
16be1a0bad rust: Rework treefile to be an object
In a later patch I'm going to add more API; basically rather
than doing the JSON parsing from C, we can add APIs to directly
access the treefile object.  This also demonstrates how we
can do more extensive APIs, in particular implement an "object"
in Rust.

The ownership across the FFI boundary becomes nicer here too,
we don't need to do a dance with the fd.

For writing this I found
http://jakegoulding.com/rust-ffi-omnibus/objects/
quite useful, as well as
https://github.com/rust-lang/regex/blob/master/regex-capi/src/rure.rs

Closes: #1474
Approved by: jlebon
2018-07-30 18:54:48 +00:00
Colin Walters
716b60b7d7 scripts: Add Lua overrides for fedora-release and also -workstation
Dusty hit this when working on Fedora CoreOS.  IMO we don't need
to do the same for a future e.g. `fedora-release-silverblue` as
we don't support someone installing it on a non-ostree system.

(Note this all only really matters for `--unified-core` on the compose
 side)

Closes: #1473
Approved by: jlebon
2018-07-30 13:12:11 +00:00
Jonathan Lebon
7911b14f49 daemon: Fix cached-update including no-op diffs
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.

We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.

Closes: #1446

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Colin Walters
831507b0f0 scripts: Add RPMOSTREE_SCRIPT_TRACE debugging envvar
I used `env RPMOSTREE_SCRIPT_TRACE='strace -f -s2048' rpm-ostree compose ...`
to debug https://bugzilla.redhat.com/show_bug.cgi?id=1548050

Closes: #1472
Approved by: jlebon
2018-07-27 13:09:49 +00:00
Colin Walters
e858a30eca compose: Stabilize --unified-core
In line with the recent trend of marking things stable, and in
preparation for stabilizing `rojig://` - Let's stabilize the `--unified-core`
option for `compose tree`.

I'm not sure we could make it the default anytime soon; today it trips
over bugs in the PAM package in RHEL7 for example.  But it
works fine for Fedora, and I think the code/design are good enough to be stable.

Closes: #1465
Approved by: jlebon
2018-07-24 23:06:50 +00:00
Colin Walters
fa29f7acfa compose: Support arch-specific packages in YAML (and in JSON again)
Follow up to: https://github.com/projectatomic/rpm-ostree/pull/1459

We now honor arch-specific packages in YAML, and reject unknown
architectures.  I looked a little bit at how to avoid having hardcoded
arch lists, but it doesn't seem worth it right now.

Closes: #1468
Approved by: jlebon
2018-07-24 22:05:06 +00:00
Jonathan Lebon
c016509548 libdnf: Bump and disable html and man pages
Skip building man pages and HTML docs for our embedded libdnf to speed
up builds.

This bump also pulls in a fix to ensure we never try to install src
packages from `dnf_context_install()`.

See: https://github.com/projectatomic/libdnf/pull/3

Update submodule: libdnf

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Colin Walters
096004426c rebase: Add support for "custom origin" descriptions
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.

Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM).  This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.

There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.

This builds on prior work in
27bd7b97bbef3b4c4b2ecd80a832d45e84f13932 from #1396 .

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
d1839378d6 compose: Add some assertions around treefile context directory
Seeing a crash on in one of our builds, not sure yet what's
going wrong as I haven't gotten a core out, but this makes things
clearer.

($kingdom for `T`/`Option<T>` instead of nullable pointers...)

Closes: #1454
Approved by: jlebon
2018-07-12 18:30:55 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
2894d61fbf build-sys: Make Rust library naming/entrypoint more generic
Prep for adding more functionality beyond treefile bits.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
bac6d634bb build-sys: Move Rust include header under rust/ directory
It makes more sense to have the include live next to the associated
code, just like we do with C, even though the `cargo build` doesn't
touch it.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
b1d7ef15d8 sysroot-core: Use new libostree sysroot pruning API
Use the new API introduced in
9131d8a4cc
which helps avoid relying on the magical deployment refs.  I thought
this would help with a livefs+staging issue, but it doesn't.  But we might
as well do it, it's adding another level of safety.

Closes: #1439
Approved by: jlebon
2018-07-03 20:18:00 +00:00
Jonathan Lebon
b94fef9f24 daemon/rollback: Really fix staged deployment case
Follow up to #1434. I was testing this inside my test VM, which didn't
have a rollback deployment (i.e. it was just the staged deployment and
the booted deployment). Testing the upcoming release on my real system
showed that this wasn't correctly fixed.

Closes: #1437
Approved by: cgwalters
2018-06-29 16:30:07 +00:00
Jonathan Lebon
beb8775d8e daemon/rollback: Error out if pending deployment is staged
It doesn't make sense to rollback a staged deployment. Give a hint
instead to use `cleanup -p` if needed. Otherwise, we dump core later on
in libostree.

This is technically a breaking change once staged deployments become the
default. However, I think it makes sense overall. We'll have to make
some noise about it.

Closes: #1434
Approved by: cgwalters
2018-06-28 22:01:14 +00:00
Jonathan Lebon
1b5ab9834e app/dbus-helpers: Return early if no diff to print
A new update might not have any package changes at all. In which case,
we shouldn't even try to print the diff, otherwise we'll fail on trying
to lookup the `upgraded` key. We did this check already from
`rpmostree_print_cached_update`, but not from `print_one_deployment`
(when printing the diff as part of the pending deployment), so we'd
error out on no-diff upgrades.

Let's just inline a check in the function directly instead of wrapping
every call site.

Closes: #1431
Approved by: cgwalters
2018-06-28 20:49:23 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Colin Walters
77e2e19c1f postprocess: Set default.target in /usr
Follow-up improvement after
https://github.com/openshift/os/pull/135

This should ensure it survives systemd's preset run on firstboot.

Although honestly...what we should *really* do is check whether
the `default.target` symlink target exists, and if not reset it
to `multi-user.target` so no one would have to care, but that'd
be conceptually separate from this, so I may do it later.

Closes: #1427
Approved by: jlebon
2018-06-25 20:45:32 +00:00
Colin Walters
d7342731ab Add a treefile option to stop creating /etc/machine-id
We actually want systemd's `ConditionFirstBoot` to fire.  The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.

However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`.  The correct thing
to do from the start is to use presets.  We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.

See: https://github.com/dustymabe/bootengine/pull/11

Closes: #1425
Approved by: jlebon
2018-06-25 16:19:00 +00:00
Jonathan Lebon
94ee42b03b daemon: Add a few missing transaction titles
Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Micah Abbott
4c61a19f42 Clarify the use of --uninstall switch
While looking at the inline help for `install`, I thought we could do
better by specifically stating that the `--uninstall` switch only
operates on layered packages.  This might prevent any users from
trying to use the `--uninstall` switch on a package that is part of
the base deployment.

Closes: #1421
Approved by: jlebon
2018-06-20 14:26:33 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
8b66c830b7 app/builtins: Consistent description for --reboot
A bunch of commands had descriptions for `--reboot` that were copied
from the `upgrade` description. Let's just make them all the same to
keep it consistent.

Closes: #1418
Approved by: cgwalters
2018-06-18 13:49:41 +00:00
Jonathan Lebon
096b0554f0 man/rpm-ostree: Tweak install/uninstall docs
Minor tweaks to try to clarify that the `install` and `uninstall`
commands are for managing an overlay. This should hopefully make it less
likely that someone tries `uninstall` when really they want
`override remove`.

Also drop the paragraph about having to specify NEVRAs for removing
local RPMs since that's no longer necessary.

Closes: #1418
Approved by: cgwalters
2018-06-18 13:49:41 +00:00
Colin Walters
57475e5cc7 daemon: Further consolidate option parsing for deploy
Fewer variables and avoid extra strdupv calls.  I also fixed
handling of refspec canonicalization that I broke with a previous
change (we weren't actually setting `self->refspec` to the canonicalized
version).

Closes: #1414
Approved by: jlebon
2018-06-15 20:19:41 +00:00
Colin Walters
5d75697dc4 status: Assert that we have deployments
I think this is what's showing up in a PR test:
21b68a68ff.2.1529072619059542132/artifacts/vmcheck/livefs.log
```
(rpm-ostree status:17017): Json-CRITICAL **: json_gvariant_serialize: assertion 'variant != NULL' failed
```

It shouldn't be happening; let's add an assertion which will give us a core
dump reliably instead of the "stumble on" `g_warning()` default.

Closes: #1416
Approved by: jlebon
2018-06-15 18:24:19 +00:00
Colin Walters
999bf3d3d9 daemon: Drop some internal options → flags conversion
It's just all-around simpler if we only parse the vardict
and avoid exploding everything into enums/parameters.  Previously
we were converting the variant into parameters and flags, and
then in most cases converting the flags back into booleans!
This should make it easier to add new parameters.

Closes: #1412
Approved by: jlebon
2018-06-15 13:25:21 +00:00
Colin Walters
eacd97c4ef daemon/txn: Hold a ref to deployment options variant
This will allow later code to parse it directly.

Closes: #1412
Approved by: jlebon
2018-06-15 13:25:21 +00:00
Ruixin Bao
ec0af351f2 passwd-util: change passwd to FALSE to allow checking groups
We are having exact same function call between rpmostree_check_groups
and rpmostree_check_passwd, which means we are ignoring all the logic
for group checking.

This commit changes the option `passwd` to FALSE to also check for groups
during compose process.

Closes: #1413
Approved by: cgwalters
2018-06-15 13:19:08 +00:00
Jonathan Lebon
445954ddb8 app/dbus-helpers: Initialize reboot & dry run vars
Minor regression from #1382. Since we no longer always include all the
option keys, we actually need to initialize the default values now.

Closes: #1410

Closes: #1411
Approved by: cgwalters
2018-06-14 18:17:08 +00:00
Jonathan Lebon
fb006b5b43 app/uninstall: Add --all switch to drop all layering
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).

Closes: #1408
Approved by: cgwalters
2018-06-14 12:36:15 +00:00
Jonathan Lebon
65d08d928e daemon/deploy: Fix off-by-one count in localinstall
We were also counting the `NULL` element we added at the end of the
array when appending how many local installs we were doing. Fix this by
just reading the array length before adding the `NULL` terminator.

Closes: #1407
Approved by: cgwalters
2018-06-13 20:32:37 +00:00
Jonathan Lebon
ca9e04714b daemon/deploy: Tweak no-op output message
We were previously relying on `!self->revision` to determine what
message to print in the no-op case. This made `override reset --all`
also print `No upgrade available.`.

Fix this by using the stronger `is_upgrade` instead.

Closes: #1407
Approved by: cgwalters
2018-06-13 20:32:37 +00:00