Commit Graph

287 Commits

Author SHA1 Message Date
Jonathan Lebon
a1014aee2f daemon: Perform kargs in cache-only mode
Users don't expect the tree to get updated here.

Closes: #1705

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
b104a289f7 daemon/kargs: Fix handling of staged deployments
When handling `GetDeploymentBootConfig()`, we would trip an assertion
when trying to read the full bootconfig from a staged deployment, which
of course doesn't have a full bootconfig yet. Rework this to add a new
`staged` key to the returned dict, in which case only `options` is
included. (Which is all `rpm-ostree kargs` needs anyway).

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
bdf3cda8db Print CVEs fixed in available updates
One question I often have when looking at the output of `status -a`:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
           Diff: 67 upgraded, 1 removed, 16 added
```

is "How serious and relevant are these advisories to me? How soon should
I reboot?". For the packages that I'm most familiar with, e.g. `kernel`
and `git-core`, I usually look up the advisory and check why it was
marked as a security update, mentioned CVEs, and how those affect me.

The updateinfo metadata includes a wealth of information that could be
useful here. In Fedora, CVEs treated by the security response team
result in RHBZs, which end up attached to the advisories and thus make
it into that metadata.

This patch tries to reduce friction in answering some of those questions
above by checking for those CVEs and printing a short description in the
output of `status -a`. Example:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                   CVE-2018-18065 CVE-2018-18066 net-snmp: various flaws [fedora-all]
                   https://bugzilla.redhat.com/show_bug.cgi?id=1637573
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                   CVE-2018-16862 kernel: cleancache: Infoleak of deleted files after reuse of old inodes
                   https://bugzilla.redhat.com/show_bug.cgi?id=1649017
                   CVE-2018-19407 kernel: kvm: NULL pointer dereference in vcpu_scan_ioapic in arch/x86/kvm/x86.c
                   https://bugzilla.redhat.com/show_bug.cgi?id=1652656
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
                   CVE-2018-19486 git: Improper handling of PATH allows for commands to executed from current directory
                   https://bugzilla.redhat.com/show_bug.cgi?id=1653143
           Diff: 67 upgraded, 1 removed, 16 added
```

Including the CVE name and RHBZ link also makes it easier to look for
more details if desired.

Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Jonathan Lebon
140704c105 daemon/transaction-types: Fix dnf cache override
Minor regression from #1587. There were places that were still doing
`dnf_context_set_cache_age()` manually, but those calls didn't exactly
have the intended effect since the core now handled caching itself.

The actual result was that the metadata was still being updated, but not
during the `dnf_repo_check` pass that the core does, but rather the
`Importing rpm-md` pass it does right after. So then, we were
incorrectly printing `(cached)` even though we'd update it afterwards.

Switch to the new way of doing things.

Closes: #1686
Approved by: cgwalters
2018-11-26 23:39:44 +00:00
Jonathan Lebon
ed2fefad85 Move reset out of experimental
This is relatively uncontroversial functionality that has already proved
useful when helping folks debug their stuff. Let's promote it to the
stable interface.

Closes: #1682
Approved by: rfairley
2018-11-22 20:47:50 +00:00
Jonathan Lebon
17359bc0cc core: Tweak "Importing" messages
To make it more obvious what the difference between "Importing metadata"
and "Importing" is, add "rpm-md" to the first and "packages" to the
second.

Closes: #1681
Approved by: cgwalters
2018-11-21 13:54:35 +00:00
Colin Walters
ce339dff9b vmcheck/multitest: Print when we're waiting
To help debug.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
c08e9620d6 vmcheck/multitest: Time execution of each test
For some reason c7 is consistently timing out, let's gather
data on how long our tests take to execute.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
33178cb621 vmcheck: Sync over libsolv and zchunk-libs too
In general our current CI/test system is susceptible to drift
between the container and AH.  The direction we should be
going is to have coreos-assembler solve this problem with
a SDK, but for now, let's ensure that the container's libsolv
makes it to the host, same thing we do for libostree.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
1dee43319c Use indicatif for progress
This turned out to be messier than I thought, because of two primary
factors; the biggest mess here of course is the indirection
through the DBus API.

The other problem is that previously we passed the string to render
each time, and with current indicatif that'd trigger a rerender.
Since (usually) don't change the "prefix string", rework the API.

Change the "percent/n_items" bits to use autocleanups as well, and
to take the prefix string as an initial argument.

Since the state expands to multiple components, also change the
API to use the `0-initialized` pattern rather than trying to
return an aggregate.

We also gain a "sub message" which we use to display e.g.
package names as we're doing checkouts.  Note this ends up
at the end, since otherwise everything else jumps around.

Closes: #1661
Approved by: rfairley
2018-11-09 13:59:43 +00:00
Jonathan Lebon
6583a557ae app/livefs: Require --i-like-danger switch
We've had multiple reports by now of folks using plain `ex livefs` and
getting their bootloader wrecked:

https://github.com/projectatomic/rpm-ostree/issues/1495
https://github.com/projectatomic/rpm-ostree/issues/1504
https://github.com/ostreedev/ostree/issues/1459

Let's require a scary switch for now to emphasize this.

Closes: #1622
Approved by: cgwalters
2018-10-17 20:10:11 +00:00
Jonathan Lebon
676519e5a5 status: Print systemd or ostree staged failure msg
Try to tease out a bit more info from the journal by looking at the
systemd message when the service transitions to the dead state or even
looking at the OSTree output itself.

Example outputs:

```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         error: opendir(ostree/deploy/fedora-atomic/deploy/887c95887a3047a60372016a0d84536530755b60df3cca33c819f7606e220adf.0): No such file or directory
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         ostree-finalize-staged.service: Failed with result 'timeout'.
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

Closes: #1601
Approved by: cgwalters
2018-10-16 17:41:09 +00:00
Jonathan Lebon
f6c3616148 status: Detect if staging failed in previous boot
Sample output:

```
$ rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

(Though open to tweaking it).

I also played with directly invoking `journalctl` for the user, but that
can get really spammy with e.g. `os-prober` output and such.

I wrote this in Rust using journal API wrappers because I also plan to
implement the `history` command in Rust and will also enhance that new
`journal` module there for that.

Requires: https://github.com/ostreedev/ostree/pull/1750
Requires: https://github.com/jmesmon/rust-systemd/pull/54
(Though I've pointed the manifest at my branch for now for CI).

Closes: #1567

Closes: #1601
Approved by: cgwalters
2018-10-16 17:41:09 +00:00
Jonathan Lebon
71588f9744 daemon/utils: Filter out rpmostree.rpmdb.pkglist
Drop the `rpmostree.rpmdb.pkglist` keys from the deployment metadata
since users shouldn't need it and it greatly increases the size of the
output.

Closes: #1577
Approved by: cgwalters
2018-09-25 15:20:57 +00:00
Jonathan Lebon
ea5bec6127 app/db-diff: Diff against rollback if no pending
Often, after rebooting from an upgrade, I want to check what was just
updated. This patch makes `db diff` do the right thing in those cases.
Specifically, before `db diff` without arguments would default to
diff'ing the pending deployment with the booted deployment and error out
otherwise. This patch extends the logic so that if there's a rollback
deployment, we default to diff'ing against that.

Closes: #1565
Approved by: cgwalters
2018-09-20 16:12:14 +00:00
Jonathan Lebon
f098d22e03 tests/vmcheck: Create correct base commits from layered
Our trick of using layered commits as base commit updates doesn't jive
well with the new opportunistic reuse of the base rpmdb. The issue is
that a layered commit includes the rpmdb of *its* base commit at the
`/usr/lib/sysimage` location. So to convert it into a proper base commit
means that the layered rpmdb should move there.

Closes: #1502
Approved by: cgwalters
2018-09-17 20:07:45 +00:00
Jonathan Lebon
a75460f538 upgrader: Reuse existing rpmdb checkout if available
Check if we can reuse the base rpmdb from the pending deployment if it
matches the base rev we're targeting. This allows us to avoid checking
out the tree early only to later on discard it. Such cases include
layering existing packages and inactive requests.

Closes: #1502
Approved by: cgwalters
2018-09-17 20:07:45 +00:00
Robert Fairley
b82db2b501 initramfs: Return error when --arg specified without --enable
This change gives an error when --arg is specified on its own.

Previously, if adding --arg without --enable, no changes occur and
only the initramfs regeneration status is printed. Since --arg only
takes effect when used with --enable, this should also give an error
(or let the user know in some way).

Closes: #1548
Approved by: jlebon
2018-09-12 20:45:19 +00:00
Robert Fairley
be7d87e4b9 initramfs: Print regeneration status for pending deployment
Previously, after enabling initramfs regeneration, the initramfs
command would indicate that initramfs regeneratin is disabled,
suggesting that the initramfs had not actually been enabled. This
happened because the initramfs regeneration status for the current
deployment was printed; initramfs regeneration is only enabled for a
pending deployment.

This change prints the regeneration for the pending deployment,
rather than the current, when no options are given to the initramfs
command.

Fixes: #1526

Closes: #1548
Approved by: jlebon
2018-09-12 20:45:19 +00:00
Micah Abbott
bce966a981 vmcheck/misc-1: skip the overlay check when needed
It's possible to run the `vmcheck` tests against an existing host that
has `rpm-ostree` already present.  We don't overlay the built binaries
in this situation, so we should not check for the presence of the
overlay in the commit meta.

Closes: #1555
Approved by: cgwalters
2018-09-12 14:48:02 +00:00
Jonathan Lebon
807f21788e Hard require staging
This removes the logic around supporting opting out of the staging
feature. We don't want to support multiple configurations here, and at
this point, staging should be considered stable.

Closes: #1546
Approved by: cgwalters
2018-09-11 20:55:48 +00:00
Jonathan Lebon
8976e502fc daemon: Make inactive requests a hard error
Flip the switch and make `rpm-ostree install glibc` a hard error if
`--allow-inactive` is not provided.

Closes: #1545
Approved by: cgwalters
2018-09-10 16:17:14 +00:00
Jonathan Lebon
651a9e01f4 core: Accumulate all missing pkgs into a single error
Accumulate the list of requested packages that were not found and error
out with the full list rather than failing early. This fixes a small UX
papercut in certain situations.

Closes: #1540

Closes: #1541
Approved by: cgwalters
2018-09-08 23:32:13 +00:00
Jonathan Lebon
86f2b67d2b vmcheck: Store temporary files in /var/tmp instead
Since `/tmp` might be on tmpfs, so we'd lose it on reboot. But we have
tests that need it to persist across reboots.

Closes: #1531
Approved by: miabbott
2018-09-07 14:20:01 +00:00
Micah Abbott
c9b4a8195e tests/vmcheck: handle missing /etc/yum.repos.d
It's possible to run the `vmcheck` tests against an existing VM which
may not have any existing `/etc/yum.repos.d` directory.  Since the
tests are providing their own repo files, we should be able to run
them in this uncommon configuration.

Closes: #1530
Approved by: jlebon
2018-09-06 20:50:25 +00:00
Jonathan Lebon
04c0678fa6 app: Add support for passing URLs to RPMs
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).

One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).

This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.

Closes: #1508
Approved by: cgwalters
2018-08-23 11:16:15 +00:00
Colin Walters
5126b2774e Stabilize stage automatic update policy
Now that staging is the default, there's no reason for this
policy not to be stable too.

Closes: #1510
Approved by: jlebon
2018-08-21 01:01:45 +00:00
Colin Walters
b6d07487d5 Turn staged deployments on by default
We've put a lot of work into staged deployments, it's time
to pull the trigger and turn them on by default.  This is
a key step for enabling `stage` mode automatic updates by
default in e.g. Fedora CoreOS/Silverblue.

We add a new `--disable-staged` build-time option to flip
things back.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
44b39a7340 livefs: Require deployment staging
Staging fixes the `/etc` bug for livefs.  There's actually more
we could do here around taking advantage of staging for livefs;
for example, I think once the livefs is complete, we could just delete
the staged deployment.  And then we don't need to render on the next
boot the live status, etc.

Anyways, all that can come in the future.  This is prep for
enabling staging by default.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Jonathan Lebon
4ccad2f64b app/status: Tweak output with --booted
Fix `--booted` was printing an extra line after `Deployments:`. While
we're there, also update the section name to `BootedDeployment:` in that
case to be more correct. (We're not printing all the deployments, only
the booted deployment.)

Closes: #1503
Approved by: cgwalters
2018-08-14 12:32:50 +00:00
Colin Walters
15aecff36a livefs: Rename --replace to --dangerous-do-not-use-replace
And don't mention "replace" in the error texts.
This is (now) known broken with kernel updates; making that not
dangerous isn't going to be easy.  I debated entirely removing it
but for now let's just make it harder to use.

Ref: https://github.com/projectatomic/rpm-ostree/issues/1495

Closes: #1497
Approved by: jlebon
2018-08-09 15:04:11 +00:00
Jonathan Lebon
67860c5ece app/pkg-builtins: Add --unchanged-exit-77
This is analogous to `upgrade --upgrade-unchanged-exit-77`, but for
`install`/`uninstall`. This way, one can determine whether the command
truly had an effect on the default deployment or not.

Since this works by comparing against the previous default deployment,
this works correctly even if we already had a pending deployment at the
beginning of the transaction.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
d35fbb665e app/pkg-builtins: Add --idempotent
Add a new `install/uninstall --idempotent` option to make it easier to
interact with the CLI through scripts. E.g. one doesn't have to check
first if a request has already been installed/uninstalled.

Closes: #1467

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
fcb061b19a app/status: Add --pending-exit-77 switch
This makes it easier for scripts to determine whether there is a pending
deployment instead of using `--json/--jsonpath`.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
7911b14f49 daemon: Fix cached-update including no-op diffs
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.

We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.

Closes: #1446

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Colin Walters
1b4a73f071 tests: Make two more tests compatible with deployment staging
Unlike the kernel args one, these two were simple.

Closes: #1458
Approved by: jlebon
2018-07-21 13:58:21 +00:00
Colin Walters
096004426c rebase: Add support for "custom origin" descriptions
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.

Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM).  This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.

There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.

This builds on prior work in
27bd7b97bb from #1396 .

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
035a2a72ba tests/kernel-args: Prep for staged
This test was hard to adapt for staging.  I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs.  Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults.  So I just deleted those bits.

Closes: #1453
Approved by: jlebon
2018-07-12 17:44:15 +00:00
Jonathan Lebon
caf66d6e45 vmcheck/autoupdate-check: Fix minor typos
Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
cfb5e7c04b libvm: add vm_get_journal_after_cursor
The cursor argument was getting munged up by SSH. Use a helper to make
sure we always get quoting right.

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
222d730cce vmcheck/misc-2: Make compatible with staged default
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system.  It's tempting to do a `reset` before each one
and reboot but this makes things work.

There's some additional assertions here as I went through and
was debugging.

Prep for making staging the default.

Closes: #1438
Approved by: jlebon
2018-07-03 19:14:54 +00:00
Colin Walters
bc52e8eaf3 tests: Update for ostree BLS file naming change
We should probably add a libtest.sh API for so for this, but this is
the quick hack.  Fallout from https://github.com/ostreedev/ostree/pull/1654

Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
fb006b5b43 app/uninstall: Add --all switch to drop all layering
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).

Closes: #1408
Approved by: cgwalters
2018-06-14 12:36:15 +00:00