Commit Graph

367 Commits

Author SHA1 Message Date
Colin Walters
75c676715a daemon: Use MountFlags=slave and opt-in to OSTree read-only /sysroot
This is all we need to tell libostree that we support a read-only
`/sysroot` and `/boot`.

See https://github.com/ostreedev/ostree/issues/1265
PR in https://github.com/ostreedev/ostree/pull/1767
2019-12-13 01:44:56 +01:00
Colin Walters
f295f54306 kargs: Support --append and --delete simultaneously
Code I wrote for the machine-config-operator expected it to
work, and I don't see a reason not to support it.

See https://github.com/openshift/machine-config-operator/issues/1265
2019-12-10 20:27:57 +01:00
Jonathan Lebon
45623a9b54 tests/vmcheck: Fix test-misc-1.sh syntax
The `EOF` needs to be alone on a line to be valid. The way to redirect
the output is unintuitively to do it at the beginning of the line
instead.
2019-10-08 14:10:53 -07:00
Colin Walters
c8113bde32 Add hidden coreos-rootfs seal command
All this does is put the immutable bit on the target directory.
The intention is to replace this bit to start:
8b205bfbb9/src/create_disk.sh (L229)

However, the real goal here is to add code in this file
to handle redeploying the rootfs for Fedora CoreOS which
combines OSTree+Ignition:
https://github.com/coreos/fedora-coreos-tracker/issues/94

Basically doing this in proper Rust is going to be a lot
nicer than shell script in dracut modules.  Among other
details, coreutils `mv` doesn't seem to do the right thing
for SELinux labels when policy isn't loaded.
2019-10-04 08:03:03 -07:00
Jonathan Lebon
10755592ea Add new ex history command
This is the rpm-ostree equivalent of `dnf history`. As opposed to the
history of the refspec (i.e. `ostree log`), this shows the history of
the system, i.e. the refspecs the host deployed, checksums, versions,
layered packages, etc... The amount of details remembered is similar to
what shows up in `status`.

There's definitely some further enhancements possible (e.g. printing
package diffs, displaying rollbacks), though this seems in good enough
shape as a first cut.

Closes: #1489

Closes: #1813
Approved by: cgwalters
2019-09-24 14:17:12 +00:00
Jonathan Lebon
9e2ceca06f app/deploy: Gate 77 exit behind --unchanged-exit-77
This has a bit of history, but essentially in 1c01141e, we made both
`upgrade` and `deploy` automatically exit 77 if there were no changes.
Then in c3f1e7c8, we only changed `upgrade` so that it became gated
behind `--upgrade-unchanged-exit-77`.

I think we should carry this forward into `deploy` as well. The way I
look at this is: the default UX shouldn't require users to care about
special exit codes. That's something scripts care about. In its vanilla
form, either a command should error out or succeed.

This patch tries to add some consistency by introducing a new
`--unchanged-exit-77` in both `deploy` and `upgrade` (where it just
replaces the previous switch). The naming here matches what `install`
has too.

So... this does break backwards compatibility for any scripts which
relied on that behaviour. Though the only app I know today which wants
deploy semantics and doesn't use the D-Bus API is Zincati, which
actually hit this issue. There's also RHCOS, though the `pivot` there
uses `rebase`, not `deploy`. So overall, I think this is worth breaking
now while we're still in a transitionary period in the downstreams?

Closes: #1906
Approved by: cgwalters
2019-09-24 08:07:04 +00:00
Jonathan Lebon
40f6b4bdc9 vmcheck: Adapt test-override-kernel.sh
This is one of the tests right now that assumes it's running on f29.
We might be sort of in this awkward dual path for a while where we want
tests to run on both f29 (i.e. FAH) and f30 (i.e. FCOS).

Closes: #1900
Approved by: cgwalters
2019-09-09 23:50:32 +00:00
Jonathan Lebon
313b3e655d app/deploy: Add --disallow-downgrade switch
In FCOS, we want to make sure that Zincati is always deploying a newer
tree to prevent downgrade attacks in certain threat models.

For completeness, also add the option to `rebase`.
2019-08-21 16:43:37 -04:00
Rafael Fonseca
d0f90ca4dd app/status: Group EVRs for RemovedBasePackages if possible
The same way we abbreviate ReplacedBasePackages when there are matching
EVR diffs, let's do something similar for RemovedBasePackages for
matching EVRs.

Solves #1784

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>

Closes: #1852
Approved by: jlebon
2019-06-18 14:34:13 +00:00
Jonathan Lebon
035ac2eaa6 tests/vmcheck: Fully drop python 3 requirement
Drop the use of Ansible everywhere. In the few cases where we really
Python, just spawn a container instead.

This is required to be able to hack on Fedora CoreOS.

Closes: #1850
Approved by: jlebon
2019-06-10 15:36:06 +00:00
Jonathan Lebon
68c416fa9c app/deploy: Add --lock-finalization switch
Of course, update agents driving rpm-ostree know exactly to which commit
they want the system to upgrade, so `upgrade --lock-finalization` is not
helpful. Teach `deploy` the `--lock-finalization` switch too.

Closes: #1846
Approved by: lucab
2019-05-27 17:18:36 +00:00
Jonathan Lebon
12fbe89aba app/db-diff: Add --format=json output
Add a new "json" output format. The "diff" format is also a mostly
machine-compatible one. But JSON is much more ubiquitous and easier to
consume.

Closes: #1844
Approved by: cgwalters
2019-05-23 20:24:11 +00:00
Jonathan Lebon
4c99cc765f Copy used command-line and set as transaction title
In the app, rebuild the exact command-line that the client used and pass
that to the daemon to be used as the transaction title. Especially in
transactions like `UpdateDeployment()`, we can avoid reverse-engineering
what the original command used was.

This will be used by the upcoming history feature to record the
command-line used in the journal.

Closes: #1824
Approved by: rfairley
2019-05-08 21:09:08 +00:00
Jonathan Lebon
d113b6a913 app/status: Make --json output pretty JSON
Yes, it's mostly for machines, but in practice it's super useful for
humans to look at esp. when debugging.

Closes: #1828
Approved by: cgwalters
2019-05-08 19:02:32 +00:00
Jonathan Lebon
206ae24d4e tests: Bump to Python 3 only
This bumps the requirement on the controlling host to Python 3 only.
It also bumps the requirement on the target host to Python 3 as well
since FCOS doesn't ship Python 2 right now.

Though we'll need to eventually drop all Python usage anyway, but at
least let's get tests passing on FCOS first. (See related previous
patch).

Closes: #1828
Approved by: cgwalters
2019-05-08 19:02:32 +00:00
Jonathan Lebon
9f618d0acc vmcheck/overlay: Drop dependency on host Python
Also switch to using `jq` on the controlling host instead of Python.

This is also prep for switching CI to FCOS which is likely to not ship
Python at all. There are still spots a bit everywhere where we currently
assume Python on the target host. We'll have to address those soon.

Closes: #1828
Approved by: cgwalters
2019-05-08 19:02:32 +00:00
Jonathan Lebon
83a267409d Add new D-Bus APIs for deployment finalization
Teach `UpdateDeployment` to make use of libostree's staging lock and
then add a `FinalizeDeployment` API to perform the final unlock &
reboot.

I also added a hidden CLI to make testing this easier, but also because
it's likely the FCOS-agent-yet-to-be-named will just end up using the
CLI to keep it simple.

Closes: #1748

Closes: #1814
Approved by: lucab
2019-04-30 08:36:28 +00:00
Jonathan Lebon
0da9f9972b tests/vmcheck: Adapt to kernel v5.0
Closes: #1810
Approved by: jlebon
2019-04-05 19:28:39 +00:00
Alexander Larsson
d5b907741a Test that /opt layering works
Closes: #1795
Approved by: jlebon
2019-03-26 13:22:32 +00:00
Jonathan Lebon
02b25c616d libpriv/kargs: Strengthen and simplify new kargs APIs
Note this patch only touches the *new* APIs that aren't part of
libostree.

Now that we can use `g_ptr_array_find_with_equal_func`, we can drop our
custom `_ostree_ptr_array_find`.

Also strengthen our handling of values everywhere to handle the `NULL`
case and properly support `KEYWORD` args. I ended up getting rid of
`_ostree_kernel_arg_query_status` in the process since it made that
assumption a lot and overall added more complexity than necessary.

Closes: #1796
Approved by: cgwalters
2019-03-23 18:48:05 +00:00
Jonathan Lebon
b324ee4807 app/status: Always print pending deployment diff
Right now we only print a diff of the pending deployment if we have a
cached update (which only happens if user just did an `upgrade`
operation). But really, we can just always print this for the pending
deployment regardless of whether there's a cached update calculated.

This is prep for changing chained operations to only show the diff
between the previous pending deployment to the new pending deployment.
With this patch, the full diff from booted to pending will always be
available through `status` (and `db diff` too though it's not as nice).

Closes: #1760
Approved by: cgwalters
2019-03-20 12:42:16 +00:00
Jonathan Lebon
ad1451fac2 Rebase to latest libdnf
This brings us back in sync with the latest libdnf git master. This
required a bunch of work both on the libdnf and rpm-ostree side to get
working. See e.g.
https://github.com/rpm-software-management/libdnf/issues/645.

A few things to adapt to:

- soname bump to `libdnf.so.2`
- `DnfAdvisory` is no longer a `GObject` (annoyingly it's not replaced
  by something we can keep a ref on, so this requires some hacks to
  steal from the `GPtrArray` -- could enhance libdnf for this later)
- disable SWDB history writing
- use new reldep public API
- update for latest `hy_subject_get_best_selector()` API

This now unlocks the possibility to add support for modules. (One can
see hints of this in the diff by the fact that `libdnf` links to
`libmodulemd1`.)

Update submodule: libdnf

Closes: #1404
Approved by: cgwalters
2019-03-19 14:29:15 +00:00
Jonathan Lebon
44110377c6 vmcheck/test-rojig-client.sh: Don't check full rojig NEVRA
Let's make this test work across major version rebases of the FAHC
treecompose job by not hardcoding a specific `%{dist}` here. It's strong
enough to check that the `deploy` operation has the expected previous
version.

I did bump the FAHC buildroot so that next version of the rojig RPM will
be f29, but I don't want to wait until the job has composed at least two
of them.

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
078268d4d3 libpriv: Add more error-prefixing in sanity checks
Currently tracing through an error around that area, and this would've
helped.

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
4ae3b174f5 ci: Bump to f29
Better late than never!

Closes: #1787
Approved by: cgwalters
2019-03-19 12:19:38 +00:00
Jonathan Lebon
3a616bd318 daemon/deployment-utils: Specify array element type
Otherwise, glib will complain if the array is empty since it can't infer
the type of the item.

While we're here, just `git grep` all instances of
`G_VARIANT_TYPE_ARRAY` and make sure they use a fully-specified format.
I added a test to sanity check that glib is happy to synthesize empty
`GVariant` arrays from `g_variant_builder()` if the format string is
specified.

Closes: #1783
Approved by: cgwalters
2019-03-13 02:46:09 +00:00
Colin Walters
e7f87b03ef tests: Don't make system user with differently-named group
Sysusers doesn't really support this, and no RPMs do it.

Closes: #1763
Approved by: jlebon
2019-02-25 14:21:28 +00:00
Jonathan Lebon
58a79056a8 libpriv/scripts: Add /run/ostree-booted for scriptlets
RPM-OSTree has been pretty good so far at consuming the exact same RPMs
used for traditional OSes without modifications. This is important,
because shielding RPMs from the OSTree abstraction means we remain
compatible with a large portion of the ecosystem.

However, there are some apps that definitely require rethinking their
approach. The example right now is akmods, which has a patch proposed to
build kmods at `%post` time on OSTree systems instead of from the
daemon.[1]

In such situations, scriptlets need something to key off of for the
OSTree-specific approach. The `/run/ostree-booted` file is the de facto
API to determine if we're running on an OSTree system or not. This patch
simply extends this API so that scriptlets can naturally make use of
them.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1667014

Closes: #1750
Approved by: cgwalters
2019-02-12 14:24:48 +00:00
Jonathan Lebon
7cceb35219 app/rebase: Support local repo remotes
Teach rpm-ostree to interpret rebases where the remote component is a
path to a local repo, e.g.:

    rpm-ostree rebase /mnt/ostree/repo:my/target/ref

Essentially, the local remote in this case is considered "ephemeral".
It's kind of the equivalent of, on traditional systems:

    dnf install --repofrompath repo,/path/to/repodata ...

The use case for this is in OpenShift v4, in which upgrades are done
from containers containing the OSTree commit. There, we want to point
RPM-OSTree directly at the repo in the mounted container and rebase to
the checksum.

For now, the option is marked experimental. One major reason for this is
that the way we pass the repo differs on RHEL7 vs other platforms. (See
comment block in `rpmostree-dbus-helpers.c` for details).

Related: https://github.com/openshift/machine-config-operator/issues/314

Co-authored-by: Colin Walters <walters@verbum.org>

Closes: #1732
Approved by: cgwalters
2019-02-05 18:03:02 +00:00
Jonathan Lebon
88c2700cdd daemon/deploy: Fix free() of override replace pkgs
We were using `g_strfreev()` to free the string array, but the strings
themselves were owned by the `modifiers` GVariantDict. Fix this and make
the comments about it more explicit. On my computer (and at least
Dusty's), this was only actually tripping up libc when passing more than
just one package on the CLI.

Closes: #1707

Closes: #1709
Approved by: cgwalters
2018-12-10 20:49:52 +00:00
Jonathan Lebon
a1014aee2f daemon: Perform kargs in cache-only mode
Users don't expect the tree to get updated here.

Closes: #1705

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
b104a289f7 daemon/kargs: Fix handling of staged deployments
When handling `GetDeploymentBootConfig()`, we would trip an assertion
when trying to read the full bootconfig from a staged deployment, which
of course doesn't have a full bootconfig yet. Rework this to add a new
`staged` key to the returned dict, in which case only `options` is
included. (Which is all `rpm-ostree kargs` needs anyway).

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
bdf3cda8db Print CVEs fixed in available updates
One question I often have when looking at the output of `status -a`:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
           Diff: 67 upgraded, 1 removed, 16 added
```

is "How serious and relevant are these advisories to me? How soon should
I reboot?". For the packages that I'm most familiar with, e.g. `kernel`
and `git-core`, I usually look up the advisory and check why it was
marked as a security update, mentioned CVEs, and how those affect me.

The updateinfo metadata includes a wealth of information that could be
useful here. In Fedora, CVEs treated by the security response team
result in RHBZs, which end up attached to the advisories and thus make
it into that metadata.

This patch tries to reduce friction in answering some of those questions
above by checking for those CVEs and printing a short description in the
output of `status -a`. Example:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                   CVE-2018-18065 CVE-2018-18066 net-snmp: various flaws [fedora-all]
                   https://bugzilla.redhat.com/show_bug.cgi?id=1637573
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                   CVE-2018-16862 kernel: cleancache: Infoleak of deleted files after reuse of old inodes
                   https://bugzilla.redhat.com/show_bug.cgi?id=1649017
                   CVE-2018-19407 kernel: kvm: NULL pointer dereference in vcpu_scan_ioapic in arch/x86/kvm/x86.c
                   https://bugzilla.redhat.com/show_bug.cgi?id=1652656
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
                   CVE-2018-19486 git: Improper handling of PATH allows for commands to executed from current directory
                   https://bugzilla.redhat.com/show_bug.cgi?id=1653143
           Diff: 67 upgraded, 1 removed, 16 added
```

Including the CVE name and RHBZ link also makes it easier to look for
more details if desired.

Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Jonathan Lebon
140704c105 daemon/transaction-types: Fix dnf cache override
Minor regression from #1587. There were places that were still doing
`dnf_context_set_cache_age()` manually, but those calls didn't exactly
have the intended effect since the core now handled caching itself.

The actual result was that the metadata was still being updated, but not
during the `dnf_repo_check` pass that the core does, but rather the
`Importing rpm-md` pass it does right after. So then, we were
incorrectly printing `(cached)` even though we'd update it afterwards.

Switch to the new way of doing things.

Closes: #1686
Approved by: cgwalters
2018-11-26 23:39:44 +00:00
Jonathan Lebon
ed2fefad85 Move reset out of experimental
This is relatively uncontroversial functionality that has already proved
useful when helping folks debug their stuff. Let's promote it to the
stable interface.

Closes: #1682
Approved by: rfairley
2018-11-22 20:47:50 +00:00
Jonathan Lebon
17359bc0cc core: Tweak "Importing" messages
To make it more obvious what the difference between "Importing metadata"
and "Importing" is, add "rpm-md" to the first and "packages" to the
second.

Closes: #1681
Approved by: cgwalters
2018-11-21 13:54:35 +00:00
Colin Walters
ce339dff9b vmcheck/multitest: Print when we're waiting
To help debug.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
c08e9620d6 vmcheck/multitest: Time execution of each test
For some reason c7 is consistently timing out, let's gather
data on how long our tests take to execute.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
33178cb621 vmcheck: Sync over libsolv and zchunk-libs too
In general our current CI/test system is susceptible to drift
between the container and AH.  The direction we should be
going is to have coreos-assembler solve this problem with
a SDK, but for now, let's ensure that the container's libsolv
makes it to the host, same thing we do for libostree.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
1dee43319c Use indicatif for progress
This turned out to be messier than I thought, because of two primary
factors; the biggest mess here of course is the indirection
through the DBus API.

The other problem is that previously we passed the string to render
each time, and with current indicatif that'd trigger a rerender.
Since (usually) don't change the "prefix string", rework the API.

Change the "percent/n_items" bits to use autocleanups as well, and
to take the prefix string as an initial argument.

Since the state expands to multiple components, also change the
API to use the `0-initialized` pattern rather than trying to
return an aggregate.

We also gain a "sub message" which we use to display e.g.
package names as we're doing checkouts.  Note this ends up
at the end, since otherwise everything else jumps around.

Closes: #1661
Approved by: rfairley
2018-11-09 13:59:43 +00:00
Jonathan Lebon
6583a557ae app/livefs: Require --i-like-danger switch
We've had multiple reports by now of folks using plain `ex livefs` and
getting their bootloader wrecked:

https://github.com/projectatomic/rpm-ostree/issues/1495
https://github.com/projectatomic/rpm-ostree/issues/1504
https://github.com/ostreedev/ostree/issues/1459

Let's require a scary switch for now to emphasize this.

Closes: #1622
Approved by: cgwalters
2018-10-17 20:10:11 +00:00
Jonathan Lebon
676519e5a5 status: Print systemd or ostree staged failure msg
Try to tease out a bit more info from the journal by looking at the
systemd message when the service transitions to the dead state or even
looking at the OSTree output itself.

Example outputs:

```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         error: opendir(ostree/deploy/fedora-atomic/deploy/887c95887a3047a60372016a0d84536530755b60df3cca33c819f7606e220adf.0): No such file or directory
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

```
[root@f28-ros ~]# rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         ostree-finalize-staged.service: Failed with result 'timeout'.
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

Closes: #1601
Approved by: cgwalters
2018-10-16 17:41:09 +00:00
Jonathan Lebon
f6c3616148 status: Detect if staging failed in previous boot
Sample output:

```
$ rpm-ostree status
State: idle
Warning: failed to finalize previous deployment
         check `journalctl -b -1 -u ostree-finalize-staged.service`
AutomaticUpdates: disabled
...
```

(Though open to tweaking it).

I also played with directly invoking `journalctl` for the user, but that
can get really spammy with e.g. `os-prober` output and such.

I wrote this in Rust using journal API wrappers because I also plan to
implement the `history` command in Rust and will also enhance that new
`journal` module there for that.

Requires: https://github.com/ostreedev/ostree/pull/1750
Requires: https://github.com/jmesmon/rust-systemd/pull/54
(Though I've pointed the manifest at my branch for now for CI).

Closes: #1567

Closes: #1601
Approved by: cgwalters
2018-10-16 17:41:09 +00:00
Jonathan Lebon
71588f9744 daemon/utils: Filter out rpmostree.rpmdb.pkglist
Drop the `rpmostree.rpmdb.pkglist` keys from the deployment metadata
since users shouldn't need it and it greatly increases the size of the
output.

Closes: #1577
Approved by: cgwalters
2018-09-25 15:20:57 +00:00
Jonathan Lebon
ea5bec6127 app/db-diff: Diff against rollback if no pending
Often, after rebooting from an upgrade, I want to check what was just
updated. This patch makes `db diff` do the right thing in those cases.
Specifically, before `db diff` without arguments would default to
diff'ing the pending deployment with the booted deployment and error out
otherwise. This patch extends the logic so that if there's a rollback
deployment, we default to diff'ing against that.

Closes: #1565
Approved by: cgwalters
2018-09-20 16:12:14 +00:00
Jonathan Lebon
f098d22e03 tests/vmcheck: Create correct base commits from layered
Our trick of using layered commits as base commit updates doesn't jive
well with the new opportunistic reuse of the base rpmdb. The issue is
that a layered commit includes the rpmdb of *its* base commit at the
`/usr/lib/sysimage` location. So to convert it into a proper base commit
means that the layered rpmdb should move there.

Closes: #1502
Approved by: cgwalters
2018-09-17 20:07:45 +00:00
Jonathan Lebon
a75460f538 upgrader: Reuse existing rpmdb checkout if available
Check if we can reuse the base rpmdb from the pending deployment if it
matches the base rev we're targeting. This allows us to avoid checking
out the tree early only to later on discard it. Such cases include
layering existing packages and inactive requests.

Closes: #1502
Approved by: cgwalters
2018-09-17 20:07:45 +00:00
Robert Fairley
b82db2b501 initramfs: Return error when --arg specified without --enable
This change gives an error when --arg is specified on its own.

Previously, if adding --arg without --enable, no changes occur and
only the initramfs regeneration status is printed. Since --arg only
takes effect when used with --enable, this should also give an error
(or let the user know in some way).

Closes: #1548
Approved by: jlebon
2018-09-12 20:45:19 +00:00
Robert Fairley
be7d87e4b9 initramfs: Print regeneration status for pending deployment
Previously, after enabling initramfs regeneration, the initramfs
command would indicate that initramfs regeneratin is disabled,
suggesting that the initramfs had not actually been enabled. This
happened because the initramfs regeneration status for the current
deployment was printed; initramfs regeneration is only enabled for a
pending deployment.

This change prints the regeneration for the pending deployment,
rather than the current, when no options are given to the initramfs
command.

Fixes: #1526

Closes: #1548
Approved by: jlebon
2018-09-12 20:45:19 +00:00
Micah Abbott
bce966a981 vmcheck/misc-1: skip the overlay check when needed
It's possible to run the `vmcheck` tests against an existing host that
has `rpm-ostree` already present.  We don't overlay the built binaries
in this situation, so we should not check for the presence of the
overlay in the commit meta.

Closes: #1555
Approved by: cgwalters
2018-09-12 14:48:02 +00:00
Jonathan Lebon
807f21788e Hard require staging
This removes the logic around supporting opting out of the staging
feature. We don't want to support multiple configurations here, and at
this point, staging should be considered stable.

Closes: #1546
Approved by: cgwalters
2018-09-11 20:55:48 +00:00
Jonathan Lebon
8976e502fc daemon: Make inactive requests a hard error
Flip the switch and make `rpm-ostree install glibc` a hard error if
`--allow-inactive` is not provided.

Closes: #1545
Approved by: cgwalters
2018-09-10 16:17:14 +00:00
Jonathan Lebon
651a9e01f4 core: Accumulate all missing pkgs into a single error
Accumulate the list of requested packages that were not found and error
out with the full list rather than failing early. This fixes a small UX
papercut in certain situations.

Closes: #1540

Closes: #1541
Approved by: cgwalters
2018-09-08 23:32:13 +00:00
Jonathan Lebon
86f2b67d2b vmcheck: Store temporary files in /var/tmp instead
Since `/tmp` might be on tmpfs, so we'd lose it on reboot. But we have
tests that need it to persist across reboots.

Closes: #1531
Approved by: miabbott
2018-09-07 14:20:01 +00:00
Micah Abbott
c9b4a8195e tests/vmcheck: handle missing /etc/yum.repos.d
It's possible to run the `vmcheck` tests against an existing VM which
may not have any existing `/etc/yum.repos.d` directory.  Since the
tests are providing their own repo files, we should be able to run
them in this uncommon configuration.

Closes: #1530
Approved by: jlebon
2018-09-06 20:50:25 +00:00
Jonathan Lebon
04c0678fa6 app: Add support for passing URLs to RPMs
This teaches the client to fetch packages from URLs directly so that one
doesn't have to `curl` first and then install. Supported anywhere
package filenames are allowed (notably: `install` and
`override replace`).

One neat things about this is that we download the file into an
`O_TMPFILE` and then pass on ownership of that fd directly to the
daemon. So at no point are the packages actually laying visible on the
system. (Assuming the filesystem supports `O_TMPFILE` that is).

This adds direct linking to libcurl and openssl, two libraries which we
were already pulling in indirectly.

Closes: #1508
Approved by: cgwalters
2018-08-23 11:16:15 +00:00
Colin Walters
5126b2774e Stabilize stage automatic update policy
Now that staging is the default, there's no reason for this
policy not to be stable too.

Closes: #1510
Approved by: jlebon
2018-08-21 01:01:45 +00:00
Colin Walters
b6d07487d5 Turn staged deployments on by default
We've put a lot of work into staged deployments, it's time
to pull the trigger and turn them on by default.  This is
a key step for enabling `stage` mode automatic updates by
default in e.g. Fedora CoreOS/Silverblue.

We add a new `--disable-staged` build-time option to flip
things back.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
44b39a7340 livefs: Require deployment staging
Staging fixes the `/etc` bug for livefs.  There's actually more
we could do here around taking advantage of staging for livefs;
for example, I think once the livefs is complete, we could just delete
the staged deployment.  And then we don't need to render on the next
boot the live status, etc.

Anyways, all that can come in the future.  This is prep for
enabling staging by default.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Jonathan Lebon
4ccad2f64b app/status: Tweak output with --booted
Fix `--booted` was printing an extra line after `Deployments:`. While
we're there, also update the section name to `BootedDeployment:` in that
case to be more correct. (We're not printing all the deployments, only
the booted deployment.)

Closes: #1503
Approved by: cgwalters
2018-08-14 12:32:50 +00:00
Colin Walters
15aecff36a livefs: Rename --replace to --dangerous-do-not-use-replace
And don't mention "replace" in the error texts.
This is (now) known broken with kernel updates; making that not
dangerous isn't going to be easy.  I debated entirely removing it
but for now let's just make it harder to use.

Ref: https://github.com/projectatomic/rpm-ostree/issues/1495

Closes: #1497
Approved by: jlebon
2018-08-09 15:04:11 +00:00
Jonathan Lebon
67860c5ece app/pkg-builtins: Add --unchanged-exit-77
This is analogous to `upgrade --upgrade-unchanged-exit-77`, but for
`install`/`uninstall`. This way, one can determine whether the command
truly had an effect on the default deployment or not.

Since this works by comparing against the previous default deployment,
this works correctly even if we already had a pending deployment at the
beginning of the transaction.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
d35fbb665e app/pkg-builtins: Add --idempotent
Add a new `install/uninstall --idempotent` option to make it easier to
interact with the CLI through scripts. E.g. one doesn't have to check
first if a request has already been installed/uninstalled.

Closes: #1467

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
fcb061b19a app/status: Add --pending-exit-77 switch
This makes it easier for scripts to determine whether there is a pending
deployment instead of using `--json/--jsonpath`.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
7911b14f49 daemon: Fix cached-update including no-op diffs
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.

We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.

Closes: #1446

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Colin Walters
1b4a73f071 tests: Make two more tests compatible with deployment staging
Unlike the kernel args one, these two were simple.

Closes: #1458
Approved by: jlebon
2018-07-21 13:58:21 +00:00
Colin Walters
096004426c rebase: Add support for "custom origin" descriptions
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.

Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM).  This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.

There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.

This builds on prior work in
27bd7b97bb from #1396 .

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
035a2a72ba tests/kernel-args: Prep for staged
This test was hard to adapt for staging.  I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs.  Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults.  So I just deleted those bits.

Closes: #1453
Approved by: jlebon
2018-07-12 17:44:15 +00:00
Jonathan Lebon
caf66d6e45 vmcheck/autoupdate-check: Fix minor typos
Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
cfb5e7c04b libvm: add vm_get_journal_after_cursor
The cursor argument was getting munged up by SSH. Use a helper to make
sure we always get quoting right.

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
222d730cce vmcheck/misc-2: Make compatible with staged default
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system.  It's tempting to do a `reset` before each one
and reboot but this makes things work.

There's some additional assertions here as I went through and
was debugging.

Prep for making staging the default.

Closes: #1438
Approved by: jlebon
2018-07-03 19:14:54 +00:00
Colin Walters
bc52e8eaf3 tests: Update for ostree BLS file naming change
We should probably add a libtest.sh API for so for this, but this is
the quick hack.  Fallout from https://github.com/ostreedev/ostree/pull/1654

Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
fb006b5b43 app/uninstall: Add --all switch to drop all layering
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).

Closes: #1408
Approved by: cgwalters
2018-06-14 12:36:15 +00:00
Jonathan Lebon
72e427cc3d app/override: Don't crash on no change
Doing `override reset --all` may be a no-op if there are no overrides
currently applied. But we were trying to always print the tree diff
under the assumption that all override invocations have to result in a
new deployment.

Fix this by using the same default deployment check that `upgrade` does
before printing.

Closes: #1407
Approved by: cgwalters
2018-06-13 20:32:37 +00:00
Jonathan Lebon
d436d48a2f libpriv/origin: Allow removing local RPMs by name only
This fixes a painful UX issue where one must use the full NEVRA when
uninstalling a locally layered RPM. Now, one can specify either the
NEVRA or the package name only. Though we still try to interpret the
request first as a NEVRA or a capability.

Closes: #1386

Closes: #1400
Approved by: cgwalters
2018-06-12 14:55:40 +00:00
Jonathan Lebon
073978aace app/override: Don't auto-enable cache-only if also installing
Otherwise, we won't be able to fetch the requested packages from the
repo.

Closes: #1384

Closes: #1402
Approved by: cgwalters
2018-06-12 14:52:27 +00:00
Colin Walters
27bd7b97bb Handle "pinned commits" specifically
There are a few scenarios today where one might deliver content
to a machine via an external transport.  For example, take the
scenario of a single server updated via USB drive.  While we
can provide a refspec...what should the remote be?  (This gets
into ostree collections).  There's nothing really that can
happen when typing `rpm-ostree upgrade` unless the USB stick
is plugged in.  That type of scenario should be emphasized
by pinning the commit - the machine is updated via an external
script.

Another case: we're experimenting embedding OSTree commits inside OCI
containers.  Here again since rpm-ostree can't understand how to
pull content from containers, it's saner to drop the refspec
bits, and pin to a commit.

Further enhancements will follow to make the admin experience more
obvious.

Closes: #1396
Approved by: jlebon
2018-06-11 20:37:58 +00:00
Colin Walters
26f04595b2 upgrade: Note if automatic updates are enabled
When e.g. FAH does the automatic update transition, let's remind
people who are doing it manually still.

Closes: #1399
Approved by: jlebon
2018-06-11 18:46:58 +00:00
Jonathan Lebon
34c3d5a02a app/install: Add --allow-inactive switch
For now all this switch does is turn off the scary warning. We also
tweak the warning to make it clear that this will be required in a
future release.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
9085e5144d tests/vmcheck: Fix rojig expected NEVRA for f28
Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
8f71aa95e6 tests/vmcheck: Disable rojig test on CentOS
I was honestly surprised to see this test was running all this time for
CentOS. As cool as it is, this test is really more meant for Fedora.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
09b2ca0482 tests/vmcheck: Fix override remove test
We were trying to test that doing `override remove` on a layered package
errored out. But the test was bogus since `foo` is a base package here.
Let's just use a brand new package that will actually be layered.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
d3d205f7af tests/vmcheck: Split basic layering tests in two
Just part of the continuing breakdown of our testsuites into smaller
pieces for more efficient allocation.

Will add a test case to the `layering-basic-2.sh` in a follow-up commit.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
bff3a54e1e daemon/upgrader: Consistently remove transient state
With the new support for pinning deployments, we need to also update
rpm-ostree to clean up the transient state as is now done in the ostree
sysroot upgrader.

This addresses that issue as well as tries to be a little cleaner in how
we clean up other transient state. Notably, we add a new helper function
to `RpmOstreeOrigin` to do this for us and use it in the upgrader. In
other cases, we do want this transient information since it allows us to
describe the deployment.

Closes: https://github.com/ostreedev/ostree/issues/1595

Closes: #1372
Approved by: cgwalters
2018-05-23 16:27:10 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Colin Walters
99901ac0dc daemon: Add an 'id' param to RegisterClient, log it
This makes the logs a bit more useful, but the ultimate goal
here is to write the originating client `id` to the cached update
data, so users know that e.g. `gnome-software` triggered it.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Colin Walters
8387f1c7c3 Add "ex-stage" update policy, support for ostree staged deployments
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.

I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.

This was all quite straightforward, just plumbing through lots
of layers.

Closes: #1321
Approved by: jlebon
2018-05-14 19:03:56 +00:00
Colin Walters
b03b6a2057 core: Tweak package checkout message, update libglnx
The common case is having layered packages and no overrides; seeing
`Applying 8 overlays` then `Building filesystem [0/8]` is redundant.
Tweak the progress to avoid the double message.  Also change the terminology
to clarify that each item is a package.

Also update libglnx to pull in progress bar newline fix.

Update submodule: libglnx

Closes: #1363
Approved by: jlebon
2018-05-10 01:13:38 +00:00
Jonathan Lebon
c1c78154b1 app/status: Print diff/advisories with pending deployment
Follow-up from #1344.

In the case where a cached update is created from an `upgrade` operation
(and soon, "stage" auto-update policy runs), we can just print the diff
and advisory info together with the pending deployment. This makes the
output look much more natural.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Colin Walters
5037c85330 tests: Extend staging to cover vmcheck/upgrades
This now works after https://github.com/ostreedev/ostree/pull/1570

However, I was hitting an issue with the temp httpd running the
tests serially, let's split them up.

Really the only sane thing is to run each of these tests in a fresh
VM; we'll do the VM-in-container pattern here hopefully soon.

Closes: #1355
Approved by: jlebon
2018-05-02 19:20:26 +00:00
Colin Walters
506910d930 Add an experimental option to use libostree's "staging" API
Now that infrastructure for this has landed in libostree,
let's make it easy for people to opt-in to testing it.  This is a distinct first
step for adding it as an update policy.

Closes: #1352
Approved by: jlebon
2018-05-02 13:46:37 +00:00
Jonathan Lebon
ee458c3c50 daemon/upgrade: Write out new cached update
Right now, cached updates generated during "check" policy runs are
completely decoupled from upgrade operations. This can lead
to the surprising situation where the "Available update" is *older* than
a freshly deployed pending tree with `rpm-ostree upgrade`.

We should just generate a cached update after upgrade operations. This
is also prep for staged deployments, where we'll want to do this as
well.

Note that we write out the cached update here even if automatic updates
are turned off since it's essentially free. I've been thinking about
always displaying that information after an `rpm-ostree upgrade` in
`status`. Though not sure if we should keep it in a separate "Available
update" section, or somehow morph it as part of the pending deployment
output.

Closes: #1344
Approved by: cgwalters
2018-04-26 21:16:52 +00:00
Colin Walters
8726be65e3 core: Finish implementing `override replace ./kernel*.x86_64.rpm
Previously we merged: #1228 AKA 12dc565b00
My recollection is that was working on it the background, while doing
something else, and I clearly didn't get to the point of testing it "for real".

There are many interlocking issues here to make this work.  For example,
the "remove RPM" logic needs special handling for the kernel, because
we also inject content into `/usr/lib/ostree-boot` and also generate
the initramfs, etc.

The architecture I chose is to have the core *detect* when a kernel
is changed, and also call into the kernel processing code when removing
a kernel package.  But the logic for doing kernel reinstallation client-side
is best alongside the initramfs generation logic which already existed
in the sysroot upgrader.

I extended the test suite to cover what was failing before, and I
tested this interactively.  But I'm uncertain about adding a test
for actually *booting* into the GA kernel as it's quite possible
some bits in userspace rely on a newer kernel.  Fixing this properly
really wants some infrastructure to better "re-version" an existing
package without changing its content.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1334

Closes: #1346
Approved by: jlebon
2018-04-26 16:49:53 +00:00