Commit Graph

248 Commits

Author SHA1 Message Date
Jonathan Lebon
cfb5e7c04b libvm: add vm_get_journal_after_cursor
The cursor argument was getting munged up by SSH. Use a helper to make
sure we always get quoting right.

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
222d730cce vmcheck/misc-2: Make compatible with staged default
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system.  It's tempting to do a `reset` before each one
and reboot but this makes things work.

There's some additional assertions here as I went through and
was debugging.

Prep for making staging the default.

Closes: #1438
Approved by: jlebon
2018-07-03 19:14:54 +00:00
Colin Walters
bc52e8eaf3 tests: Update for ostree BLS file naming change
We should probably add a libtest.sh API for so for this, but this is
the quick hack.  Fallout from https://github.com/ostreedev/ostree/pull/1654

Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
fb006b5b43 app/uninstall: Add --all switch to drop all layering
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).

Closes: #1408
Approved by: cgwalters
2018-06-14 12:36:15 +00:00
Jonathan Lebon
72e427cc3d app/override: Don't crash on no change
Doing `override reset --all` may be a no-op if there are no overrides
currently applied. But we were trying to always print the tree diff
under the assumption that all override invocations have to result in a
new deployment.

Fix this by using the same default deployment check that `upgrade` does
before printing.

Closes: #1407
Approved by: cgwalters
2018-06-13 20:32:37 +00:00
Jonathan Lebon
d436d48a2f libpriv/origin: Allow removing local RPMs by name only
This fixes a painful UX issue where one must use the full NEVRA when
uninstalling a locally layered RPM. Now, one can specify either the
NEVRA or the package name only. Though we still try to interpret the
request first as a NEVRA or a capability.

Closes: #1386

Closes: #1400
Approved by: cgwalters
2018-06-12 14:55:40 +00:00
Jonathan Lebon
073978aace app/override: Don't auto-enable cache-only if also installing
Otherwise, we won't be able to fetch the requested packages from the
repo.

Closes: #1384

Closes: #1402
Approved by: cgwalters
2018-06-12 14:52:27 +00:00
Colin Walters
27bd7b97bb Handle "pinned commits" specifically
There are a few scenarios today where one might deliver content
to a machine via an external transport.  For example, take the
scenario of a single server updated via USB drive.  While we
can provide a refspec...what should the remote be?  (This gets
into ostree collections).  There's nothing really that can
happen when typing `rpm-ostree upgrade` unless the USB stick
is plugged in.  That type of scenario should be emphasized
by pinning the commit - the machine is updated via an external
script.

Another case: we're experimenting embedding OSTree commits inside OCI
containers.  Here again since rpm-ostree can't understand how to
pull content from containers, it's saner to drop the refspec
bits, and pin to a commit.

Further enhancements will follow to make the admin experience more
obvious.

Closes: #1396
Approved by: jlebon
2018-06-11 20:37:58 +00:00
Colin Walters
26f04595b2 upgrade: Note if automatic updates are enabled
When e.g. FAH does the automatic update transition, let's remind
people who are doing it manually still.

Closes: #1399
Approved by: jlebon
2018-06-11 18:46:58 +00:00
Jonathan Lebon
34c3d5a02a app/install: Add --allow-inactive switch
For now all this switch does is turn off the scary warning. We also
tweak the warning to make it clear that this will be required in a
future release.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
9085e5144d tests/vmcheck: Fix rojig expected NEVRA for f28
Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
8f71aa95e6 tests/vmcheck: Disable rojig test on CentOS
I was honestly surprised to see this test was running all this time for
CentOS. As cool as it is, this test is really more meant for Fedora.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
09b2ca0482 tests/vmcheck: Fix override remove test
We were trying to test that doing `override remove` on a layered package
errored out. But the test was bogus since `foo` is a base package here.
Let's just use a brand new package that will actually be layered.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
d3d205f7af tests/vmcheck: Split basic layering tests in two
Just part of the continuing breakdown of our testsuites into smaller
pieces for more efficient allocation.

Will add a test case to the `layering-basic-2.sh` in a follow-up commit.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
bff3a54e1e daemon/upgrader: Consistently remove transient state
With the new support for pinning deployments, we need to also update
rpm-ostree to clean up the transient state as is now done in the ostree
sysroot upgrader.

This addresses that issue as well as tries to be a little cleaner in how
we clean up other transient state. Notably, we add a new helper function
to `RpmOstreeOrigin` to do this for us and use it in the upgrader. In
other cases, we do want this transient information since it allows us to
describe the deployment.

Closes: https://github.com/ostreedev/ostree/issues/1595

Closes: #1372
Approved by: cgwalters
2018-05-23 16:27:10 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Colin Walters
99901ac0dc daemon: Add an 'id' param to RegisterClient, log it
This makes the logs a bit more useful, but the ultimate goal
here is to write the originating client `id` to the cached update
data, so users know that e.g. `gnome-software` triggered it.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Colin Walters
8387f1c7c3 Add "ex-stage" update policy, support for ostree staged deployments
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.

I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.

This was all quite straightforward, just plumbing through lots
of layers.

Closes: #1321
Approved by: jlebon
2018-05-14 19:03:56 +00:00
Colin Walters
b03b6a2057 core: Tweak package checkout message, update libglnx
The common case is having layered packages and no overrides; seeing
`Applying 8 overlays` then `Building filesystem [0/8]` is redundant.
Tweak the progress to avoid the double message.  Also change the terminology
to clarify that each item is a package.

Also update libglnx to pull in progress bar newline fix.

Update submodule: libglnx

Closes: #1363
Approved by: jlebon
2018-05-10 01:13:38 +00:00
Jonathan Lebon
c1c78154b1 app/status: Print diff/advisories with pending deployment
Follow-up from #1344.

In the case where a cached update is created from an `upgrade` operation
(and soon, "stage" auto-update policy runs), we can just print the diff
and advisory info together with the pending deployment. This makes the
output look much more natural.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Colin Walters
5037c85330 tests: Extend staging to cover vmcheck/upgrades
This now works after https://github.com/ostreedev/ostree/pull/1570

However, I was hitting an issue with the temp httpd running the
tests serially, let's split them up.

Really the only sane thing is to run each of these tests in a fresh
VM; we'll do the VM-in-container pattern here hopefully soon.

Closes: #1355
Approved by: jlebon
2018-05-02 19:20:26 +00:00
Colin Walters
506910d930 Add an experimental option to use libostree's "staging" API
Now that infrastructure for this has landed in libostree,
let's make it easy for people to opt-in to testing it.  This is a distinct first
step for adding it as an update policy.

Closes: #1352
Approved by: jlebon
2018-05-02 13:46:37 +00:00
Jonathan Lebon
ee458c3c50 daemon/upgrade: Write out new cached update
Right now, cached updates generated during "check" policy runs are
completely decoupled from upgrade operations. This can lead
to the surprising situation where the "Available update" is *older* than
a freshly deployed pending tree with `rpm-ostree upgrade`.

We should just generate a cached update after upgrade operations. This
is also prep for staged deployments, where we'll want to do this as
well.

Note that we write out the cached update here even if automatic updates
are turned off since it's essentially free. I've been thinking about
always displaying that information after an `rpm-ostree upgrade` in
`status`. Though not sure if we should keep it in a separate "Available
update" section, or somehow morph it as part of the pending deployment
output.

Closes: #1344
Approved by: cgwalters
2018-04-26 21:16:52 +00:00
Colin Walters
8726be65e3 core: Finish implementing `override replace ./kernel*.x86_64.rpm
Previously we merged: #1228 AKA 12dc565b00
My recollection is that was working on it the background, while doing
something else, and I clearly didn't get to the point of testing it "for real".

There are many interlocking issues here to make this work.  For example,
the "remove RPM" logic needs special handling for the kernel, because
we also inject content into `/usr/lib/ostree-boot` and also generate
the initramfs, etc.

The architecture I chose is to have the core *detect* when a kernel
is changed, and also call into the kernel processing code when removing
a kernel package.  But the logic for doing kernel reinstallation client-side
is best alongside the initramfs generation logic which already existed
in the sysroot upgrader.

I extended the test suite to cover what was failing before, and I
tested this interactively.  But I'm uncertain about adding a test
for actually *booting* into the GA kernel as it's quite possible
some bits in userspace rely on a newer kernel.  Fixing this properly
really wants some infrastructure to better "re-version" an existing
package without changing its content.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1334

Closes: #1346
Approved by: jlebon
2018-04-26 16:49:53 +00:00
Colin Walters
9df2428a1a tests: Move upgrade/rebase tests from unit to vmcheck/test-upgrades
Now we stop running rpm-ostreed as non-root, which is going to be
a maintenance pain going forward.  If we do introduce non-VM based
tests I think we should look to doing in-container testing.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
a50a6cb959 tests: Move more update-generation bits to libvm.sh
The autoupdate test had a lot of useful infrastructure.  Prep
for removing `tests/check/test-upgrade-rebase.sh`.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
1521dc32c9 status: Show rpmmd-repo data in verbose mode
Building on:

 - 9cbec27d4c
 - e7a42f70a9

I was looking at a rpm-ostree run that imports a variety of rpmmd-repos,
and information about the source repositories is really useful for determining
the up-to-dateness.  We've been capturing this data for a while, it's
about time we started showing it somewhere.

This does make `status --verbose` notably more verbose, but eh, that
seems fine for now.

See also https://github.com/projectatomic/rpm-ostree/issues/774

Closes: #1345
Approved by: jlebon
2018-04-25 14:52:23 +00:00
Colin Walters
e34f17fc3e core: Don't rm-rf directories when doing overrides
It's very normal for base packages to own directories with
dependent packages installing files there.  Doing an rm-rf
for directories was just wrong.  Concretely this fixes
an `override replace ./systemd-*.rpm`.

librpm is also pretty conservative here (for good reason)
and just ignores `ENOTEMPTY`, so let's match that.

I opted to split things up so we remove not-directories
in a first pass, then remove all directories we can in the
second.  This should maximize our chances of removing what we can
in a scenario where e.g. two co-dependent packages install files
to a directory one of them owns.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1340

Closes: #1341
Approved by: jlebon
2018-04-20 16:07:44 +00:00
Colin Walters
1c8c755e81 tests: Split test-basic into misc-{1,2}
Our test suite originated when package layering was still being
developed, but now that that's mature, the logic where layering
tests are distinct makes less sense.

The `basic` test had grown to really be a collection of many
miscellaneous things.  Let's make that more explicit.  Further,
let's avoid having each test suite grow too large; when a single
test fails we don't have an easy way to rerun just that test,
so a crude way to have faster local iteration is to split into groups.

My plan is to reintroduce a `basic` test that covers the basics
of all functionality - update, deploy, layering, etc.  The advanced/corner
cases of layering like the `rm -rf /` test would still live in a
`test-layering.sh` or so.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Colin Walters
73fa64eb12 tests: Move ostree-update creation functions into libvm.sh
So they can be reused outside of just the autoupdate test.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Colin Walters
4f42b94da7 tests: Migrate some basic unit tests that start daemon to vmcheck
The unit tests run an rpm-ostree daemon as non-root, which worked
surprisingly well for quite a while.  But it started failing when
working on a patch which adds caching that writes to `/var`.

Since we have the vmcheck system now, let's switch over to that.
This PR moves the random "basic" tests we'd accumulated like
one to verify `StateRoot` is only in `status --verbose`, but not
the tests for the `rebase` command etc.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Jonathan Lebon
60a59cbbe4 vmcheck/override-replace: Fix status jq test
We should be checking that there are no requested base replacements
here, not removals. This was likely a copy-paste slip-up from copying
the override-remove tests. I still kept those checks for sanity, and
just tacked on the correct ones.

Closes: #1323
Approved by: cgwalters
2018-04-05 15:26:46 +00:00
Jonathan Lebon
bb07a29bc2 overrides: Allow resetting inactive overrides
This fixes a small regression from #852 which prevented inactive
overrides to be reset. Which is funny, because that's exactly the most
likely time when you would want to reset an override.

Basically, the `!is_layered --> no overrides` doesn't make sense for
inactive overrides. I suspect most people worked around this by just
using `reset --all`.

Closes: #1323
Approved by: cgwalters
2018-04-05 15:26:46 +00:00
Colin Walters
defa1dc38a core/scripts: Support /var/lib/rpm-state
I was trying a `--ex-unified-core` compose of FAW, and things fell over
on `urw-base35-fonts` which does a dance of setting a stamp file in
`%post` and checking it in `%posttrans`.

This whole pattern should be considered deprecated by file triggers. But let's
support it for now.

Note there's a lot of parameter passing as we need a single directory which is
held across multiple script invocations.

Closes: #1319
Approved by: jlebon
2018-03-28 18:37:17 +00:00
Colin Walters
779f764fcd app: Call Reload right after getting sysroot proxy
A final followup to our pile of "add reload" PRs.  This one is
necessary to fix the `status` race.

Closes: #1312
Approved by: jlebon
2018-03-26 13:05:40 +00:00
Jonathan Lebon
706506b2d1 app/dbus: Always call Reload() after transaction
Specifically in this case, this allows us to close a race condition
during `upgrade --check` where the `CachedUpdate` property might not
have been updated yet when we read it after finishing the transaction.

Closes: #1311
Approved by: cgwalters
2018-03-23 21:50:33 +00:00
Colin Walters
60aa9a4a33 daemon: Automatically reload sysroot before txn
In this PR: https://github.com/projectatomic/rpm-ostree/pull/1309
I was hitting race conditions running `ostree admin pin` then
`rpm-ostree cleanup` as it was possible that the daemon hadn't handled
the inotify on the sysroot and reloaded the deployment state before
the txn request came in.

Close this race by doing an implicit `reload` before starting a txn.
This is a pretty efficient operation because for the sysroot we're
just doing a `stat()` and comparing mtime.

Implementation wise, change the external API to drop the "did change"
boolean as nothing outside of the `sysroot.c` file used it.

A followup to this would be changing the `status` CLI to call a
(new) DBus API like `RequestReload` that at least did the sysroot
reload if the daemon was otherwise idle or so?  And it'd be available
to unprivileged users.

Closes: #1311
Approved by: cgwalters
2018-03-23 21:50:33 +00:00
Colin Walters
9ddf65aade daemon: Retain pinned deployments
Followup to: https://github.com/ostreedev/ostree/pull/1464

Ideally, we'd delegate more logic around these things to libostree, but we're
not there yet.

This means e.g. `rpm-ostree cleanup -r` for a pinned rollback will just silently
skip it.  It'd be nicer to emit an error probably, but that'd be quite a bit
more work.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1293

Closes: #1309
Approved by: jlebon
2018-03-23 13:25:33 +00:00
Jonathan Lebon
4291500368 daemon: Also generate CachedUpdate in DownloadUpdateRpmDiff
Moving to caching the GVariant to disk rather than during RPMOSTreeOS
reloads broke the legacy `DownloadUpdateRpmDiff` path because it relied
on the implied recalculations that occur on reloads to update
`CachedUpdate`.

This patch series was initially going to be about just migrating all the
legacy APIs to make use of the new metadata, which would have fixed this
properly. But we first need some real coverage in that aread, which is
very poor right now. I'd like to investigate integrating the Cockpit
tests (at least the ostree-specific parts) into our CI to remedy this.

Anyways, for now at least, let's fix Cockpit.

Closes: #1300

Closes: #1303
Approved by: cgwalters
2018-03-17 20:17:32 +00:00
Jonathan Lebon
bc97c3aa53 app/db-list: Use pkglist metadata when possible
This patch teaches `db list` to also use the pkglist metadata when it
can, just like we did for `db diff`. To spell it out: this then allows
`db list` to work on commits for which we only have the commit object.

I went for the surgical incision here and didn't try to support
invocations which use fnmatch patterns for now. Definitely possible,
though it didn't feel like it was worth the effort given that the common
case is just a raw `db list` (I'd wager most people are probably
hard-wired to pipe to `grep` anyway for filtering).

Also fix the usage string, which had the arguments flipped.

Closes: #1299
Approved by: cgwalters
2018-03-15 15:49:44 +00:00
Jonathan Lebon
d6172ddee2 app/db-diff: Add --base switch
Support a `--base` switch to perform the diff on the base layer if
deployment checksum was locally composed. This is useful to filter out
from the diff changes due to layered packages.

Closes: #1299
Approved by: cgwalters
2018-03-15 15:49:44 +00:00
Jonathan Lebon
aaf2c30807 vmcheck: Obey VMCHECK_DEBUG for rpm-ostreed.conf
Closes: #1304
Approved by: cgwalters
2018-03-14 21:49:16 +00:00
Colin Walters
f6d08183b6 daemon: Don't error out if a remote isn't found
This came up in a few places recently; it happens for RHEL in some
cases, and in general we don't want to completely fail the daemon
start if someone messes up their remote config.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1301

Closes: #1302
Approved by: jlebon
2018-03-14 18:50:13 +00:00