Commit Graph

2154 Commits

Author SHA1 Message Date
Jonathan Lebon
bff3a54e1e daemon/upgrader: Consistently remove transient state
With the new support for pinning deployments, we need to also update
rpm-ostree to clean up the transient state as is now done in the ostree
sysroot upgrader.

This addresses that issue as well as tries to be a little cleaner in how
we clean up other transient state. Notably, we add a new helper function
to `RpmOstreeOrigin` to do this for us and use it in the upgrader. In
other cases, we do want this transient information since it allows us to
describe the deployment.

Closes: https://github.com/ostreedev/ostree/issues/1595

Closes: #1372
Approved by: cgwalters
2018-05-23 16:27:10 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Colin Walters
23129a15b7 compose: Add a missing return in error path
Was hitting a `SEGV` in a build that *might* be this.

Closes: #1371
Approved by: jlebon
2018-05-23 13:24:44 +00:00
Colin Walters
99901ac0dc daemon: Add an 'id' param to RegisterClient, log it
This makes the logs a bit more useful, but the ultimate goal
here is to write the originating client `id` to the cached update
data, so users know that e.g. `gnome-software` triggered it.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Colin Walters
7a7d7ad886 daemon: Load sd unit for callers, log it
The high level goal is to render in a better way what caused an
update: https://github.com/projectatomic/rpm-ostree/issues/247#issuecomment-386615707

This gets us for Cockpit:
`Initiated txn DownloadUpdateRpmDiff for client(dbus:1.28 unit:session-6.scope uid:0): /org/projectatomic/rpmostree1/fedora_atomic`
which isn't as good as I'd hoped; I was thinking we'd get `cockpit.service`
but actually Cockpit does invocations as a real login for good reason.

We get a similar result from the CLI.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Jonathan Lebon
377b1674da ci: Drop f27-sanity testsuite
This started out as an experiment to see how useful it'd be in
rpm-ostree. Again, the goal wasn't to leverage the rpm-ostree-specific
tests of a-h-t here, but rather running the *other* tests to ensure that
we weren't inadvertedly impacting userspace. In the end, as far as I
remember, it hasn't brought up anything since adding it here, so let's
bring the experiment to an end.

It might still be worth doing this same experiment in the ostree CI
instead, though I don't want to add more load there until we fully
switch over to the new CI infra.

Closes: #1369
Approved by: cgwalters
2018-05-16 15:19:13 +00:00
Jonathan Lebon
bb7b489d17 Release 2018.5
Tree seems sane, tests are passing, and no major outstanding issues.
Let's get all those new goodies out!

Closes: #1367
Approved by: cgwalters
2018-05-15 13:07:37 +00:00
Colin Walters
e57504a88a ci: Split codestyle checks into separate context
Prep for reworking the primary test to do vm-in-container, which
will temporarily be vm-in-container-in-vm.

See https://github.com/projectatomic/rpm-ostree/pull/1362

Closes: #1364
Approved by: jlebon
2018-05-14 19:50:36 +00:00
Colin Walters
8387f1c7c3 Add "ex-stage" update policy, support for ostree staged deployments
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.

I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.

This was all quite straightforward, just plumbing through lots
of layers.

Closes: #1321
Approved by: jlebon
2018-05-14 19:03:56 +00:00
Jonathan Lebon
38b11d34e5 ci: Use continuous branch for now
We are hitting issues with a-h-t on CAHC right now failing due to an
`atomic scan` issue and preventing promotion of continuous content to
the smoketested branch. This is then blocking our CI which needs a newer
tree with the latest glib2 baked in. Let's unblock our CI for now by
rebasing on the continuous branch. We'll drop that hack once the a-h-t
issues are ironed out.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Jonathan Lebon
169fab6f4f ci: Drop BDB1539 hack
This shouldn't be needed anymore.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Colin Walters
b03b6a2057 core: Tweak package checkout message, update libglnx
The common case is having layered packages and no overrides; seeing
`Applying 8 overlays` then `Building filesystem [0/8]` is redundant.
Tweak the progress to avoid the double message.  Also change the terminology
to clarify that each item is a package.

Also update libglnx to pull in progress bar newline fix.

Update submodule: libglnx

Closes: #1363
Approved by: jlebon
2018-05-10 01:13:38 +00:00
Jonathan Lebon
6fbf405be0 libglnx: Bump for copy_file_range fix
Fixes compilation on f28. Also prep for #1358.

Update submodule: libglnx

Closes: #1361
Approved by: cgwalters
2018-05-04 20:18:12 +00:00
Jonathan Lebon
beee865ed2 app/status: Add --advisories flag
The `status --verbose` output is getting quite long now that we print
repo information and diffs. Let's add a switch for only making
advisories more verbose so that it's easy to get more information
without having to use `-v` and scroll up if the diff is massive.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Jonathan Lebon
c1c78154b1 app/status: Print diff/advisories with pending deployment
Follow-up from #1344.

In the case where a cached update is created from an `upgrade` operation
(and soon, "stage" auto-update policy runs), we can just print the diff
and advisory info together with the pending deployment. This makes the
output look much more natural.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Jonathan Lebon
33ea6fbc96 daemon/upgrader: Reuse sack from core for advisories
In #1344, we changed `upgrade` to always write the cached update. One
thing I hadn't noticed was that with pkglayering, loading the sack twice
meant we printed all that gory repo info twice.

Really, the much nicer thing to do is just to make an effort to try to
*keep* the original sack that the core used so that we skip all that
overhead completely.

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Jonathan Lebon
ab5452bc3a libpriv/core: Always load updateinfo when preparing
It just makes the sack more useful for other purposes this way. Callers
can just use `download_metadata` directly if they'd like to override
this. Prep for a follow-up patch.

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Jonathan Lebon
974a9f6f71 app/dbus-helpers: Factor out diff/advisory printing
Prep for printing this info as part of the deployments. I also snuck in
a small fix so that we don't print the `Diff:` line if there's no diff
(e.g. it's a no-op update).

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Jonathan Lebon
a7adf82c95 app/status: Drop unused argument
Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Colin Walters
f39eeee671 db: Add some error prefixing around rpmdb loading
In https://github.com/cockpit-project/cockpit/pull/9090#issuecomment-386564044
we're seeing:
`Txn DownloadUpdateRpmDiff ... failed: open(O_TMPFILE): No such file or directory`

Looking at the error paths in the rpmdb diff code, there's not really
any error prefixing.  Let's add some so this is easier to debug.

Closes: #1356
Approved by: jlebon
2018-05-04 13:02:19 +00:00
Colin Walters
5037c85330 tests: Extend staging to cover vmcheck/upgrades
This now works after https://github.com/ostreedev/ostree/pull/1570

However, I was hitting an issue with the temp httpd running the
tests serially, let's split them up.

Really the only sane thing is to run each of these tests in a fresh
VM; we'll do the VM-in-container pattern here hopefully soon.

Closes: #1355
Approved by: jlebon
2018-05-02 19:20:26 +00:00
Colin Walters
0dfe0de0e0 daemon: Add some error prefixing around reloading
I was hitting an error during sysroot loading while playing
with deployment staging, and it wasn't initially
clear which phase was causing it (a client-initated reload versus
one we do before or after deploying).  Add some error prefixing.

Closes: #1355
Approved by: jlebon
2018-05-02 19:20:26 +00:00
Colin Walters
506910d930 Add an experimental option to use libostree's "staging" API
Now that infrastructure for this has landed in libostree,
let's make it easy for people to opt-in to testing it.  This is a distinct first
step for adding it as an update policy.

Closes: #1352
Approved by: jlebon
2018-05-02 13:46:37 +00:00
Micah Abbott
b12899a7f2 man/rpm-ostree: --check always updates RPM metadata
Add a short note explaining that the RPM metadata is always refreshed
when using the `--check` flag.

Closes: #1353
Approved by: jlebon
2018-05-01 14:25:08 +00:00
Colin Walters
30e29dfc0e syscore: Only do a single repo prune for base+pkgcache
Since we unified the pkgcache repo in 7056e6b726
there's no longer a reason to perform two repository prunes.

Change things so that the first phase is regenerating all of the
refs (in a single libostree txn), then perform a single prune.

This is preparation for reworking how we do prunes, which is
going to be useful for staged deployments.

Closes: #1351
Approved by: jlebon
2018-04-30 19:01:30 +00:00
Jonathan Lebon
ee458c3c50 daemon/upgrade: Write out new cached update
Right now, cached updates generated during "check" policy runs are
completely decoupled from upgrade operations. This can lead
to the surprising situation where the "Available update" is *older* than
a freshly deployed pending tree with `rpm-ostree upgrade`.

We should just generate a cached update after upgrade operations. This
is also prep for staged deployments, where we'll want to do this as
well.

Note that we write out the cached update here even if automatic updates
are turned off since it's essentially free. I've been thinking about
always displaying that information after an `rpm-ostree upgrade` in
`status`. Though not sure if we should keep it in a separate "Available
update" section, or somehow morph it as part of the pending deployment
output.

Closes: #1344
Approved by: cgwalters
2018-04-26 21:16:52 +00:00
Colin Walters
8726be65e3 core: Finish implementing `override replace ./kernel*.x86_64.rpm
Previously we merged: #1228 AKA 12dc565b00
My recollection is that was working on it the background, while doing
something else, and I clearly didn't get to the point of testing it "for real".

There are many interlocking issues here to make this work.  For example,
the "remove RPM" logic needs special handling for the kernel, because
we also inject content into `/usr/lib/ostree-boot` and also generate
the initramfs, etc.

The architecture I chose is to have the core *detect* when a kernel
is changed, and also call into the kernel processing code when removing
a kernel package.  But the logic for doing kernel reinstallation client-side
is best alongside the initramfs generation logic which already existed
in the sysroot upgrader.

I extended the test suite to cover what was failing before, and I
tested this interactively.  But I'm uncertain about adding a test
for actually *booting* into the GA kernel as it's quite possible
some bits in userspace rely on a newer kernel.  Fixing this properly
really wants some infrastructure to better "re-version" an existing
package without changing its content.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1334

Closes: #1346
Approved by: jlebon
2018-04-26 16:49:53 +00:00
Colin Walters
9df2428a1a tests: Move upgrade/rebase tests from unit to vmcheck/test-upgrades
Now we stop running rpm-ostreed as non-root, which is going to be
a maintenance pain going forward.  If we do introduce non-VM based
tests I think we should look to doing in-container testing.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
a50a6cb959 tests: Move more update-generation bits to libvm.sh
The autoupdate test had a lot of useful infrastructure.  Prep
for removing `tests/check/test-upgrade-rebase.sh`.

Closes: #1339
Approved by: cgwalters
2018-04-25 20:28:56 +00:00
Colin Walters
39186e3330 docs/compose: Mention caching, privileged containers, example code
And also drop the `-z2` I saw show up in another project.

Closes: #1347
Approved by: jlebon
2018-04-25 18:43:36 +00:00
Colin Walters
1521dc32c9 status: Show rpmmd-repo data in verbose mode
Building on:

 - 9cbec27d4c
 - e7a42f70a9

I was looking at a rpm-ostree run that imports a variety of rpmmd-repos,
and information about the source repositories is really useful for determining
the up-to-dateness.  We've been capturing this data for a while, it's
about time we started showing it somewhere.

This does make `status --verbose` notably more verbose, but eh, that
seems fine for now.

See also https://github.com/projectatomic/rpm-ostree/issues/774

Closes: #1345
Approved by: jlebon
2018-04-25 14:52:23 +00:00
Colin Walters
e34f17fc3e core: Don't rm-rf directories when doing overrides
It's very normal for base packages to own directories with
dependent packages installing files there.  Doing an rm-rf
for directories was just wrong.  Concretely this fixes
an `override replace ./systemd-*.rpm`.

librpm is also pretty conservative here (for good reason)
and just ignores `ENOTEMPTY`, so let's match that.

I opted to split things up so we remove not-directories
in a first pass, then remove all directories we can in the
second.  This should maximize our chances of removing what we can
in a scenario where e.g. two co-dependent packages install files
to a directory one of them owns.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1340

Closes: #1341
Approved by: jlebon
2018-04-20 16:07:44 +00:00
Colin Walters
4c3d9cdbce tests: Split unit test-basic.sh into introspection + upgrade-rebase
Following up to: https://github.com/projectatomic/rpm-ostree/pull/1336

It makes sense to keep the library tests as unit tests (although
we should also support doing them installed).

The upgrade-rebase tests will move into vmcheck/ soon.

Closes: #1338
Approved by: jlebon
2018-04-17 13:08:24 +00:00
Colin Walters
1c8c755e81 tests: Split test-basic into misc-{1,2}
Our test suite originated when package layering was still being
developed, but now that that's mature, the logic where layering
tests are distinct makes less sense.

The `basic` test had grown to really be a collection of many
miscellaneous things.  Let's make that more explicit.  Further,
let's avoid having each test suite grow too large; when a single
test fails we don't have an easy way to rerun just that test,
so a crude way to have faster local iteration is to split into groups.

My plan is to reintroduce a `basic` test that covers the basics
of all functionality - update, deploy, layering, etc.  The advanced/corner
cases of layering like the `rm -rf /` test would still live in a
`test-layering.sh` or so.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Colin Walters
73fa64eb12 tests: Move ostree-update creation functions into libvm.sh
So they can be reused outside of just the autoupdate test.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Colin Walters
4f42b94da7 tests: Migrate some basic unit tests that start daemon to vmcheck
The unit tests run an rpm-ostree daemon as non-root, which worked
surprisingly well for quite a while.  But it started failing when
working on a patch which adds caching that writes to `/var`.

Since we have the vmcheck system now, let's switch over to that.
This PR moves the random "basic" tests we'd accumulated like
one to verify `StateRoot` is only in `status --verbose`, but not
the tests for the `rebase` command etc.

Closes: #1336
Approved by: jlebon
2018-04-16 17:53:20 +00:00
Matthew Leeds
b11d1fc1e3 README.md: Fix youtube link
Closes: #1335
Approved by: cgwalters
2018-04-14 15:24:54 +00:00
Colin Walters
0d0b9c85ee daemon: Simplify how we load deployments
I was going to add a `StagedDeployment` property and
found the code here confusing in the way we were walking
the whole deployment list.  Let's use the
`ostree_sysroot_query_deployments_for()` API for all
cases here.  Then the special case is just:
"if no booted deployment, default == pending".

Also change the code style to declare-and-initialize.

Prep for staging deployments.

Closes: #1327
Approved by: jlebon
2018-04-11 21:00:36 +00:00
Colin Walters
2b624acff0 core,scripts: Move creation of /var/tmp into core
In prep for cleaning up how we manage mounts.  If we're looking
at a real existing `/var`, then it must have the directory.  The
only case where we don't is in pkglayering, so move the special
case there.

Prep for https://github.com/projectatomic/rpm-ostree/issues/1329

Closes: #1333
Approved by: jlebon
2018-04-11 13:09:17 +00:00
Colin Walters
abcaf42d55 compose: Add environment variable to skip tmpdir cleanup
This was useful for me to directly invoke bwrap against the target
rootfs.

Part of working on: https://github.com/projectatomic/rpm-ostree/issues/1329

Closes: #1333
Approved by: jlebon
2018-04-11 13:09:17 +00:00
Colin Walters
e38c64c330 bwrap: Split out a "base" constructor
The bwrap code has some "opinionated" setup around e.g. /etc and
such that I'd like to centralize even more.  However, the dracut
case of taking the host's `/etc` is the unusual standout.  Let's
split out a base constructor.

Prep for https://github.com/projectatomic/rpm-ostree/issues/1329

Closes: #1333
Approved by: jlebon
2018-04-11 13:09:16 +00:00
Colin Walters
4be214a978 bin/status: Factor out deployment printing into helper function
Exactly like: https://github.com/ostreedev/ostree/pull/1504

Prep for staged deployments, where it's a new deployment that isn't
in the deployment array (since it's not in the bootloader order).

Closes: #1328
Approved by: jlebon
2018-04-09 18:48:50 +00:00
Jonathan Lebon
60a59cbbe4 vmcheck/override-replace: Fix status jq test
We should be checking that there are no requested base replacements
here, not removals. This was likely a copy-paste slip-up from copying
the override-remove tests. I still kept those checks for sanity, and
just tacked on the correct ones.

Closes: #1323
Approved by: cgwalters
2018-04-05 15:26:46 +00:00
Jonathan Lebon
bb07a29bc2 overrides: Allow resetting inactive overrides
This fixes a small regression from #852 which prevented inactive
overrides to be reset. Which is funny, because that's exactly the most
likely time when you would want to reset an override.

Basically, the `!is_layered --> no overrides` doesn't make sense for
inactive overrides. I suspect most people worked around this by just
using `reset --all`.

Closes: #1323
Approved by: cgwalters
2018-04-05 15:26:46 +00:00
Jonathan Lebon
cb99fb44b2 buildutils: Add libglnx.m4 to .gitignore
Not sure why this one isn't being auto-generated.

Closes: #1323
Approved by: cgwalters
2018-04-05 15:26:46 +00:00
Colin Walters
d25588722b ci: Ensure we exclude ostree from base fedora too
rdgo uses an earlier git tag due to branches, until we have proper
repo priorities that don't care about versions, let's do this
hack.

Same thing as https://pagure.io/fedora-atomic-host-continuous/c/a7145410fbc7e73188fde449628fa0343b05e4a4?branch=master

Closes: #1326
Approved by: jlebon
2018-04-05 15:26:23 +00:00
Jonathan Lebon
28a00f8a13 daemon: Factor out function for retrieving DnfSack
Factor out a helper function to create a `DnfSack` for the booted
deployment. Prep for more work.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
29826564d4 daemon: Create cache dir before writing cached update
If the user never ran e.g. `rpm-ostree install`, then the cache dir will
not even exist yet. Let's make sure it's created before writing the
cached update into it.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
3a1fd2c139 tests/libvm: Support SSH_CONFIG env var
When juggling multiple test VMs for different purposes, it's useful to
be able to easily e.g. `make vmcheck` a specific one by overriding the
ssh-config file to use, rather than editing the latter each time.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
bd2920ad01 daemon/upgrader: Return new deployment in deploy()
Enhance `rpmostree_sysroot_upgrader_deploy` to also return the newly
created deployment on success. Prep for more work.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00