Commit Graph

470 Commits

Author SHA1 Message Date
Jonathan Lebon
8c11060f75 daemon/deploy: Fix transaction title on uninstalls
We were labeling pure `uninstall` transactions as `install`. Tease out
that distinction. (Though if a `--install` switch is passed, then we
still just label it as `install` -- there's no difference between that
and `install --uninstall` anyway).

Closes: #1407
Approved by: cgwalters
2018-06-13 20:32:37 +00:00
Colin Walters
2c8a144c1b daemon: Move deploy parsing down into _new_deploy()
Follow to previous change; this should make it easier to add
a new parameter.  It also just makes more sense to centralize
the parsing here.

Closes: #1403
Approved by: jlebon
2018-06-12 20:29:56 +00:00
Colin Walters
826177c2a0 daemon: Push deploy GVariant parsing down a level
Subsequent patches will move it down farther; the high level
goal is to avoid touching many functions to add a parameter.

Closes: #1403
Approved by: jlebon
2018-06-12 20:29:56 +00:00
Colin Walters
27bd7b97bb Handle "pinned commits" specifically
There are a few scenarios today where one might deliver content
to a machine via an external transport.  For example, take the
scenario of a single server updated via USB drive.  While we
can provide a refspec...what should the remote be?  (This gets
into ostree collections).  There's nothing really that can
happen when typing `rpm-ostree upgrade` unless the USB stick
is plugged in.  That type of scenario should be emphasized
by pinning the commit - the machine is updated via an external
script.

Another case: we're experimenting embedding OSTree commits inside OCI
containers.  Here again since rpm-ostree can't understand how to
pull content from containers, it's saner to drop the refspec
bits, and pin to a commit.

Further enhancements will follow to make the admin experience more
obvious.

Closes: #1396
Approved by: jlebon
2018-06-11 20:37:58 +00:00
Jonathan Lebon
eeb5213ddd upgrader: Add missing values to flags GType
We were missing a bunch of flags as part of the property GType. I was
getting a warning that's become fatal because we set
`G_DEBUG=fatal-warnings` for `make vmsync`, which is how I was testing
this:

```
rpm-ostree[1813]: value "((RpmOstreeSysrootUpgraderFlags) 48)" of type 'RpmOstreeSysrootUpgraderFlags' is invalid or out of range for property 'flags' of type 'RpmOstreeSysrootUpgraderFlags'
kernel: traps: pool[1816] trap int3 ip:7f47f4b4b6d5 sp:7f47e989d1f0 error:0 in libglib-2.0.so.0.5600.1[7f47f4af9000+115000]
```

I'm not quite sure why this is happening now though, since I use
`make vmsync` a lot. My guess would be something to do with how we
changed flag passing in d568e54d from "all of them" to "only those we
need", but I didn't verify that.

Closes: #1398
Approved by: cgwalters
2018-06-11 17:50:34 +00:00
Jonathan Lebon
4249dc8c63 libpriv: Directly parse NEVRAs, don't use branches
When I initially added support for local RPM layering (#657), I was
operating under the assumption that there is no reliable way to parse a
NEVRA back to its constituent elements. So we worked around this by
doing lookups in the pkgcache and comparing against branches converted
back to their NEVRAs, which was pretty hacky.

Later on, I learned that you *can* reliably parse a NEVRA back to its
elements because RPM forbids all fields from having a colon `:` and all
fields other than the package name to have a dash `-`. In fact, there is
a helper from `libdnf` to do exactly this.

This patch swaps out the old way of decomposing NEVRAs and converting
NEVRAs to their cache branch notation with a simpler more direct
approach using the helper function from `libdnf`.

Closes: #1395
Approved by: cgwalters
2018-06-08 20:51:30 +00:00
Jonathan Lebon
34c3d5a02a app/install: Add --allow-inactive switch
For now all this switch does is turn off the scary warning. We also
tweak the warning to make it clear that this will be required in a
future release.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
625c17f9b0 daemon/deploy: Re-use base layer rsack
Follow-up to previous commit. Rather than allocating and destroying
RefSacks multiple times, hoist it out of those blocks so it can be
re-used.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
3c12a58b25 daemon/deploy: Warn on immediately inactive requests
This has been a huge point of confusion for users with the new
declarative way of specifying package requests (see #1216, #1287).
Essentially, doing `rpm-ostree install glibc` will work, but it's not
very useful because we don't really expect glibc to ever go away. As a
heuristic, let's work towards forbidding package requests that would
immediately become inactive. For now, we just print a warning.

Note that this still leaves the door open for layered pkgs that e.g.
become inactive during a rebase to a tree which has it baked in, and
then active again when rebasing back; supporting this was the major
advantage of switching to a declarative approach.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
bff3a54e1e daemon/upgrader: Consistently remove transient state
With the new support for pinning deployments, we need to also update
rpm-ostree to clean up the transient state as is now done in the ostree
sysroot upgrader.

This addresses that issue as well as tries to be a little cleaner in how
we clean up other transient state. Notably, we add a new helper function
to `RpmOstreeOrigin` to do this for us and use it in the upgrader. In
other cases, we do want this transient information since it allows us to
describe the deployment.

Closes: https://github.com/ostreedev/ostree/issues/1595

Closes: #1372
Approved by: cgwalters
2018-05-23 16:27:10 +00:00
Colin Walters
99901ac0dc daemon: Add an 'id' param to RegisterClient, log it
This makes the logs a bit more useful, but the ultimate goal
here is to write the originating client `id` to the cached update
data, so users know that e.g. `gnome-software` triggered it.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Colin Walters
7a7d7ad886 daemon: Load sd unit for callers, log it
The high level goal is to render in a better way what caused an
update: https://github.com/projectatomic/rpm-ostree/issues/247#issuecomment-386615707

This gets us for Cockpit:
`Initiated txn DownloadUpdateRpmDiff for client(dbus:1.28 unit:session-6.scope uid:0): /org/projectatomic/rpmostree1/fedora_atomic`
which isn't as good as I'd hoped; I was thinking we'd get `cockpit.service`
but actually Cockpit does invocations as a real login for good reason.

We get a similar result from the CLI.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Colin Walters
8387f1c7c3 Add "ex-stage" update policy, support for ostree staged deployments
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.

I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.

This was all quite straightforward, just plumbing through lots
of layers.

Closes: #1321
Approved by: jlebon
2018-05-14 19:03:56 +00:00
Jonathan Lebon
c1c78154b1 app/status: Print diff/advisories with pending deployment
Follow-up from #1344.

In the case where a cached update is created from an `upgrade` operation
(and soon, "stage" auto-update policy runs), we can just print the diff
and advisory info together with the pending deployment. This makes the
output look much more natural.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Jonathan Lebon
33ea6fbc96 daemon/upgrader: Reuse sack from core for advisories
In #1344, we changed `upgrade` to always write the cached update. One
thing I hadn't noticed was that with pkglayering, loading the sack twice
meant we printed all that gory repo info twice.

Really, the much nicer thing to do is just to make an effort to try to
*keep* the original sack that the core used so that we skip all that
overhead completely.

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Colin Walters
0dfe0de0e0 daemon: Add some error prefixing around reloading
I was hitting an error during sysroot loading while playing
with deployment staging, and it wasn't initially
clear which phase was causing it (a client-initated reload versus
one we do before or after deploying).  Add some error prefixing.

Closes: #1355
Approved by: jlebon
2018-05-02 19:20:26 +00:00
Colin Walters
506910d930 Add an experimental option to use libostree's "staging" API
Now that infrastructure for this has landed in libostree,
let's make it easy for people to opt-in to testing it.  This is a distinct first
step for adding it as an update policy.

Closes: #1352
Approved by: jlebon
2018-05-02 13:46:37 +00:00
Colin Walters
30e29dfc0e syscore: Only do a single repo prune for base+pkgcache
Since we unified the pkgcache repo in 7056e6b726
there's no longer a reason to perform two repository prunes.

Change things so that the first phase is regenerating all of the
refs (in a single libostree txn), then perform a single prune.

This is preparation for reworking how we do prunes, which is
going to be useful for staged deployments.

Closes: #1351
Approved by: jlebon
2018-04-30 19:01:30 +00:00
Jonathan Lebon
ee458c3c50 daemon/upgrade: Write out new cached update
Right now, cached updates generated during "check" policy runs are
completely decoupled from upgrade operations. This can lead
to the surprising situation where the "Available update" is *older* than
a freshly deployed pending tree with `rpm-ostree upgrade`.

We should just generate a cached update after upgrade operations. This
is also prep for staged deployments, where we'll want to do this as
well.

Note that we write out the cached update here even if automatic updates
are turned off since it's essentially free. I've been thinking about
always displaying that information after an `rpm-ostree upgrade` in
`status`. Though not sure if we should keep it in a separate "Available
update" section, or somehow morph it as part of the pending deployment
output.

Closes: #1344
Approved by: cgwalters
2018-04-26 21:16:52 +00:00
Colin Walters
8726be65e3 core: Finish implementing `override replace ./kernel*.x86_64.rpm
Previously we merged: #1228 AKA 12dc565b00
My recollection is that was working on it the background, while doing
something else, and I clearly didn't get to the point of testing it "for real".

There are many interlocking issues here to make this work.  For example,
the "remove RPM" logic needs special handling for the kernel, because
we also inject content into `/usr/lib/ostree-boot` and also generate
the initramfs, etc.

The architecture I chose is to have the core *detect* when a kernel
is changed, and also call into the kernel processing code when removing
a kernel package.  But the logic for doing kernel reinstallation client-side
is best alongside the initramfs generation logic which already existed
in the sysroot upgrader.

I extended the test suite to cover what was failing before, and I
tested this interactively.  But I'm uncertain about adding a test
for actually *booting* into the GA kernel as it's quite possible
some bits in userspace rely on a newer kernel.  Fixing this properly
really wants some infrastructure to better "re-version" an existing
package without changing its content.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1334

Closes: #1346
Approved by: jlebon
2018-04-26 16:49:53 +00:00
Colin Walters
1521dc32c9 status: Show rpmmd-repo data in verbose mode
Building on:

 - 9cbec27d4c
 - e7a42f70a9

I was looking at a rpm-ostree run that imports a variety of rpmmd-repos,
and information about the source repositories is really useful for determining
the up-to-dateness.  We've been capturing this data for a while, it's
about time we started showing it somewhere.

This does make `status --verbose` notably more verbose, but eh, that
seems fine for now.

See also https://github.com/projectatomic/rpm-ostree/issues/774

Closes: #1345
Approved by: jlebon
2018-04-25 14:52:23 +00:00
Colin Walters
0d0b9c85ee daemon: Simplify how we load deployments
I was going to add a `StagedDeployment` property and
found the code here confusing in the way we were walking
the whole deployment list.  Let's use the
`ostree_sysroot_query_deployments_for()` API for all
cases here.  Then the special case is just:
"if no booted deployment, default == pending".

Also change the code style to declare-and-initialize.

Prep for staging deployments.

Closes: #1327
Approved by: jlebon
2018-04-11 21:00:36 +00:00
Jonathan Lebon
bb07a29bc2 overrides: Allow resetting inactive overrides
This fixes a small regression from #852 which prevented inactive
overrides to be reset. Which is funny, because that's exactly the most
likely time when you would want to reset an override.

Basically, the `!is_layered --> no overrides` doesn't make sense for
inactive overrides. I suspect most people worked around this by just
using `reset --all`.

Closes: #1323
Approved by: cgwalters
2018-04-05 15:26:46 +00:00
Jonathan Lebon
28a00f8a13 daemon: Factor out function for retrieving DnfSack
Factor out a helper function to create a `DnfSack` for the booted
deployment. Prep for more work.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
29826564d4 daemon: Create cache dir before writing cached update
If the user never ran e.g. `rpm-ostree install`, then the cache dir will
not even exist yet. Let's make sure it's created before writing the
cached update into it.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
bd2920ad01 daemon/upgrader: Return new deployment in deploy()
Enhance `rpmostree_sysroot_upgrader_deploy` to also return the newly
created deployment on success. Prep for more work.

Closes: #1324
Approved by: cgwalters
2018-04-04 20:48:39 +00:00
Jonathan Lebon
37306e36ee daemon: Add new Reload D-Bus method
Add a new `Reload` method as a softer alternative to `ReloadConfig`.
We'll also make use of this on the client side to sync with the daemon.

Closes: #1311
Approved by: cgwalters
2018-03-23 21:50:33 +00:00
Jonathan Lebon
747236d593 daemon: Rename sysroot_reload to be more descriptive
Follow-up to previous commit. Since we have so many concepts called
"sysroot" and "configs", let's be explicit with our naming here to be
sure we know what we're calling.

Closes: #1311
Approved by: cgwalters
2018-03-23 21:50:33 +00:00
Colin Walters
60aa9a4a33 daemon: Automatically reload sysroot before txn
In this PR: https://github.com/projectatomic/rpm-ostree/pull/1309
I was hitting race conditions running `ostree admin pin` then
`rpm-ostree cleanup` as it was possible that the daemon hadn't handled
the inotify on the sysroot and reloaded the deployment state before
the txn request came in.

Close this race by doing an implicit `reload` before starting a txn.
This is a pretty efficient operation because for the sysroot we're
just doing a `stat()` and comparing mtime.

Implementation wise, change the external API to drop the "did change"
boolean as nothing outside of the `sysroot.c` file used it.

A followup to this would be changing the `status` CLI to call a
(new) DBus API like `RequestReload` that at least did the sysroot
reload if the daemon was otherwise idle or so?  And it'd be available
to unprivileged users.

Closes: #1311
Approved by: cgwalters
2018-03-23 21:50:33 +00:00
Colin Walters
9ddf65aade daemon: Retain pinned deployments
Followup to: https://github.com/ostreedev/ostree/pull/1464

Ideally, we'd delegate more logic around these things to libostree, but we're
not there yet.

This means e.g. `rpm-ostree cleanup -r` for a pinned rollback will just silently
skip it.  It'd be nicer to emit an error probably, but that'd be quite a bit
more work.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1293

Closes: #1309
Approved by: jlebon
2018-03-23 13:25:33 +00:00
Jonathan Lebon
4291500368 daemon: Also generate CachedUpdate in DownloadUpdateRpmDiff
Moving to caching the GVariant to disk rather than during RPMOSTreeOS
reloads broke the legacy `DownloadUpdateRpmDiff` path because it relied
on the implied recalculations that occur on reloads to update
`CachedUpdate`.

This patch series was initially going to be about just migrating all the
legacy APIs to make use of the new metadata, which would have fixed this
properly. But we first need some real coverage in that aread, which is
very poor right now. I'd like to investigate integrating the Cockpit
tests (at least the ostree-specific parts) into our CI to remedy this.

Anyways, for now at least, let's fix Cockpit.

Closes: #1300

Closes: #1303
Approved by: cgwalters
2018-03-17 20:17:32 +00:00
Jonathan Lebon
e4dd2c4d23 daemon: Write to journal on successful transaction
We would write to the journal when we initiated a new transaction as
well as when that transaction errored. This just completes the picture
by also writing when the transaction finished successfully. My ulterior
motive is to make testing easier.

Also make sure to print the method name, which is more telling than the
object path.

Closes: #1303
Approved by: cgwalters
2018-03-17 20:17:32 +00:00
Jonathan Lebon
45fb100826 daemon/deploy: Factor out update variant generation
No functional change. Prep for next patch.

Closes: #1304
Approved by: cgwalters
2018-03-14 21:49:16 +00:00
Colin Walters
f6d08183b6 daemon: Don't error out if a remote isn't found
This came up in a few places recently; it happens for RHEL in some
cases, and in general we don't want to completely fail the daemon
start if someone messes up their remote config.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1301

Closes: #1302
Approved by: jlebon
2018-03-14 18:50:13 +00:00
Colin Walters
b509af739a daemon: Clean up how we load deployment GPG results
Mostly prep for further work.  But it's just nicer anyways to
directly write into the dict.

Closes: #1302
Approved by: jlebon
2018-03-14 18:50:13 +00:00
Colin Walters
60d4470e05 status: Show deployment pinned 📌 state
Implemented in libostree in https://github.com/ostreedev/ostree/pull/1464
Let's display it - wrapping the command will come later.

I also just noticed `rpmostree_syscore_filter_deployments()` at least is
going to have to learn about pinning; will need to improve the test suite
around this too.

Closes: #1292
Approved by: jlebon
2018-03-08 13:43:50 +00:00
Jonathan Lebon
05e09cd72c auto-updates: Cache cached-update GVariant to disk
Rather than recalculating `cached-update` as part of transaction
cleanups and RpmostreedOS internal reloads, write it directly to a file
from `deploy_transaction_execute`. This gives two major benefits:

1. Auto-updates now has virtually zero impact to daemon startup time.
2. We get to directly use the `DnfSack` created during metadata refresh
   rather than reconstructing it later on. This greatly simplifies code.

This makes use of new APIs in libdnf to skip filelists and load
updateinfo metadata right from the start.

Closes: #1268
Approved by: cgwalters
2018-03-05 22:42:26 +00:00
Jonathan Lebon
1b580ce201 libpriv/core: Use new download_metadata libdnf API
Allow callers to pass flags directly to libdnf depending on their needs.
Make use of this in `refresh-md` at least so that we don't even bother
loading the rpmdb since we're just interested in downloading fresh bits.

Closes: #1268
Approved by: cgwalters
2018-03-05 22:42:26 +00:00
Jonathan Lebon
a05a21edd3 daemon/refresh-md: Describe transaction as "refresh-md"
Not sure what I was thinking using "makecache" as the human-friendly
title for this transaction, when the well-known name is "refresh-md".
Let's just make these match.

Closes: #1284
Approved by: cgwalters
2018-03-01 23:29:17 +00:00
Jonathan Lebon
f6fb505cac daemon: Complete output-to-self rename
To reduce confusion in future versions of myself and other readers,
complete the suggested rename from "redirect-output" to "output-to-self"
to internal variable names as well.

Closes: #1283
Approved by: cgwalters
2018-03-01 23:04:11 +00:00
Colin Walters
76919a0c4b rojig-rename: API functions in core/origin, pruning refspec
This renames the API functions in the core and origin, and also fixes up the
fact that we were still looking for `jigdo/` refs in the pruning code.

Closes: #1279
Approved by: jlebon
2018-03-01 22:35:46 +00:00
Jonathan Lebon
83a94ffe95 daemon/os: Check deployment before using it
If one isn't actually booted in a deployment, we'd get `NULL` from
libostree. Check for it before trying to use it. Something something
Rust.

This will all go away soon with #1268, though this quick fix should
allow anyone hitting this to use FAHC RPMs to move forward.

Closes: #1282
Approved by: cgwalters
2018-02-28 17:52:42 +00:00
Jonathan Lebon
ccae8ca977 auto-updates: Redirect output to self if timer
As mentioned during code review, it feels odd to have our automatic
service collect output under its name rather than the main daemon. It
shouldn't be thought of as a daemon with its own logs, it's really just
a trigger.

We change this here so that for automatic updates, output is redirected
to the daemon stdout, which of course goes to the journal. This should
make logs in case of errors more discoverable as well since
`rpm-ostreed` is the well-known name.

I drilled down the notion directly into `RpmostreedTransaction` since I
think it's a useful property that might be handy at a more general
level.

Closes: #1278
Approved by: cgwalters
2018-02-28 14:13:47 +00:00
Jonathan Lebon
80bd997212 daemon/os: Check transaction before connecting signal
Didn't hit this, but clearly we should check if there's even a
transaction to listen to before trying to connect a signal.

Closes: #1278
Approved by: cgwalters
2018-02-28 14:13:47 +00:00
Jonathan Lebon
26f3784972 daemon/transaction: drop emit_message_printf() helper
This was superseded by `rpmostree_output_message`. This is prep for
being able to "silence" transactions and instead have all the output go
to the daemon.

Closes: #1277
Approved by: cgwalters
2018-02-27 21:14:55 +00:00
Colin Walters
b304a433a0 Second renaming pass for "rojig"
We're continuing an incremental renaming process; previously we changed
the most user-visible strings.  Now we're doing some internal variables,
and notably the cached refs and the origin files - the latter set is
things that end up on disk.

This leaves the biggest items; renaming APIs, files, and tests.

Closes: #1276
Approved by: jlebon
2018-02-27 18:45:10 +00:00
Jonathan Lebon
55040a56ad daemon/sysroot: Emit UPDATED signal for configs
Let's also make sure we refresh `RPMOSTreeOS` objects when configs
changed. Specifically, what they export to their D-Bus interfaces may be
dependent on daemon settings, as is the case with auto-updates.

We might want to decouple this into two separate signals in the future
(one for sysroot changes, and one for config changes), though given that
only `RPMOSTreeOS` listens for `UPDATED` right now, we can just get away
with a single one for the time being.

Closes: #1261
Approved by: cgwalters
2018-02-17 21:15:30 +00:00
Jonathan Lebon
df80e15002 auto-updates: Use service type simple
I initially used `oneshot` for this since it felt like a better fit for
non-daemon services. But the only practical difference between the two
is that systemd will wait until we exit before running dependent
follow-up units.

I'd like to change this to `simple` because we can expect users to force
a trigger run using `systemctl start rpm-ostree-automatic`, which will
hang until completion in `oneshot` mode. And given that we're not part
of the boot process and no other unit depends on us, the distinction
makes no semantic difference to us.

Closes: #1261
Approved by: cgwalters
2018-02-17 21:15:30 +00:00
Jonathan Lebon
5d1c043058 daemon/os: Do less work on startup if auto-updates off
While playing with the freshly built v2018.2 Fedora packages on my
Atomic Workstation, I realized that we paid the auto-updates startup
cost even when it was disabled. I've had auto-updates on for so long, I
hadn't seen this pass by me. 🙈

The long-term fix here is caching the update to disk rather than
recalculating it on startup everytime (as suggested during review). Will
work on this as follow-up. Though for now, I'd like to get this smaller
patch in and backported to Fedora so only folks opting in to try it out
experience the >1s startup delay.

Closes: #1261
Approved by: cgwalters
2018-02-17 21:15:30 +00:00
Colin Walters
a0b6d241d2 status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo ♲📦:
https://github.com/projectatomic/rpm-ostree/issues/1081#issuecomment-348540604

Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.

(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.

Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.

Closes: #1240
Approved by: jlebon
2018-02-16 17:02:47 +00:00
Jonathan Lebon
0729487ae5 Check and display pending security advisories
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.

This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.

Closes: #1249
Approved by: cgwalters
2018-02-15 15:30:26 +00:00
Jonathan Lebon
45a3b53558 daemon/api: fix legacy D-Bus API and add coverage
Fix logic to make sure we check if the refspec is of type `ostree://`
even when it's explicitly specified. Also fix `Deploy` in the case where
we didn't just `Download` the RPM diff by adding a new @checksum
parameter to the higher-level API.

Finally, add a basic test for the `GetCached*RpmDiff` APIs so we have at
least *some* coverage. This is also good prep for making sure we don't
break anything when we convert those APIs to use the more efficient
pkglist metadata. The tests completely ignore the `DownloadRpmDiff`
paths for now though.

Closes: #1250

Closes: #1253
Approved by: cgwalters
2018-02-15 15:03:05 +00:00
Colin Walters
528456a59f daemon: Add an origin_is_rojig() helper
Suggested in another PR review, it makes the code nicer.

Closes: #1252
Approved by: jlebon
2018-02-14 21:54:35 +00:00
Colin Walters
799a809c2d Add support for deploy <version> in rojig:// mode
This fleshes out an important piece of the story, showing that
we can support history versioning the same way that we did with
ostree.

Also it's very useful for testing; I'm going to extend the suite after this to
deploy the previous version, clean everything up, then upgrade and verify we
only download changed RPMs.

Closes: #1232
Approved by: jlebon
2018-02-14 21:54:30 +00:00
Jonathan Lebon
57ab9844b9 daemon/autoupdates: Don't insert rpm-diff if empty
Canonicalize to "absent" to make it easier for clients.

Closes: #1248
Approved by: cgwalters
2018-02-14 19:02:02 +00:00
Colin Walters
0a4f6bdab4 jigdo: Use separate rpmostree/jigdo/... refs
The way we import packages in jigdo mode is different from package layering; we
may only import a subset of files for example. In general, we need to treat
jigdo differently.

Related: https://github.com/projectatomic/rpm-ostree/issues/1197

Closes: #1238
Approved by: jlebon
2018-02-13 20:08:13 +00:00
Colin Walters
782c49b748 upgrader: In jigdo mode, don't check rpm-md repos twice for layers
Typing `rpm-ostree upgrade` was quite verbose with layered packages, we'd see
the rpm-md repos twice. The better fix would be to pass the context/sack from
one stage to the other, but this is a quick simple fix to at least reduce
verbosity (and potentially avoid extra network requests).

Closes: #1241
Approved by: jlebon
2018-02-13 18:24:47 +00:00
Jonathan Lebon
92029de009 daemon: fix automatic update service switch
I missed this one when renaming the switch during the review of the
initial code drop.

Closes: #1242
Approved by: cgwalters
2018-02-13 17:30:12 +00:00
Jonathan Lebon
dc63bba78d daemon/dbus: remove reboot mode from comments
This is no longer supported.

Closes: #1242
Approved by: cgwalters
2018-02-13 17:30:12 +00:00
Colin Walters
b881f33ba7 Add support for rebase rojig://
This is an initial drop of support for:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`.  We also
then support `rpm-ostree upgrade` from that.

There's a lot that could be improved here; the test coverage is relatively
minimal. A blocking issue there is having a realistic jigdo setup, and that's
going to require changing how we do testing. For now, this means that if we want
to e.g. change the format we'll have to temporarily disable this test, get the
format change in, update FAHC, then re-enable the test.

Closes: #1166
Approved by: jlebon
2018-02-02 20:18:58 +00:00
Colin Walters
f6dee42496 sysroot: LOG_ERR failure to read state, not g_critical()
If we can't read the system state, that's an *external* problem with e.g. files
most likely, not a situation in which we should abort.

This came up while playing with `rojig://` where we seem to write
the origin file incorrectly.

Closes: #1166
Approved by: jlebon
2018-02-02 20:18:58 +00:00
Jonathan Lebon
553a68c32a libpriv: add new rpmostree-types.h
There are types that we want to share between the daemon and the client
for deduplication. Those are, as expected, related to D-Bus things like
formats and enums. Let's create a new file for it rather than shove it
in `rpmostree-util.h`. As mentioned in the file, some of these probably
belong better directly in the public API.

Closes: #1147
Approved by: cgwalters
2018-01-27 23:52:43 +00:00
Jonathan Lebon
51fb641305 Initial support for automatic updates
This patch introduces a new `AutomaticUpdatePolicy` configuration. This
was a long time coming for rpm-ostree, given that its update model makes
it extremely apt for such a feature.

The config supports a `check` mode, which should be very useful to
Atomic Workstation users, as well as a `reboot` mode, which could be
used in its present form in simple single node Atomic Host situations.

There is still a lot of work to be done, including integrating
advisories, and supporting a `deploy` mode. This feature hopefully will
be leveraged as well by higher-level projects like GNOME Software and
Cockpit.

Closes: #1147
Approved by: cgwalters
2018-01-27 23:52:43 +00:00
Jonathan Lebon
183cb952e3 daemon: print warning only if val couldn't be parsed
We don't want to print a warning if the setting is missing from the
config file. That's totally normal (e.g. the config we ship has all its
configs commented out). We *do* want to print a warning if the config is
provided, but it couldn't be parsed as a proper `uint64`.

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
812daf3902 libpriv/util: add function to get deployment root
Prep for auto-updates.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
688f28c072 daemon: tweak transaction titles for deploy and refresh_md
Prep for auto-updates. Factor out a bunch of flags into nice booleans,
and tweak the transaction title a bit to provide more information about
flags provided.

This will be useful for when the user does a `status` while the daemon
is running due to the automatic updates trigger firing.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Colin Walters
2c270a6644 rebase: Add support for providing ostree:// prefix
This fixes an asymmetry we introduced when rendering using the `ostree://`
prefix; we now support parsing it as well.  We reject the `rojig://` prefix
explicitly for now.

On the implementation side...things are messy. My thought is that we do our best
to canonicalize at the entrypoints for both the client side as well as the
server side. However, for backwards compatibility we can't go all the way to
writing out `ostree://` to disk in the origin files.

I'm actually uncertain if Cockpit will deal with this...need to test that.

Closes: #1210
Approved by: jlebon
2018-01-17 14:19:39 +00:00
Colin Walters
6df154e2e3 daemon: Pass GVariant deployment modifier down another level
Redo the internal API to pass GVariant down another level; this will be more
obviously useful as we move it much farther along through the layers.

Closes: #1170
Approved by: jlebon
2018-01-16 17:16:51 +00:00
Colin Walters
7276bcd1a0 daemon: Add an internal typedef for GVariant modifiers
Since we have a lot of variants, I'm thinking about using
this typedef approach to clarify things a bit more.  We could
even potentially do codegen around this, but let's start with
just a typedef.

Closes: #1170
Approved by: jlebon
2018-01-16 17:16:51 +00:00
Jonathan Lebon
cf1038e804 daemon/config: add new IdleExitTimeout config
Allow users to specify how long the daemon should idle for before
exiting. This was *mostly* an excuse for me to set up a somewhat useful
config file ahead of the auto-update work. Though I think allowing this
config makes sense as well. I like it better than the secret env var for
development purposes. PackageKit also has a similar `ShutdownTimeout`
configuration.

Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
b9bae8fc5b daemon/config: add new rpm-ostreed.conf
Making this a separate commit, since it's a first for rpm-ostree. We now
have a conf file complete with man page! No options yet though.

Interestingly, there was a function called `rpmostreed_reload_config`
which was declared but never defined. Didn't look too much into that.

We make sure that the config is part of the things we reload when users
call `rpm-ostree reload`.

Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
57eb370eb4 daemon/sysroot: minor indentation fix
Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
c80a4168de dbus: document missing key from CachedUpdate
Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Jonathan Lebon
4ad627f69b daemon: bump idle exit timeout to 60s
Since it usually takes more than 10s for users to enter consecutive
commands, let's bump the timeout to 60s. That way, we avoid the churn of
starting up twice and e.g. polluting the journal.

On a user interface level, this doesn't make a big difference: a
`status` from cold takes around 100ms, whereas with the daemon running,
it takes slightly less than 50ms. Slightly noticeable, but a non-issue.

However, auto-update will require some more work at startup, and a cold
`status` will bump to about 350ms, which is definitely more noticeable.
Bumping the timeout will ensure that at least within the span of one
"interaction" (multiple commands), we only do this work once.

Closes: #1192
Approved by: cgwalters
2018-01-10 18:53:50 +00:00
Colin Walters
054b48d55b core,compose: Fix unified core pkgcache labeling
Basically the `rpmostree_context_relabel()` call we had in the treecompose path
for unified core didn't actually have any effect as the core code did a relabel
and unset the array.

I think this may actually be a regression from: https://github.com/projectatomic/rpm-ostree/pull/1137
though I didn't verify.

Anyways looking at this, the code is a lot simpler if we change the API so that
the "normal" relabeling is folded into `rpmostree_context_assemble()`. Then we
change the public relabel API to be "force relabel" which we use in the unified
core 🌐 treecompose path.

This shrinks the jigdoRPM for FAH from 90MB to 68MB.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1172

Closes: #1173
Approved by: jlebon
2018-01-09 16:59:19 +00:00
Jonathan Lebon
def21255ac lib/db: use new pkglist metadata API if possible
Add a function that can smartly perform diff operations on sorted
RpmOstreePackage arrays and make the db API use that. This allows us to
immediately take advantage of the benefits in a few places where diffs
are performed, including post-deployment tree diffs, and the legacy db
diff variant API. The upcoming `CachedUpdate` rework will also make use
of this (but with the notable difference of setting `allow_noent` to
`TRUE`).

Note this introduces a new `rpm_ostree_db_diff_ext` public API which has
the same interface as `rpm_ostree_db_diff` but also takes flags.

Closes: #1162
Approved by: cgwalters
2017-12-30 11:32:38 +00:00
Jonathan Lebon
addc41ad12 libpriv/rpm-util: drop get_pkglist_for_root
This function returned both an RpmOstreeRefsack and a pkglist bound to
the refsack. Interestingly, there were only two users of it, and one of
them didn't even make use of the pkglist functionality. Since the
lifetime semantics of this function are tricky, let's drop it and
introduce a dedicated function just for returning package lists.

I also dropped the `GCancellable` argument, since it isn't/can't easily
be used by those code paths.

Closes: #1158
Approved by: cgwalters
2017-12-20 13:10:36 +00:00
Jonathan Lebon
97133bd028 daemon: avoid using floating GVariant refs
It makes it harder to keep track of ownership except in the trivial
`_new ()` case for which they were designed (i.e. shoving straight into
another glib GVariant function that takes ownership of it right away).

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
608ab302e8 daemon/sysroot: reload repo before deployments
In the case of a rebase to a fresh remote, we want to make sure our
`OstreeRepo` picks up the new remote before we try to parse its
deployment. Otherwise, we'll error out when trying to fetch configs for
it.

This was picked up after turning on strict errors in the GPG
verification process.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
5fb153601a daemon/deployment-utils: some new style porting
Prep for more work.

We actually pass a `GError` to the gpg results function and fail for any
other failures than gpg signature verification. (In that case, we want
to make sure that information gets to the D-Bus API level, so killing
the daemon would be the wrong choice).

I also factored out all the logic in
`rpmostreed_commit_generate_cached_details_variant` into a separate
function. This is prep for reusing it elsewhere.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
9ad0829330 daemon/os: reorder iface functions
This is minor, though the list of methods we support is getting long and
will get longer. Let's just add some order to it to make it easier for
humans to parse.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
8e1a769734 daemon/package-variant: fix off-by-one error
We're definitely interested in the rpmdb diff even if there's just one
package changed.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
5018583637 daemon/package-variant: some more porting to new style
Prep for more work.

This also involved changing the functions from pointer return values to
`gboolean`.

I also made functions that didn't really need to be public become
private and fixed some indentation.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
5ff69e10db daemon/os: check for download-only and dry-run conflict
Minor follow-up to previous commit. It seems a bit confusing to allow
specifying `download-only` and `dry-run`. The former already includes
all the steps in the latter but goes further, as documented. Let's check
for this combination.

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Jonathan Lebon
183399e1cd daemon/deploy: fix --dry-run downloads packages
Minor regression that crept in during the `--download-only` work. We
would download and import packages even when `--dry-run` was given.
Make sure we stop right after printing the transaction.

Related: #1128

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Jonathan Lebon
7f0737e7fc daemon/os: fix complete function for SetInitramfsState()
We were calling the wrong completer function for `SetInitramfsState()`.
Not that it mattered much in compiled form since both ways ended up
calling the same internal gdbus function with the same arguments.

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Jonathan Lebon
d5e92df5c1 rpm-ostreed.service: tweak description and add doc
This is highly dependent of the outcome of [1], though until that's
settled there, let's at least update the description to something a
little more apt. It feels more appropriate to consider rpm-ostree as a
"system manager" than just a "package manager" (which it certainly is
too of course). Also use Title Case convention which seems more popular
overall and looks nicer.

[1] https://github.com/projectatomic/rpm-ostree/issues/405

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Colin Walters
dac5ccc76e Improve progress output
This rolls up several libglnx changes: https://github.com/GNOME/libglnx/pull/101

Now of course things are trickier here because we have an internal
abstraction over directly emitting to a console versus sending the
result over DBus.  Further complicating things is that some things
call into libdnf and thus *require* use of `DnfState` which does
not give us the "n items" information, versus other parts which
we implement and can do what we want.

Even *further* complicating things is that we have to take care around non-CLI
callers like Cockpit; so I didn't try to pass the "n items" over DBus, rather
just reimplemented the "insert into text" that libglnx is doing.

Anyways overall this looks better IMO for all cases.

Update submodule: libglnx

Closes: #1143
Approved by: jlebon
2017-12-14 03:32:22 +00:00
Colin Walters
a16422484e txn: Let first error win vs cancellation
Basically since we're doing internal async ops which set the cancellable on
failure, we still want the first error to win since it'll be more useful. See
the docs for `g_task_set_check_cancellable()` for more.

Closes: #1137
Approved by: jlebon
2017-12-12 19:21:07 +00:00
Colin Walters
88cb736404 daemon/sysroot: Drop internal cancellable
I believe this is a leftover vestige, and it was adding confusion when I was
debugging `rpmostree-core.c` async ops and cancellation.

Now the only cancellables in the daemon are created by transaction ops.

Closes: #1137
Approved by: jlebon
2017-12-12 19:21:07 +00:00
Colin Walters
5f03181c0a core: c++ build: Use enum-NONE value instead of zero
In the whole libdnf/C++ discussion I experimented with trying to build
rpm-ostree as C++. There's a whole ton of stuff there. I'm going to punt for
now, but let's land this one change so some progress was made.

Closes: #1141
Approved by: jlebon
2017-12-12 17:26:19 +00:00
Colin Walters
95b423afe9 Add cancel verb and DBus API
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).

Closes: #1019
Approved by: jlebon
2017-12-07 20:12:52 +00:00
Colin Walters
1c0e354571 importer: Rework API
Now that the importer *only* imports into OSTree repos, let's
clean up the API so that the `OstreeRepo` and `OstreeSePolicy`
are passed as constructor args.

Also rework things so there's only one constructor API that
steals the fd.

This is prep for adding another async import API.

Closes: #1124
Approved by: jlebon
2017-12-07 19:44:19 +00:00
Colin Walters
f8bc9a1c9d compose/dracut: Use a host tmpdir for dracut
In unified core mode, this avoids an intense spam of errors from `cp`
because `tmpfs` doesn't support the `user.` xattr namespace, and
since [this dracut commit](61c761bc2c)
dracut tries to copy all xattrs, which was just done for IMA.
There's no point to having the SELinux labels or other xattrs
in the initramfs.

The real fix here is dracut should learn to *only* copy the IMA
xattrs, or even better disable IMA enforcement for the dracut
run or something.

Closes: #1126
Approved by: jlebon
2017-12-04 15:37:16 +00:00
Jonathan Lebon
7056e6b726 daemon: Unify pkgcache with system repo
We originally needed the pkgcache to be a separate repo due to ostree's
overzealous pruning policies. The idea was to maintain multiple commits
in each pkg branch for different SELinux policies. In practice, there's
not much use in maintaining old copies and it's just easier to always
relabel on the fly. So then, the need for a separate repo completely
melts away.

This helps simplify the mental model a bit and allows us to avoid subtle
issues like #1047. Note however that the core is still capable of
handling split repos for the `--ex-unified-core` compose use case. Once
that and the jigdo work are a bit more settled, we can have a clearer
picture of how to simplify the core further.

The tricky bit is migrating the cache. When deploying, we check if a
pkgcache repo exists and migrate its refs if so. We then leave behind a
symlink to the system repo to remain compatible with older rpm-ostrees.

Closes: #1055
Approved by: cgwalters
2017-12-01 15:41:40 +00:00
Jonathan Lebon
4d72a73b80 daemon/sysroot: Port to new style
No functional changes.
Prep for unified repo work.

Closes: #1055
Approved by: cgwalters
2017-12-01 15:41:40 +00:00
Jonathan Lebon
5860897092 tree: account for ephemeral nevra string
When writing this code, I made the false assumption that the nevra
string lives as long as the pool does, i.e. as long as we have a
reference to its `DnfSack`.

In fact, they have undefined lifetimes. Notably any place in which one
calls `dnf_package_get_nevra` a lot may result in the invalidation of
previously returned nevras.

This patch ensures that we copy the string in the few places where we
are susceptible to this.

There is a related libdnf patch[1] which tightens the definition here so
that we can assume the string at least lives as long as its
`DnfPackage`. It turns out that the callsites addressed in this patch
are also those in which we would break that assumption. IOW, this patch
is needed regardless of how [1] goes.

[1] https://github.com/rpm-software-management/libdnf/pull/388

Closes: #1119
Approved by: cgwalters
2017-11-29 20:05:39 +00:00
Colin Walters
9c004e1833 Change unpacking to use a single ostree txn
Right now each ostree txn incurs a `syncfs()`; see
https://github.com/ostreedev/ostree/issues/1184
And before this patch, we were doing a txn per package import.

We can really do better in libostree - we'll fix that, but in the short term
let's use a bigger txn for every package. However, the obvious change here of
simply hoisting up the txn is that on failure for imports, we'd discard all
downloaded packages.  We fix that by changing the auto txn API to have
a `commit_on_failure` boolean, and use it in cases where we're doing
imports.

This is prep work for jigdo, where we'll be using the import path all the time.
My bigger plan is to do multithreaded imports.

Closes: #1116
Approved by: jlebon
2017-11-29 16:48:19 +00:00
Colin Walters
f58cb44fb2 txn: For --download-only, print 'changed' message if changed
I was playing with `--download-only` a bit with an eye to
having something like this be used by Cockpit/gnome-software instead
of what it's doing now, but a problem is that at the moment we
don't have a way to reflect the "changed" state back to clients.

This is a first step towards that by simply printing a different
message.

I think really to make all of this work more nicely though, including
supporting e.g. rpm database diffs, we are going to have to instead
work on the [pending deployment](https://github.com/ostreedev/ostree/issues/545)
path.  That way we'll have done the depsolve, stored repo timestamps
etc.; we'll be able to accurately show what *did* change rather than
try to recreate what will happen on the next `rpm-ostree upgrade --cache-only`.

Closes: #1118
Approved by: jlebon
2017-11-29 15:29:49 +00:00
Colin Walters
366826132a libpriv: Rename "unpacker" to "importer", split off "unpacker-core"
In the jigdo path we don't actually want to import the OIRPM literally
into ostree.  I considered adding jigdo logic into `rpmostree-unpacker.c`
but it'd be a mess as the functionality is quite logically separate
from importing.

So split off an `unpacker-core.c` file which has the bare libarchive+RPM
helpers, and rename `RpmOstreeUnpacker` to `RpmOstreeImporter`.

Closes: #1110
Approved by: jlebon
2017-11-20 20:57:33 +00:00