Commit Graph

2500 Commits

Author SHA1 Message Date
Jonathan Lebon
ccae8ca977 auto-updates: Redirect output to self if timer
As mentioned during code review, it feels odd to have our automatic
service collect output under its name rather than the main daemon. It
shouldn't be thought of as a daemon with its own logs, it's really just
a trigger.

We change this here so that for automatic updates, output is redirected
to the daemon stdout, which of course goes to the journal. This should
make logs in case of errors more discoverable as well since
`rpm-ostreed` is the well-known name.

I drilled down the notion directly into `RpmostreedTransaction` since I
think it's a useful property that might be handy at a more general
level.

Closes: #1278
Approved by: cgwalters
2018-02-28 14:13:47 +00:00
Jonathan Lebon
f0910c2b5c libpriv/output: Also quiet down progress if not tty
Teach the RpmOstreeOutput default handler to be nice to non-TTYs as
well, just like we did in #1225. We should look at somehow unifying this
with the code in `rpmostree-dbus-helpers.c`.

Closes: #1278
Approved by: cgwalters
2018-02-28 14:13:47 +00:00
Jonathan Lebon
80bd997212 daemon/os: Check transaction before connecting signal
Didn't hit this, but clearly we should check if there's even a
transaction to listen to before trying to connect a signal.

Closes: #1278
Approved by: cgwalters
2018-02-28 14:13:47 +00:00
Jonathan Lebon
26f3784972 daemon/transaction: drop emit_message_printf() helper
This was superseded by `rpmostree_output_message`. This is prep for
being able to "silence" transactions and instead have all the output go
to the daemon.

Closes: #1277
Approved by: cgwalters
2018-02-27 21:14:55 +00:00
Colin Walters
aba3f444d6 scripts: Support RPMOSTREE_SCRIPT_DEBUG
This makes it easy to get an interactive shell instead of executing a specific
script. I used this while debugging `build-locale-archive`. This is better than
`bwrap-script-shell.sh` as we'll get the real rofiles-fuse etc., and
additionally have the root filesystem at the exact same point.

Closes: #1275
Approved by: jlebon
2018-02-27 19:16:18 +00:00
Colin Walters
b304a433a0 Second renaming pass for "rojig"
We're continuing an incremental renaming process; previously we changed
the most user-visible strings.  Now we're doing some internal variables,
and notably the cached refs and the origin files - the latter set is
things that end up on disk.

This leaves the biggest items; renaming APIs, files, and tests.

Closes: #1276
Approved by: jlebon
2018-02-27 18:45:10 +00:00
Jonathan Lebon
829a746821 lib/package: Handle arch pkg transitions
Let's try to match expectations a bit from the dnf/yum world and
describe e.g. a `noarch` package become archful as an upgrade rather
than a removal and an addition. This was originally implicitly supported
(before PR #1230) by the fact that we didn't compare arches at all (and
in fact, arches don't even show up in a `db diff` output for modified
packages).

We bring this back here, but only in the simple case that it's a single
package. We still don't try to do any fancy handling for packages of the
same name.

Closes: #1272

Closes: #1274
Approved by: cgwalters
2018-02-27 15:44:11 +00:00
Jonathan Lebon
64968436e6 lib/package: Re-order arch comparison branches
Making this a separate commit to increase the odds we get this right.
Prep for next commit.

Closes: #1274
Approved by: cgwalters
2018-02-27 15:44:11 +00:00
Jonathan Lebon
56c192f5cc lib/package: Fix logic error in diff algorithm
Classic indentation/bracing error causing logic error. This made us skip
over some packages which then showed up wrongly as removed. This will be
implicitly tested in a new test case added in a following commit.

Closes: #1274
Approved by: cgwalters
2018-02-27 15:44:11 +00:00
Jonathan Lebon
0874152ba6 vmcheck: Drop selinux-policy hack
This shouldn't be necessary anymore.

Closes: #1274
Approved by: cgwalters
2018-02-27 15:44:11 +00:00
Micah Abbott
ad8e9b6de8 man: add example usage for automatic update policy
The man page was a little weak on how to configure and enable the
automatic update policy of `rpm-ostreed`.  I've added an `EXAMPLE`
section that shows the necessary steps to get this working.

Closes: #1273
Approved by: jlebon
2018-02-26 20:21:43 +00:00
Colin Walters
3dd132285a Initial renaming pass of "jigdo" to "rojig"
I noticed that Ubuntu also uses the original "jigdo", so let's start
pulling off the band-aid here and do a mass rename.

For this first pass I'm focusing on CLI entrypoints and docs, as that's what
people are going to see; renaming all of the internal C functions, structure
variables etc. can come later.

Closes: #1269
Approved by: jlebon
2018-02-26 15:32:50 +00:00
Colin Walters
4af55efffd main: Fix wording of ex command description
Previous wording was awkward; noticed this in a CI error.

Closes: #1271
Approved by: jlebon
2018-02-23 18:21:17 +00:00
pcahyna
1ea5426751 fix "make tags"
src/libpriv/rpmostree-cleanup.h was removed in commit f14d1a3536 and its presence in Makefile causes "make tags" to fail.
Closes: #1270
Approved by: jlebon
2018-02-23 16:35:48 +00:00
Colin Walters
88324984ab README.md: Update "projects using rpm-ostree" section
Mention Kube over Docker, and also drop the "immutable" bit as I think it's a
misleading term.

Closes: #1264
Approved by: jlebon
2018-02-20 15:42:35 +00:00
Colin Walters
d03cd8a7b7 tests/jigdo-client: Re-enable
FAHC is rebuilt now with v5.

Closes: #1263
Approved by: jlebon
2018-02-20 14:27:54 +00:00
Jonathan Lebon
cf6f704ee4 Release 2018.3
Bugfix release to fix minor performance regression from auto-updates
work. Plus a low-risk enhancement to `status` to print only the booted
deployment with `--booted`.

Also minor refresh of release workflow documentation.

Closes: #1262
Approved by: cgwalters
2018-02-18 18:20:34 +00:00
Jonathan Lebon
55040a56ad daemon/sysroot: Emit UPDATED signal for configs
Let's also make sure we refresh `RPMOSTreeOS` objects when configs
changed. Specifically, what they export to their D-Bus interfaces may be
dependent on daemon settings, as is the case with auto-updates.

We might want to decouple this into two separate signals in the future
(one for sysroot changes, and one for config changes), though given that
only `RPMOSTreeOS` listens for `UPDATED` right now, we can just get away
with a single one for the time being.

Closes: #1261
Approved by: cgwalters
2018-02-17 21:15:30 +00:00
Jonathan Lebon
df80e15002 auto-updates: Use service type simple
I initially used `oneshot` for this since it felt like a better fit for
non-daemon services. But the only practical difference between the two
is that systemd will wait until we exit before running dependent
follow-up units.

I'd like to change this to `simple` because we can expect users to force
a trigger run using `systemctl start rpm-ostree-automatic`, which will
hang until completion in `oneshot` mode. And given that we're not part
of the boot process and no other unit depends on us, the distinction
makes no semantic difference to us.

Closes: #1261
Approved by: cgwalters
2018-02-17 21:15:30 +00:00
Jonathan Lebon
5d1c043058 daemon/os: Do less work on startup if auto-updates off
While playing with the freshly built v2018.2 Fedora packages on my
Atomic Workstation, I realized that we paid the auto-updates startup
cost even when it was disabled. I've had auto-updates on for so long, I
hadn't seen this pass by me. 🙈

The long-term fix here is caching the update to disk rather than
recalculating it on startup everytime (as suggested during review). Will
work on this as follow-up. Though for now, I'd like to get this smaller
patch in and backported to Fedora so only folks opting in to try it out
experience the >1s startup delay.

Closes: #1261
Approved by: cgwalters
2018-02-17 21:15:30 +00:00
Colin Walters
2e25732ba1 status: Add -b/--booted to only print that deployment
It's usually what I want to paste into bug reports for example; I rarely want to
show the pending state or rollback.

Closes: #1260
Approved by: jlebon
2018-02-17 19:00:24 +00:00
Jonathan Lebon
f54779df20 configure.ac: Bump ostree build dep to v2018.2
We make use of the new `OstreeRepoCheckoutFilterResult` type.

Closes: #1259
Approved by: cgwalters
2018-02-16 22:07:10 +00:00
Jonathan Lebon
b65f696b0b Release 2018.2
We have lots of goodies in the bag worthy of a release. More rojig
improvements, initial auto-updates support, and many fixes.

Closes: #1258
Approved by: cgwalters
2018-02-16 20:28:53 +00:00
Jonathan Lebon
ff9c19acd3 app/override: allow removing and replacing atomically
This is an essential functionality rather than a nicety. Some
replacements can *only* be done without conflicts if we can remove
packages at the same time.

I do like that this has to be done explicitly, though OTOH, I can
definitely see folks wanting an `--allow-removals` type of switch in the
future.

Closes: #1255

Closes: #1257
Approved by: cgwalters
2018-02-16 19:15:18 +00:00
Colin Walters
5879b96a64 jigdo V5: Use number of objects as cache invalidation trigger
Changes in a server-side tree can cause the need for clients to import different
objects from packages. For example, turning on documentation. Another more
subtle case is where an object might "move" from package A to B by being deleted
from A - then the jigdo build process will pick the B version.

We need a "cache validation key"; a way for the server to tell the client that
the objects it should import from the package have changed. Initially I was
thinking of using the libostree "content hash" but that would be awkward as we'd
have to do an import on the server side too.

After more consideration I realized a simple *count* of the number of objects
actually works, because (as I note in a comment) changing a file in the tree
will result in it ending up in the jigdoRPM (and count as a deletion).  And
obviously adding or removing objects changes the count too.

In fact we could have done this *without* breaking the format by just having the
client start recording the number of xattr entries, but this adds greater
flexibility down the line since we can in theory change how we do cache
invalidation if we *really* need to (but at the cost of triggering clients to
redownload packages).

Note the client logic got moved around as now we need to parse all the xattrs
before we decide what packages to download.

My test case here is turning on docs - I noticed this actually affects *every*
package which was surprising to me; I expected at least some packages wouldn't
have docs. I'll double check this.

It'd be good to have a "moving object" case too which I may look at.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1197

Closes: #1256
Approved by: jlebon
2018-02-16 18:45:51 +00:00
Colin Walters
a4d8980d59 tests/jigdo-e2e: Strictly require npkgs to import
Since we changed things to have `jigdoSet = pkgSet`, we can just require exactly
`${npkgs}` here on import, which is what we found from `db list`.

Closes: #1256
Approved by: jlebon
2018-02-16 18:45:51 +00:00
Colin Walters
64c4fe8559 origin: Rename jigdo-version key to jigdo-override-version
Suggested in PR review, it matches the current `override-version`
key better.

Closes: #1240
Approved by: jlebon
2018-02-16 17:02:47 +00:00
Colin Walters
a0b6d241d2 status: Render jigdo mode using package NEVRA
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`.  However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.

This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo ♲📦:
https://github.com/projectatomic/rpm-ostree/issues/1081#issuecomment-348540604

Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.

(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.

Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.

Closes: #1240
Approved by: jlebon
2018-02-16 17:02:47 +00:00
Jonathan Lebon
0729487ae5 Check and display pending security advisories
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.

This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.

Closes: #1249
Approved by: cgwalters
2018-02-15 15:30:26 +00:00
Jonathan Lebon
45a3b53558 daemon/api: fix legacy D-Bus API and add coverage
Fix logic to make sure we check if the refspec is of type `ostree://`
even when it's explicitly specified. Also fix `Deploy` in the case where
we didn't just `Download` the RPM diff by adding a new @checksum
parameter to the higher-level API.

Finally, add a basic test for the `GetCached*RpmDiff` APIs so we have at
least *some* coverage. This is also good prep for making sure we don't
break anything when we convert those APIs to use the more efficient
pkglist metadata. The tests completely ignore the `DownloadRpmDiff`
paths for now though.

Closes: #1250

Closes: #1253
Approved by: cgwalters
2018-02-15 15:03:05 +00:00
Colin Walters
528456a59f daemon: Add an origin_is_rojig() helper
Suggested in another PR review, it makes the code nicer.

Closes: #1252
Approved by: jlebon
2018-02-14 21:54:35 +00:00
Colin Walters
799a809c2d Add support for deploy <version> in rojig:// mode
This fleshes out an important piece of the story, showing that
we can support history versioning the same way that we did with
ostree.

Also it's very useful for testing; I'm going to extend the suite after this to
deploy the previous version, clean everything up, then upgrade and verify we
only download changed RPMs.

Closes: #1232
Approved by: jlebon
2018-02-14 21:54:30 +00:00
Colin Walters
c5939b7ca7 tests: Add RPMOSTREE_TEST_NO_OVERLAY to skip overlay
The use case here is to run our tests against the shipped tree;
this could be used by a-h-t for example:
https://github.com/projectatomic/atomic-host-tests/issues/74

I tried this with just `TESTS=basic` and it failed for `usroverlay`. So we'd
have to start adding feature detection to the test suite to make this truly
useful, but let's at least start with the basic bits now to play with it.

Closes: #1251
Approved by: jlebon
2018-02-14 20:00:16 +00:00
Colin Walters
f669e5157c jigdo: Clean up client update output a bit
I added the "commit already written" print for the very special case
when doing a `rebase --experimental rojig://` and the target jigdoRPM
is for the same commit you're on.  The semantics today of that are
weird because it's the *next* update where you download all the RPMs...

Anyways that case aside, let's only print when something did change,
and drop the checksum (same rationale as status PR) and also drop
the "from %u packages" since one sees that in the downloader.

Closes: #1244
Approved by: jlebon
2018-02-14 19:32:58 +00:00
Jonathan Lebon
8b72dcbde2 tests/libtest: Fix bash syntax error
I think the logic there just worked because we don't actually use
`noarch` RPMs in the vmcheck suite.

Closes: #1248
Approved by: cgwalters
2018-02-14 19:02:02 +00:00
Jonathan Lebon
57ab9844b9 daemon/autoupdates: Don't insert rpm-diff if empty
Canonicalize to "absent" to make it easier for clients.

Closes: #1248
Approved by: cgwalters
2018-02-14 19:02:02 +00:00
Colin Walters
f0e381392e README.md: Add arch diagram, tweak wording, add talk links
There's a lot more we could do here but I think the architecture
diagram in particular helps a lot.

Closes: #1245
Approved by: jlebon
2018-02-14 17:36:29 +00:00
Colin Walters
12dc565b00 core: Set installonly to ensure we only have one kernel
The goal here is to support `override replace kernel.x86_64.rpm`.

There's a whole lot of logic in libdnf to support having multiple
kernel packages installed.  AIUI, that was implemented because:

1) The kernel is the biggest source of regressions
2) It'd be quite noticeable if all of your kernel modules were deleted

Of course point 2) applies to a lot of userspace too...that's something
rpm-ostree fixes of course.

Anyways, in some testing all we need to do really is just turn that
logic off unconditionally.

Closes: https://github.com/projectatomic/rpm-ostree/issues/946

Closes: #1228
Approved by: jlebon
2018-02-14 16:43:08 +00:00
Colin Walters
40b010a7ea util: Initialize sack pointer to NULL
Otherwise we'll be freeing uninitialized memory on error (this happened in CI
somehow).

Closes: #1246
Approved by: jlebon
2018-02-14 16:17:52 +00:00
Jonathan Lebon
99aa6f89d2 ci: expand rootfs on CentOS
The new kernel tests require a little bit more space.

Closes: #1247
Approved by: cgwalters
2018-02-14 16:16:52 +00:00
Jonathan Lebon
75f4636320 core/commit: clean up rpmdb leftover files
While debugging the recent `BDB0087 DB_RUNRECOVERY` issues that cropped
up recently, I came upon the fact that we're leaving leftover rpmdb
files in the rootfs on client-side assemblies. Let's clean those up too.

We do this in the `commit` phase, *after* the final time we access the
rpmdb to generate the pkglist.

Closes: #1247
Approved by: cgwalters
2018-02-14 16:16:52 +00:00
Colin Walters
0a4f6bdab4 jigdo: Use separate rpmostree/jigdo/... refs
The way we import packages in jigdo mode is different from package layering; we
may only import a subset of files for example. In general, we need to treat
jigdo differently.

Related: https://github.com/projectatomic/rpm-ostree/issues/1197

Closes: #1238
Approved by: jlebon
2018-02-13 20:08:13 +00:00
Colin Walters
cd9e0c8ec7 util: Don't do libsolv cache for temporary sacks
We should really only generate a cache for the current root's rpmdb; this
function can be used for e.g. `db diff` and it doesn't make sense to generate a
libsolv cache for arbitrary commits since it's unlikely they get reused.

Closes: #1238
Approved by: jlebon
2018-02-13 20:08:13 +00:00
Colin Walters
782c49b748 upgrader: In jigdo mode, don't check rpm-md repos twice for layers
Typing `rpm-ostree upgrade` was quite verbose with layered packages, we'd see
the rpm-md repos twice. The better fix would be to pass the context/sack from
one stage to the other, but this is a quick simple fix to at least reduce
verbosity (and potentially avoid extra network requests).

Closes: #1241
Approved by: jlebon
2018-02-13 18:24:47 +00:00
Jonathan Lebon
aa87fe7d1a app/status: also check for timer state
I spent quite some time trying to figure out why the last run timestamp
of the autoupdate service wasn't showing up in `status`. It turned out
to be that systemd only keeps that information around if an associated
timer unit is active.

Thinking more, I think it makes sense to split out the case where the
timer isn't even running explicitly, rather than just reporting "no runs
since boot".

Closes: #1242
Approved by: cgwalters
2018-02-13 17:30:12 +00:00
Jonathan Lebon
92029de009 daemon: fix automatic update service switch
I missed this one when renaming the switch during the review of the
initial code drop.

Closes: #1242
Approved by: cgwalters
2018-02-13 17:30:12 +00:00
Jonathan Lebon
dc63bba78d daemon/dbus: remove reboot mode from comments
This is no longer supported.

Closes: #1242
Approved by: cgwalters
2018-02-13 17:30:12 +00:00
Colin Walters
852b574d01 tests/compose: f is valid in a hex checksum
I *think* this is why our tests started failing recently. It seems somehow very
unlikely to me though that we'd somehow managed to avoid `f` in the boot
checksums until now, but without doing some math...it seems plausible.

Closes: #1243
Approved by: jlebon
2018-02-13 17:26:54 +00:00
Colin Walters
f1e9f53480 scripts: Buffer output in non-journal case, prefix print with id
In the server-side `compose tree` case, this makes it obvious which package is
causing error spew. No particular burning reason, this was just bothering me.

Closes: #1235
Approved by: jlebon
2018-02-12 19:07:11 +00:00
Colin Walters
f26dcd59a2 Add rpm-ostree usroverlay
I saw kalev's slides reference `rpm-ostree unlock`; this patch makes it exist.
In general, people have a hard time (understandably) grasping the distinction
between ostree and rpm-ostree; along with the goal of making ostree really
"libostree", let's start wrapping more commands where it makes sense.

I also took this opportunity to have a more descriptive name; it's important
to note that it *doesn't* overlay `/etc`, `/var`, or `/boot` for example.

Closes: #1233
Approved by: jlebon
2018-02-11 18:03:06 +00:00