Commit Graph

1968 Commits

Author SHA1 Message Date
Colin Walters
7276bcd1a0 daemon: Add an internal typedef for GVariant modifiers
Since we have a lot of variants, I'm thinking about using
this typedef approach to clarify things a bit more.  We could
even potentially do codegen around this, but let's start with
just a typedef.

Closes: #1170
Approved by: jlebon
2018-01-16 17:16:51 +00:00
Jonathan Lebon
cf1038e804 daemon/config: add new IdleExitTimeout config
Allow users to specify how long the daemon should idle for before
exiting. This was *mostly* an excuse for me to set up a somewhat useful
config file ahead of the auto-update work. Though I think allowing this
config makes sense as well. I like it better than the secret env var for
development purposes. PackageKit also has a similar `ShutdownTimeout`
configuration.

Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
b9bae8fc5b daemon/config: add new rpm-ostreed.conf
Making this a separate commit, since it's a first for rpm-ostree. We now
have a conf file complete with man page! No options yet though.

Interestingly, there was a function called `rpmostreed_reload_config`
which was declared but never defined. Didn't look too much into that.

We make sure that the config is part of the things we reload when users
call `rpm-ostree reload`.

Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
57eb370eb4 daemon/sysroot: minor indentation fix
Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
0cc50bb043 vmcheck/overlay: use --keep-metadata for source-title
One issue right now is that repeated applications of `make vmoverlay`
will result in losing the source title.

Let's use the new `--keep-metadata` in subsequent `make vmoverlay`
invocations to make sure it gets transferred over correctly.

Closes: #1205
Approved by: cgwalters
2018-01-16 15:32:32 +00:00
Jonathan Lebon
54502eb8bf vmcheck/overlay: add version and timestamp in source title
This makes it much easier to know how old the base we're working from is
and whether it's time to upgrade before hacking some more.

Closes: #1205
Approved by: cgwalters
2018-01-16 15:32:32 +00:00
Jonathan Lebon
4f9066d247 vmcheck/overlay: drop temporary hacks
By now, these should have percolated down to all the streams.

Closes: #1205
Approved by: cgwalters
2018-01-16 15:32:32 +00:00
Jonathan Lebon
5c9a9c30f7 ci/build-check: add -Wno-error=deprecated-declarations
For libdnf.

Closes: #1114
Approved by: jlebon
2018-01-16 14:14:33 +00:00
Colin Walters
cf58640cb3 Switch libdnf to projectatomic branch
We previously added a projectatomic/libdnf fork. I just pushed an
`rpmostree-branch` to there with one PR on top. This is an unfortunate turn but
the C++ thing still needs to be worked out.

Obviously we don't want a permanent fork, but I find it hard to justify spending
a whole lot of time pitching in to help with the libdnf C++ conversion versus
all the other work I could be doing.

Update submodule: libdnf

Closes: #1114
Approved by: jlebon
2018-01-16 14:14:32 +00:00
Colin Walters
e6a65b80df Release 2018.1
Note this runtime-requires libostree 2018.1, but not (AFAIK)
build time.

Closes: #1207
Approved by: jlebon
2018-01-15 19:03:11 +00:00
Jonathan Lebon
f320478cd6 tests/vmcheck: expand a bit more --copyup tests
Check that appending actually worked and check that editing a copied-up
file from a required package works.

Closes: #1203
Approved by: cgwalters
2018-01-12 19:48:32 +00:00
Jonathan Lebon
183976d76d lib/db: actually allow-none for out parameters
The docs specify the `out_` parameters as `allow-none`, but we didn't
actually support it. We'll need this for auto-updates too, so let's
support specifying only some of them (but at least one).

Closes: #1202
Approved by: cgwalters
2018-01-12 17:48:31 +00:00
Jonathan Lebon
699cc89198 libpriv/rpm-util: encode epoch as a string
This is essentially a revert of #1190. I initially changed from storing
the epoch, version, and release separately to together to avoid endian
issues with epoch. This works around that instead by just encoding it as
a string.

As is done in our custom NEVRA printer (which before #1190 we used
here), we make sure that epochs of "0" are not printed as part of the
`nevra` or `evr` members.

Closes: #1198
Approved by: cgwalters
2018-01-11 21:58:49 +00:00
Colin Walters
bc5237ebf8 tests: Add a test case for epoch 0 in rpmdb pkglist
Basically we're choosing the libdnf behavior of omitting `Epoch: 0` which IMO is
the sane thing to do even though in *theory* they're different RPMs but
who would ever ship both without incrementing release/etc?

More information in: https://github.com/projectatomic/rpm-ostree/pull/187

Closes: #1198
Approved by: cgwalters
2018-01-11 21:58:49 +00:00
Jonathan Lebon
738fbc633b tests/libtest-core: support multiple literal checks
`grep` supports checking multiple fixed strings separated by newlines,
but it's mostly just easier to pass them as separate arguments, so let's
support that. This is now at parity with the similar
`assert_file_has_content`.

Will upstream this to ostree as well once reviewed.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
9f7511a5d8 libpriv/rpm-util: add autoptr support to RpmHeaders
Prep for follow up patches.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
542e4c153b libpriv/rpm-util: factor refts from rootfs helper
This will be used later. Also make it take either a `GLnxTmpDir` or
a path to the rootfs.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
436f7a1d4c app/status: factor out some print functions
With automatic updates, I'd like to make the outputs of `upgrade
--check/--preview` very similar to parts of `status`. E.g. timestamp,
version, GPG signature printing. So let's factor those out into
`libbuiltin`.

Note that unlike the others, `rpmostree_print_gpg_info` deals directly
with a `GVariant` received from D-Bus, so it felt more appropriate to
place it in `dbus-helpers`. Plus, `rpmostree_print_signatures` was
already there, so it simplified things a bit.

This also sneaks in a nicer way of printing padded strings using the
argument field width syntax.

Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Jonathan Lebon
778a78bb9f app/status: factor out escape chars logic
Make use of the new `glnx_stdout_is_console ()` rather than caching our
own result of `isatty()`. Add helper functions in `libbuiltin.h` to
retrieve escape characters. Prep for using them from other files.

Update submodule: libglnx

Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Jonathan Lebon
c80a4168de dbus: document missing key from CachedUpdate
Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Colin Walters
7e44c032c8 experiments-and-demos: New subdir with skopeo2ostree Dockerfile
Before I went all in on jigdo I was playing around with doing
a build from `Dockerfile` of something that looked like Atomic Host,
and then importing that build into an ostree repo.

For a lot of reasons, I think this is not the way to go in general
right now, but let's keep this demo around so we can pick
things up back later if we want to (or someone else does).

Closes: #1193
Approved by: jlebon
2018-01-11 14:07:17 +00:00
Colin Walters
c0cb60bd5e core: Don't try to relabel if no SELinux policy
This tripped up some local `ex container` work after:
https://github.com/projectatomic/rpm-ostree/pull/1173

Would be nice to have tests but...let's slide this one through for now since I
plan to get a lot more coverage as `ex container` morphs into a `mock`
replacement.

Closes: #1194
Approved by: jlebon
2018-01-10 22:10:13 +00:00
Colin Walters
9f777c26ee compose: Add --ex-jigdo-output-set
Following up on `--ex-jigdo-output-rpm`, add support for writing the entire set
to an output directory. This is intended for use cases like FAHC, where we're
generating data outside of the upstream Fedora infrastructure. Further, we want
to support having our own history stored reliably, even if upstream prunes RPMs.

Now, this can be interesting even for upstreams like Fedora, as it naturally
captures just the subset of RPMs; doing full history support for that would
likely be a lot more palatable than for Everything.

Closes: #1165
Approved by: jlebon
2018-01-10 19:18:40 +00:00
Colin Walters
79f4b4b861 compose: Add --ex-jigdo-output-rpm
Part of the goal of jigdo ♲📦 is to support organizations switching to *only*
providing RPMs. An intermediate step there is to "lock" the repo and jigdo
together; we don't want to update the ref if building the jigdoRPM fails.

Add an option to perform `rpm-ostree compose tree` and `rpm-ostree ex
commit2jigdo` together; notably we generate a commit, but only update the ref
once the jigdoRPM is built.

Closes: #1165
Approved by: jlebon
2018-01-10 19:18:40 +00:00
Jonathan Lebon
4ad627f69b daemon: bump idle exit timeout to 60s
Since it usually takes more than 10s for users to enter consecutive
commands, let's bump the timeout to 60s. That way, we avoid the churn of
starting up twice and e.g. polluting the journal.

On a user interface level, this doesn't make a big difference: a
`status` from cold takes around 100ms, whereas with the daemon running,
it takes slightly less than 50ms. Slightly noticeable, but a non-issue.

However, auto-update will require some more work at startup, and a cold
`status` will bump to about 350ms, which is definitely more noticeable.
Bumping the timeout will ensure that at least within the span of one
"interaction" (multiple commands), we only do this work once.

Closes: #1192
Approved by: cgwalters
2018-01-10 18:53:50 +00:00
Jonathan Lebon
7f946130ef app/main: also return the bus type on option parsing
This is prep for automatic updates. There, we want to know which D-Bus
we're connected to and e.g. only try to reach out to other services like
systemd if we're on the system bus.

Closes: #1191
Approved by: cgwalters
2018-01-10 17:20:33 +00:00
Jonathan Lebon
370679717b libpriv/rpm-util: just encode evr in pkglist
I initially wanted to include all of `epoch`, `version`, and `release`
separately in the rpmdb.pkglist metadata in case we needed them
separately. Thinking more on this, I can't think of a really good reason
to have them, so since we're not public yet, let's just encode the `evr`
as a single string.

The reason I'm revisiting this is because it took me a while to hunt
down issues when handling epoch, which turned out to be the fact that we
weren't consistently committing as big endian (i.e. we did so in the
`compose tree` case, but not the layering case), and neither were we
consistently converting back from big endian. It's doable of course, but
the added gymnastics doesn't feel justified for the gains here.

This also nicely cleans up the `RpmOstreePackage` implementation to be
leaner and faster.

Closes: #1190
Approved by: cgwalters
2018-01-10 17:18:09 +00:00
Colin Walters
b1c92a6a1d tests/compose: Various fixes
- Actually use separate `${test_tmpdir}` for test setup (closes a race)
 - Merge stdout/stderr (more readable)
 - Ensure logs are renamed to `.txt` even on failure
 - Use `--progress` for some feedback
 - Use `-j +1` so that even on unicore machines we get at least 2
   jobs (and in general NCPUS+1)

Closes: #1188
Approved by: jlebon
2018-01-10 15:16:18 +00:00
Colin Walters
c82a7b53a0 man: Various tweaks/updates to the man page
I initially started this since I wanted to have the client-side
commands first, but ended up splitting them into a separate section.

Various other things:

 - Update the intro
 - Add jlebon to authors

Closes: #1189
Approved by: jlebon
2018-01-09 22:38:19 +00:00
Colin Walters
054b48d55b core,compose: Fix unified core pkgcache labeling
Basically the `rpmostree_context_relabel()` call we had in the treecompose path
for unified core didn't actually have any effect as the core code did a relabel
and unset the array.

I think this may actually be a regression from: https://github.com/projectatomic/rpm-ostree/pull/1137
though I didn't verify.

Anyways looking at this, the code is a lot simpler if we change the API so that
the "normal" relabeling is folded into `rpmostree_context_assemble()`. Then we
change the public relabel API to be "force relabel" which we use in the unified
core 🌐 treecompose path.

This shrinks the jigdoRPM for FAH from 90MB to 68MB.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1172

Closes: #1173
Approved by: jlebon
2018-01-09 16:59:19 +00:00
Colin Walters
a3e0b47e6b ci/papr: Bump compose tests to 4 min vCPUs
Compose is a slow test right now.  Down the line what I'd
like to do is: https://github.com/projectatomic/papr/pull/70
Since this job can be scheduled as a container, not a VM.  There's
no reason to grab a whole 8GB of RAM for it, but we *do* want multiple
CPUs.  Containers do that by default.

Closes: #1187
Approved by: jlebon
2018-01-09 16:26:32 +00:00
Colin Walters
84f0b017d9 bwrap: Use rofiles-fuse --copyup by default
This fixes a large swath of compatibility issues, for the same reasons as
overlayfs makes a lot of things Just Work.  The ugly part of course is
doing hidden copyups inside the filesystem.

We've gone quite a long time with the "pure rofiles" mode, and have made changes
to various bits of userspace to be compatible with it. But what finally made me
give up on that is glibc's locale-archive; there's a patch for it that
is stalled, but even if it was applied we would still need to work with
older glibc.

This issue comes to the fore in unified core 🌐 mode, as without this
we won't get a correct locale archive.

Closes: #1171
Approved by: jlebon
2018-01-09 15:36:08 +00:00
Colin Walters
0e16be8273 tests: Run compose tests in parallel ∥
Over a year later, the "opening the host rpmdb" bug is fixed,
so we can do composes in parallel ∥, hooray!

I'm dusting this off since we were running into CI (PAPR) timeouts
when I was adding more to the compose tests.

Closes: #545
Approved by: jlebon
2018-01-09 14:19:31 +00:00
Colin Walters
02c945cd4f core: Say "Installing %u packages", not "overlays" for containers
Minor, but it was annoying me perhaps somewhat unreasonably.  There's
no "overlay" going on in `ex container` runs, so let's just use the
term "installing".

Closes: #1186
Approved by: jlebon
2018-01-08 19:22:46 +00:00
Colin Walters
e834f4dd27 scripts: Only append "run journalctl" message if using journal
For the `ex container` case we don't log to the journal by default,
so this message was misleading.

Closes: #1181
Approved by: jlebon
2018-01-08 15:14:45 +00:00
Colin Walters
7fa55f7e5f lib/core: Add API to retrieve all pkgs
Will be used for `compose tree --ex-output-jigdo-set`. Also probably more things
could use this. I first tried just doing another query in the higher level code
but that fails for this case as `sort_packages() reimplements
`dnf_transaction_ensure_repo_list()` which we need to get the `DnfRepo` (which
itself feels like a hack, we should be maintaining that dynamically or have a
hash accessor).

Closes: #1184
Approved by: jlebon
2018-01-08 14:41:34 +00:00
Colin Walters
032c1556f4 core,util: Lower some DnfPackage* bits into utils
Ideally we'd be fixing upstream libdnf but that's a bit blocked
right now.  I need these at a higher level to implement
`rpm-ostree compose tree --ex-output-jigdo-set` which needs to
link/copy the input RPMs.

Closes: #1184
Approved by: jlebon
2018-01-08 14:41:34 +00:00
Colin Walters
7ffb544e35 tests/compose: Rename jigdo.sh to jigdo-e2e.sh
I plan to add another jigdo test, and the naming is clearer.

Closes: #1184
Approved by: jlebon
2018-01-08 14:41:34 +00:00
Colin Walters
855d1293ab cli: Rework exit status processing
We've had many bugs from internal helpers using `return EXIT_FAILURE` rather
than `return FALSE`.  The reason we need exit codes is to handle the
`RPM_OSTREE_EXIT_UNCHANGED` case. I realized recently that we had the handy
`RpmOstreeCommandInvocation` which we can use to signal back this special case.
Then all of our functions otherwise are just normal `GError`.

One minor wart here is the two cases of "usage error" versus "command
invocation" in `main.c`, but IMO the general cleanup is well worth that.

Closes: #1169
Approved by: jlebon
2018-01-06 15:30:34 +00:00
Colin Walters
affa50fbc1 Lower jigdo client logic into core
Introduce a new `rpmostree_context_execute_jigdo()` that fills the same role as
`ostree_repo_pull_with_options()`. This will be used by the sysroot upgrader.

I didn't change the jigdo client code much yet; as a TODO says there's a lot
more we can do to improve things. Some of the public APIs we added to the core
no longer need to be public, such as `rpmostree_context_set_packages()`. But
let's try to do things incrementally.

I did at least change the `g_print()`s to `rpmostree_output_message()`.  I
dropped the `commit_and_print()`; at some point will come back and clean things
up so we consistently journal/print stats.

Closes: #1168
Approved by: jlebon
2018-01-05 17:02:53 +00:00
Colin Walters
a55e923eae core: Split out a -private.h header with struct
Prep for doing the "single object, multiple .c files" pattern like is done in
e.g. libostree with OstreeRepo and `ostree-repo-{refs,commit.c}` etc. For jigdo
we'll need to split things up (see what I did there?).

Closes: #1168
Approved by: jlebon
2018-01-05 17:02:52 +00:00
Colin Walters
8f9be2e54b scripts: Suppress microcode_ctl posttrans
Only applies to CentOS7.  Currently the package unconditionally
runs `dracut -f` which we need to handle internally.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1199582

Closes: #1174
Approved by: jlebon
2018-01-05 16:27:26 +00:00
Colin Walters
80e3857720 tests/compose: Pull in Fedora updates
In particular this gets us the selinux-policy fix for:
https://github.com/projectatomic/rpm-ostree/pull/1173#issuecomment-355014583

We might as well do updates since they have to work anyways.

Closes: #1177
Approved by: jlebon
2018-01-05 15:20:42 +00:00
Micah Abbott
f4877419ac docs: update branch examples for 'rebase' command
The old branch examples use Fedora 26 which is almost EOL.  The new
Fedora 27 examples show off the various `testing` and `updates`
branches, as well as the support for different arches.

Closes: #1175
Approved by: cgwalters
2018-01-03 16:05:15 +00:00
Jonathan Lebon
37273705dd app/db-diff: make use of new db API
This is the first step towards unifying how we introspect packages from
a specific commit. We currently do this in three ways: libdnf, librpm,
and now `rpmostree.rpmdb.pkglist`. I'd like to get to a point where we
only have `rpmostree.rpmdb.pkglist` and libdnf, the latter only when
more complex queries are required.

This patch teaches the `db diff` command to make use of the new db diff
API so that it can work even on metadata-only commits. This is relevant
for use cases mentioned in #558.

I didn't get rid of the `rpmhdrs_diff` functions right now because of
the `--changelogs` option: libdnf currently does not expose this, so we
fall back to the previous API in that case. OTOH, I wonder how much it's
actually used in the wild; maybe we could just nix it?

Closes: #1162
Approved by: cgwalters
2017-12-30 11:32:38 +00:00
Jonathan Lebon
def21255ac lib/db: use new pkglist metadata API if possible
Add a function that can smartly perform diff operations on sorted
RpmOstreePackage arrays and make the db API use that. This allows us to
immediately take advantage of the benefits in a few places where diffs
are performed, including post-deployment tree diffs, and the legacy db
diff variant API. The upcoming `CachedUpdate` rework will also make use
of this (but with the notable difference of setting `allow_noent` to
`TRUE`).

Note this introduces a new `rpm_ostree_db_diff_ext` public API which has
the same interface as `rpm_ostree_db_diff` but also takes flags.

Closes: #1162
Approved by: cgwalters
2017-12-30 11:32:38 +00:00
Jonathan Lebon
18d36afdf9 lib/package: teach about rpmostree.rpmdb.pkglist
In order to make use the new pkglist metadata more consummable, let's
add a function to create RpmOstreePackage arrays from it. Since some
APIs need to keep working as before, we use a `allow_noent` parameter to
distinguish between the two cases.

This is prep trying our best to make use of this metadata when possible
rather than checking out the rpmdb and initializing a `DnfSack`.

Closes: #1162
Approved by: cgwalters
2017-12-30 11:32:38 +00:00
Colin Walters
5a6c779850 tests: Make use of commit --no-bindings if available
The way we alias a commit with e.g. `vmcheck_orig` trips
up ref bindings.  Perhaps we should rework the test suite
but for now let's just mark `vmcheck` as aliasable.

See:
https://github.com/ostreedev/ostree/pull/1380
https://github.com/ostreedev/ostree/pull/1379

Closes: #1167
Approved by: jlebon
2017-12-22 14:07:07 +00:00
Jonathan Lebon
e20768dded app/status: support JSONPath queries
The underlying library already supports this. Let's expose it to make it
easier for humans to query the metadata.

Closes: #1164
Approved by: cgwalters
2017-12-21 03:59:54 +00:00
Jonathan Lebon
a3b5f54fbd app/status: wrap package lists nicely if too long
Rather than letting the terminal wrap our line and go unindented on the
next line, let's do the wrapping ourselves to tidy up the output.

Closes: #1159
Approved by: cgwalters
2017-12-20 17:06:31 +00:00