Commit Graph

2292 Commits

Author SHA1 Message Date
Colin Walters
a20fac3e9c core: Fix override-remove from UsrMove compat paths /lib,/bin etc
I'm working on supporting `override replace ./kernel-*.rpm`:
https://github.com/projectatomic/rpm-ostree/issues/946

But after battling the "installonly" logic in libdnf, I was confused why we
still had the files in `/usr/lib/modules`. It turned out to be because we only
remove files in `/usr`, but the code didn't handle UsrMove compat links.

There are a variety of approaches to fix this.  Obviously a lot
of things get nicer in jigdo mode as we do UsrMove canonicalization
on import, and we could change this code to walk the imported pkg
ostree ref.

Another approach would be to walk the initial symlink, and check whether or not
it's the same as `/usr` (i.e. let the kernel do it).

For now though, what I chose to do was to scan the rootfs and find the UsrMove
compat links (i.e. we avoid hardcoding them again here).  This is
fewer syscalls than the above and works well in practice.

Closes: #1226
Approved by: jlebon
2018-01-31 13:09:49 +00:00
Colin Walters
d3c1003079 client: Don't output progress bars on non-ttys
This is what a lot of other tools do. It can get very verbose, with a
potentially huge amount of output if things are trickling in.  This way
we're at least more friendly to someone running `cmd: rpm-ostree upgrade`
via Ansible or equivalent.

The slight hack here is that we *do* output `100%` on non-ttys to ensure we
print the result of the task.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1183

Closes: #1225
Approved by: jlebon
2018-01-29 21:28:07 +00:00
Jonathan Lebon
553a68c32a libpriv: add new rpmostree-types.h
There are types that we want to share between the daemon and the client
for deduplication. Those are, as expected, related to D-Bus things like
formats and enums. Let's create a new file for it rather than shove it
in `rpmostree-util.h`. As mentioned in the file, some of these probably
belong better directly in the public API.

Closes: #1147
Approved by: cgwalters
2018-01-27 23:52:43 +00:00
Jonathan Lebon
51fb641305 Initial support for automatic updates
This patch introduces a new `AutomaticUpdatePolicy` configuration. This
was a long time coming for rpm-ostree, given that its update model makes
it extremely apt for such a feature.

The config supports a `check` mode, which should be very useful to
Atomic Workstation users, as well as a `reboot` mode, which could be
used in its present form in simple single node Atomic Host situations.

There is still a lot of work to be done, including integrating
advisories, and supporting a `deploy` mode. This feature hopefully will
be leveraged as well by higher-level projects like GNOME Software and
Cockpit.

Closes: #1147
Approved by: cgwalters
2018-01-27 23:52:43 +00:00
Colin Walters
3e9c6cf230 Fix "releasever" option, test it by default
In #875 AKA b46fc35901 we
added support for the `releasever` option in treefiles.  I am
pretty sure it worked at the time...but I didn't add tests.

Either it never worked or some refactoring broke it. The whole chain of
`GKeyFile` → `GVariant` is so confusing. Anyways fix it by copying the string.
Now let's use it by default in the compose tests, and while we're here bump
those to F27.

I'm doing this patch now as I was playing with doing a compose from
the `/usr/share/rpm-ostree/treefile.json` and wanted to use the stock
`.repo` files.

Closes: #1220
Approved by: jlebon
2018-01-23 15:18:52 +00:00
Jonathan Lebon
4f025a6609 tree: nuke TODO file
We don't do a good job of keeping this file up to date. It might end up
doing more harm than good if it confuses newcomers.

Anyway, many of these are either no longer relevant or already
implemented. Let's agree that the canonical source of TODOs is the issue
tracker?

Closes: #1222
Approved by: cgwalters
2018-01-22 17:49:25 +00:00
Kalev Lember
79da2e2468 lib: Fix the library name to link to in the pkg-config file
This lets other programs actually link with the shared library.

Closes: #1221
Approved by: jlebon
2018-01-22 16:05:52 +00:00
Colin Walters
48e0cac9fb scripts: Ignore %verifyscript
It obviously shouldn't block the ability to install, and anyways
the single use in `ksh` is not compelling.  If someone comes up with
one we can revisit supporting this.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1216

Closes: #1218
Approved by: jlebon
2018-01-19 15:32:09 +00:00
Colin Walters
b85ae9e1d6 jigdo: V4: Use archful provides for jigdoRPM Requires
When I tried to use my WIP client patches to do:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`,
I got a missing file object which turned out to
be the client importing the i686 RPMs.

This was passing in the test suite because we don't mirror i686 of course, but
on the client side right now we end up using all enabled repos, and since Fedora
is multiarch, the behavior is going to be...not predictable.

Thinking a bit about on this problem I actually happened to recall
the RPM `%{_isa}` macro which is used in Fedora in various places;
for example to "arch bind" `-devel` packages to their base.  See
for example [this case](33c7dc02bc/f/ostree.spec (_79)) in libostree.

As I noted at first, the core problem here is that the "final"
RPM architecture field is not symmetric in any way with the definition
of that `%{_isa}` macro.  See:

d9d47e0114/installplatform (L25)

The *third* solution I ended up on here is to iterate over the
`Provides` on the server side and we take the first thing
that matches `Provides: %{name}(whatever)`.

I briefly thought about trying to somehow drive into libsolv the
logic to prefer the jigdoRPM's native architecture...IIRC yum did
something like that in the past but it was never done in libsolv?
Anyways the dependencies here are now more correct, so other tools
will also handle it.

Closes: #1213
Approved by: jlebon
2018-01-19 14:18:19 +00:00
Colin Walters
4008bcb27c build-sys: Fix trailing comma causing syntax error
Amazingly things seemed to stumble on from here, but this
started actively breaking when I was trying to change the code below.
It took me a while to realize it wasn't my changes introducing
the error message 😭.

Also indent so it's clearer where the conditional ends.

Closes: #1213
Approved by: jlebon
2018-01-19 14:18:19 +00:00
Jonathan Lebon
500dc02182 vmcheck/overlay: always use --consume --no-bindings
For CI purposes at least, these features are already part of FAH and the
latest CAH smoketested. Anyway, we're already hacking the latest ostree
into place for `--keep-metadata`. If it already supports the latter,
then clearly it'll support those.

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
183cb952e3 daemon: print warning only if val couldn't be parsed
We don't want to print a warning if the setting is missing from the
config file. That's totally normal (e.g. the config we ship has all its
configs commented out). We *do* want to print a warning if the config is
provided, but it couldn't be parsed as a proper `uint64`.

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
9e24e9c793 Revert "tests/libvm: write rpm build logs to file"
This reverts commit 1ef259ed76.

Basically, for historical POSIX compatibility, `errexit` doesn't work at
all inside functions called from an if-statement/boolean context. This
is something I had already learned (and forgotten) when hacking on PAPR.

There are ways around this, but it's just not worth the added complex
shell goop for what it gives us.

http://mywiki.wooledge.org/BashFAQ/105

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
83fb40593f ci: explicitly pass --sysconfdir=/etc to configure
I spent some time trying to figure out why on CentOS my changes to
`/etc/rpm-ostreed.conf` didn't seem to have any effect. It turns out
that for some reason I didn't spend too much time looking into, the
autoconf in CentOS defaults to `${prefix}/etc` when `--prefix` is given.

This is also why I was under the erroneous assumption that D-Bus config
files were in `/usr/etc` in CentOS. This patch fixes that as well.

Closes: #1215
Approved by: cgwalters
2018-01-18 20:44:07 +00:00
Jonathan Lebon
c01d4a8d2b vmcheck: always use base config file
Let's standardize on the default config file when running tests. We copy
the original out of the way and install the default one so tests can do
whatever they want with it.

This also strengthens the post-test cleanup to make sure we rebase back
to the local vmcheck branch, in case we're somehow on a different branch
with the exact same commit.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
1ef259ed76 tests/libvm: write rpm build logs to file
The output from `rpmbuild` makes test results harder to comb through
when debugging. Let's just dump all that to file and only output it if
something goes wrong.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
812daf3902 libpriv/util: add function to get deployment root
Prep for auto-updates.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
688f28c072 daemon: tweak transaction titles for deploy and refresh_md
Prep for auto-updates. Factor out a bunch of flags into nice booleans,
and tweak the transaction title a bit to provide more information about
flags provided.

This will be useful for when the user does a `status` while the daemon
is running due to the automatic updates trigger firing.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
a70fd4e960 libpriv/rpm-util: add path param for pkglist creation
Prep for auto-updates.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
2697af3135 app/status: no longer print pending info
Prep for auto-updates. Let's not print the "Pending*" keys anymore. With
automatic updates, we'll have a whole other section of the output just
for that.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Jonathan Lebon
a211e4670d app/status: separate state from deployments printing
Prep for auto-updates. Use different functions for printing the daemon
state and printing the deployments themselves.

Closes: #1212
Approved by: cgwalters
2018-01-17 22:45:24 +00:00
Colin Walters
2c270a6644 rebase: Add support for providing ostree:// prefix
This fixes an asymmetry we introduced when rendering using the `ostree://`
prefix; we now support parsing it as well.  We reject the `rojig://` prefix
explicitly for now.

On the implementation side...things are messy. My thought is that we do our best
to canonicalize at the entrypoints for both the client side as well as the
server side. However, for backwards compatibility we can't go all the way to
writing out `ostree://` to disk in the origin files.

I'm actually uncertain if Cockpit will deal with this...need to test that.

Closes: #1210
Approved by: jlebon
2018-01-17 14:19:39 +00:00
Colin Walters
567bedf035 core,status: Add API to parse refspecs, use in status
In preparation for rojig (rpm-ostree jigdo) client work.  Changing
all of the code where we currently use refspecs (i.e. pure ostree)
to also learn about jigdo would be really painful - basically adding
new DBus APIs and all of the places we pass through data.

A simpler path then is to overload the meaning of refspecs to have an (optional)
URI scheme. If we don't recognize the scheme, assume it's ostree for now.

Change the status command to parse and then render as a way of "canonicalizing"
so that we have the same behavior as before of rendering with `ostree://`.

Closes: #1208
Approved by: jlebon
2018-01-16 17:43:46 +00:00
Colin Walters
6df154e2e3 daemon: Pass GVariant deployment modifier down another level
Redo the internal API to pass GVariant down another level; this will be more
obviously useful as we move it much farther along through the layers.

Closes: #1170
Approved by: jlebon
2018-01-16 17:16:51 +00:00
Colin Walters
7276bcd1a0 daemon: Add an internal typedef for GVariant modifiers
Since we have a lot of variants, I'm thinking about using
this typedef approach to clarify things a bit more.  We could
even potentially do codegen around this, but let's start with
just a typedef.

Closes: #1170
Approved by: jlebon
2018-01-16 17:16:51 +00:00
Jonathan Lebon
cf1038e804 daemon/config: add new IdleExitTimeout config
Allow users to specify how long the daemon should idle for before
exiting. This was *mostly* an excuse for me to set up a somewhat useful
config file ahead of the auto-update work. Though I think allowing this
config makes sense as well. I like it better than the secret env var for
development purposes. PackageKit also has a similar `ShutdownTimeout`
configuration.

Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
b9bae8fc5b daemon/config: add new rpm-ostreed.conf
Making this a separate commit, since it's a first for rpm-ostree. We now
have a conf file complete with man page! No options yet though.

Interestingly, there was a function called `rpmostreed_reload_config`
which was declared but never defined. Didn't look too much into that.

We make sure that the config is part of the things we reload when users
call `rpm-ostree reload`.

Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
57eb370eb4 daemon/sysroot: minor indentation fix
Closes: #1204
Approved by: cgwalters
2018-01-16 16:51:01 +00:00
Jonathan Lebon
0cc50bb043 vmcheck/overlay: use --keep-metadata for source-title
One issue right now is that repeated applications of `make vmoverlay`
will result in losing the source title.

Let's use the new `--keep-metadata` in subsequent `make vmoverlay`
invocations to make sure it gets transferred over correctly.

Closes: #1205
Approved by: cgwalters
2018-01-16 15:32:32 +00:00
Jonathan Lebon
54502eb8bf vmcheck/overlay: add version and timestamp in source title
This makes it much easier to know how old the base we're working from is
and whether it's time to upgrade before hacking some more.

Closes: #1205
Approved by: cgwalters
2018-01-16 15:32:32 +00:00
Jonathan Lebon
4f9066d247 vmcheck/overlay: drop temporary hacks
By now, these should have percolated down to all the streams.

Closes: #1205
Approved by: cgwalters
2018-01-16 15:32:32 +00:00
Jonathan Lebon
5c9a9c30f7 ci/build-check: add -Wno-error=deprecated-declarations
For libdnf.

Closes: #1114
Approved by: jlebon
2018-01-16 14:14:33 +00:00
Colin Walters
cf58640cb3 Switch libdnf to projectatomic branch
We previously added a projectatomic/libdnf fork. I just pushed an
`rpmostree-branch` to there with one PR on top. This is an unfortunate turn but
the C++ thing still needs to be worked out.

Obviously we don't want a permanent fork, but I find it hard to justify spending
a whole lot of time pitching in to help with the libdnf C++ conversion versus
all the other work I could be doing.

Update submodule: libdnf

Closes: #1114
Approved by: jlebon
2018-01-16 14:14:32 +00:00
Colin Walters
e6a65b80df Release 2018.1
Note this runtime-requires libostree 2018.1, but not (AFAIK)
build time.

Closes: #1207
Approved by: jlebon
2018-01-15 19:03:11 +00:00
Jonathan Lebon
f320478cd6 tests/vmcheck: expand a bit more --copyup tests
Check that appending actually worked and check that editing a copied-up
file from a required package works.

Closes: #1203
Approved by: cgwalters
2018-01-12 19:48:32 +00:00
Jonathan Lebon
183976d76d lib/db: actually allow-none for out parameters
The docs specify the `out_` parameters as `allow-none`, but we didn't
actually support it. We'll need this for auto-updates too, so let's
support specifying only some of them (but at least one).

Closes: #1202
Approved by: cgwalters
2018-01-12 17:48:31 +00:00
Jonathan Lebon
699cc89198 libpriv/rpm-util: encode epoch as a string
This is essentially a revert of #1190. I initially changed from storing
the epoch, version, and release separately to together to avoid endian
issues with epoch. This works around that instead by just encoding it as
a string.

As is done in our custom NEVRA printer (which before #1190 we used
here), we make sure that epochs of "0" are not printed as part of the
`nevra` or `evr` members.

Closes: #1198
Approved by: cgwalters
2018-01-11 21:58:49 +00:00
Colin Walters
bc5237ebf8 tests: Add a test case for epoch 0 in rpmdb pkglist
Basically we're choosing the libdnf behavior of omitting `Epoch: 0` which IMO is
the sane thing to do even though in *theory* they're different RPMs but
who would ever ship both without incrementing release/etc?

More information in: https://github.com/projectatomic/rpm-ostree/pull/187

Closes: #1198
Approved by: cgwalters
2018-01-11 21:58:49 +00:00
Jonathan Lebon
738fbc633b tests/libtest-core: support multiple literal checks
`grep` supports checking multiple fixed strings separated by newlines,
but it's mostly just easier to pass them as separate arguments, so let's
support that. This is now at parity with the similar
`assert_file_has_content`.

Will upstream this to ostree as well once reviewed.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
9f7511a5d8 libpriv/rpm-util: add autoptr support to RpmHeaders
Prep for follow up patches.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
542e4c153b libpriv/rpm-util: factor refts from rootfs helper
This will be used later. Also make it take either a `GLnxTmpDir` or
a path to the rootfs.

Closes: #1200
Approved by: cgwalters
2018-01-11 20:17:50 +00:00
Jonathan Lebon
436f7a1d4c app/status: factor out some print functions
With automatic updates, I'd like to make the outputs of `upgrade
--check/--preview` very similar to parts of `status`. E.g. timestamp,
version, GPG signature printing. So let's factor those out into
`libbuiltin`.

Note that unlike the others, `rpmostree_print_gpg_info` deals directly
with a `GVariant` received from D-Bus, so it felt more appropriate to
place it in `dbus-helpers`. Plus, `rpmostree_print_signatures` was
already there, so it simplified things a bit.

This also sneaks in a nicer way of printing padded strings using the
argument field width syntax.

Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Jonathan Lebon
778a78bb9f app/status: factor out escape chars logic
Make use of the new `glnx_stdout_is_console ()` rather than caching our
own result of `isatty()`. Add helper functions in `libbuiltin.h` to
retrieve escape characters. Prep for using them from other files.

Update submodule: libglnx

Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Jonathan Lebon
c80a4168de dbus: document missing key from CachedUpdate
Closes: #1196
Approved by: cgwalters
2018-01-11 15:37:57 +00:00
Colin Walters
7e44c032c8 experiments-and-demos: New subdir with skopeo2ostree Dockerfile
Before I went all in on jigdo I was playing around with doing
a build from `Dockerfile` of something that looked like Atomic Host,
and then importing that build into an ostree repo.

For a lot of reasons, I think this is not the way to go in general
right now, but let's keep this demo around so we can pick
things up back later if we want to (or someone else does).

Closes: #1193
Approved by: jlebon
2018-01-11 14:07:17 +00:00
Colin Walters
c0cb60bd5e core: Don't try to relabel if no SELinux policy
This tripped up some local `ex container` work after:
https://github.com/projectatomic/rpm-ostree/pull/1173

Would be nice to have tests but...let's slide this one through for now since I
plan to get a lot more coverage as `ex container` morphs into a `mock`
replacement.

Closes: #1194
Approved by: jlebon
2018-01-10 22:10:13 +00:00
Colin Walters
9f777c26ee compose: Add --ex-jigdo-output-set
Following up on `--ex-jigdo-output-rpm`, add support for writing the entire set
to an output directory. This is intended for use cases like FAHC, where we're
generating data outside of the upstream Fedora infrastructure. Further, we want
to support having our own history stored reliably, even if upstream prunes RPMs.

Now, this can be interesting even for upstreams like Fedora, as it naturally
captures just the subset of RPMs; doing full history support for that would
likely be a lot more palatable than for Everything.

Closes: #1165
Approved by: jlebon
2018-01-10 19:18:40 +00:00
Colin Walters
79f4b4b861 compose: Add --ex-jigdo-output-rpm
Part of the goal of jigdo ♲📦 is to support organizations switching to *only*
providing RPMs. An intermediate step there is to "lock" the repo and jigdo
together; we don't want to update the ref if building the jigdoRPM fails.

Add an option to perform `rpm-ostree compose tree` and `rpm-ostree ex
commit2jigdo` together; notably we generate a commit, but only update the ref
once the jigdoRPM is built.

Closes: #1165
Approved by: jlebon
2018-01-10 19:18:40 +00:00
Jonathan Lebon
4ad627f69b daemon: bump idle exit timeout to 60s
Since it usually takes more than 10s for users to enter consecutive
commands, let's bump the timeout to 60s. That way, we avoid the churn of
starting up twice and e.g. polluting the journal.

On a user interface level, this doesn't make a big difference: a
`status` from cold takes around 100ms, whereas with the daemon running,
it takes slightly less than 50ms. Slightly noticeable, but a non-issue.

However, auto-update will require some more work at startup, and a cold
`status` will bump to about 350ms, which is definitely more noticeable.
Bumping the timeout will ensure that at least within the span of one
"interaction" (multiple commands), we only do this work once.

Closes: #1192
Approved by: cgwalters
2018-01-10 18:53:50 +00:00
Jonathan Lebon
7f946130ef app/main: also return the bus type on option parsing
This is prep for automatic updates. There, we want to know which D-Bus
we're connected to and e.g. only try to reach out to other services like
systemd if we're on the system bus.

Closes: #1191
Approved by: cgwalters
2018-01-10 17:20:33 +00:00