Commit Graph

345 Commits

Author SHA1 Message Date
Jonathan Lebon
def21255ac lib/db: use new pkglist metadata API if possible
Add a function that can smartly perform diff operations on sorted
RpmOstreePackage arrays and make the db API use that. This allows us to
immediately take advantage of the benefits in a few places where diffs
are performed, including post-deployment tree diffs, and the legacy db
diff variant API. The upcoming `CachedUpdate` rework will also make use
of this (but with the notable difference of setting `allow_noent` to
`TRUE`).

Note this introduces a new `rpm_ostree_db_diff_ext` public API which has
the same interface as `rpm_ostree_db_diff` but also takes flags.

Closes: #1162
Approved by: cgwalters
2017-12-30 11:32:38 +00:00
Jonathan Lebon
addc41ad12 libpriv/rpm-util: drop get_pkglist_for_root
This function returned both an RpmOstreeRefsack and a pkglist bound to
the refsack. Interestingly, there were only two users of it, and one of
them didn't even make use of the pkglist functionality. Since the
lifetime semantics of this function are tricky, let's drop it and
introduce a dedicated function just for returning package lists.

I also dropped the `GCancellable` argument, since it isn't/can't easily
be used by those code paths.

Closes: #1158
Approved by: cgwalters
2017-12-20 13:10:36 +00:00
Jonathan Lebon
97133bd028 daemon: avoid using floating GVariant refs
It makes it harder to keep track of ownership except in the trivial
`_new ()` case for which they were designed (i.e. shoving straight into
another glib GVariant function that takes ownership of it right away).

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
608ab302e8 daemon/sysroot: reload repo before deployments
In the case of a rebase to a fresh remote, we want to make sure our
`OstreeRepo` picks up the new remote before we try to parse its
deployment. Otherwise, we'll error out when trying to fetch configs for
it.

This was picked up after turning on strict errors in the GPG
verification process.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
5fb153601a daemon/deployment-utils: some new style porting
Prep for more work.

We actually pass a `GError` to the gpg results function and fail for any
other failures than gpg signature verification. (In that case, we want
to make sure that information gets to the D-Bus API level, so killing
the daemon would be the wrong choice).

I also factored out all the logic in
`rpmostreed_commit_generate_cached_details_variant` into a separate
function. This is prep for reusing it elsewhere.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
9ad0829330 daemon/os: reorder iface functions
This is minor, though the list of methods we support is getting long and
will get longer. Let's just add some order to it to make it easier for
humans to parse.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
8e1a769734 daemon/package-variant: fix off-by-one error
We're definitely interested in the rpmdb diff even if there's just one
package changed.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
5018583637 daemon/package-variant: some more porting to new style
Prep for more work.

This also involved changing the functions from pointer return values to
`gboolean`.

I also made functions that didn't really need to be public become
private and fixed some indentation.

Closes: #1160
Approved by: cgwalters
2017-12-20 13:09:42 +00:00
Jonathan Lebon
5ff69e10db daemon/os: check for download-only and dry-run conflict
Minor follow-up to previous commit. It seems a bit confusing to allow
specifying `download-only` and `dry-run`. The former already includes
all the steps in the latter but goes further, as documented. Let's check
for this combination.

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Jonathan Lebon
183399e1cd daemon/deploy: fix --dry-run downloads packages
Minor regression that crept in during the `--download-only` work. We
would download and import packages even when `--dry-run` was given.
Make sure we stop right after printing the transaction.

Related: #1128

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Jonathan Lebon
7f0737e7fc daemon/os: fix complete function for SetInitramfsState()
We were calling the wrong completer function for `SetInitramfsState()`.
Not that it mattered much in compiled form since both ways ended up
calling the same internal gdbus function with the same arguments.

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Jonathan Lebon
d5e92df5c1 rpm-ostreed.service: tweak description and add doc
This is highly dependent of the outcome of [1], though until that's
settled there, let's at least update the description to something a
little more apt. It feels more appropriate to consider rpm-ostree as a
"system manager" than just a "package manager" (which it certainly is
too of course). Also use Title Case convention which seems more popular
overall and looks nicer.

[1] https://github.com/projectatomic/rpm-ostree/issues/405

Closes: #1155
Approved by: cgwalters
2017-12-15 15:23:17 +00:00
Colin Walters
dac5ccc76e Improve progress output
This rolls up several libglnx changes: https://github.com/GNOME/libglnx/pull/101

Now of course things are trickier here because we have an internal
abstraction over directly emitting to a console versus sending the
result over DBus.  Further complicating things is that some things
call into libdnf and thus *require* use of `DnfState` which does
not give us the "n items" information, versus other parts which
we implement and can do what we want.

Even *further* complicating things is that we have to take care around non-CLI
callers like Cockpit; so I didn't try to pass the "n items" over DBus, rather
just reimplemented the "insert into text" that libglnx is doing.

Anyways overall this looks better IMO for all cases.

Update submodule: libglnx

Closes: #1143
Approved by: jlebon
2017-12-14 03:32:22 +00:00
Colin Walters
a16422484e txn: Let first error win vs cancellation
Basically since we're doing internal async ops which set the cancellable on
failure, we still want the first error to win since it'll be more useful. See
the docs for `g_task_set_check_cancellable()` for more.

Closes: #1137
Approved by: jlebon
2017-12-12 19:21:07 +00:00
Colin Walters
88cb736404 daemon/sysroot: Drop internal cancellable
I believe this is a leftover vestige, and it was adding confusion when I was
debugging `rpmostree-core.c` async ops and cancellation.

Now the only cancellables in the daemon are created by transaction ops.

Closes: #1137
Approved by: jlebon
2017-12-12 19:21:07 +00:00
Colin Walters
5f03181c0a core: c++ build: Use enum-NONE value instead of zero
In the whole libdnf/C++ discussion I experimented with trying to build
rpm-ostree as C++. There's a whole ton of stuff there. I'm going to punt for
now, but let's land this one change so some progress was made.

Closes: #1141
Approved by: jlebon
2017-12-12 17:26:19 +00:00
Colin Walters
95b423afe9 Add cancel verb and DBus API
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).

Closes: #1019
Approved by: jlebon
2017-12-07 20:12:52 +00:00
Colin Walters
1c0e354571 importer: Rework API
Now that the importer *only* imports into OSTree repos, let's
clean up the API so that the `OstreeRepo` and `OstreeSePolicy`
are passed as constructor args.

Also rework things so there's only one constructor API that
steals the fd.

This is prep for adding another async import API.

Closes: #1124
Approved by: jlebon
2017-12-07 19:44:19 +00:00
Colin Walters
f8bc9a1c9d compose/dracut: Use a host tmpdir for dracut
In unified core mode, this avoids an intense spam of errors from `cp`
because `tmpfs` doesn't support the `user.` xattr namespace, and
since [this dracut commit](61c761bc2c)
dracut tries to copy all xattrs, which was just done for IMA.
There's no point to having the SELinux labels or other xattrs
in the initramfs.

The real fix here is dracut should learn to *only* copy the IMA
xattrs, or even better disable IMA enforcement for the dracut
run or something.

Closes: #1126
Approved by: jlebon
2017-12-04 15:37:16 +00:00
Jonathan Lebon
7056e6b726 daemon: Unify pkgcache with system repo
We originally needed the pkgcache to be a separate repo due to ostree's
overzealous pruning policies. The idea was to maintain multiple commits
in each pkg branch for different SELinux policies. In practice, there's
not much use in maintaining old copies and it's just easier to always
relabel on the fly. So then, the need for a separate repo completely
melts away.

This helps simplify the mental model a bit and allows us to avoid subtle
issues like #1047. Note however that the core is still capable of
handling split repos for the `--ex-unified-core` compose use case. Once
that and the jigdo work are a bit more settled, we can have a clearer
picture of how to simplify the core further.

The tricky bit is migrating the cache. When deploying, we check if a
pkgcache repo exists and migrate its refs if so. We then leave behind a
symlink to the system repo to remain compatible with older rpm-ostrees.

Closes: #1055
Approved by: cgwalters
2017-12-01 15:41:40 +00:00
Jonathan Lebon
4d72a73b80 daemon/sysroot: Port to new style
No functional changes.
Prep for unified repo work.

Closes: #1055
Approved by: cgwalters
2017-12-01 15:41:40 +00:00
Jonathan Lebon
5860897092 tree: account for ephemeral nevra string
When writing this code, I made the false assumption that the nevra
string lives as long as the pool does, i.e. as long as we have a
reference to its `DnfSack`.

In fact, they have undefined lifetimes. Notably any place in which one
calls `dnf_package_get_nevra` a lot may result in the invalidation of
previously returned nevras.

This patch ensures that we copy the string in the few places where we
are susceptible to this.

There is a related libdnf patch[1] which tightens the definition here so
that we can assume the string at least lives as long as its
`DnfPackage`. It turns out that the callsites addressed in this patch
are also those in which we would break that assumption. IOW, this patch
is needed regardless of how [1] goes.

[1] https://github.com/rpm-software-management/libdnf/pull/388

Closes: #1119
Approved by: cgwalters
2017-11-29 20:05:39 +00:00
Colin Walters
9c004e1833 Change unpacking to use a single ostree txn
Right now each ostree txn incurs a `syncfs()`; see
https://github.com/ostreedev/ostree/issues/1184
And before this patch, we were doing a txn per package import.

We can really do better in libostree - we'll fix that, but in the short term
let's use a bigger txn for every package. However, the obvious change here of
simply hoisting up the txn is that on failure for imports, we'd discard all
downloaded packages.  We fix that by changing the auto txn API to have
a `commit_on_failure` boolean, and use it in cases where we're doing
imports.

This is prep work for jigdo, where we'll be using the import path all the time.
My bigger plan is to do multithreaded imports.

Closes: #1116
Approved by: jlebon
2017-11-29 16:48:19 +00:00
Colin Walters
f58cb44fb2 txn: For --download-only, print 'changed' message if changed
I was playing with `--download-only` a bit with an eye to
having something like this be used by Cockpit/gnome-software instead
of what it's doing now, but a problem is that at the moment we
don't have a way to reflect the "changed" state back to clients.

This is a first step towards that by simply printing a different
message.

I think really to make all of this work more nicely though, including
supporting e.g. rpm database diffs, we are going to have to instead
work on the [pending deployment](https://github.com/ostreedev/ostree/issues/545)
path.  That way we'll have done the depsolve, stored repo timestamps
etc.; we'll be able to accurately show what *did* change rather than
try to recreate what will happen on the next `rpm-ostree upgrade --cache-only`.

Closes: #1118
Approved by: jlebon
2017-11-29 15:29:49 +00:00
Colin Walters
366826132a libpriv: Rename "unpacker" to "importer", split off "unpacker-core"
In the jigdo path we don't actually want to import the OIRPM literally
into ostree.  I considered adding jigdo logic into `rpmostree-unpacker.c`
but it'd be a mess as the functionality is quite logically separate
from importing.

So split off an `unpacker-core.c` file which has the bare libarchive+RPM
helpers, and rename `RpmOstreeUnpacker` to `RpmOstreeImporter`.

Closes: #1110
Approved by: jlebon
2017-11-20 20:57:33 +00:00
Colin Walters
9d4b760754 daemon,libpriv: Some porting to decl-after-stmt
Not immediate prep for anything, I'd just been looking at this
code recently.

(Also a bit of `glnx_throw_errno` porting)

Closes: #1107
Approved by: jlebon
2017-11-17 16:27:53 +00:00
Colin Walters
7ab8869cdd tree-wide: s/hif/dnf/
No real particular reason, just decided to do this while the outstanding patches
don't conflict too much.

Closes: #1106
Approved by: jlebon
2017-11-17 14:46:57 +00:00
Colin Walters
7ea2a5afce (refactor) Move pkgcache helper function to sysroot-core
Trying to drain the `util.c` file; this one lives more logically
there.

Closes: #1086
Approved by: jlebon
2017-11-08 03:11:36 +00:00
Colin Walters
8b4c5e0901 tree-wide: Use glnx_close_fd()
It's cleaner, and we get `EBADF` checks, etc.

Closes: #1085
Approved by: jlebon
2017-11-08 02:50:12 +00:00
Colin Walters
a42c45f726 lib/core: Synthesize tmpdir for commit if no base provided
The `ex container` path was already doing this - lower the logic
into the core so I can use it for the jigdo path.  I think it's just
clearer as well even for the sysroot pkglayering path.

Closes: #1083
Approved by: jlebon
2017-11-08 02:12:52 +00:00
Colin Walters
170f74c282 lib/core: Add API to set devino cache
Rather than passing it to the assemble functions.

Prep for jigdo work.  I'm trying to move more logic
from the system upgrader path to the core, since some of
it is needed for jigdo and "unified core".

Closes: #1083
Approved by: jlebon
2017-11-08 02:12:52 +00:00
Ruixin Bao
27de772c4f libpriv/vmcheck: support replace option for karg command
Refactor the similar code portion from delete option into
a new function, and reuse it in the replace logic.

Add the replace functionality to allow kernel arguments
replacement. Now support two different types of replacement:

1: replace by key=value when there is only one key existing
e.g we have 'hi=hello' as our one of the arguments (hi only has
one value associated with it) , and 'rpm-ostree ex kargs --replace hi=new'
will replace the value and change hi=hello into hi=new.

2: replace by key=value=new_value for all other replacements
( this will work for swapping single value pair too !)

Some tests for rpm-ostree ex kargs --replace are added
for future regression.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Ruixin Bao
09cf438f04 vmcheck: Add tests for rpm-ostree ex kargs delete and append
Tests are introduced in this commit to test the basic
functionality for rpm-ostree ex kargs --append,
and rpm-ostree ex kargs --delete.

Those tests are added for future regression.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Ruixin Bao
50863c370d daemon: add handlation for kargs append
API functions from ostree-kernel-args.c
are copied to libpriv. The append functionality
reuses  _ostree_kernel_args_append_argv() for
collecting added kernel arguments.

Also added handlation in rpm-ostree upgrader
to allow deployments happen with kernel arguments.

Now, the user is able to add kernel arguments via
'rpm-ostree ex kargs --append key=value'
or 'rpm-ostree ex kargs --append key' if they
want to have an empty value with key.

The user is also able to display the current
kernel arguments via 'rpm-ostree ex kargs'

In addition, this functionality will create a pending deployment,
and will update the conf file in /boot/loader/entries/ostree-$osname-0.conf upon success.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Ruixin Bao
fa593afd89 app/daemon: add groundwork for rpm-ostree kargs command
This is initial groundwork for https://github.com/projectatomic/rpm-ostree/issues/594.

This commit sets up most of the required
front end logic( arg parsing, transaction handling), and will
be used in the following commits.

There is nothing really fancy in this commit, as most of the code
shares the similar style between other dbus related commands.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Colin Walters
9ec45d3c89 daemon: Add commit metadata to deployment properties
In another PR I did the manual bridging of commit metadata to deployment
property, but that's annoying. Let's just bridge all commit metadata.

Closes: #1069
Approved by: jlebon
2017-10-25 03:37:05 +00:00
Jonathan Lebon
c107a05b8e tree: pass NULL to glnx_fstatat_allow_noent when needed
Now that libglnx allows it, we can skip declaring a `struct stat` we
don't actually need just to check if a file exists.

Closes: #1066
Approved by: cgwalters
2017-10-17 20:48:13 +00:00
Colin Walters
94527ab69c tree-wide: Use glnx_autofd
It's a better name.  Already ported ostree.

Closes: #1063
Approved by: jlebon
2017-10-17 16:01:34 +00:00
Jonathan Lebon
4d1b5b4636 libpriv/core: use cached repodata checksum for goal
An interesting behaviour arises from the new `--cache-only` option. We
would essentially consider two goals with the same exact pkglist as
different depending on whether a package came from a yum repo or our
pkgcache repo. This meant that `rpm-ostree upgrade` followed by
`rpm-ostree upgrade -C` was never detected as a no-op and always
resulted in a relayer.

This was due to the fact that the checksum libsolv calculated was based
on our chopped off pkgcache meta-RPMs. This of course won't match the
checksum from the repodata, which is of the whole RPM.

Thankfully, we already save the repodata checksum when unpacking RPMs.
This patch simply detects the case where we're trying to checksum a pkg
originating from the pkgcache, and reads in the saved repodata checksum
rather than the wrong libsolv-derived one.

Closes: #1049
Approved by: cgwalters
2017-10-16 17:49:51 +00:00
Jonathan Lebon
e49f7cdd81 app: add --download-only option
To complement the new `--cache-only` option, add a `--download-only`
option. This does exactly what it says: we download the ostree, download
and import packages, but don't actually commit & deploy. This can be
used to effectively prime a follow-up `--cache-only` operation that can
be done during a more convenient/safer maintenance window.

I debated naming the two options `--pull-only` and `--deploy-only` like
the ostree equivalents. Though "pull" felt like the wrong word given
that it's associated more with ostree pulling but rpm-ostree also
downloads & imports RPMs. As for `--deploy-only` vs `--cache-only`, it
seems like `--cache-only` is a more accurate description of the
functionality (i.e. rather than describing an action, it describes a
mode). I also considered `--no-download` to make the synergy with
`--download-only` more obvious. Maybe that's better? Naming is hard...

Closes: #713

Closes: #1049
Approved by: cgwalters
2017-10-16 17:49:51 +00:00
Jonathan Lebon
b811eb61c0 app: support full offline operations with --cache-only
As Colin mentioned in #1035, the new `--cache-only` implemented only the
rpmmd half of the story. Here we complete that story by also ensuring
that when in cache-only mode, we don't download new ostree data nor new
packages. We try to complete the requested operation with what we have.

To do this, we add support for the same `SYNTHETIC` pull that was added
in ostree[1] so that we don't actually pull, but still perform timestamp
checking.

On the pkgcache side, we disable all remote repos and instead insert all
our cached RPMs into the `DnfSack`. Care is taken to still perform
SHA256 verification for local pkg installs/replacements.

[1] https://github.com/ostreedev/ostree/pull/642

Closes: #687

Closes: #1049
Approved by: cgwalters
2017-10-16 17:49:51 +00:00
Colin Walters
aea688fa01 Update libglnx
I want to use `glnx_tmpdir_unset()` in the `compose commit` PR.

Update submodule: libglnx

Closes: #1056
Approved by: jlebon
2017-10-16 13:22:30 +00:00
Jonathan Lebon
dbceb2aa51 daemon/deploy: remove duplicate call to set_origin
We already set the origin unconditionally right after this block. No
point in `dup`'ing twice.

Closes: #1053
Approved by: cgwalters
2017-10-13 15:00:25 +00:00
Jonathan Lebon
74eaa2ba59 daemon: add RPMOSTREE_DEBUG_DISABLE_DAEMON_IDLE_EXIT
The new idle exit behaviour is nice, but it makes debugging it harder
because you have to be fast enough to attach or place your breakpoints
and trigger it before it auto-exits. Add a compile-time flag that
developers can easily turn on to disable the auto-exit behaviour.

Closes: #1052
Approved by: cgwalters
2017-10-12 15:35:44 +00:00
Colin Walters
4a4f9952c2 tree-wide: Use a #define for /usr/share/rpm location
In prep for potentially changing it:
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html

Of course actually doing a transition would be harder than this, as we'd need to
add a compat symlink, and even that wouldn't quite be enough as e.g. the
"preview" code would need to learn how to follow the symlink (or just try both
locations).

In practice I think we'd need to land the code to handle both locations, let
that trickle out for e.g. 3 months, then make the switch in our treecomposes.

But, might as well make this change now; using a `#define` makes it slightly
easier to find places that need changing later.

Closes: #1048
Approved by: jlebon
2017-10-10 19:59:21 +00:00
Jonathan Lebon
4442a0c362 app: add -C flag to always use cached metadata
This is the equivalent version of `yum/dnf -C`. It goes together with
the new `makecache` command to allow completely asynchronous cache
update and usage.

Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Jonathan Lebon
8285ef2a15 app: add 'refresh-md' command
This is essentially the `dnf/yum makecache` equivalent for rpm-ostree.
To complete the picture, this goes hand in hand with the `-C`
equivalent, which is added in the next patch.

Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Jonathan Lebon
8fae56d120 libpriv/core: teach core to use deployment for yum repos
Add a function in the core that knows how to take a deployment and point
its passwd dir and libdnf's repos dir to it. We call it the
"configuration deployment", because those are configuration files which
should be inherited from the config merge, not the origin merge.

Prep for `makecache`.

Closes: #1038
Approved by: cgwalters
2017-10-05 18:00:54 +00:00
Colin Walters
95227f079a WIP: Implement ex livefs --replace
Lots of tradeoffs in this.  See the comments in the code for
more info.  WIP for now.

Closes: #1028
Approved by: jlebon
2017-10-05 13:19:20 +00:00
Jonathan Lebon
4501791dc2 daemon: fix uint negative comparison
Closes: #1036
Approved by: cgwalters
2017-10-04 21:10:55 +00:00