Commit Graph

1203 Commits

Author SHA1 Message Date
Colin Walters
3f5cc629dc lib/unpacker: Add a SKIP_EXTRANEOUS flag, use it for filesystem+rootfiles
Prep for unified core. For packages we know ship stuff we want to skip, enable
this flag.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
acbf00724d lib/unpacker: Drop ostree convention flag, always enable it
The core use case always enabled this; we exposed it on the command line, but
honestly I think it's a lot simpler if our imports always enable this. If we
need to *undo* it (for example, generating a "traditional feeling" OCI/Docker
image a la <https://github.com/projectatomic/rpm-ostree/pull/367>), we can
easily do that as postprocessing afterwards.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
5633424628 lib/unpacker: Fix theoretical error handling case
We should be using `g_propagate_error()` to ensure we handle the case where the
caller specifies `NULL` for the error.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Jonathan Lebon
e605386818 daemon: make use of the new deployment flags
Now that we have the semantics we need in libostree, let's just use that
and drop the logic here.

Closes: #942
Approved by: cgwalters
2017-08-25 18:59:24 +00:00
Jonathan Lebon
b05b9679f9 scripts: fix argument ordering
Just noticed this while trying out the unified core work. We were
passing arguments in the wrong order (or alternatively, receiving them
in the wrong order).

Closes: #952
Approved by: cgwalters
2017-08-24 21:48:14 +00:00
Colin Walters
00231446b3 app: Clarify message on repeated Ctrl-C
While we have cancellation issues¹, let's help the
user understand that we heard them the first time.

¹ <https://github.com/projectatomic/rpm-ostree/issues/897>

Closes: #948
Approved by: jlebon
2017-08-24 17:30:10 +00:00
Colin Walters
06f3a1811a lib/passwd: Port passwd checking to fd-relative
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
a4ca8c8888 app/compose: Drop GFile usage from package install path
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
addc53a707 tree-wide: Fix extra colon in error prefixes
This is a trap when doing style porting; we added a rule to ostree's `maint.mk`
to catch it but not here. Not sure I want to add `maint.mk` here yet.

Note that in newer libglnx `glnx_opendirat()` does error prefixing, so
drop the prefixing there.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
278ed216d0 tree-wide: A bit more libglnx porting
Mostly for `glnx_renameat()`.

Closes: #941
Approved by: jlebon
2017-08-24 13:33:59 +00:00
Colin Walters
3840fad28e tree-wide: Fully purge the rest of "noscripts" support
It was partially removed in 325ee354e9
from PR #873 - this removes all of the rest of the layers in
the daemon and core for this.

Note that while working on this I found it astoundingly confusing that
`rpmts_add_install()` and `add_install()` had separate orderings for `gboolean
noscripts, is_install`. That's fixed by having the latter lose the `noscripts`
flag entirely, but while we're here change `rpmts_add_install()` to use flags
for clarity.

Closes: #938
Approved by: jlebon
2017-08-23 16:31:27 +00:00
Colin Walters
4d32f736e1 lib/unpacker: Port unpack function to new style
Missed this in the tmpdir porting since it used `g_mkdtemp_full()`.
We also now have autocleanups for `OstreeRepoCommitModifier`.

Closes: #939
Approved by: jlebon
2017-08-23 15:14:53 +00:00
Colin Walters
4be8658f65 lib/passwd: Error if presented with an unknown check-passwd value
Prep for sysusers work; let's cleanly reject things we don't know.

Closes: #935
Approved by: jlebon
2017-08-23 14:51:08 +00:00
Colin Walters
5f7d860a33 app: Set GSETTINGS_BACKEND=memory early on if root
I was about to copy this bit from the daemon for subprocess work, so let's
centralize it. Further, we should invoke `setenv()` as early as possible; see
<https://sourceware.org/bugzilla/show_bug.cgi?id=15607#c2>.

Closes: #934
Approved by: jlebon
2017-08-22 21:18:08 +00:00
Colin Walters
6b3a99c3a9 lib/bwrap: Use --die-with-parent
See <https://github.com/projectatomic/bubblewrap/pull/165>; really every
bwrap use case I can think of should specify this (including ours), it's
just not the default out of conservatism.

This way if the daemon happens to e.g. SEGV it'll also cleanly `SIGKILL` any
outstanding scripts.

Was just looking at our bwrap usage for multiprocess work.

Closes: #936
Approved by: jlebon
2017-08-22 18:58:58 +00:00
Colin Walters
0428275a3c tree-wide: Port to new libglnx tmpdir API
Lots of cleanups, and this also allows us to fully port to new style in several
places.

We can drop our own `rpmostree_mkdtemp()`.

Update submodule: libglnx

Closes: #928
Approved by: jlebon
2017-08-22 17:21:37 +00:00
Colin Walters
82f95e7761 daemon/upgrader: Use new libostree timestamp-check option
Since we have a copy of this libostree code, pick up the new
changes from <https://github.com/ostreedev/ostree/pull/1055>.

Note the added test doesn't really test our logic since
we're only doing local pulls, but at least we have something.

Closes: #932
Approved by: jlebon
2017-08-22 16:08:21 +00:00
Colin Walters
ce2033c182 lib/core: Unlink to-import RPMs before unpacking
Was just reading the code. We have similar logic in libostree for individual
objects and deltas.

Closes: #933
Approved by: jlebon
2017-08-22 15:51:05 +00:00
Colin Walters
83bd3c8d13 bin/compose: Move /etc/{passwd,group} injection later
There's no reason to touch things in the target root *before*
we've even downloaded the repodata.  Further, doing so has
the potential trigger the "existing root" logic in libdnf if
it detects `/usr`.  This doesn't matter now since we're writing
to `/etc/`, but it will for later work, so let's put these bits
in the right place.

Prep for <https://github.com/projectatomic/rpm-ostree/pull/927>

Closes: #931
Approved by: jlebon
2017-08-22 01:02:52 +00:00
Colin Walters
129f95b6d8 bin/compose: Move treefile variable into self struct
Prep for a future patch.

Closes: #931
Approved by: jlebon
2017-08-22 01:02:52 +00:00
Colin Walters
b419cb9071 lib/passwd: Misc code cleanup and style porting
I was briefly thinking about sysusers work, and decided to do some quick cleanup
here.

Closes: #929
Approved by: jlebon
2017-08-21 16:59:46 +00:00
Colin Walters
f51611dc74 tree-wide: Some misc libglnx porting
Port away from `glnx_set_error_from_errno` to prefixing errors or libglnx
wrappers as appropriate.

Closes: #926
Approved by: jlebon
2017-08-18 17:38:49 +00:00
Colin Walters
fe1be77025 sysroot: Prefix error when loading deployment with index
This would have helped me debug <https://github.com/projectatomic/rpm-ostree/issues/923>.

Closes: #924
Approved by: jlebon
2017-08-17 16:53:48 +00:00
Colin Walters
8bb604ce67 tree-wide: Port to ostree_repo_{open,create}_at()
A lot of code gets nicer.

Closes: #922
Approved by: jlebon
2017-08-17 15:28:14 +00:00
Colin Walters
1f0a76985e daemon: Use journal over syslog
systemd has to go to a bit more effort to integrate `syslog()`, let's
use the journal directly for consistency.  Just happened to be
reading the code.

Closes: #921
Approved by: jlebon
2017-08-17 13:47:33 +00:00
Jonathan Lebon
ef889b0b3a daemon: use new journal-msg signal
Make use of the new journal-msg signal that OstreeSysroot emits. Since
we now also only rely on the output API to print messages, this allows
us to drop the outrageous hacks we had to do to watch our own stdout,
part of which Coverity didn't like.

Coverity CID: 163694

Closes: #919
Approved by: cgwalters
2017-08-16 16:05:06 +00:00
Jonathan Lebon
1fb9dc57c9 daemon: ensure we consistently use output API
There were places where we were relying on the stdout self read hack to
relay messages to clients. Fix those by explicitly using the output API
in all spots where the daemon may pass. Really, we should exclusively be
using the output API in the daemon and libpriv trees, though many other
instances exist for now.

Closes: #919
Approved by: cgwalters
2017-08-16 16:05:06 +00:00
Jonathan Lebon
7b774cee7e output: add support for messages
This is already supported by the D-Bus API, though not in our output
wrapper.

Closes: #919
Approved by: cgwalters
2017-08-16 16:05:06 +00:00
Jonathan Lebon
35d69c1577 container-builtins: squash a -Wmaybe-uninitialized
Closes: #920
Approved by: cgwalters
2017-08-16 15:48:49 +00:00
Colin Walters
b6705f3feb daemon: Check for updated rpms when upgrading
This closes a longstanding bug - since package layering first
landed, we only checked for newer RPMs if the base tree changed.
In some scenarios like RHELAH, this doesn't matter much by default
since they move at the same cadence.  Except if you use EPEL for example.
In Fedora, today the FAH releases are async of the rpm-md repos, and
there's also COPR which can update more than once a day even.

We should check for both update sources. Luckily we'd already introduced logic
for this in the treecompose case (checksumming the depsolved package sack). We
just need to start using it for client side assembly too.

Closes: https://github.com/projectatomic/rpm-ostree/issues/391

Closes: #911
Approved by: jlebon
2017-08-16 15:33:11 +00:00
Jonathan Lebon
8dddefbad3 unpacker: properly check for glib error
`g_mkdtemp_full` returns a pointer to the template var itself or NULl on
error. We were treating it like an integer.

Coverity CID: 163698

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Jonathan Lebon
8ee1385492 scripts: check var before use
We were using the `bwrap` var before checking it for `NULL`.

Coverity CID: 163704

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Jonathan Lebon
b4411f5fc0 core: log error to journal
I hesitated on this one. Really, we *could* just explicitly cast the
result to mark it as used, though OTOH, we really should at least drop
something in the journal. It's not fatal, but it shouldn't happen so we
should know about it.

Coverity CID: 163717

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Jonathan Lebon
7553b0000b livefs: comment out dead code
The `replacing` var was already checked to be FALSE higher up.

Coverity CID: 163689

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Jonathan Lebon
cf410211d2 dbus-helpers: initialize bus name
In the event that we're not on a message bus, we need to make sure we're
passing NULL.

Coverity CID: 163684

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Jonathan Lebon
c59c11f789 container-builtins: initialize fds to -1
1. We weren't initializing all fds to -1, but 0 is a valid fd number.
2. We were checking for nonzero rather than explicitly for -1.

Coverity CID: 163691
Coverity CID: 163700
Coverity CID: 163714

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Ruixin
534fc30308 app: provide command description in a better place
Before, when using rpm-ostree -h command, the command
description was shown together with the command itself

Now, we separate the command description out, providing
user a better view of the description.

Closes: #916
Approved by: jlebon
2017-08-15 13:09:09 +00:00
Ruixin
a2b2758e0f app: add more rpm-ostree -h output
This is brought up by https://github.com/projectatomic/rpm-ostree/issues/806.

When user types in rpm-ostree [subcommands] -h, the user can now see
more useful information for the subcommands.

Closes: #916
Approved by: jlebon
2017-08-15 13:09:09 +00:00
Colin Walters
ee1613e4bf livefs: Stop changing immutable bit on deployment roots
This is no longer necessary since we switched to tracking livefs state in the
origin, and we don't support changing content outside of `/usr` right now
via livefs.

Just noticed this while looking at the livefs code.

Closes: #912
Approved by: jlebon
2017-08-10 13:58:41 +00:00
Colin Walters
f41183e0e5 app/ex-container: Port to new style
Was just looking at this code as part of unified core work. It was an easy port.

Closes: #913
Approved by: jlebon
2017-08-10 13:39:08 +00:00
Colin Walters
0ff4403253 bin/compose: Port to new code style
Move basically everything except early option
processing into a helper function.

Also update a few other misc bits above that to new style.

Closes: #904
Approved by: jlebon
2017-08-08 17:09:13 +00:00
Colin Walters
162b8d4898 bin/compose: Silently ignore --workdir-tmpfs
Down the line when we have a unified core, this will be totally unnecessary.
Even without that, it's not worth the maintenance pain for us. Anyone who wants
this can easily on their own mount a tmpfs and point us at it with --workdir.

Prep for further code cleanup.

Closes: #904
Approved by: jlebon
2017-08-08 17:09:13 +00:00
Colin Walters
39e36db741 postprocess: Port a bit more to new style
Was looking at this file to change how we do kernels, and noticed these two were
straightforward ports.

Closes: #909
Approved by: jlebon
2017-08-08 16:24:45 +00:00
Colin Walters
787c880b64 bin/rebase: Add -b and -m options
The rebase command syntax has confused people a lot.  Let's follow
git here and add a `-b/--branch` option and encourage people to use
that.  The case of switching remotes is `-m/--remote`; it's definitely
unfortunate that `-r` is already taken for `--reboot`.

One thing I'm a little bit unhappy about is how we're doing logic
on the client side here.  Changing the DBus API for this would
also be awkward though.

Closes: https://github.com/projectatomic/rpm-ostree/issues/886

Closes: #890
Approved by: jlebon
2017-08-08 16:02:15 +00:00
Colin Walters
9b22c2c516 scripts: Journal execution of %post/%posttrans, and times for all scripts
The script delay is annoying; some of them take over half a second here. This
helps with debugging. Another enhancement we could make after this is aborting
scripts that appear to be hung after e.g. 30 seconds, and logging that.

Closes: #905
Approved by: jlebon
2017-08-03 16:58:52 +00:00
Jonathan Lebon
dc24dd3105 rollback: allow users to undo a rollback
The new API to find pending and rollback deployments do so relative to
the booted deployment. This caused an interesting behaviour: the first
time a user uses "rpm-ostree rollback", it would (as expected) move the
previous deployment first. but the second call to "rpm-ostree rollback"
would fail since there were now no more rollback deployments.

We fine tune the logic here to allow this, as well as the more general
case of putting the booted deployment back on top.

This fixes a subtle regression from b7cf58e
(https://github.com/projectatomic/rpm-ostree/pull/767).

Closes: https://github.com/projectatomic/rpm-ostree/issues/906

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Jonathan Lebon
d9c1e4a5d3 daemon: use upstreamed function to find rollback
Now that the query_deployment() function was upstreamed in ostree (see
https://github.com/ostreedev/ostree/pull/897), let's just use it from
there and drop our copy here.

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Colin Walters
0515f59d70 bin/compose: Use autocleanups for context structure
Prep for more code cleanup here and porting to new style.  We
had a mix of owner pointers.  Fix this by consistently having
the `self` struct own the data.

Closes: #903
Approved by: jlebon
2017-08-02 17:41:43 +00:00
Colin Walters
08e3f5de7f lib: Log task output for pre/post scripts
Some of the scripts can be expensive (`shared-mime-info` for example), and we
appear to be hanging for at least a few seconds even on fast hardware. I'd like
to have a lot more logging here...potentially something like showing
individual package names live in the terminal, but this is a start.

Closes: #902
Approved by: jlebon
2017-08-02 17:16:28 +00:00
Ruixin
53c39632d4 Deployment: add endoflife
When commit metadata contains ostree.endoflife attribute,
its information will be added to the deployment Variant,
which will later be shown as a red & bold message when
'rpm-ostree status' command is called.

A test is added for future regression

Closes: #889
Approved by: cgwalters
2017-08-02 13:53:10 +00:00
Colin Walters
0d4d6be94f Implement file triggers (%transfiletriggerin) for layered pkgs
File triggers are a post-RHEL7 thing; more information at
http://rpm.org/user_doc/file_triggers.html

There are two notable users I've been testing this with;
`glib2` and `vagrant`.  The `vagrant` one is more immediately urgent,
since it makes `vagrant-libvirt` work, which I currently rely on
for my workstation dev.

I've tested things successfully with `vagrant`, and I did verify that we run the
`glib2` ones when doing `rpm-ostree ex container`.

Long term, more transaction file triggers are likely to live in
"base" packages like `glib2`.  We don't implement those yet, but
extending this to do that shouldn't be too hard.

There was *significant* what I'd call reverse engineering of the
implementation in librpm.  The file triggers code there is spread out
and abstracted in a few different places in the code.  I found
trying to understand what header values were involved to be quite
tricky.

There are some corner cases like multiple patterns that I *think*
this does correctly, but could use more validation.  The main
question I had was - is it required that the patterns for e.g.
`%transfiletriggerin` and `%transfiletriggerun` be identical?

Closes: https://github.com/projectatomic/rpm-ostree/issues/648

Closes: #869
Approved by: jlebon
2017-07-27 20:58:09 +00:00
Jonathan Lebon
d4effe8f47 dbus: don't register if not active
Follow-up tweak to #894. Make the client smarter so we only register
when we know we can. We could be more sophisticated here and e.g.
introduce the concept of "read-only" clients in the daemon to only allow
access to non-mutating methods, though let's delay that discussion at
least until the daemon learns to auto-exit.

Closes: #898

Closes: #900
Approved by: cgwalters
2017-07-27 20:34:22 +00:00
Colin Walters
34b5a004a8 daemon: Add a sanitycheck(/bin/true) before we deploy a tree
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.

We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).

There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.

Closes: #892
Approved by: jlebon
2017-07-27 17:58:58 +00:00
Jonathan Lebon
577aef98e2 app: spawn polkit authentication agent
Spawn pkttyagent when trying to call a method that may require
authentication to give users a chance to provide auth right from the
terminal.

Since we're now relying on polkit for authorizing most of the OS
interface methods, let's drop the root check on those.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
21109e18ae dbus: drop root check on RegisterClient()
Drop the root check for RegisterClient() so that any UID can use it to
then safely invoke methods. UIDs which already have an active session do
not need to authenticate for RegisterClient().

In order to implement this, we move the polkit authority setup into the
RpmostreedSysroot object and enable polkit on the interface.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
3ade73dedd daemon: factor out uid fetching
Make a method specialized in fetching the UID of a D-Bus client.

Prep for polkit tty auth agent.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Colin Walters
5763027b5f Update libglnx, do some porting to new APIs
In particular I wanted to start using `GLNX_AUTO_PREFIX_ERROR`.

Update submodule: libglnx

Closes: #893
Approved by: jlebon
2017-07-21 22:37:17 +00:00
Colin Walters
2082b3f8fb lib: Add version macros and version checking function
The version checking function in particular is really useful for people doing
`from gi.repository import RpmOstree`, which we'd like at least some things like
Anaconda and Pungi to do.

Closes: #891
Approved by: jlebon
2017-07-21 20:35:26 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
4222407657 core/scripts: Cleanup refactor
No functional changes; the main thing is to change the "ignore" case to an early
return, and also introduce a "medium level" function that does argument parsing.

Prep for file trigger work.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
325ee354e9 core/scripts: Drop support for ignore-scripts
This was a transient hack that I don't think we need any longer.
Since then we've now committed to e.g. ignoring `%pretrans` etc.,
and we also have the hardcoded blacklist.

It never really worked in a complete sense since one would have to re-specify
the ignore on every operation, which was not sustainable.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
f38bb61f91 Update libglnx
And port the passwd util.

Update submodule: libglnx

Closes: #867
Approved by: jlebon
2017-07-18 16:43:40 +00:00
Matthew Barnes
c7024d69ae daemon: Avoid "Updating from: (null)" message
Check for a NULL refspec before emitting message in
package_diff_transaction_execute().

Closes: #876
Approved by: cgwalters
2017-07-13 20:14:31 +00:00
Jonathan Lebon
50ad253ebe codebase: partial porting to new style
Just some minor porting of the less glamorous parts of the codebase.

Closes: #865
Approved by: cgwalters
2017-07-07 20:03:28 +00:00
Jonathan Lebon
f1437028cf core: only update repos enabled for packages
We don't care about repos enabled only for METADATA. This works around
an interesting libdnf behaviour in which all repos in redhat.repo are
enabled for metadata, even if disabled.

Downstream RHBZ: #1468302

Closes: #863
Approved by: cgwalters
2017-07-06 18:59:16 +00:00
Colin Walters
f599f4bc7c libpriv: Some style porting
Scattered bits, just keeping up some momentum.

Closes: #861
Approved by: jlebon
2017-07-06 14:29:38 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
7b4134c8e6 core: make cachebranch finder function public
Prep for replacement overrides.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
63405d655b core: plug leak from ignore_scripts hash table
Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
2da68cb510 sysroot-core: minor style updates
This function was already mostly in the new style. Just moving the
declarations closer to where they are used. No functional changes. The
logic will be tweaked in an upcoming commit.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Colin Walters
1f33f42c7a scripts: Port some to new style, minor cleanup
Prep for further work.

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Colin Walters
137271efee libpriv/passwd: Port to new style
All straightforward. Just chipping away at the porting.

Closes: #851
Approved by: jlebon
2017-06-26 18:03:12 +00:00
Colin Walters
c7c89e8735 status: Use more concise GPGSignature without --verbose
Since it takes up a lot of room and should be something that's just always
working.

Closes: https://github.com/projectatomic/rpm-ostree/issues/842

Closes: #848
Approved by: jlebon
2017-06-26 15:51:46 +00:00
Colin Walters
4d72f8d787 cmdline: Stop printing GPG results during pulls
It's really verbose for deploy, and in general, let's follow
the Unix philosophy of silence = success.

Closes: https://github.com/projectatomic/rpm-ostree/issues/843

Closes: #850
Approved by: jlebon
2017-06-26 14:36:41 +00:00
Jonathan Lebon
453e9ca570 rpm-util: factor out get_by_pkgname
This will be needed multiple times in the future.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
3ae90ce5fc upgrader: make metadata tmp dir part of upgrader
I initially planned to use the metadata tmp dir for another reason than
just in finalize_overlays(), but I will not anymore. Regardless, this
seems like a nice patch which also allows us to complete the style
conversion in this file.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
853737f667 unpacker: encode NEVRA info in commit metadata
It seems silly that to find out more detailed information about the
NEVRA of a cached pkg, we have to resort to write out the header to
disk, then reading it back in with librpm in order to tease out the info
we want. Let's just encode that information directly in the commit
metadata and provide a helper to fetch it.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
4507c581cf core: factor out pkgcache nevra to branch lookup
This will be useful in the future for another function.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
7b6237c14f postprocess: prefix more errors
This is a follow-up to #845. Judging from the output of the erroring
compose[1], the error seems to happen after committing all the objects.
Add more detailed prefixes there.

[1] https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/thread/2MZ5KFEVITKWJZYOTSAYCZ7XYAJCWZCT/

Closes: #846
Approved by: cgwalters
2017-06-23 15:48:05 +00:00
Colin Walters
64b049b8eb postprocess: Style porting
Prep for debugging: https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/thread/2MZ5KFEVITKWJZYOTSAYCZ7XYAJCWZCT/
Almost everything is converted in this file now.

Closes: #845
Approved by: jlebon
2017-06-23 13:50:57 +00:00
Colin Walters
675066a9c7 daemon: Log caller uid
This is an extension to the previous change to distingush between
"caller" and "client".  Now for clients we log the uid (both
in the message and structured).

This is a natural followon from the polkit work, since now different
uids can invoke us.

Closes: #838
Approved by: jlebon
2017-06-22 21:17:10 +00:00
Colin Walters
7d3b72c814 Remove all tabs ⭾ in *.[ch] and add a CI check for them
There weren't many left, and I happened to have one of these files open and it
was distracting.

Closes: #839
Approved by: jlebon
2017-06-22 01:20:55 +00:00
Colin Walters
46f4b62f3b daemon: Differentiate in logs between "client" and "caller"
A "caller" is a bus address that hasn't called `RegisterClient`.

Prep for things like unifying the bus name tracking between the transaction and
the daemon; right now it's a bit silly that the txn code also monitors the
caller's bus name. So down the line we could change that code to call into the
daemon and use its tracking for registered clients.

Closes: #837
Approved by: jlebon
2017-06-20 21:50:39 +00:00
Colin Walters
009e1f78d6 daemon: Render txn data a bit more nicely
Separate the different fields. Prep for adding more metadata like the caller's
uid.

Closes: #837
Approved by: jlebon
2017-06-20 21:50:39 +00:00
Jonathan Lebon
0b1c5eda17 override remove: allow inactive removals
The property of removal overrides dropping out if the package was
removed from the base layer felt a bit too magical and hacky. We really
should remember that wish and re-apply it if the pkg comes back. This is
similar to package layering: requests can become inactive (seems like a
better word than "dormant") if the package is already part of the base
layer, but they don't really go away.

This patch reworks the logic so that removal overrides work the same
way. In the status output, we now have both "RemovedBasePackages" and
"InactiveBaseRemovals" (which is only printed in verbose mode),
similarly to how we have "LayeredPackages" and "InactiveRequests". And
similarly, we also print out in the upgrader during a transaction all
the inactive base removals.

Another cool thing is that we now allow any pattern to be specified at
the CLI. E.g. `ex override remove /usr/bin/strace` will resolve to
strace.

Closes: #836
Approved by: cgwalters
2017-06-20 21:24:05 +00:00
Jonathan Lebon
61560e0686 status: rename RequestedPackages to InactiveRequests
In the JSON output, the "requested-packages" holds *all* the requests
for package layering, whereas the "packages" key holds only those
requests which actually caused a package to be layered.

When outputting RequestedPackages, we try to be nice and print only the
"dormant" packages, i.e. those requests which did not require any
packages to be layered since the base layer already fulfilled them. This
is the subset of "requested-packages" not also in "packages". But then,
we could call this subset something much more appropriate than
"RequestedPackages" when outputting.

This patch renames it to "InactiveRequests" to reflect the "dormant"
aspect of them. We also make it only be output in verbose mode, since
it's not as interesting as what was *actually* layered when trying to
understand what the host is currently running. But we do adapt the
upgrader so that it prints out inactive requests during transactions.

Closes: #836
Approved by: cgwalters
2017-06-20 21:24:05 +00:00
Jonathan Lebon
4ffe12a6af libglnx: bump for GLNX_HASH_TABLE_FOREACH macros
Example of what the new macros look like when applied on a whole
codebase. It definitely feels much nicer to work with!

Update submodule: libglnx

Requires: https://github.com/GNOME/libglnx/pull/55

Closes: #834
Approved by: cgwalters
2017-06-20 14:09:24 +00:00
Kalev Lember
368b1c36f6 Don't use polkit when running on the session bus
This is the case when we're running self tests.

Closes: #825
Approved by: cgwalters
2017-06-19 21:19:42 +00:00
Kalev Lember
84f6bcb4f0 Add polkit support
This allows non-root users access to the rpm-ostree daemon, which is
a pre-requirement for gnome-software rpm-ostree support.

Closes: #745

Closes: #825
Approved by: cgwalters
2017-06-19 21:19:42 +00:00
Jonathan Lebon
c1e6685767 upgrader: fix missing goto out
Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
782edff91a deploy transaction: factor out local RPM importing
This will be useful in the future for local replacement overrides.

Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
f0ed8cdb88 rpm-utils: fix pkglist printout
Noticed this while doing composes recently.

Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
6b8f0f6b23 core: fix error message construction
In the error path when trying to remove a base package, we would try to
print a DnfPackage as char*, which of course didn't result in any
coherent output.

Closes: #833
Approved by: cgwalters
2017-06-19 15:19:59 +00:00
Jonathan Lebon
f5227c8d51 postprocess: add explanatory comment re. /var/run
Include the findings from
https://github.com/projectatomic/rpm-ostree/pull/831 as a comment for
future investigations.

Closes: #832
Approved by: cgwalters
2017-06-16 13:33:49 +00:00
Jonathan Lebon
57cf8d9bc0 postprocess: always nuke /var/run
If `/var/run` exists, it will be a dir, so we'll fail the `unlinkat`
with `EISDIR`. Let's just explicitly nuke it.

Closes: #831
Approved by: cgwalters
2017-06-15 19:27:14 +00:00
Jonathan Lebon
943138b132 postprocess: stronger handling for sepolicy in /var
We shouldn't just check that the "targeted" dir exists, but rather that
the actual directory where the modules are stored exists. This fixes a
regression on RHEL in which the new selinux-policy-targeted lists some
%ghost files under /var/lib/selinux and as a result think that the
policy is in /var.

Closes: #831
Approved by: cgwalters
2017-06-15 19:27:14 +00:00
Jonathan Lebon
4a5e9c3086 postprocess: use glnx_throw helpers to add details
Closes: #831
Approved by: cgwalters
2017-06-15 19:27:14 +00:00
Colin Walters
9f671efc60 treecompose: Add various g_prefix_error()
To help debug a problem which turned out to be in SELinux postprocessing for
RHELAH.

Closes: #830
Approved by: jlebon
2017-06-15 17:40:33 +00:00
Jonathan Lebon
4a61e55663 deploy: use consistent argument names
This is just a cosmetic patch that ensures we use the same names for the
same things everywhere to make it easier to follow along.

Closes: #829
Approved by: cgwalters
2017-06-13 13:25:24 +00:00
Jonathan Lebon
4ee57a6dbd rpmostree-dbus-helpers: generalize pkg sorting
Minor prep for replace overrides.

Let rpmostree_sort_pkgs_strv work on an existing fd list rather than
creating its own, since it may be called multiple times on different
pkgs arrays.

Factor out the logic that calls to sort and inserts the values in the
final modifiers variant dict to prepare for being able to do this
operation multiple times.

Closes: #829
Approved by: cgwalters
2017-06-13 13:25:24 +00:00
Jonathan Lebon
1331ac837d deploy transaction: append to txn title in piecemeal
Only add information to it re. installing/uninstall/local installs after
we check that the corresponding arrays are not `NULL`.

Closes: #826

Closes: #827
Approved by: cgwalters
2017-06-09 20:12:36 +00:00
Jonathan Lebon
afcd99c1bf status: don't use autoptr() on generated objects
This would make a hard dependency on glib > 2.50, which we're not ready
for yet.

Closes: #824
Approved by: cgwalters
2017-06-08 21:14:58 +00:00
Jonathan Lebon
e6a86c1c8e util.h: add helper macro for proper English pluralization
This is to assuage my pet peeve of bad grammar output. It's of course
English-centric, and would require some tweaking if/when we support
internalionalization systems like gettext. Though I don't think it
makes it more difficult to do so in the future.

Closes: #824
Approved by: cgwalters
2017-06-08 21:14:58 +00:00
Jonathan Lebon
cba24435b0 core: don't use rpmfiles if missing
Starting from v4.12, rpmteFI has been deprecated in favour of
rpmteFiles. Make use of it if we can, otherwise fall back to the older
API.

Closes: #824
Approved by: cgwalters
2017-06-08 21:14:58 +00:00
Colin Walters
1977e3cfba scripts: Unlink our temporary post scripts
I just happened to do an `ls /usr` on my workstation. Sigh 😣

Closes: #822
Approved by: jlebon
2017-06-08 15:55:57 +00:00
Colin Walters
5da409fcad upgrader: Add user-inaccessible private dir for rootfs checkouts
This is part of the saga of permissions and checkouts that
came about thinking about flatpak, but suid and world-writable dirs
are also an issue for us.

There's no reason to make suid binaries accessible temporarily
to users while we're computing a new root.  Similarly, we don't
want anyone to actually *write* to our temporary `/tmp`.  The
simple fix is to make an intermediate dir that's `0700`.

See: https://github.com/ostreedev/ostree/pull/909
See: https://github.com/flatpak/flatpak/pull/843

Closes: #821
Approved by: jlebon
2017-06-08 15:24:02 +00:00
Colin Walters
69cc3a13a2 upgrader: Remove duplicate #defines
These were moved to `sysroot-core.h`, but the originals weren't
deleted.  Do so now.  Prep for more work here.

Closes: #821
Approved by: jlebon
2017-06-08 15:24:02 +00:00
Jonathan Lebon
4e2936f085 origin: avoid double lookup on hash table removals
More efficient *and* prettier! So much win!

Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Jonathan Lebon
cde3295e26 app: add experimental support for pkg removals
This is one more step towards making rpm-ostree more powerful in its
quest to be the ultimate *hybrid* image/package system. Package layering
allows us to add packages on top of the base package set received from
the content provider. However, we're not able to remove or replace
packages in the base set itself.

This patch introduces a new `override` command, which is for now nested
under the experimental `ex` command. The `override` command will allow
users to modify the base package set itself. The first implemented
subcommands are `remove` and `reset`.

A stub has been provided for the more useful `replace` subcommand,
though much of the needed logic for that operation are implemented in
this patch as part of the `remove` subcommand.

Part of: https://github.com/projectatomic/rpm-ostree/issues/485

Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Colin Walters
7d1424e0c7 compose: Error out early if treecompose-post isn't executable
Closes: https://github.com/projectatomic/rpm-ostree/issues/817

Closes: #818
Approved by: jlebon
2017-06-05 19:18:20 +00:00
Jonathan Lebon
a6c9b6213c origin: drop GCancellable arg and rename function
1. There's no point in passing a GCancellable in those cases. All the
   manipulations should be short-lived.
2. Rename delete_packages to remove_packages, since "remove" is the
   proper antonym of "add".

Closes: #819
Approved by: cgwalters
2017-06-05 18:47:14 +00:00
Colin Walters
02f1fa678e daemon: Plug leak of AddMatch rules for RegisterClient
I noticed this when I set up a `while true; do rpm-ostree status; done` loop
to test: https://github.com/projectatomic/rpm-ostree/pull/814
This is a local, authenticated DoS of sorts, but there are honestly a lot of
those, IMO not worth a CVE.

Closes: https://github.com/projectatomic/rpm-ostree/issues/815

Closes: #816
Approved by: jlebon
2017-06-02 17:57:14 +00:00
Colin Walters
e5294829b8 Add human-readable Title to transaction, render in status
There's a lot that could be done to improve this; we're not setting a title for
`rollback` etc. But I think in practice right now the "deploy" path (which
includes upgrade/install) etc. is most important.

Re-synthesizing a human readable string here is definitely a bit fragile and
going to be a maintenance pain. One thing I debated is having the client send
its commandline as a string. But that would only work for `/usr/bin/rpm-ostree`,
not e.g. Cockpit.

Anyways for now, this is useful and we can always improve it later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/454

Closes: #814
Approved by: jlebon
2017-06-02 17:27:10 +00:00
Colin Walters
d5a493f8b1 Update libglnx
This adapts us to the new dfd API.

Update submodule: libglnx

Closes: #813
Approved by: jlebon
2017-05-31 20:56:27 +00:00
Jonathan Lebon
cdd4b4148a core: drop unused argument to assemble_tmprootfs
Closes: #808
Approved by: cgwalters
2017-05-31 15:19:09 +00:00
Jonathan Lebon
af982ff156 livefs: fix nested continue bug in optimization
There's a subtle but classic issue in this code: calling `continue`
really just continued the inner loop, whereas we meant continuing the
outer loop.

Fix this by making the prefix lookup check a proper predicate function.
Place that function somewhere public, because we'll make use of it in
base overrides as well.

Closes: #807
Approved by: cgwalters
2017-05-31 14:47:54 +00:00
Jonathan Lebon
b800fae0cc upgrader: refactor to improve legibility
At this point, there are many helper functions that really should just
be made upgrader-aware to make it easier to follow. For example, now
both find_missing_pkgs_in_rpmdb() and generate_treespec() are passed the
Upgrader object directly, allowing us to avoid having to thread through
multiple arguments everywhere. Another example is that the Upgrader now
carries the list of final packages that need to be overlaid rather than
passing it around.

We also rename the functions to better reflect what they actually do.

Closes: #804
Approved by: cgwalters
2017-05-30 16:42:59 +00:00
Jonathan Lebon
33935173fa upgrader: drop unnecessary malloc for tmprootfs
Since we now use a predictable directory for the rootfs checkout and
assembly operation, there's no need for a tmprootfs variable anymore.

Closes: #804
Approved by: cgwalters
2017-05-30 16:42:59 +00:00
Jonathan Lebon
5735447f2a origin: add helper to check if local assembly needed
It's less cumbersome to check for this in the origin. Factor it out
there so that it's easier to modify it in the future.

Closes: #804
Approved by: cgwalters
2017-05-30 16:42:59 +00:00
Jonathan Lebon
3fea05d35f upgrader: factor out sack_has_subject
In preparation for using it outside the upgrader.

Closes: #804
Approved by: cgwalters
2017-05-30 16:42:59 +00:00
Jonathan Lebon
c491114720 core: auto-call download_metadata()
In all cases, we always do download_metadata() followed by prepare().
Let's just make life easier for callers by inlining this in prepare(),
much like what libdnf does. The function is still public in case some
use case calls for making them separate.

Closes: #805
Approved by: cgwalters
2017-05-30 15:17:07 +00:00
Jonathan Lebon
b804ae8ae3 core: absorb RpmOstreeInstall into RpmOstreeContext
There's not much use in having RpmOstreeInstall be separate now that
it's completely private. Let's just make it part of the main context
object rather than making callers allocate space for it and pass it to
the various context methods.

We also rename `prepare_install` to `prepare` since it will (soon) do
more than just installs (and sometimes only pure uninstalls).

Closes: #805
Approved by: cgwalters
2017-05-30 15:17:07 +00:00
Jonathan Lebon
d1608ba066 app: more conversion to new style
No big surprises there. A few goto cleanups remain in rpm-util.c
functions which use tmpdirs.

Closes: #803
Approved by: cgwalters
2017-05-30 14:47:31 +00:00
Jonathan Lebon
83aeb018c1 main: deduplicate subcommand handling
We had a lot of duplicate code that handled subcommand handling. Though
now that we have proper structs to represent commands and command
invocations, we can easily unify these.

Closes: #803
Approved by: cgwalters
2017-05-30 14:47:31 +00:00
Jonathan Lebon
fa62981693 main: delete duplicate GPG sig printing function
We already have a `rpmostree_print_signatures` function that the
`status` command uses to print GPG signatures.

Closes: #803
Approved by: cgwalters
2017-05-30 14:47:31 +00:00
Jonathan Lebon
e852bf33d6 main: make rpm subcommand a proper alias
We were manually aliasing `rpm` to `db`, but we now have a nicer way to
define legacy aliases. Use it.

Closes: #803
Approved by: cgwalters
2017-05-30 14:47:31 +00:00
Colin Walters
51ae1e080e postprocess: Some porting to new style
Not all of it, but I did a few things since the GLnxTmpfile bit
let us port one of the functions.

Closes: #800
Approved by: jlebon
2017-05-29 14:24:43 +00:00
Colin Walters
0ea649d80b Bump libglnx, port to new tmpfile API
Update submodule: libglnx

Closes: #800
Approved by: jlebon
2017-05-29 14:24:43 +00:00
Colin Walters
7aba1b91dc Move rpm-md repo status display into core
Part of: https://github.com/projectatomic/rpm-ostree/issues/774

Basically, for `rpm-ostree status` to accurately help people
understand whether or not their system is up to date, we need
to be showing repository timestamps.

Ideally, we'd change the libdnf API to support what we're doing
here better.  But, this works for now.

Closes: #791
Approved by: jlebon
2017-05-26 19:11:17 +00:00
Colin Walters
2c509a768f core: Enable repo caching (one day)
This is prep for a rework of rpm-md downloads (visualizing state better + prep
for parallelization). libdnf defaults to a week which is just too long
(particularly for async security updates).

See also the linked https://github.com/rpm-software-management/libdnf/issues/291

Closes: #791
Approved by: jlebon
2017-05-26 19:11:17 +00:00
Colin Walters
ec5f95b0e3 lib/core: Define a helper to get enabled rpm-md repos
We do this in a few places. Prep for a future patch around metadata downloads.

Closes: #791
Approved by: jlebon
2017-05-26 19:11:17 +00:00
Colin Walters
15b8dd2b7c postprocess: A few more builtin /var suppressions
And tweak the comment; I'm not blindly adding everything from
e.g. a Fedora compose here, since while in practice we omit them,
the ugliness serves as a reminder to fix the packages.

(For example, pam should use tmpfiles.d snippets for `/var/log/tallylog`,
 or %ghost it)

Closes: #795
Approved by: jlebon
2017-05-26 14:26:25 +00:00
Colin Walters
9980448464 lib/util: Add a DECLARE_RPMSIGHANDLER_RESET to pacify clang
We need to add an `__attribute((used))` to the autocleanup variable in order to
pacify its set-but-unused, so make a macro to simplify callers. This is a bit
like systemd's `PRESERVE_ERRNO` cleanup.

Closes: #794
Approved by: jlebon
2017-05-25 20:53:08 +00:00
Colin Walters
a56a98152a core: Use a single tmpdir
This way we only have one directory to `rm_rf` at cleanup time.
A next steps might be teaching libdnf to use this.  Futher, we could
consider inheriting a tmpdir from the transaction.

Closes: #792
Approved by: jlebon
2017-05-25 19:18:38 +00:00
Colin Walters
6efa2de279 core: Pass self object instead of tmpdir dfd
In various places we thread down the tmpdir dfd. But since its lifecycle is
bound to the context, we might as well pass the context itself. This just ends
up being cleaner, and is also prep for changing how we do the tmpdir.

Closes: #792
Approved by: jlebon
2017-05-25 19:18:38 +00:00
Colin Walters
6d78f3504f core: Minor refactoring prep in local pkg install
Make a single place where we generate filenames, as prep for a future patch to
change it.

Closes: #792
Approved by: jlebon
2017-05-25 19:18:38 +00:00
Colin Walters
53d259b3c3 lib: Define an autocleanup for RPM signal handling
Allows us to convert the metadata download func to new code style, and is
generally in line with our cleanup-focused philosophy on code, life,
and the meaning of the universe.

Closes: #789
Approved by: jlebon
2017-05-25 14:41:01 +00:00
Colin Walters
6210f34096 lib/core: Use autoptr to port tmprootfs_assemble() to new code style
I was going to make other changes here, and noticed the only
blocker was removed in https://github.com/projectatomic/rpm-ostree/pull/771

Closes: #787
Approved by: jlebon
2017-05-24 21:49:40 +00:00
Jonathan Lebon
86f82cf648 upgrader: switch to new code style
Splitting this out before I keep hacking on this file. Pretty
straightforward stuff. There is one legitimate use of the `goto out`
pattern left in which we clean up a tmpdir.

Closes: #788
Approved by: cgwalters
2017-05-24 21:19:49 +00:00
Colin Walters
f62c7665f7 livefs: Optimize etc merge by avoiding duplicate checkouts
This came up in review of https://github.com/projectatomic/rpm-ostree/pull/783

The implementation isn't the most beautiful thing in the world,
but we can revisit later.

Closes: #786
Approved by: jlebon
2017-05-24 18:49:29 +00:00
Colin Walters
d2bd8500da compose: Add tmp-is-dir option to make /tmp a directory
There are a few reasons to do this. First, systemd changed to refuse mounts on
symlinks, and hence if one *wants* "/tmp-on-tmpfs", one would need to write a
different `sysroot-tmp.mount` unit.

Second, the original rationale for having this symlink was that if you had
multiple ostree stateroots ("osnames"), it's nicer if they had the same `/tmp`
to avoid duplication. But in practice today that's already an issue due to
`/var/tmp`, and further the multiple-stateroot case is pretty unusual. And that
case is *further* broken by SELinux (if one wanted to have e.g. an Ubuntu and
Fedora) stateroots.  So let's fully decouple this and make `/tmp` a plain
old directory by default, so systemd's `tmp.mount` can become useful.

Now, things get interesting for the case where someone wants a physical `/tmp`
that *does* persist across reboots. Right now, if one just did a `systemctl mask
tmp.mount` as we do in Fedora Atomic Host's cloud images, you'd get a semantic
where `/tmp` stays per-deployment, which is weird.  Our recommendation for
that should likely be to set up a bind mount for `/tmp` → `/var/tmp`.

For now, this stays an option to ensure compatibility; if FAH Cloud images
want to stay with "physical /tmp", then we'd have to change the kickstart.

Closes: https://github.com/projectatomic/rpm-ostree/issues/669

Closes: #778
Approved by: jlebon
2017-05-24 17:49:36 +00:00
Colin Walters
2adc1cf246 core: Ignore overrides for nonexistent %ghost files in /etc
As seen in e.g. `ipa-client-common`.  We expect ghosts 👻 to not
exist.

Closes: https://github.com/projectatomic/rpm-ostree/issues/784

Closes: #785
Approved by: jlebon
2017-05-24 15:53:07 +00:00
Colin Walters
a18c2ecbf7 livefs: Fix etc merge with subdirectories
This was a kind of last-minute bug introduced when I tweaked the
checkout to use `.` to avoid a `mkdir()` for files.  But there were
multiple bugs with that; for files that are in subdirectories of `/etc`
we obviously need to get the right subdir and not use `/etc`.

Second, we need to handle the case where we're adding new subdirectories.

This change fixes `rpm-ostree install firewalld + rpm-ostree ex livefs`.

Closes: #783
Approved by: jlebon
2017-05-23 20:34:47 +00:00
Jonathan Lebon
174510fc5d core: allow no enabled repos for local pkgs
This is a valid case when layering local RPMs. Otherwise (e.g. if
specific non-local packages are requested), give the nicer error rather
than letting libdnf throw something less useful.

Closes: #780

Closes: #781
Approved by: cgwalters
2017-05-19 16:08:18 +00:00
Colin Walters
82fbb5342f postprocess: Some porting to new code style
Prep for future work.  No surprises here.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
024f953124 core: Log to journal rpm-md repo timestamps
This is part of: https://github.com/projectatomic/rpm-ostree/issues/774

Basically, it's very useful for debugging since one could get an out-of-date
mirror or the like.

Down the line we may more formally store this data in a permanently-persistent
way, but this is a simple extension of the journaling we do now.

Closes: #775
Approved by: jlebon
2017-05-12 18:09:02 +00:00
Jonathan Lebon
ab1176f682 transaction-types.c: convert to new style
Pretty straightforward stuff.

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
1105ef872d pkg-builtins: rename internal functions
The pkg-add/remove names are legacy now.

s/pkg_add/install/
s/pkg_remove/uninstall/

This is mostly for cosmetics.

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
da4c25fea1 libglnx: bump and use new helpers
Update submodule: libglnx

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
3fae2eaf89 daemon/core: Fix leaks in metadata cleanup
I noticed I had `/tmp/rpmostree-metadata*` directories, and there
were two cases where we created those dirs.  Astonishingly,
*both* had bugs that prevented cleaning up.

Rename one of them so in the future it'll be clearer which is buggy (or both).

Closes: #772
Approved by: jlebon
2017-05-12 14:05:58 +00:00
Jonathan Lebon
96f75519f3 core: partial switch to new style
Switch a couple of core functions to the new style in preparation for
some other work there. This doesn't tackle the larger assemble()
function.

Another function which escaped for now is one that uses a tmpdir. Makes
me wonder if we should add a "GlnxTmpDir" type thing which simplifies
the mkdtempat/opendirat pattern and has a shutil_rmrf cleanup.

Closes: #771
Approved by: cgwalters
2017-05-10 20:46:43 +00:00
Jonathan Lebon
143b1a5bd5 rpm-util: use glib auto cleanups macros
Rather than defining our own cleanup functions. And it's easier on the
eyes!

Closes: #771
Approved by: cgwalters
2017-05-10 20:46:43 +00:00
Jonathan Lebon
8c3e8b24ad core: invalidate pkgcache based on RPM checksum
Now that the unpacker actually tracks what the RPM checksum was at
import time, we can make use of that to determine whether we should
download the newer RPM, even if it has the same NEVRA.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
eff00053ef unpacker: embed RPM checksum in metadata
In addition to the header checksum, we also want a checksum of the RPM
file itself. This will allow us to know right away whether an RPM with
the same NEVRA in the repos is actually the same one we already
imported.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
fa10f36fb6 core: write number of imported packages to journal
The actual number of imported packages is not printed right now on the
terminal. But it's useful information to have, so stick it in the
journal.

Also make both the ENABLED_REPOS and ENABLED_REPOS_SOLVABLES
comma-separated lists. This essentially makes them valid YAML/JSON,
which is a nice property to have for e.g. test/shell interactions.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
04adff5d75 rpmostreed-os.c: remove unused variable
Not sure why the clang tester didn't pick this up.

Closes: #768
Approved by: cgwalters
2017-05-08 21:11:46 +00:00
Colin Walters
b7cf58efc1 daemon: Rework API for rollback deployment lookup
Before this commit, in the case where we have 3 deployments, the rollback code
would pick the merge/pending deployment. This is understandable since that logic
predates us potentially creating 3 deployments.

For livefs, I want rollback to actually go to the rollback, not the pending.

Add an API to find the pending/rollback, and put it in syscore, since we have
other similar functions to this there. (Perhaps in the future we'll figure out a
way to dedup the deployment walking/filtering logic a bit).

Port the two callers to it.

Closes: #767
Approved by: jlebon
2017-05-08 14:33:11 +00:00
Colin Walters
3f3090ee7e daemon: Reimplement ClearRollbackTarget in terms of cleanup txn
I didn't realize we had this dbus method.  It was added in the initial
drop of the DBus API, and AFAICS nothing uses it; it's not exposed
via the cmdline, and a quick `git grep` in Cockpit shows it's
not used there either.

It's tempting to delete it.

However, before we take that step, let's reimplement it in terms of the new,
shiny, more powerful "cleanup" txn.

The particular reason I'm doing this is I want to change how we handle
rollbacks.

Closes: #767
Approved by: jlebon
2017-05-08 14:33:11 +00:00
Colin Walters
e7a16c561b os: Use correct completion for cleanup txn
Spotted while doing further work.

Closes: #767
Approved by: jlebon
2017-05-08 14:33:11 +00:00
Colin Walters
670b134cb2 postprocess: Convert a few funcs to new code style
Prep for more work here.

Closes: #766
Approved by: jlebon
2017-05-08 13:34:29 +00:00
Jonathan Lebon
752368eb5b rebase: allow rebasing to a local branch
This is a follow-up to commit 77acf62. There, we added support for
rebasing from a local branch to another local branch. But in testing,
it's also really useful to be able to rebase from a remote-based refspec
to a local branch. We allow this here by slightly expanding the syntax
of allowed refspecs.

Now, we can use rpm-ostree all the time rather than fallback to `ostree
admin deploy`, which isn't pkg-aware.

Closes: #764
Approved by: cgwalters
2017-05-05 21:01:26 +00:00
Jonathan Lebon
a54efdff8a rpmostreed-utils.c: switch to new style
Since we're going to do some work there. Also fix not always translating
the error from ostree_parse_refspec.

Closes: #764
Approved by: cgwalters
2017-05-05 21:01:26 +00:00
Colin Walters
11309cab97 core: Ignore %pretrans
This was argued on list a while ago: http://lists.rpm.org/pipermail/rpm-ecosystem/2016-August/000391.html
Basically, every use of `%pretrans` is a workaround for some in-place
upgrade problem.  But we never do inplace updates, we always assemble
a new tree.  Hence, there is no point for us to ever execute these.

Let's stop erroring out on them. If there does turn out to be some problem with
a particular package (e.g. the `openjdk-copy-configs` or whatever), we would
likely end up needing to fix that package's `%post`, not start executing its
`%pretrans`.

Closes: #763
Approved by: jlebon
2017-05-01 18:07:06 +00:00
Colin Walters
081c9dc65b core: Journal selinux relabeling stats
An updated version of: https://github.com/projectatomic/rpm-ostree/pull/316
using the journal (and structured logging) instead.

I was just curious.  And yeah, in some quick testing via deploying an old
FAH `25.67`, there are zero relabels.

We should likely have a test that actually forces a relabel...would require
custom SELinux policy build?  Or hm, maybe we could override a file context.

Closes: #760
Approved by: jlebon
2017-04-28 21:42:39 +00:00
Jonathan Lebon
f58600a348 util: fix potential overflow in var substitution
When appending the trailing bytes, we were passing (p - s), but p by
definition is always NULL at that point. Chaos ensues.

The really evil part about this is that the len is passed signed and
glib treats negative values to mean it should lookup the length itself,
so this worked *most of the time*. Though I'm guessing if the address at
s is large enough, (p - s) can wrap around and become positive again,
thus causing a massive allocation. Anyway, I didn't actually check this
(esp. since the report was for ARM), but the patch is clearly right.

I think this may be the cause for
https://bugzilla.redhat.com/show_bug.cgi?id=1381357.

Closes: #756
Approved by: cgwalters
2017-04-27 20:55:52 +00:00
Colin Walters
6f3f20a900 utils: Mostly convert to new code style
I was looking at the varsubst code, doing a style conversion first.

Closes: #755
Approved by: jlebon
2017-04-27 18:57:10 +00:00
Colin Walters
7d98e3efc8 libpriv: Add a helper function to print pkg diff
I want to use this in livefs, where I'll end up doing some diff
computations on the server and am currently rendering text there.

It might also be a step towards using this in `db diff`.

Closes: #709
Approved by: jlebon
2017-04-26 14:26:00 +00:00
Colin Walters
999e0034ef daemon/cleanup: Make use of new libostree API to fix TODO
We now avoid doing cleanup twice.

Closes: #709
Approved by: jlebon
2017-04-26 14:26:00 +00:00
Colin Walters
9260d828a8 daemon: Move deployment cleanup logic into core
While nothing else besides cleanup right now would call this directly, the code
should live close to the other very similar function:
`rpmostree_syscore_add_deployment()`.

Closes: #709
Approved by: jlebon
2017-04-26 14:25:59 +00:00
Colin Walters
386ac7537b daemon/core: Move some deployment logic here, also add bump_mtime()
This is prep for livefs.  We need to tweak the logic from what
core libostree has in `ostree_sysroot_simple_write_deployment()`,
and while we could land improved logic there, I think it makes
sense to carry this here until we're confident enough in the logic
to make it ABI.

This does depend on a [new libostree API](https://github.com/ostreedev/ostree/pull/745)
that allows writing deployments without doing cleanup.

The `bump_mtime()` bit is also prep for livefs, carrying in this patch to avoid
splitting things too much.

Closes: #709
Approved by: jlebon
2017-04-26 14:25:59 +00:00
Colin Walters
6485a8b306 core: Port a few functions to new style
I was looking at livefs executing scripts, realized I may need to
do some refactoring here, and decided to do some style updates.

Also, we make use of the `g_autoptr()` for ostree types in a few
places.

Closes: #752
Approved by: jlebon
2017-04-26 13:26:49 +00:00
Colin Walters
c8546b62a3 core: Fix error handling in hardlink changes for directory iteration
This function wasn't like the others.  If an error happened during
directory iteration, we'd still return TRUE, but with an error set.

Noticed while I was going to do some code style porting.

Closes: #752
Approved by: jlebon
2017-04-26 13:26:49 +00:00
Colin Walters
c87151b910 app: Prefix error when trying to connect to the bus
I typed `rpm-ostree status` in a container, and today we print:
`error: Could not connect: No such file or directory` which isn't
quite informative enough for me.

Closes: #751
Approved by: jlebon
2017-04-24 20:12:40 +00:00
Colin Walters
06e4ba2602 scripts: Ignore openjdk pretrans
I honestly didn't really dive into this, but it looks like
this is yet another "hack upgrades in the yum case", which we
should be able to ignore since we always do reassembly.

I at least tested `java -version` works with this.

This is a band-aid for the bigger issue of:
https://github.com/projectatomic/rpm-ostree/issues/749

(Doing this one since an AtomicWS user reported it)

Closes: #750
Approved by: jlebon
2017-04-24 14:38:33 +00:00
Colin Walters
65cfd22314 libpriv: Delete some dead code
I happened to notice these functions are unused. Purge them from existence.

Closes: #746
Approved by: jlebon
2017-04-21 13:54:05 +00:00
Colin Walters
a180a35a48 status: Do show Commit with --verbose, don't show StateRoot without it
`Commit` is useful in cases where one wants to e.g. run `rpm-ostree db diff`.
`StateRoot` however is going to be in the vast majority of cases identical
across deployments.

Closes: #743
Approved by: jlebon
2017-04-21 13:31:21 +00:00
Colin Walters
b2f9c5ed82 status: Rename OSName → StateRoot in the human-readable text
See: https://github.com/ostreedev/ostree/pull/794

I originally tried to do this ~a year ago in:
https://github.com/projectatomic/rpm-ostree/pull/295

But Micah pushed back at the time.  I'd still like to do this
though, since I personally find `OSName` *more* confusing
than `StateRoot`.

One specific rationale I have for making this change is that
I rebased a home server from CentOSAH to FAH - and I feel
it's just saner to see `StateRoot: centos-atomic-host` instead
of `OSName: centos-atomic-host` underneath the fedora refspec.

Closes: #743
Approved by: jlebon
2017-04-21 13:31:21 +00:00
Colin Walters
9ab46f5221 Require ostree 2017.5
This is preparatory for future changes which will make use the newer sysroot
writing API. But in this change, we can drop all of our version ifdef'd code.

Closes: #740
Approved by: jlebon
2017-04-20 20:58:51 +00:00
Colin Walters
b21b08c770 app: Only call RegisterClient() if we're root
Otherwise we can't do e.g. `rpm-ostree status`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/738

Closes: #739
Approved by: dustymabe
2017-04-14 02:30:19 +00:00
Colin Walters
0c99106a99 Update libdnf submodule
To pull in the GPG https:// key fix.

Closes: https://github.com/projectatomic/rpm-ostree/issues/715

Closes: #719
Approved by: jlebon
2017-04-13 15:15:03 +00:00
Jonathan Lebon
f13db897f6 core: handle enabled_metadata repos
In #728, we started querying enabled repos using get_n_solvables().
However, there are different kinds of enabled repos, and
`dnf_repo_get_enabled()` reflects that through the bitmask it returns:
  - DNF_REPO_ENABLED_NONE: repo disabled
  - DNF_REPO_ENABLED_PACKAGES: repo enabled for package installs
  - DNF_REPO_ENABLED_METADATA: repo enabled for metadata

We were treating it as a boolean, though really, we should only print
data about repos with ENABLED_PACKAGES on, which are the actual repos
libdnf can fetch packages from. Repos with only ENABLED_METADATA on are
not fetched by default, and thus will cause SIGSEGV when trying to
get_n_solvables().

I ran into this while trying to debug #720 on F25 AH, which has this
repo by default:

  [fedora-cisco-openh264]
  name=Fedora $releasever openh264 (From Cisco) - $basearch
  baseurl=https://codecs.fedoraproject.org/openh264/$releasever/$basearch/
  enabled=0
  enabled_metadata=1

Closes: #736
Approved by: cgwalters
2017-04-13 14:06:04 +00:00
Colin Walters
530ab23cb8 core: Add journal message for pkg state (repos, solvables)
I'm debugging a depsolve failure right now, and just like for
the treecompose case, it's useful to see the information about
enabled repositories.  (We should actually maintain this
better in the daemon, but this is a start)

Closes: #728
Approved by: jlebon
2017-04-11 19:24:33 +00:00
Colin Walters
7819b85a7d treecompose: Prepare device API mounts
This gives scripts access to e.g. `/dev/urandom`.  Short term
hack until we implement https://github.com/projectatomic/rpm-ostree/issues/729

The reason we don't need to explicitly clean these up before committing is right
now for treecompose we only lift `/usr` from the RPM content, so we don't run
into ostree refusing to commit devices.

Closes: https://github.com/projectatomic/rpm-ostree/issues/727

Closes: #730
Approved by: jlebon
2017-04-07 17:06:36 +00:00
Colin Walters
b060ecda21 unpacker: Also set flag to make failure to label fatal
There's no immediate need for this; was just a lead
I was chasing down while debugging
https://github.com/ostreedev/ostree/pull/775

Closes: #724
Approved by: jlebon
2017-04-05 13:19:39 +00:00
Jonathan Lebon
d5eeea646a rpmostree-kernel.c: always get kver from /lib/modules
The kver that rpmostree_find_kernel() returned would include the csum
that ostree tacks on. This would then cause dracut to fail to find the
modules for the given kver.

The issue is that rpmostree_find_kernel() can be called for both the
ostree checkout case (pkglayering) and the yumroot case (treecompose).
Rather than using a heuristic to determine whether there is a trailing
csum we should trim, we instead always fetch the kver from the dirname
in /usr/lib/modules, which will work in both cases and is stronger.

This also fixes a few cosmetic issues. Notably the final initramfs and
vmlinuz filenames in /usr/lib/ostree-boot would have *two* chekcsums
tacked on, rather than just one.

Closes: #722
Approved by: cgwalters
2017-04-04 18:47:31 +00:00
Jonathan Lebon
7b9f4870bd dracut: always specify kver
Otherwise dracut will just default to the running kernel, which will be
wrong if the pending deployment doesn't have the same kernel. Use --kver
rather than appending to be more explicit.

Closes: #722
Approved by: cgwalters
2017-04-04 18:47:31 +00:00
Jonathan Lebon
b4fba5e7d2 initramfs: add --os switch
This is already standard in the other commands, and also comes in useful
in chroot situations like anaconda.

Closes: #717
Approved by: cgwalters
2017-04-04 15:52:12 +00:00
Jonathan Lebon
008b885026 start-daemon: add metavar for --dbus-peer option
Closes: #717
Approved by: cgwalters
2017-04-04 15:52:12 +00:00
Colin Walters
471f02b528 daemon: Fix systemctl reload rpm-ostreed
I think this was a conceptual merge conflict from
https://github.com/projectatomic/rpm-ostree/pull/292
and
https://github.com/projectatomic/rpm-ostree/pull/598

Also I noticed we weren't consistently using `@primaryname@`...
will fix that when we get back to the renaming PR.

Closes: #716
Approved by: jlebon
2017-04-04 15:28:17 +00:00
Jonathan Lebon
c35622dada app: make use of new UpdateDeployment()
Add support for --install/--uninstall to upgrade/deploy/rebase by
calling out to UpdateDeployment(). Since it's not released yet, back out
the support for install-local-packages in the options variant for
PkgChange(), and make it instead use UpdateDeployment() as needed.

Closes: #711
Approved by: cgwalters
2017-03-31 14:58:38 +00:00
Jonathan Lebon
5e75dcf656 D-Bus: add new UpdateDeployment() method
The new UpdateDeployment() method wraps the full capability of the
deploy transaction handler. Modifiers indicate how one wants to change
the origin, and options indicate extra behaviours.

One interesting note here is that all the options "come together" for
the first time: some of them were user-specified options (reboot,
skip-purge, dry-run), some were internal only (no-pull-base), and some
were both (allow-downgrade).

This means we now have to handle interesting cases, e.g.:
  - We check for option conflicts between "no-pull-base" and
    "set-refspec"/"set-revision" (similarly for "skip-purge").
  - We enable "allow-downgrade" by default if the refspec or revision is
    changing.
  - Previously, "dry-run" could only be specified through PkgChange().
    Rather than making it only valid when pkgs are overlayed (which
    itself wasn't that meaningful since the introduction of dormant
    package requests), we generalize it so that "dry-run" just means:
    stop right before actually deploying the tree.

Closes: #711
Approved by: cgwalters
2017-03-31 14:58:38 +00:00
Colin Walters
c1872bc29e postprocess: Tell ostree to error out on missing labels
See https://github.com/ostreedev/ostree/pull/768 - basically
`/proc` is unlabeled, and this gets annoying for livefs work
since it shows up as a diff.

Plus at some point in the future `ostree fsck` might actually learn to verify
deployment directories.

Closes: #714
Approved by: jlebon
2017-03-31 14:30:03 +00:00
Colin Walters
f71e1b8b4d all: Conditionally use new ostree_sepolicy_new_at()
If available, using the also-new `OSTREE_CHECK_VERSION`. I dropped the `path`
argument from one of the internal APIs since it made the code simpler, and every
caller except one was passing `.`.

Closes: #712
Approved by: jlebon
2017-03-30 13:06:48 +00:00
Colin Walters
2abb3a547a util: Stub out OSTREE_CHECK_VERSION if we don't have it
Prep for using it in other places.

Closes: #712
Approved by: jlebon
2017-03-30 13:06:48 +00:00
Jonathan Lebon
d9c6733718 RpmostreedOS: share more between deployment handlers
All the handlers that eventually create a deploy transaction can share
much more code. This will make it easier to implement pkg
install/uninstall for all of them in one place.

Closes: #710
Approved by: cgwalters
2017-03-28 14:02:29 +00:00
Colin Walters
1b2a7cd90c daemon: Introduce a "sysroot core" file
Right now, the `rpm-ostree cleanup` logic calls into a "public static" method of
`RpmOstreeSysrootUpgrader`. That's because today, a lot of the high level "core
server side" logic lived in that class. However, it's only for upgrades (well,
really "generating pending deployments").

As I'm working on "livefs" which is really something different, I find it needs
to interact with some of `SysrootUpgrader`'s logic. Let's introduce a "sysroot
core" which is just a set of static functions taking an `OstreeSysroot*`, and
then all 3 of cleanup, (upgrade|deploy) and livefs can use these.

Closes: #708
Approved by: jlebon
2017-03-27 19:22:32 +00:00
Colin Walters
75d91a1757 daemon: Add a sysroot argument to state → GVariant functions
Prep for livefs; there we're storing live update state as xattrs on the
deployment root, and hence we need access to the sysroot to query it.

Closes: #706
Approved by: jlebon
2017-03-27 18:57:45 +00:00
Colin Walters
c1f15cd38b app: Add a helper to get a proxy for OSExperimental too
Prep work for livefs.  Also port the function to new style.

Closes: #706
Approved by: jlebon
2017-03-27 18:57:45 +00:00
Jonathan Lebon
2672dc1255 pkg-builtins: factor out sorting function
Rework pkg_change() so that the core functionality is implemented by a
helper function. Also convert it to the new style.

I also renamed the "local-packages" to "install-local-packages".
Contrast with the future values: "install-packages" and
"uninstall-packages".

Closes: #704
Approved by: cgwalters
2017-03-23 21:49:50 +00:00
Jonathan Lebon
071aa93f03 app: convert many builtins to new return style
One interesting note here was that I couldn't use glnx_throw in as many
places as I wanted because EXIT_SUCCESS/EXIT_FAILURE are the exact
opposite of that semantic. It was still convenient to use though as a
short-form for g_set_error().

Closes: #704
Approved by: cgwalters
2017-03-23 21:49:50 +00:00
Jonathan Lebon
d592c1874c D-Bus: enable fd passing for deploy/rebase/upgrade
This is the same change that was done for PkgChange(). It will be
required if we want to support specifying local packages through e.g.
--install switches.

Closes: #704
Approved by: cgwalters
2017-03-23 21:49:50 +00:00
Colin Walters
3e833659b7 daemon: Add a separate OSExperimental interface
Just like `rpm-ostree ex`, for things like `ex livefs` that have DBus
interfaces, we should segregate these off so that people know they're unstable.
And conversely that they can test for the presence of the method on the main
interface for stability.

I initially tried having the same `RpmostreeOS` object implement both
but couldn't work out how to do that; see https://mail.gnome.org/archives/gtk-app-devel-list/2017-March/msg00161.html

Closes: #701
Approved by: jlebon
2017-03-23 19:24:41 +00:00
Colin Walters
1969b96f91 libpriv/origin: Convert to new code style
This is a small one, but I like the momentum. Also note my use of the `,`
operator here. A little unusual maybe but I think it makes sense.

Closes: #700
Approved by: jlebon
2017-03-22 20:42:38 +00:00
Jonathan Lebon
702ccadb64 core: convert some functions to new style
Closes: #699
Approved by: cgwalters
2017-03-22 17:07:10 +00:00
Colin Walters
8b6c55ca60 core: Also apply mode overrides to symlinks
I originally thought there was a bug here, but missed we were skipping
symlinks earlier above.  See [previous discussion][1].

Now, I'm not aware right now of something this patch actively fixes, but I think
it makes sense to be consistent in our handling of things here with respect to
symlinks.

1: 29dd1bd801..8158dcfb47 (r95017893)

Closes: #689
Approved by: jlebon
2017-03-21 21:35:37 +00:00
Jonathan Lebon
d2ea6cf1b1 rpm-ostree install: pass local packages by fd
This is a follow-up to commit 81c43e8 (#657). That commit extended the
definition of "packages_added" to also support local RPMs. We revert
that here, and instead open the RPM file ourselves and send the fd over
D-Bus. We add support for a "local-packages" option containing the fd
indices to process.

It's just cleaner and safer overall; the daemon and client might not
even be sharing the same view of the filesystem!

Closes: #696
Approved by: cgwalters
2017-03-21 21:09:34 +00:00
Jonathan Lebon
64b7dfcfb0 dbus: enable fd list passing for PkgChange
Add an annotation so that the generated D-Bus code allows rpm-ostree to
pass fds through messages. Note that this doesn't change the API itself,
only how rpm-ostree uses the GDBus library. Clients should still be able
to invoke PkgChange() as before.

Closes: #696
Approved by: cgwalters
2017-03-21 21:09:34 +00:00
Colin Walters
35436827af status: Only show one of BaseCommit/Commit
Following up on the previous commit regarding timestamps - the checksum of the
derived commit isn't very useful. It's not going to be the same between any two
people's systems. What we should show instead is the rpm-md repository
timestamps.

In the short term though, I think omitting this even without the rpm-md
timestamps is better.  Less visual clutter.

Closes: #697
Approved by: jlebon
2017-03-21 20:33:24 +00:00
Colin Walters
77548d06ee status: Use the timestamp from the base, not layered
Currently we propagate the version field from the base to derived,
but the status UI shows the timestamp from the derived commit.

I found this *extremely* confusing when I was trying to reason about the
up-to-dateness of my system. The version and timestamp should be locked
together so we display them consistently.

When we generated the local commit isn't really useful information; conceptually
we have the base timestamp, plus timestamps for the rpm-md repos. We should
display the latter instead.

(This also adds `base-timestamp` and `base-verison` to the JSON; only
 the former is new information, but the latter helps us be consistent)

Closes: #697
Approved by: jlebon
2017-03-21 20:33:24 +00:00
Jonathan Lebon
df8bd53845 pkg-builtins: remove more code for no scripts
This takes commit c437f90 a bit further. We delete more #ifdef 0 blocks
which can just be revived later on if we need it.

Closes: #695
Approved by: cgwalters
2017-03-21 16:53:56 +00:00
Colin Walters
a05e256177 compose: Delete /usr/etc/passwd- (and the other variants)
There's no point to shipping these backup files in the base tree. We already had
code to delete them for the package layering case where they caused active harm.
At the point we added that code we really should have *also* changed treecompose
to delete them. Better late than never.

The reason I'm doing this now is because having them in the base tree causes `ex
livefs` to spuriously think that layering a package that *doesn't* change `/etc`
as if it does, because the layering code deletes the backup files.

Closes: #693
Approved by: jlebon
2017-03-20 16:35:17 +00:00
Jonathan Lebon
bde8456da6 compose: consistent error messages across subcommands
Make `rpm-ostree compose bad-command` and `rpm-ostree ex bad-command`
have the same error message formatting.

Closes: #688
Approved by: cgwalters
2017-03-16 21:39:33 +00:00
Jonathan Lebon
152e7bf1a6 CLI: use proper command name in experimental notice
This is a follow-up to #683. All the experimental commands will be under
`rpm-ostree ex`, so we remove the flag and instead print the notice
before dispatching the subcommand, where we can print the correct name.

Closes: #688
Approved by: cgwalters
2017-03-16 21:39:33 +00:00
Colin Walters
710ea30612 main: Move "rpm-ostree container" → "rpm-ostree ex container"
Now all of our "experimental" bits are under one roof, so we get consistent
handling for them.

Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
8dfed46e26 app: Rename internals → ex
The goal is to consolidate our "experimental" functionality under one
subcommand.  This makes it easier to determine when things "graduate"
to permanent-stability status under the main command line.

Closes: https://github.com/projectatomic/rpm-ostree/issues/682

Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
7226b4135a app: Make hidden/experimental flags for command parsing
I think this ends up being cleaner - the properties of
a command are now consistently represented as flags.

Prep work for https://github.com/projectatomic/rpm-ostree/issues/682

Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
7453c955df main: Remove unused "preview" commands
Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
12c34bb249 Move flags into command struct, pass down through builtins
This is prep work for https://github.com/projectatomic/rpm-ostree/issues/682

Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
62a93c7850 postprocess: Handle f26 /etc/nsswitch.conf configuration
F26 put sss first, which broke our regexp. When we switch to sysusers, man it'll
be nice to dump ♲ this.

Closes: https://github.com/projectatomic/rpm-ostree/issues/685

Closes: #686
Approved by: jlebon
2017-03-16 18:39:38 +00:00
Colin Walters
fc9396372f app: Dedup code for subcommand parsing
We had lots of copies of this code, and while I'm not planning on changing it, I
decided to dedup on general principle.

Closes: #681
Approved by: jlebon
2017-03-15 21:11:42 +00:00
Colin Walters
7e4f56a862 app: Switch subcommands to use "base" RpmOstreeCommand struct
They were just duplicates.  Prepartory cleanup for future patches.

Closes: #681
Approved by: jlebon
2017-03-15 21:11:42 +00:00
Colin Walters
24faf942df compose: Add --add-metadata-from-json
I'd like to embed structured metadata about the originating git
repository.  See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master).  However, I think what we really
want here is a *single* value which has subkeys.

One thing in the back of my mind too is...we could use this to
enhance our "change detection".  Right now we checksum the sack,
treefile, and treecompose-post.  But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.

In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).

Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.

Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.

I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.

Closes: #676
Approved by: jlebon
2017-03-14 18:17:07 +00:00
Colin Walters
479667807c compose: Canonicalize user-provided metadata to big endian
Should have no effect right now, since there's actually
no way we expose at the moment to write anything other
than strings.

But it will be useful when we add a way to write arbitrary metadata; see
https://github.com/projectatomic/rpm-ostree/pull/676

Closes: #679
Approved by: jlebon
2017-03-14 15:27:47 +00:00
Colin Walters
5142b8b4c2 compose-tree: Clean up metadata parsing
Preparatory work for a future patch; by having a hash it's easier to handle
duplicates. Note I started trying to use decl-after-stmt but hit the fact we're
still using `goto out` here.

Closes: #679
Approved by: jlebon
2017-03-14 15:27:47 +00:00
Colin Walters
05a59e5af5 core: Don't try to chmod() symlinks
I just noticed this while reading the code. Right now we're actually following
links (since `AT_SYMLINK_NOFOLLOW` isn't implemented), which could definitely
lead to bad things if e.g. one has a symlink pointing to a file which shouldn't
be executable or world-readable etc.

(Also I noticed our hardlink-breaking-only-if-regular logic is now broken;
 I added a TODO)

Closes: #677
Approved by: jlebon
2017-03-13 17:53:51 +00:00
Colin Walters
77fa10cfbe core: Skip (and warn on) rpmfi overrides outside of /{usr,etc,var}
I still need to query whether Fedora has anything like this; maybe `/boot`? But
things are not likely to work right now if we do, so let's ignore it.

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
ab7431c458 core: Apply rpmfi overrides for /etc to /usr/etc
This fixes installing `mock`, which for some reason has files in `/etc/mock`
owned by `root:mock`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/671

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
d8b96a8531 core: Include filename in error messages for rpmfi overrides
This gives me now:

```
error: While applying overrides for pkg mock: fstatat: etc/mock/custom-1-aarch64.cfg: No such file or directory
```

Which is definitely more useful for debugging 💣.

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
62320b1920 rebase: Update man page for syntax, print if just changing remote
The rationale for just printing if changing remote is I think
it's really only that version that feels "magical".

Closes: https://github.com/projectatomic/rpm-ostree/issues/569

Closes: #678
Approved by: jlebon
2017-03-13 14:45:30 +00:00
Jonathan Lebon
81c43e81fb PkgChange: add support for layering local RPMs
Allow users to directly specify an RPM file on the command-line. The
"packages_added" array of the PkgChange() method can now contain
absolute paths to RPM files.

Grow the origin format to have a new "requested-local" key. This is
similar to the "requested" key, except that the packages are always
installed from cache. The "requested-local" array values also embed the
SHA-256 of the header we expect.

There is now a new "LocalPackages" line in the status. These packages
are a subset of the "packages" element (which are printed as
"LayeredPackages") and represent the packages that are explicitly marked
for installing from cache.

Interesting design choices/notes:
  - Just as before, even with foo-1.0-1.x86_64 installed from RPM, a
    user can still request "/usr/bin/foo": it will be made dormant. As
    soon as foo stops being explicitly layered from the RPM, it will try
    to fulfill the request by going to the repos. This allows users to
    "pin" a layered package to a certain RPM, and then unpin it.
  - The strings/NEVRAs in "requested" and "requested-local" are strictly
     distinct. This allows us to be able to tell what the user means
     exactly when they do "rpm-ostree uninstall".

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Jonathan Lebon
5c5ebc1b0a core: add support for installing from cache
We start by adding support in the core for installing packages strictly
from the cache repo. We fool the libdnf stack by re-exporting the header
as an RPM, and explicitly marking it for install. The treefile format
supports specifying the expected SHA-256 of the metadata header, in case
the cache for a specific NEVRA changed.

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Colin Walters
e86dc13c49 bwrap: Don't use --unshare-net in nspawn by default
This will fix rpm-ostree-in-mock-in-koji. The drawback is minor: post scripts
will have network access. But we're going to be testing the no-network case in
our Docker-based builds, so that's fine.

Closes: #672
Approved by: jlebon
2017-03-10 17:27:56 +00:00
Jonathan Lebon
49cbdb739a status: always include the packages entries
Pull #646 introduced a subtle regression: we went from always including
a "packages" entry to only including it if there are packages present.
Albeit it's easy to guard against, though to be nice, let's make it
easier for consumers by always including it.

Reported-by: Micah Abbott <miabbott@redhat.com>

Closes: #670
Approved by: cgwalters
2017-03-10 14:54:45 +00:00
Colin Walters
14a9135ea0 main: Output YAML for --version
Equivalent of https://github.com/ostreedev/ostree/pull/691
Basically, it's nicer to parse.

Closes: #665
Approved by: jlebon
2017-03-09 14:23:34 +00:00
Colin Walters
7cf366497f daemon: Log txn initiation
Down the line we should have a saner stringification, and also
log things like the invoking `loginuid` etc.

But this gets us something.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
fd6b30746e daemon: Move mainctx iteration out of main() into object
Preparation for idle exit - the daemon knows when it should idle exit. It's
tempting to move all of the mainloop logic there, but it'd be more code churn
than I want to do right now.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
927d9d061a daemon: Track transaction status, render it in systemd status
This is part of the idle exit work - the higher level daemon logic needs to
track running transaction state, so we can combine that with whether we have `> 0` clients.

(Yes, we need a better stringification of txns)

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
33e9f2a4cd daemon: use sd_notify() when we receieve a signal
Just to keep the status display up to date.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
d3772ffc4f daemon: Use sd_notify(STATUS=) to print how many clients are active
Now `systemctl status rpm-ostreed` will show it. In future patches I plan to add
whether we have an active txn and our pending idle exit timeout.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
98c1f43326 daemon: Add RegisterClient API
This is like what bluez does. With this, we have a stronger mechanism to avoid
races with future work to auto-exit on idle.  Registered clients
hold a reference to the daemon effectively.

Note that calling `UnregisterClient` is optional if the calling process is going
to exit soon - as is the case for using the command line binary via e.g.
`rpm-ostree status`.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
d4789e0886 daemon: use systemd notify(STOPPING=1) + ReleaseName combo
More info here:

 - https://lists.freedesktop.org/archives/dbus/2015-May/016671.html
 - https://github.com/cgwalters/test-exit-on-idle

This is preparatory work for doing exit-on-idle, which we want to do because we
currently leak memory badly. Even if we didn't leak, it's still better to not
have services sit around resident.

While I had the patient open, I did some code cleanup such as switching to
direct `g_main_context_iteration()` and using `sd_journal_print()` instead of
`g_warning()` etc.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
7b8e05ffc2 Allow and start using C99 declaration-after-statement
The equivalent of https://github.com/ostreedev/ostree/pull/718
but for this codebase.

I just picked one example at random, there's plenty of others, but I don't want
to do any kind of tree-wide conversion since we have lots of outstanding
patches.

Closes: #664
Approved by: jlebon
2017-03-06 23:16:30 +00:00
Colin Walters
79ee224709 core: Log commit stats into journal
It's useful to see how many new objects were added and how much storage they
use. Just part of a general trend where I'd like to make a lot more extensive
use of the journal, particularly with structured logging.

Closes: #661
Approved by: jlebon
2017-03-06 22:19:10 +00:00
Colin Walters
e1e2bf9fbe compose-tree: Print nsolvables, consolidate disabled repos
This should help us if we ever find corrupt repodata; I thought
that might be happening in https://pagure.io/releng/issue/6602
Also, consolidate the disabled repos printing, since I think
we don't need to be verbose about those.

Closes: #613
Approved by: jlebon
2017-03-06 21:21:10 +00:00
Colin Walters
f9944e6d96 daemon: Remove several unused instance variables
These were leftovers from the very original code which did support exit-on-idle;
we're going to do it differently now.

Closes: #659
Approved by: jlebon
2017-03-06 16:14:38 +00:00
Colin Walters
a00b9f94fb daemon: Tell systemd when we're shutting down
This won't actually help with much right now; just splitting out this change to
highlight it as part of future work for exit-on-idle.

Closes: #659
Approved by: jlebon
2017-03-06 16:14:38 +00:00