Commit Graph

963 Commits

Author SHA1 Message Date
Colin Walters
0ff4403253 bin/compose: Port to new code style
Move basically everything except early option
processing into a helper function.

Also update a few other misc bits above that to new style.

Closes: #904
Approved by: jlebon
2017-08-08 17:09:13 +00:00
Colin Walters
162b8d4898 bin/compose: Silently ignore --workdir-tmpfs
Down the line when we have a unified core, this will be totally unnecessary.
Even without that, it's not worth the maintenance pain for us. Anyone who wants
this can easily on their own mount a tmpfs and point us at it with --workdir.

Prep for further code cleanup.

Closes: #904
Approved by: jlebon
2017-08-08 17:09:13 +00:00
Colin Walters
39e36db741 postprocess: Port a bit more to new style
Was looking at this file to change how we do kernels, and noticed these two were
straightforward ports.

Closes: #909
Approved by: jlebon
2017-08-08 16:24:45 +00:00
Colin Walters
787c880b64 bin/rebase: Add -b and -m options
The rebase command syntax has confused people a lot.  Let's follow
git here and add a `-b/--branch` option and encourage people to use
that.  The case of switching remotes is `-m/--remote`; it's definitely
unfortunate that `-r` is already taken for `--reboot`.

One thing I'm a little bit unhappy about is how we're doing logic
on the client side here.  Changing the DBus API for this would
also be awkward though.

Closes: https://github.com/projectatomic/rpm-ostree/issues/886

Closes: #890
Approved by: jlebon
2017-08-08 16:02:15 +00:00
Colin Walters
9b22c2c516 scripts: Journal execution of %post/%posttrans, and times for all scripts
The script delay is annoying; some of them take over half a second here. This
helps with debugging. Another enhancement we could make after this is aborting
scripts that appear to be hung after e.g. 30 seconds, and logging that.

Closes: #905
Approved by: jlebon
2017-08-03 16:58:52 +00:00
Jonathan Lebon
dc24dd3105 rollback: allow users to undo a rollback
The new API to find pending and rollback deployments do so relative to
the booted deployment. This caused an interesting behaviour: the first
time a user uses "rpm-ostree rollback", it would (as expected) move the
previous deployment first. but the second call to "rpm-ostree rollback"
would fail since there were now no more rollback deployments.

We fine tune the logic here to allow this, as well as the more general
case of putting the booted deployment back on top.

This fixes a subtle regression from b7cf58e
(https://github.com/projectatomic/rpm-ostree/pull/767).

Closes: https://github.com/projectatomic/rpm-ostree/issues/906

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Jonathan Lebon
d9c1e4a5d3 daemon: use upstreamed function to find rollback
Now that the query_deployment() function was upstreamed in ostree (see
https://github.com/ostreedev/ostree/pull/897), let's just use it from
there and drop our copy here.

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Colin Walters
0515f59d70 bin/compose: Use autocleanups for context structure
Prep for more code cleanup here and porting to new style.  We
had a mix of owner pointers.  Fix this by consistently having
the `self` struct own the data.

Closes: #903
Approved by: jlebon
2017-08-02 17:41:43 +00:00
Colin Walters
08e3f5de7f lib: Log task output for pre/post scripts
Some of the scripts can be expensive (`shared-mime-info` for example), and we
appear to be hanging for at least a few seconds even on fast hardware. I'd like
to have a lot more logging here...potentially something like showing
individual package names live in the terminal, but this is a start.

Closes: #902
Approved by: jlebon
2017-08-02 17:16:28 +00:00
Ruixin
53c39632d4 Deployment: add endoflife
When commit metadata contains ostree.endoflife attribute,
its information will be added to the deployment Variant,
which will later be shown as a red & bold message when
'rpm-ostree status' command is called.

A test is added for future regression

Closes: #889
Approved by: cgwalters
2017-08-02 13:53:10 +00:00
Colin Walters
0d4d6be94f Implement file triggers (%transfiletriggerin) for layered pkgs
File triggers are a post-RHEL7 thing; more information at
http://rpm.org/user_doc/file_triggers.html

There are two notable users I've been testing this with;
`glib2` and `vagrant`.  The `vagrant` one is more immediately urgent,
since it makes `vagrant-libvirt` work, which I currently rely on
for my workstation dev.

I've tested things successfully with `vagrant`, and I did verify that we run the
`glib2` ones when doing `rpm-ostree ex container`.

Long term, more transaction file triggers are likely to live in
"base" packages like `glib2`.  We don't implement those yet, but
extending this to do that shouldn't be too hard.

There was *significant* what I'd call reverse engineering of the
implementation in librpm.  The file triggers code there is spread out
and abstracted in a few different places in the code.  I found
trying to understand what header values were involved to be quite
tricky.

There are some corner cases like multiple patterns that I *think*
this does correctly, but could use more validation.  The main
question I had was - is it required that the patterns for e.g.
`%transfiletriggerin` and `%transfiletriggerun` be identical?

Closes: https://github.com/projectatomic/rpm-ostree/issues/648

Closes: #869
Approved by: jlebon
2017-07-27 20:58:09 +00:00
Jonathan Lebon
d4effe8f47 dbus: don't register if not active
Follow-up tweak to #894. Make the client smarter so we only register
when we know we can. We could be more sophisticated here and e.g.
introduce the concept of "read-only" clients in the daemon to only allow
access to non-mutating methods, though let's delay that discussion at
least until the daemon learns to auto-exit.

Closes: #898

Closes: #900
Approved by: cgwalters
2017-07-27 20:34:22 +00:00
Colin Walters
34b5a004a8 daemon: Add a sanitycheck(/bin/true) before we deploy a tree
This is a followup to https://github.com/projectatomic/rpm-ostree/pull/888
but more comprehensive; in the layering case, the sanitycheck runs
after all the `%posttrans` scripts, so we'll get a consistent error message
for the `rm -rf /` test.

We also do the sanitycheck for the "pure ostree" case, as well as cases
where we didn't actually layer packages (including `ex override remove` as
well as simply regenerating an initrd).

There's obviously a lot more we could do in a sanitycheck; as I say in the
comment it's tempting to consider trying to boot systemd (in a fully volatile
config), but for now let's do this. In the end of course the admin has rollback
too.

Closes: #892
Approved by: jlebon
2017-07-27 17:58:58 +00:00
Jonathan Lebon
577aef98e2 app: spawn polkit authentication agent
Spawn pkttyagent when trying to call a method that may require
authentication to give users a chance to provide auth right from the
terminal.

Since we're now relying on polkit for authorizing most of the OS
interface methods, let's drop the root check on those.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
21109e18ae dbus: drop root check on RegisterClient()
Drop the root check for RegisterClient() so that any UID can use it to
then safely invoke methods. UIDs which already have an active session do
not need to authenticate for RegisterClient().

In order to implement this, we move the polkit authority setup into the
RpmostreedSysroot object and enable polkit on the interface.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
3ade73dedd daemon: factor out uid fetching
Make a method specialized in fetching the UID of a D-Bus client.

Prep for polkit tty auth agent.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Colin Walters
5763027b5f Update libglnx, do some porting to new APIs
In particular I wanted to start using `GLNX_AUTO_PREFIX_ERROR`.

Update submodule: libglnx

Closes: #893
Approved by: jlebon
2017-07-21 22:37:17 +00:00
Colin Walters
2082b3f8fb lib: Add version macros and version checking function
The version checking function in particular is really useful for people doing
`from gi.repository import RpmOstree`, which we'd like at least some things like
Anaconda and Pungi to do.

Closes: #891
Approved by: jlebon
2017-07-21 20:35:26 +00:00
Colin Walters
fee6d06bf4 lib: Expose new API around basearch
For https://pagure.io/atomic-wg/issue/299 we need to make it
more convenient to substitute the architecture in an installation
context.  I plan to use this API inside `rpmostreepayload` in Anaconda,
so we can substitute the same value of `${basearch}` we use in treefiles
since https://github.com/projectatomic/rpm-ostree/pull/305

Now, you might wonder - why do we need an API wrapping libdnf? It's because
libdnf is not API stable yet. We're just exposing a tiny subset. In theory we
could use the Python dnf bindings in Anaconda, but things get slightly weird if
rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for
now this a small API surface to maintain (forever).

This change reworks the internal `varsubst` bits to take a pure `DnfContext`,
since we don't want to spin up a whole `RpmOstreeContext` just to do some
string substitutions.

Closes: #877
Approved by: jlebon
2017-07-21 16:02:41 +00:00
Colin Walters
b46fc35901 Add releasever opt, avoid opening up host's rpmdb in treecompose
Closes: https://github.com/projectatomic/rpm-ostree/issues/546

Previously, we'd open up the host's rpmdb for both `compose tree`
and `ex container`.  In the first case, because we require root, we'd
succeed.  For `ex container`, we'd spew an error.

Fixing this was trickier than I thought.  First because there was
*also* a libdnf bug here: https://github.com/rpm-software-management/libdnf/pull/307

Second, there's a compatibility hazard here for anyone using `.repo` files that
reference `$releasever`. This actually happened to me with `ex container` as I'd
just done a `ln -s /etc/yum.repos.d/fedora.repo rpmmd.repos.d`.  I fixed
that first by doing a `sed -i -e 's,$releasever,26,' rpmmd.repos.d/*.repo`.

As far as I can see today, none of Fedora Atomic or CentOS AH rely on this. But
in order to enhance compatibility, let's add a "releasever" option. This makes
it easier again to reuse stock `.repo` files if we wanted to do so.

(Also, I realized we can just use `/usr/share/empty` as *the* canonical immutable
 empty directory)

Closes: #875
Approved by: jlebon
2017-07-21 15:02:59 +00:00
Colin Walters
1f3ebba982 scripts: Honor the -e flag for scripts
This is required for glibc-all-langpacks at least:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Otherwise, its usage is...extraordinarily rare. In fact looking at a snapshot of
`rpm-specs-20170518.tar.xz` from Fedora, the only other use is in
`postfix.spec`, and it appears bogus (the value is already expanded at build
time).

But the glibc case is special, as the value of `install_langs` is indeed
potentially dynamic per system.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
4222407657 core/scripts: Cleanup refactor
No functional changes; the main thing is to change the "ignore" case to an early
return, and also introduce a "medium level" function that does argument parsing.

Prep for file trigger work.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
325ee354e9 core/scripts: Drop support for ignore-scripts
This was a transient hack that I don't think we need any longer.
Since then we've now committed to e.g. ignoring `%pretrans` etc.,
and we also have the hardcoded blacklist.

It never really worked in a complete sense since one would have to re-specify
the ignore on every operation, which was not sustainable.

Closes: #873
Approved by: jlebon
2017-07-18 19:21:15 +00:00
Colin Walters
f38bb61f91 Update libglnx
And port the passwd util.

Update submodule: libglnx

Closes: #867
Approved by: jlebon
2017-07-18 16:43:40 +00:00
Matthew Barnes
c7024d69ae daemon: Avoid "Updating from: (null)" message
Check for a NULL refspec before emitting message in
package_diff_transaction_execute().

Closes: #876
Approved by: cgwalters
2017-07-13 20:14:31 +00:00
Jonathan Lebon
50ad253ebe codebase: partial porting to new style
Just some minor porting of the less glamorous parts of the codebase.

Closes: #865
Approved by: cgwalters
2017-07-07 20:03:28 +00:00
Jonathan Lebon
f1437028cf core: only update repos enabled for packages
We don't care about repos enabled only for METADATA. This works around
an interesting libdnf behaviour in which all repos in redhat.repo are
enabled for metadata, even if disabled.

Downstream RHBZ: #1468302

Closes: #863
Approved by: cgwalters
2017-07-06 18:59:16 +00:00
Colin Walters
f599f4bc7c libpriv: Some style porting
Scattered bits, just keeping up some momentum.

Closes: #861
Approved by: jlebon
2017-07-06 14:29:38 +00:00
Jonathan Lebon
5b541f4859 app: support ex override replace for local RPMs
Add experimental support for replacing packages from the base layer with
local RPMs. This is useful for example, to cherry pick a fixed package,
or to roll back to a previous package version. Like with pkg removals,
only files in /usr are actually replaced.

This patch also contains a few usability improvements as well, e.g.
showing the full NEVRA of removed packages rather than just their names,
and support for resetting overrides using either the pkgname or NEVRA.

Closes: #852
Approved by: cgwalters
2017-07-05 20:59:35 +00:00
Jonathan Lebon
7b4134c8e6 core: make cachebranch finder function public
Prep for replacement overrides.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
63405d655b core: plug leak from ignore_scripts hash table
Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Jonathan Lebon
2da68cb510 sysroot-core: minor style updates
This function was already mostly in the new style. Just moving the
declarations closer to where they are used. No functional changes. The
logic will be tweaked in an upcoming commit.

Closes: #859
Approved by: cgwalters
2017-07-05 14:02:21 +00:00
Colin Walters
14431f7f09 lib/scripts: Handle script interpreters
Seen in the wild with `vagrant`'s use of `%post -p /usr/bin/ruby`. This was a
very easy fix, and actually makes the code a little bit nicer, as we no longer
need to explicitly make the script executable, since we now pass it as
`argv[1]`, the same way librpm does. That in turn would make it possible to fix
the TODO and use `bwrap --file`, but that can come later.

Closes: https://github.com/projectatomic/rpm-ostree/issues/856

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Colin Walters
1f33f42c7a scripts: Port some to new style, minor cleanup
Prep for further work.

Closes: #858
Approved by: jlebon
2017-07-04 15:51:20 +00:00
Colin Walters
137271efee libpriv/passwd: Port to new style
All straightforward. Just chipping away at the porting.

Closes: #851
Approved by: jlebon
2017-06-26 18:03:12 +00:00
Colin Walters
c7c89e8735 status: Use more concise GPGSignature without --verbose
Since it takes up a lot of room and should be something that's just always
working.

Closes: https://github.com/projectatomic/rpm-ostree/issues/842

Closes: #848
Approved by: jlebon
2017-06-26 15:51:46 +00:00
Colin Walters
4d72f8d787 cmdline: Stop printing GPG results during pulls
It's really verbose for deploy, and in general, let's follow
the Unix philosophy of silence = success.

Closes: https://github.com/projectatomic/rpm-ostree/issues/843

Closes: #850
Approved by: jlebon
2017-06-26 14:36:41 +00:00
Jonathan Lebon
453e9ca570 rpm-util: factor out get_by_pkgname
This will be needed multiple times in the future.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
3ae90ce5fc upgrader: make metadata tmp dir part of upgrader
I initially planned to use the metadata tmp dir for another reason than
just in finalize_overlays(), but I will not anymore. Regardless, this
seems like a nice patch which also allows us to complete the style
conversion in this file.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
853737f667 unpacker: encode NEVRA info in commit metadata
It seems silly that to find out more detailed information about the
NEVRA of a cached pkg, we have to resort to write out the header to
disk, then reading it back in with librpm in order to tease out the info
we want. Let's just encode that information directly in the commit
metadata and provide a helper to fetch it.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
4507c581cf core: factor out pkgcache nevra to branch lookup
This will be useful in the future for another function.

Closes: #847
Approved by: cgwalters
2017-06-23 17:47:22 +00:00
Jonathan Lebon
7b6237c14f postprocess: prefix more errors
This is a follow-up to #845. Judging from the output of the erroring
compose[1], the error seems to happen after committing all the objects.
Add more detailed prefixes there.

[1] https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/thread/2MZ5KFEVITKWJZYOTSAYCZ7XYAJCWZCT/

Closes: #846
Approved by: cgwalters
2017-06-23 15:48:05 +00:00
Colin Walters
64b049b8eb postprocess: Style porting
Prep for debugging: https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/thread/2MZ5KFEVITKWJZYOTSAYCZ7XYAJCWZCT/
Almost everything is converted in this file now.

Closes: #845
Approved by: jlebon
2017-06-23 13:50:57 +00:00
Colin Walters
675066a9c7 daemon: Log caller uid
This is an extension to the previous change to distingush between
"caller" and "client".  Now for clients we log the uid (both
in the message and structured).

This is a natural followon from the polkit work, since now different
uids can invoke us.

Closes: #838
Approved by: jlebon
2017-06-22 21:17:10 +00:00
Colin Walters
7d3b72c814 Remove all tabs ⭾ in *.[ch] and add a CI check for them
There weren't many left, and I happened to have one of these files open and it
was distracting.

Closes: #839
Approved by: jlebon
2017-06-22 01:20:55 +00:00
Colin Walters
46f4b62f3b daemon: Differentiate in logs between "client" and "caller"
A "caller" is a bus address that hasn't called `RegisterClient`.

Prep for things like unifying the bus name tracking between the transaction and
the daemon; right now it's a bit silly that the txn code also monitors the
caller's bus name. So down the line we could change that code to call into the
daemon and use its tracking for registered clients.

Closes: #837
Approved by: jlebon
2017-06-20 21:50:39 +00:00
Colin Walters
009e1f78d6 daemon: Render txn data a bit more nicely
Separate the different fields. Prep for adding more metadata like the caller's
uid.

Closes: #837
Approved by: jlebon
2017-06-20 21:50:39 +00:00
Jonathan Lebon
0b1c5eda17 override remove: allow inactive removals
The property of removal overrides dropping out if the package was
removed from the base layer felt a bit too magical and hacky. We really
should remember that wish and re-apply it if the pkg comes back. This is
similar to package layering: requests can become inactive (seems like a
better word than "dormant") if the package is already part of the base
layer, but they don't really go away.

This patch reworks the logic so that removal overrides work the same
way. In the status output, we now have both "RemovedBasePackages" and
"InactiveBaseRemovals" (which is only printed in verbose mode),
similarly to how we have "LayeredPackages" and "InactiveRequests". And
similarly, we also print out in the upgrader during a transaction all
the inactive base removals.

Another cool thing is that we now allow any pattern to be specified at
the CLI. E.g. `ex override remove /usr/bin/strace` will resolve to
strace.

Closes: #836
Approved by: cgwalters
2017-06-20 21:24:05 +00:00
Jonathan Lebon
61560e0686 status: rename RequestedPackages to InactiveRequests
In the JSON output, the "requested-packages" holds *all* the requests
for package layering, whereas the "packages" key holds only those
requests which actually caused a package to be layered.

When outputting RequestedPackages, we try to be nice and print only the
"dormant" packages, i.e. those requests which did not require any
packages to be layered since the base layer already fulfilled them. This
is the subset of "requested-packages" not also in "packages". But then,
we could call this subset something much more appropriate than
"RequestedPackages" when outputting.

This patch renames it to "InactiveRequests" to reflect the "dormant"
aspect of them. We also make it only be output in verbose mode, since
it's not as interesting as what was *actually* layered when trying to
understand what the host is currently running. But we do adapt the
upgrader so that it prints out inactive requests during transactions.

Closes: #836
Approved by: cgwalters
2017-06-20 21:24:05 +00:00
Jonathan Lebon
4ffe12a6af libglnx: bump for GLNX_HASH_TABLE_FOREACH macros
Example of what the new macros look like when applied on a whole
codebase. It definitely feels much nicer to work with!

Update submodule: libglnx

Requires: https://github.com/GNOME/libglnx/pull/55

Closes: #834
Approved by: cgwalters
2017-06-20 14:09:24 +00:00