Commit Graph

374 Commits

Author SHA1 Message Date
Colin Walters
3f367dbce2 scripts+bwrap: Make script execution cancellable
Prep for implementing `rpm-ostree cancel`, but this works with the way we handle
`Ctrl-C` interactively on a client as well. Being able to cancel a script
execution is quite nice; some of them are expensive, and having one loop forever
has been known to happen.

Closes: #1025
Approved by: jlebon
2017-10-03 14:24:07 +00:00
Jonathan Lebon
a9c38d33b8 vmcheck: add SELinux labeling tests
There's a lot of paths in the core related to SELinux policy changes and
relabeling packages. We currently have no test coverage for them. We add
support in the test libraries here to build such packages.

We also add a test that checks both that we correctly relabel RPMs when
the policy changes and that we handle layered packages that install
SELinux packages properly.

Closes: #999
Approved by: cgwalters
2017-10-03 01:01:14 +00:00
Jonathan Lebon
4ad5747b96 libpriv/core: use final sepolicy when doing final commit
It's possible that a layered package installed an SELinux package which
can change the labeling of some files. The problem is that imported
packages will not have used the updated policy and thus may have some of
its paths mislabeled. We fix this here by always loading the SELinux
policy from the tmprootfs and using that for the final commit.

Closes: #999
Approved by: cgwalters
2017-10-03 01:01:14 +00:00
Colin Walters
d4af84d428 Change ex-container to use BARE_USER_ONLY, update core creation API
Switch `ex container` to `OSTREE_REPO_MODE_BARE_USER_ONLY`; this is a good
match, for the same reasons as flatpak. We don't want suid binaries, we don't
want to chown anything, no SELinux labeling, etc.  There's no reason to write the
metadata to the `BARE_USER` xattrs.

The "unprivileged" context as used for `ex container` was a flag that was passed
down, but I think it's a lot easier if things just infer this state by looking
at whether the target repo is `bare-user-only`. Now we just have an
`rpmostree_context_new_tree()` API that handles both container and compose.

Closes: #1026
Approved by: jlebon
2017-10-02 16:13:52 +00:00
Colin Walters
acbbda8214 tree-wide: Some more misc libglnx porting
More use of the `glnx_unlinkat()`, `glnx_fstatat()`, and `glnx_renameat()`
wrappers.

Closes: #1027
Approved by: jlebon
2017-10-02 13:41:01 +00:00
Jonathan Lebon
2cf87ba897 tree: some more glnx_fstatat_allow_noent porting
Closes: #1022
Approved by: cgwalters
2017-09-28 21:15:10 +00:00
Colin Walters
d6de31d90d libpriv/scripts: Only log to journal if uid == 0
The previous change to log to the journal broke running `ex container` as
non-root with scripts. Yes, I really desperately need to set up real tests for
that, like we have for composes. And obviously checking `getuid() == 0` is a bit
crude but doing better would require plumbing through quite a bit of of
knowledge from the toplevel down into script execution.

Closes: #1021
Approved by: jlebon
2017-09-28 20:59:18 +00:00
Colin Walters
c0ef5eec4f libpriv/core: Check cancellable before/after downloading rpm-md
This is a quick band-aid to improve Ctrl-C handling in this area:
https://github.com/projectatomic/rpm-ostree/issues/897

At least this way we'll exit earlier rather than waiting until all the repos are
downloaded (and imported into libsolv, which is itself slow).

Closes: #1018
Approved by: jlebon
2017-09-28 17:55:18 +00:00
Jonathan Dieter
fd6109af1d compose: Copy in additional files before postprocessing
Sometimes it's useful to have access to the additional files when running
the post script, so this re-orders the compose process to copy the
additional files in before the post script runs

Signed-off-by: Jonathan Dieter <jdieter@lesbg.com>

Closes: #997
Approved by: jlebon
2017-09-28 15:38:55 +00:00
Colin Walters
2e59a2748d postprocess: Allow add-files into /etc
Even though it's really `/usr/etc`. This is for greater consistency with
`postprocess-script` where it appears as `/etc`.

Closes: #997
Approved by: jlebon
2017-09-28 15:38:55 +00:00
Colin Walters
45287828e5 lib/compose: Maintain /etc as /usr/etc more consistently
Lots of confusion in the codebase about this. The basic problem is that in
*most* cases, our code doesn't care; it's conceptually operating on `/usr/etc`,
which we could maintain as `/etc` and just rename it back at the very end.

The exceptions though are the `/etc/passwd` handling and livefs. And of course
libostree needs to handle `/usr/etc` vs `/etc` for config merging.

I considered trying to keep things the other way, but while I think we have some
ugly added here in this patch for things where we need to maintain an external
view (`remove-files` and `remove-from-packages`, and boy am I glad we had tests
for those), this ends up being mostly more consistent elsewhere.

One thing that might help is to maintain a fd for it; but that'd be an even more
invasive change.

This also ends up rolling in some unified core prep from
https://github.com/projectatomic/rpm-ostree/pull/940 in the form of
`rename_if_exists()` - basically for some minimal rootfs we may not have
`/boot`, or for that matter potentially even `etc`.

Prep for https://github.com/projectatomic/rpm-ostree/pull/997

Closes: #997
Approved by: jlebon
2017-09-28 15:38:55 +00:00
Colin Walters
11d9a78094 libpriv/core: Add some comments (and a bit of style porting)
I was reading some of the core code recently, and in general I think we need
more comments. I made a pass from the top of the file and got halfway down.

Closes: #1017
Approved by: jlebon
2017-09-28 13:04:29 +00:00
Jonathan Lebon
cb7e84c4a6 libpriv/scripts: redirect scriptlet output to journal
Rather than just letting the scriptlets inherit the daemon's
stdout/stderr, redirect their outputs so that we can set a customized
identifier to make it easier to distinguish from the daemon output.

Also print out the `journalctl` command needed so that users can
investigate the output themselves.

Closes: #998
Approved by: cgwalters
2017-09-27 19:01:45 +00:00
Jonathan Lebon
c92ff926f6 app/status: cleaner diff output for replaced pkgs
I suspect a common pattern with local replacement overrides is to
simultaneously replace a group of packages that depend on each other in
one shot, as is the case with docker, docker-common, and
docker-rhel-push-plugin currently in Fedora Atomic Host. In such cases,
we can print a cleaner diff in the status to make it easier to grok.

Before:

     ReplacedBasePackages: strace 4.18-1.fc26 -> 4.19-1.fc26, docker-common 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, docker 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, docker-rhel-push-plugin 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26

After:

     ReplacedBasePackages: docker-common docker docker-rhel-push-plugin 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, strace 4.18-1.fc26 -> 4.19-1.fc26

Closes: #1004
Approved by: cgwalters
2017-09-27 16:22:50 +00:00
Colin Walters
58f8f892fb lib/postprocess: Use O_TMPFILE, not O_APPEND for tmpfiles.d writing
The comment here was wrong; we don't rely on `O_APPEND` here for package
layering since we convert on import.  I noticed this while I was doing
a grep for `O_APPEND` in the codebase as part of unified core work.

Fix this by converting to `O_TMPFILE`+`GLNX_LINK_TMPFILE_NOREPLACE`.

Prep for unified core.

Closes: #1009
Approved by: jlebon
2017-09-26 18:27:00 +00:00
Colin Walters
57fcd9e1b0 libpriv: Remove unused OSTREE_GIO_FAST_QUERYINFO
Hooray, it's dead (here, but not in ostree). Noticed this while working on
<https://github.com/projectatomic/rpm-ostree/pull/997>

Closes: #1012
Approved by: jlebon
2017-09-26 14:54:52 +00:00
Colin Walters
5eab29846a libpriv/passwd: Use GLNX_FILE_COPY_NOXATTRS
Prep for unified core work. This was failing due to a bug in libglnx
<https://github.com/GNOME/libglnx/pull/80> but I think this change is also
correct. There's no good reason for us to copy xattrs like the SELinux label
here - rather we want the labels to be reset during commit.

I did a tree-wide grep for other users and the only other case that is odd is
the treecompose `add-files`; I'd say we should change this but out of (a likely
excess of) conservatism I just left a "FIXME" for now.

Closes: #1008
Approved by: jlebon
2017-09-26 13:07:59 +00:00
Jonathan Lebon
c10a45aa5e libpriv/core: squash a -Wmaybe-uninitialized
Noticed this pass by while doing a `fedpkg local`. It doesn't actually
matter much since there are no exits possible between the point it's
declared and when it's actually assigned, which I will assume is why our
clang suite didn't pick it up, maybe? Not gonna look into that right
now.

Closes: #1010
Approved by: cgwalters
2017-09-25 17:39:42 +00:00
Jonathan Lebon
167fd4c1dd libpriv/scripts: use rofiles-fuse on /etc too
We were directly bind mounting the checked out `/usr/etc` onto `/etc`
which was exposing us to corruption from scriptlets. Since we already
have an rofiles-fuse mount for `/usr`, let's just re-use its `etc/`
subdir and bind mount that instead.

Closes: #1003
Approved by: cgwalters
2017-09-24 13:11:21 +00:00
Jonathan Lebon
f089b8de1f libpriv/postprocess: also delete semanage lock files
We don't need those in the tree, so let's nuke them. This also fixes
subtle compatibility issues between hardlinks and lock files (see #999).

Closes: #1002
Approved by: cgwalters
2017-09-21 15:28:55 +00:00
Jonathan Lebon
25ba196c27 libpriv/core: also sort update/downgrade packages
Now that we support overrides, we need the sorter to make sure it goes
through the override packages in case they need to be relabeled.

Closes: #961

Closes: #993
Approved by: cgwalters
2017-09-18 14:53:50 +00:00
Jonathan Lebon
e78807143f libpriv/core: fix tmpdir making logic
Even if we're already initialized, we still need to create the subdir
requested. This would cause relabeling to fail because we try to create
the "relabel" subdir, which never actually gets created.

Closes: #993
Approved by: cgwalters
2017-09-18 14:53:50 +00:00
Colin Walters
4a9aebcc02 core: Don't search for base filetriggers if no base rpmdb
Otherwise librpm whines on stderr about a missing database.

Related: https://github.com/projectatomic/rpm-ostree/issues/987

Closes: #989
Approved by: jlebon
2017-09-15 12:03:12 +00:00
Ruixin
56ef177235 tests/core: allow merging identical files during installation
This PR uses https://github.com/ostreedev/ostree/pull/1156
to ensure that when installing packages containing files with
exact same content, the files will be merged.

When installing packages containing conflicting files,
the error will still be detected at the ostree side.

The checkout overwrite option at rpm-ostree side is also
modified to accomodate the changes made in ostree side.

A test is added for regression
2017-09-13 20:41:15 +00:00
Colin Walters
f113fc5e27 Rework treecompose kernel processing
Prep for changing `boot_location: new` to use `/usr/lib/ostree-boot`
and `/usr/lib/modules`.  Rework our kernel postprocessing
so that we unify the `boot_location` handling with initramfs generation.

Instead of doing the initramfs first in postprocessing, we do it nearly last,
after e.g. `etc` is renamed to `usr/etc`. This has some consequences, such as
the fact that `run_bwrap_mutably()` is now called in both situations. In
general, our handling of `etc` is inconsistent, although understandably so.

As part of this, I finally got around to implementing the bit from
https://github.com/systemd/systemd/pull/4174 however suboptimal it is; need the
unified core so we can cleanly ignore the posttrans like we do others.  We
intentionally keep the file around in the generated tree so that installing a
kernel RPM per client doesn't try to do any of this either.

This all gets folded together so that the logic for handling the bootloader gets
simpler - in the Fedora case, we now know to find kernels in `/usr/lib/modules`
and can ignore `/boot`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
3ff48589b3 compose: Make boot_location: legacy be an alias for both
We have 3 locations to find kernels now; I can't think of
a reason to support placing kernels *only* in `/boot`.  The
original commit
15ecaacd36
doesn't give a reason, and I certainly can't think of one now.

This makes `legacy` be an alias for `both`, which should be fully compatible.

Prep for further refactoring towards changing `new` to mean both
`/usr/lib/ostree-boot` *and* `/usr/lib/modules`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
150de0087e scripts: Support overriding lua, use for glibc-all-langpacks
Today in Fedora the `glibc-all-langpacks.posttrans` is implemented
in lua, for no good reason.  See:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Since that's stalled out, let's add support for overrides.  This
is obviously a much bigger step with more long term maintenance
implications over our current "ignore scripts" list.  But we can't
block either.

This is needed for unified core work:
https://github.com/projectatomic/rpm-ostree/issues/729

(We also override `fedora-release-atomichost` but I'll likely
 submit a patch for that upstream)

Closes: #980
Approved by: jlebon
2017-09-08 18:02:22 +00:00
Colin Walters
e40eaebbe0 core: Use fd-relative access to rpmdb
I was linking to this code from elsewhere and noticed that
for our hardlink breaks we were not using fd-relative even
though we can.  Down the line if we fork librpm into a separate
process and do e.g. `--dbpath=.` it'll do it too.

(Side note, I verified that commenting out the hardlink breaking
 here was caught by the `ostree fsck` I added to the test suite)

Closes: #979
Approved by: jlebon
2017-09-07 22:54:40 +00:00
Colin Walters
38d1876270 Update libglnx, do some fstatat-noent porting
Started on porting to the new `glnx_fstatat_allow_noent()`.  The
usage varies a lot and it felt easy to screw up, so I'm just
starting by doing a few of them.

Update submodule: libglnx

Closes: #978
Approved by: jlebon
2017-09-07 22:30:34 +00:00
Colin Walters
79086f8c3d core: Use a single tmpdir/ostreetxn for relabeling
I'd like to cut down on the number of tempdirs we make; when
debugging with gdb I often use e.g. `r` to restart forcibly, and
that leaves the dirs around.

Previously we'd make a tmpdir per relabeled package; this hoists it into a
well-known "relabel/" subdir of the per-core tmpdir we make. (A next step here
is to dedup the core tmpdir with the tmpdir that the upgrader makes)

Further while I'm here, let's use a single OSTree transaction for all of the
relabeling; the rationale here is that it's better to do fewer `syncfs()`
operations.

Closes: #970
Approved by: jlebon
2017-09-07 19:04:08 +00:00
Colin Walters
f69717bf69 libpriv+daemon: Use autocleanup to abort ostree txn
This allows porting one function each in the unpacker and sysroot to new style.
There were also two cases in the core where we were missing an
`abort_transaction()` invocation.

libostree version in: https://github.com/ostreedev/ostree/pull/1132
Prep for: https://github.com/projectatomic/rpm-ostree/pull/970

Closes: #972
Approved by: jlebon
2017-09-07 13:50:19 +00:00
Colin Walters
20abc85243 unpacker: Translate /boot → /usr/lib/ostree-boot
At one point `rpm-ostree install libvirt` dragged in libguestfs which in turn
brought in `syslinux-extlinux-nonlinux` which has files in `/boot/extlinux`,
which we rejected.  (That dependency chain appears to have been fixed currently)

For the general case, this is just a partial fix in that we haven't nailed down
the semantics of how updates for `/boot` work.  But in this particular case,
we'll just break libguestfs' `extlinux` verb, which I'm OK with.

Another case is `fwupdate-efi` - we require manual intervention to copy the
data into `/boot` after installing the package.

This is also preparation for [unified core](https://github.com/projectatomic/rpm-ostree/issues/729)
in that we now ensure imported kernels don't end up in `/boot` unless
explicitly configured.

Closes: https://github.com/projectatomic/rpm-ostree/issues/853

Closes: #969
Approved by: jlebon
2017-09-07 13:30:36 +00:00
Ruixin Bao
d6b3217a86 lib/core: Detect file name conflict before rpmdb
This PR uses https://github.com/ostreedev/ostree/pull/1116
to ensure that when installing packages containing conflicting files, the
error will be detected at the ostree side, rather than when
writing the rpm database.

Closes: https://github.com/projectatomic/rpm-ostree/issues/365

Closes: #974
Approved by: cgwalters
2017-09-07 13:13:12 +00:00
Jonathan Lebon
5294c1ea69 scripts: squash a -Wmaybe-uninitialized
Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Colin Walters
a78a88844c libpriv/scripts: Use gperf comments, not C comments
I noticed our C-style comments were actually showing up in the generated hash
table. Consistently use gperf's `#`, and also move all comments to the line
above the value for more readability.

Closes: #967
Approved by: jlebon
2017-09-01 17:58:51 +00:00
Colin Walters
213a656ddf postprocess: Unlink our treecompose-post out of the final /bin
Another variant of <https://github.com/projectatomic/rpm-ostree/pull/822> 😭

Closes: #964
Approved by: jlebon
2017-08-31 03:06:11 +00:00
Colin Walters
3047513509 core: Run %post before %posttrans
While working on unified core and the Fedora Atomic Host content set, I hit a
dependency between `docker.posttrans` which tries to read `/etc/os-release`, and
`fedora-release-atomichost.post` which creates that symlink.

It seems best practice to me to run `%post`s strictly before
`%posttrans`; we're not likely to do parallelization anytime
soon anyways.

While here I cleaned things up by having an enum for the script kind,
rather than multiple functions, otherwise we would have had another
wrapper in core.c.

Closes: #963
Approved by: jlebon
2017-08-30 15:50:09 +00:00
Colin Walters
8a87887988 libpriv/postprocess: Port add-files to fd-relative
This is the last bit that used a `yumroot` variable.

Closes: #950
Approved by: jlebon
2017-08-28 13:46:44 +00:00
Colin Walters
d469323693 libpriv/passwd: Fully complete fd-relative porting
I initially thought I wouldn't need this but it turns out it makes things nicer
unified core. In the legacy path we'll need to write
`etc/passwd`, but now we need `usr/etc/passwd` in the unified path, and
we can start to match up with the other code paths which are fd-relative.

Closes: #950
Approved by: jlebon
2017-08-28 13:46:44 +00:00
Colin Walters
3f5cc629dc lib/unpacker: Add a SKIP_EXTRANEOUS flag, use it for filesystem+rootfiles
Prep for unified core. For packages we know ship stuff we want to skip, enable
this flag.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
acbf00724d lib/unpacker: Drop ostree convention flag, always enable it
The core use case always enabled this; we exposed it on the command line, but
honestly I think it's a lot simpler if our imports always enable this. If we
need to *undo* it (for example, generating a "traditional feeling" OCI/Docker
image a la <https://github.com/projectatomic/rpm-ostree/pull/367>), we can
easily do that as postprocessing afterwards.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
5633424628 lib/unpacker: Fix theoretical error handling case
We should be using `g_propagate_error()` to ensure we handle the case where the
caller specifies `NULL` for the error.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Jonathan Lebon
b05b9679f9 scripts: fix argument ordering
Just noticed this while trying out the unified core work. We were
passing arguments in the wrong order (or alternatively, receiving them
in the wrong order).

Closes: #952
Approved by: cgwalters
2017-08-24 21:48:14 +00:00
Colin Walters
06f3a1811a lib/passwd: Port passwd checking to fd-relative
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
addc53a707 tree-wide: Fix extra colon in error prefixes
This is a trap when doing style porting; we added a rule to ostree's `maint.mk`
to catch it but not here. Not sure I want to add `maint.mk` here yet.

Note that in newer libglnx `glnx_opendirat()` does error prefixing, so
drop the prefixing there.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
278ed216d0 tree-wide: A bit more libglnx porting
Mostly for `glnx_renameat()`.

Closes: #941
Approved by: jlebon
2017-08-24 13:33:59 +00:00
Colin Walters
3840fad28e tree-wide: Fully purge the rest of "noscripts" support
It was partially removed in 325ee354e9
from PR #873 - this removes all of the rest of the layers in
the daemon and core for this.

Note that while working on this I found it astoundingly confusing that
`rpmts_add_install()` and `add_install()` had separate orderings for `gboolean
noscripts, is_install`. That's fixed by having the latter lose the `noscripts`
flag entirely, but while we're here change `rpmts_add_install()` to use flags
for clarity.

Closes: #938
Approved by: jlebon
2017-08-23 16:31:27 +00:00
Colin Walters
4d32f736e1 lib/unpacker: Port unpack function to new style
Missed this in the tmpdir porting since it used `g_mkdtemp_full()`.
We also now have autocleanups for `OstreeRepoCommitModifier`.

Closes: #939
Approved by: jlebon
2017-08-23 15:14:53 +00:00
Colin Walters
4be8658f65 lib/passwd: Error if presented with an unknown check-passwd value
Prep for sysusers work; let's cleanly reject things we don't know.

Closes: #935
Approved by: jlebon
2017-08-23 14:51:08 +00:00
Colin Walters
6b3a99c3a9 lib/bwrap: Use --die-with-parent
See <https://github.com/projectatomic/bubblewrap/pull/165>; really every
bwrap use case I can think of should specify this (including ours), it's
just not the default out of conservatism.

This way if the daemon happens to e.g. SEGV it'll also cleanly `SIGKILL` any
outstanding scripts.

Was just looking at our bwrap usage for multiprocess work.

Closes: #936
Approved by: jlebon
2017-08-22 18:58:58 +00:00