Commit Graph

2031 Commits

Author SHA1 Message Date
Colin Walters
f113fc5e27 Rework treecompose kernel processing
Prep for changing `boot_location: new` to use `/usr/lib/ostree-boot`
and `/usr/lib/modules`.  Rework our kernel postprocessing
so that we unify the `boot_location` handling with initramfs generation.

Instead of doing the initramfs first in postprocessing, we do it nearly last,
after e.g. `etc` is renamed to `usr/etc`. This has some consequences, such as
the fact that `run_bwrap_mutably()` is now called in both situations. In
general, our handling of `etc` is inconsistent, although understandably so.

As part of this, I finally got around to implementing the bit from
https://github.com/systemd/systemd/pull/4174 however suboptimal it is; need the
unified core so we can cleanly ignore the posttrans like we do others.  We
intentionally keep the file around in the generated tree so that installing a
kernel RPM per client doesn't try to do any of this either.

This all gets folded together so that the logic for handling the bootloader gets
simpler - in the Fedora case, we now know to find kernels in `/usr/lib/modules`
and can ignore `/boot`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
93d3fc6617 tests/compose: Add env variables to force caching and fast fail
This helped me debug/fix the tests faster.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
3ff48589b3 compose: Make boot_location: legacy be an alias for both
We have 3 locations to find kernels now; I can't think of
a reason to support placing kernels *only* in `/boot`.  The
original commit
15ecaacd36
doesn't give a reason, and I certainly can't think of one now.

This makes `legacy` be an alias for `both`, which should be fully compatible.

Prep for further refactoring towards changing `new` to mean both
`/usr/lib/ostree-boot` *and* `/usr/lib/modules`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
aab7571fc1 ci: Fixate centos container image to 7.3 until rpm-md repos update
Sigh.

Closes: #985
Approved by: jlebon
2017-09-12 13:45:50 +00:00
Jonathan Lebon
bed10f650a Makefile-tests.am: fail on first command
The shell invocations from make don't have errexit on, so just chain
commands with && manually.

Closes: #983
Approved by: cgwalters
2017-09-11 18:38:43 +00:00
Jonathan Lebon
bae4faea2b app/upgrade: use signal to determine if changed
Make `upgrade` use the same trick as `deploy` to determine if a new
deployment was laid down. Apart from those two, all other operations
that can lay down a new deployment always do so in the happy path.

Prep for further work.

Closes: #983
Approved by: cgwalters
2017-09-11 18:38:43 +00:00
Jonathan Lebon
df515fd2f1 app/libbuiltin: convert to new style
Pretty straightforward stuff. Prep for further work.

Closes: #983
Approved by: cgwalters
2017-09-11 18:38:43 +00:00
Colin Walters
150de0087e scripts: Support overriding lua, use for glibc-all-langpacks
Today in Fedora the `glibc-all-langpacks.posttrans` is implemented
in lua, for no good reason.  See:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Since that's stalled out, let's add support for overrides.  This
is obviously a much bigger step with more long term maintenance
implications over our current "ignore scripts" list.  But we can't
block either.

This is needed for unified core work:
https://github.com/projectatomic/rpm-ostree/issues/729

(We also override `fedora-release-atomichost` but I'll likely
 submit a patch for that upstream)

Closes: #980
Approved by: jlebon
2017-09-08 18:02:22 +00:00
Colin Walters
e40eaebbe0 core: Use fd-relative access to rpmdb
I was linking to this code from elsewhere and noticed that
for our hardlink breaks we were not using fd-relative even
though we can.  Down the line if we fork librpm into a separate
process and do e.g. `--dbpath=.` it'll do it too.

(Side note, I verified that commenting out the hardlink breaking
 here was caught by the `ostree fsck` I added to the test suite)

Closes: #979
Approved by: jlebon
2017-09-07 22:54:40 +00:00
Colin Walters
38d1876270 Update libglnx, do some fstatat-noent porting
Started on porting to the new `glnx_fstatat_allow_noent()`.  The
usage varies a lot and it felt easy to screw up, so I'm just
starting by doing a few of them.

Update submodule: libglnx

Closes: #978
Approved by: jlebon
2017-09-07 22:30:34 +00:00
Colin Walters
79086f8c3d core: Use a single tmpdir/ostreetxn for relabeling
I'd like to cut down on the number of tempdirs we make; when
debugging with gdb I often use e.g. `r` to restart forcibly, and
that leaves the dirs around.

Previously we'd make a tmpdir per relabeled package; this hoists it into a
well-known "relabel/" subdir of the per-core tmpdir we make. (A next step here
is to dedup the core tmpdir with the tmpdir that the upgrader makes)

Further while I'm here, let's use a single OSTree transaction for all of the
relabeling; the rationale here is that it's better to do fewer `syncfs()`
operations.

Closes: #970
Approved by: jlebon
2017-09-07 19:04:08 +00:00
Colin Walters
b5c98ec4f1 daemon: Add a policy for reload-config
This fixes `rpm-ostree reload` as root, and supports configuring
it to be enabled for other users as well.  This was overlooked
in the polkit work originally.

Closes: https://github.com/projectatomic/rpm-ostree/issues/976

Closes: #977
Approved by: jlebon
2017-09-07 15:11:00 +00:00
Jonathan Lebon
b0c75ae900 vmcheck: use --selinux-policy when committing
Let's make SELinux and atomic-host-tests happy with trees we concoct
ourselves by using the new --selinux-policy. (Specifically, we want our
sync'ed binaries to have install_exec_t).

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Jonathan Lebon
50ecc04cb6 overlay.sh: drop hack for backup passwd files
We shouldn't need this anymore.

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Jonathan Lebon
b22d7d764d vmcheck: rename build.sh to install.sh
I had meant for this to be in the other PR#968. I originally did both
`make` and `make install` there, but now it only does `make install`, so
let's just rename it to make that more obvious.

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Colin Walters
c093a587a2 tests: Add a test case for lua scripts
This is currently unsupported, let's add a test case. Prep for support for
script overrides.

Closes: #973
Approved by: jlebon
2017-09-07 14:09:47 +00:00
Colin Walters
f69717bf69 libpriv+daemon: Use autocleanup to abort ostree txn
This allows porting one function each in the unpacker and sysroot to new style.
There were also two cases in the core where we were missing an
`abort_transaction()` invocation.

libostree version in: https://github.com/ostreedev/ostree/pull/1132
Prep for: https://github.com/projectatomic/rpm-ostree/pull/970

Closes: #972
Approved by: jlebon
2017-09-07 13:50:19 +00:00
Colin Walters
20abc85243 unpacker: Translate /boot → /usr/lib/ostree-boot
At one point `rpm-ostree install libvirt` dragged in libguestfs which in turn
brought in `syslinux-extlinux-nonlinux` which has files in `/boot/extlinux`,
which we rejected.  (That dependency chain appears to have been fixed currently)

For the general case, this is just a partial fix in that we haven't nailed down
the semantics of how updates for `/boot` work.  But in this particular case,
we'll just break libguestfs' `extlinux` verb, which I'm OK with.

Another case is `fwupdate-efi` - we require manual intervention to copy the
data into `/boot` after installing the package.

This is also preparation for [unified core](https://github.com/projectatomic/rpm-ostree/issues/729)
in that we now ensure imported kernels don't end up in `/boot` unless
explicitly configured.

Closes: https://github.com/projectatomic/rpm-ostree/issues/853

Closes: #969
Approved by: jlebon
2017-09-07 13:30:36 +00:00
Ruixin Bao
d6b3217a86 lib/core: Detect file name conflict before rpmdb
This PR uses https://github.com/ostreedev/ostree/pull/1116
to ensure that when installing packages containing conflicting files, the
error will be detected at the ostree side, rather than when
writing the rpm database.

Closes: https://github.com/projectatomic/rpm-ostree/issues/365

Closes: #974
Approved by: cgwalters
2017-09-07 13:13:12 +00:00
Jonathan Lebon
48ddca9280 ci: workaround broken fedora:26 image
This is essentially the same workaround as
https://github.com/ostreedev/ostree/pull/1143.

See https://bugzilla.redhat.com/show_bug.cgi?id=1483553.

Closes: #975
Approved by: cgwalters
2017-09-07 12:27:54 +00:00
Jonathan Lebon
7fbf9f32d4 hacking: make it easier to use a custom tree
Let's make using a custom install tree easier and document the process.
We split out the insttree step into `build.sh` so that we no longer have
to `flock(1)` around it, and also share between `overlay.sh` and
`sync.sh`.

Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Jonathan Lebon
5294c1ea69 scripts: squash a -Wmaybe-uninitialized
Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Colin Walters
a78a88844c libpriv/scripts: Use gperf comments, not C comments
I noticed our C-style comments were actually showing up in the generated hash
table. Consistently use gperf's `#`, and also move all comments to the line
above the value for more readability.

Closes: #967
Approved by: jlebon
2017-09-01 17:58:51 +00:00
Colin Walters
213a656ddf postprocess: Unlink our treecompose-post out of the final /bin
Another variant of <https://github.com/projectatomic/rpm-ostree/pull/822> 😭

Closes: #964
Approved by: jlebon
2017-08-31 03:06:11 +00:00
Colin Walters
3047513509 core: Run %post before %posttrans
While working on unified core and the Fedora Atomic Host content set, I hit a
dependency between `docker.posttrans` which tries to read `/etc/os-release`, and
`fedora-release-atomichost.post` which creates that symlink.

It seems best practice to me to run `%post`s strictly before
`%posttrans`; we're not likely to do parallelization anytime
soon anyways.

While here I cleaned things up by having an enum for the script kind,
rather than multiple functions, otherwise we would have had another
wrapper in core.c.

Closes: #963
Approved by: jlebon
2017-08-30 15:50:09 +00:00
Colin Walters
4cbdcf93c8 daemon/syscore: Add some error prefixing to cleanup paths
Would likely have helped clarify this issue where an ostree repo txn failed
post-bootloader txn: <https://github.com/projectatomic/rpm-ostree/issues/958>

Closes: #960
Approved by: jlebon
2017-08-29 13:24:06 +00:00
Jonathan Lebon
44ea9f35a1 ci: also compile tests
This should make sure we catch test errors such as
https://github.com/GNOME/libglnx/pull/76 when bumping submodules (until
we eventually get a PR tester on libglnx).

Requires: https://github.com/GNOME/libglnx/pull/76

Update submodule: libglnx

Closes: #954
Approved by: cgwalters
2017-08-29 13:07:21 +00:00
Colin Walters
8a87887988 libpriv/postprocess: Port add-files to fd-relative
This is the last bit that used a `yumroot` variable.

Closes: #950
Approved by: jlebon
2017-08-28 13:46:44 +00:00
Colin Walters
d469323693 libpriv/passwd: Fully complete fd-relative porting
I initially thought I wouldn't need this but it turns out it makes things nicer
unified core. In the legacy path we'll need to write
`etc/passwd`, but now we need `usr/etc/passwd` in the unified path, and
we can start to match up with the other code paths which are fd-relative.

Closes: #950
Approved by: jlebon
2017-08-28 13:46:44 +00:00
Colin Walters
3f5cc629dc lib/unpacker: Add a SKIP_EXTRANEOUS flag, use it for filesystem+rootfiles
Prep for unified core. For packages we know ship stuff we want to skip, enable
this flag.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
acbf00724d lib/unpacker: Drop ostree convention flag, always enable it
The core use case always enabled this; we exposed it on the command line, but
honestly I think it's a lot simpler if our imports always enable this. If we
need to *undo* it (for example, generating a "traditional feeling" OCI/Docker
image a la <https://github.com/projectatomic/rpm-ostree/pull/367>), we can
easily do that as postprocessing afterwards.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
5633424628 lib/unpacker: Fix theoretical error handling case
We should be using `g_propagate_error()` to ensure we handle the case where the
caller specifies `NULL` for the error.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Jonathan Lebon
e605386818 daemon: make use of the new deployment flags
Now that we have the semantics we need in libostree, let's just use that
and drop the logic here.

Closes: #942
Approved by: cgwalters
2017-08-25 18:59:24 +00:00
Jonathan Lebon
2f6f8a910d libvm: set up ControlPath socket in /var/tmp
It turns out there's a much smaller limit than PATH_MAX for Unix
sockets. On Linux, it's 108 characters. It took me some time to figure
out why `vmcheck` would sometimes fail depending on where the src
directory is and how ${topsrcdir} is defined. Let's just make things
safer by just using /var/tmp.

Closes: #949
Approved by: cgwalters
2017-08-24 22:12:17 +00:00
Jonathan Lebon
b05b9679f9 scripts: fix argument ordering
Just noticed this while trying out the unified core work. We were
passing arguments in the wrong order (or alternatively, receiving them
in the wrong order).

Closes: #952
Approved by: cgwalters
2017-08-24 21:48:14 +00:00
Jonathan Lebon
f5bcd62364 ci: run the improved-sanity-test
Start running the improved-sanity-test from atomic-host-tests to make
sure the system works. This is an easy way to get PR-level comprehensive
integration tests for free.

But note that we *don't* mark it as required since the tests are not
stored here and it can happen that they need to be adjusted for new
rpm-ostree behaviours. In this way, this added check also allows us to
give a heads-up that breaking changes are coming.

Closes: #899
Approved by: cgwalters
2017-08-24 20:43:16 +00:00
Jonathan Lebon
c7837d003a Makefile-tests.am: drop vmshell target
I actually stopped using this a long time ago. Looking at it now, it
doesn't make much sense in multi-host situations. Let's just nix it.

Closes: #899
Approved by: cgwalters
2017-08-24 20:43:16 +00:00
Jonathan Lebon
a1b5109e8b docs/manual: be more explicit about bootloader order
Make it more clear that bootloader order implies what the default
deployment is.

Closes: #947
Approved by: cgwalters
2017-08-24 20:13:05 +00:00
Colin Walters
00231446b3 app: Clarify message on repeated Ctrl-C
While we have cancellation issues¹, let's help the
user understand that we heard them the first time.

¹ <https://github.com/projectatomic/rpm-ostree/issues/897>

Closes: #948
Approved by: jlebon
2017-08-24 17:30:10 +00:00
Colin Walters
06f3a1811a lib/passwd: Port passwd checking to fd-relative
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
a4ca8c8888 app/compose: Drop GFile usage from package install path
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
addc53a707 tree-wide: Fix extra colon in error prefixes
This is a trap when doing style porting; we added a rule to ostree's `maint.mk`
to catch it but not here. Not sure I want to add `maint.mk` here yet.

Note that in newer libglnx `glnx_opendirat()` does error prefixing, so
drop the prefixing there.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
278ed216d0 tree-wide: A bit more libglnx porting
Mostly for `glnx_renameat()`.

Closes: #941
Approved by: jlebon
2017-08-24 13:33:59 +00:00
Colin Walters
3840fad28e tree-wide: Fully purge the rest of "noscripts" support
It was partially removed in 325ee354e9
from PR #873 - this removes all of the rest of the layers in
the daemon and core for this.

Note that while working on this I found it astoundingly confusing that
`rpmts_add_install()` and `add_install()` had separate orderings for `gboolean
noscripts, is_install`. That's fixed by having the latter lose the `noscripts`
flag entirely, but while we're here change `rpmts_add_install()` to use flags
for clarity.

Closes: #938
Approved by: jlebon
2017-08-23 16:31:27 +00:00
Colin Walters
4d32f736e1 lib/unpacker: Port unpack function to new style
Missed this in the tmpdir porting since it used `g_mkdtemp_full()`.
We also now have autocleanups for `OstreeRepoCommitModifier`.

Closes: #939
Approved by: jlebon
2017-08-23 15:14:53 +00:00
Colin Walters
4be8658f65 lib/passwd: Error if presented with an unknown check-passwd value
Prep for sysusers work; let's cleanly reject things we don't know.

Closes: #935
Approved by: jlebon
2017-08-23 14:51:08 +00:00
Colin Walters
5f7d860a33 app: Set GSETTINGS_BACKEND=memory early on if root
I was about to copy this bit from the daemon for subprocess work, so let's
centralize it. Further, we should invoke `setenv()` as early as possible; see
<https://sourceware.org/bugzilla/show_bug.cgi?id=15607#c2>.

Closes: #934
Approved by: jlebon
2017-08-22 21:18:08 +00:00
Jonathan Lebon
c2b422a19b libdnf: bump for metadata_expire fix
To officially bring in
https://github.com/rpm-software-management/libdnf/pull/320.

Update submodule: libdnf

Closes: #930

Closes: #937
Approved by: cgwalters
2017-08-22 19:49:55 +00:00
Colin Walters
6b3a99c3a9 lib/bwrap: Use --die-with-parent
See <https://github.com/projectatomic/bubblewrap/pull/165>; really every
bwrap use case I can think of should specify this (including ours), it's
just not the default out of conservatism.

This way if the daemon happens to e.g. SEGV it'll also cleanly `SIGKILL` any
outstanding scripts.

Was just looking at our bwrap usage for multiprocess work.

Closes: #936
Approved by: jlebon
2017-08-22 18:58:58 +00:00
Colin Walters
0428275a3c tree-wide: Port to new libglnx tmpdir API
Lots of cleanups, and this also allows us to fully port to new style in several
places.

We can drop our own `rpmostree_mkdtemp()`.

Update submodule: libglnx

Closes: #928
Approved by: jlebon
2017-08-22 17:21:37 +00:00