Commit Graph

1638 Commits

Author SHA1 Message Date
Jonathan Lebon
2c514f446c ci: don't run clang on CentOS
The CentOS tester is currently failing because clang is being passed an
argument it doesn't know about:

  clang: error: unknown argument: '-fstack-protector-strong'

We get our version of clang from EPEL, which currently has 3.4.2. The
Fedora clang does have support for the flag (4.0.1).

Anyway, the point of this check is just to find unused/uninitialized
vars and other gotchas. In that sense, we're more interested in what the
latest version of clang has to say.

Closes: #992
Approved by: cgwalters
2017-09-14 21:02:46 +00:00
Ruixin
56ef177235 tests/core: allow merging identical files during installation
This PR uses https://github.com/ostreedev/ostree/pull/1156
to ensure that when installing packages containing files with
exact same content, the files will be merged.

When installing packages containing conflicting files,
the error will still be detected at the ostree side.

The checkout overwrite option at rpm-ostree side is also
modified to accomodate the changes made in ostree side.

A test is added for regression
2017-09-13 20:41:15 +00:00
Jonathan Lebon
f62f6b7086 livefs: tweak warning re. /etc
Be more explicit about what the warning means.

Closes: #988
Approved by: dustymabe
2017-09-13 13:51:06 +00:00
Jonathan Lebon
2a5ecf7a3e livefs: add warning that /etc won't be preserved
Since the whole premise of livefs is that you stay in your deployment
just a little longer, it becomes much more likely for people to run into
https://github.com/projectatomic/rpm-ostree/issues/40 (myself included).

Printing such a notice was discussed in the initial livefs design
discussions: https://github.com/projectatomic/rpm-ostree/issues/639.

Closes: #986
Approved by: cgwalters
2017-09-12 19:51:48 +00:00
Jonathan Lebon
4ad3ea58c6 app: smarter deployment change detection
Commands like `upgrade` and `deploy` need to know if a new deployment
was actually laid down so that it may print a pkg diff if so. This is
implemented by listening for changes to the DefaultDeployment D-Bus
property. D-Bus emits a signal when the deployment variant changes
value.

However, in #595, with the introduction of `pending-*` related keys, the
deployment variant no longer represents data solely tied to that
specific deployment. In this case, because `deploy` operations currently
set the ref to the resolved checksum, it can happen that deploying the
same base commit when the current refspec *isn't* pointing to that base
commit will result in the `pending-*` keys dropping out and a default
deployment change notification going out.

In this patch, we strengthen how we determine whether a new deployment
was laid down by actually looking at the deployment id, rather than just
assuming that a change to the property implies a new deployment.

Closes: #981

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Jonathan Lebon
077d7c1a9e daemon/deploy: allow redeploying livefs booted commit
Also something I noticed while working on #981. When sitting on a livefs
commit, once a user does `rpm-ostree cleanup --pending --rollback`, it's
impossible to redeploy the same booted commit. Let's allow users to do
this.

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Jonathan Lebon
c08ca8f922 daemon/upgrader: make use of override-commit-ids
Currently, when setting the `override-commit` key in the origin, the
upgrader pulls that commit checksum directly and then updates the
refspec to point to it. This behaviour was inherited from its ostree
version; at the time it was implemented, the pull code didn't support
passing a specific commit for a given refspec. However, we now have
the override-commit-ids option, which will make libostree update the ref
for us.

We change the code here to make use of it and simplify the function.
This also fixes the corner case of local branches: we shouldn't change
the ref if we're on a local branch. This is actually what drove me to
this patch as I was debugging #981.

(Aside: I'm still not convinced updating the refspec is always the
correct thing to do even in the remote case, though it's a bit messy to
disentangle).

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Colin Walters
f113fc5e27 Rework treecompose kernel processing
Prep for changing `boot_location: new` to use `/usr/lib/ostree-boot`
and `/usr/lib/modules`.  Rework our kernel postprocessing
so that we unify the `boot_location` handling with initramfs generation.

Instead of doing the initramfs first in postprocessing, we do it nearly last,
after e.g. `etc` is renamed to `usr/etc`. This has some consequences, such as
the fact that `run_bwrap_mutably()` is now called in both situations. In
general, our handling of `etc` is inconsistent, although understandably so.

As part of this, I finally got around to implementing the bit from
https://github.com/systemd/systemd/pull/4174 however suboptimal it is; need the
unified core so we can cleanly ignore the posttrans like we do others.  We
intentionally keep the file around in the generated tree so that installing a
kernel RPM per client doesn't try to do any of this either.

This all gets folded together so that the logic for handling the bootloader gets
simpler - in the Fedora case, we now know to find kernels in `/usr/lib/modules`
and can ignore `/boot`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
93d3fc6617 tests/compose: Add env variables to force caching and fast fail
This helped me debug/fix the tests faster.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
3ff48589b3 compose: Make boot_location: legacy be an alias for both
We have 3 locations to find kernels now; I can't think of
a reason to support placing kernels *only* in `/boot`.  The
original commit
15ecaacd36
doesn't give a reason, and I certainly can't think of one now.

This makes `legacy` be an alias for `both`, which should be fully compatible.

Prep for further refactoring towards changing `new` to mean both
`/usr/lib/ostree-boot` *and* `/usr/lib/modules`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
aab7571fc1 ci: Fixate centos container image to 7.3 until rpm-md repos update
Sigh.

Closes: #985
Approved by: jlebon
2017-09-12 13:45:50 +00:00
Jonathan Lebon
bed10f650a Makefile-tests.am: fail on first command
The shell invocations from make don't have errexit on, so just chain
commands with && manually.

Closes: #983
Approved by: cgwalters
2017-09-11 18:38:43 +00:00
Jonathan Lebon
bae4faea2b app/upgrade: use signal to determine if changed
Make `upgrade` use the same trick as `deploy` to determine if a new
deployment was laid down. Apart from those two, all other operations
that can lay down a new deployment always do so in the happy path.

Prep for further work.

Closes: #983
Approved by: cgwalters
2017-09-11 18:38:43 +00:00
Jonathan Lebon
df515fd2f1 app/libbuiltin: convert to new style
Pretty straightforward stuff. Prep for further work.

Closes: #983
Approved by: cgwalters
2017-09-11 18:38:43 +00:00
Colin Walters
150de0087e scripts: Support overriding lua, use for glibc-all-langpacks
Today in Fedora the `glibc-all-langpacks.posttrans` is implemented
in lua, for no good reason.  See:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585

Since that's stalled out, let's add support for overrides.  This
is obviously a much bigger step with more long term maintenance
implications over our current "ignore scripts" list.  But we can't
block either.

This is needed for unified core work:
https://github.com/projectatomic/rpm-ostree/issues/729

(We also override `fedora-release-atomichost` but I'll likely
 submit a patch for that upstream)

Closes: #980
Approved by: jlebon
2017-09-08 18:02:22 +00:00
Colin Walters
e40eaebbe0 core: Use fd-relative access to rpmdb
I was linking to this code from elsewhere and noticed that
for our hardlink breaks we were not using fd-relative even
though we can.  Down the line if we fork librpm into a separate
process and do e.g. `--dbpath=.` it'll do it too.

(Side note, I verified that commenting out the hardlink breaking
 here was caught by the `ostree fsck` I added to the test suite)

Closes: #979
Approved by: jlebon
2017-09-07 22:54:40 +00:00
Colin Walters
38d1876270 Update libglnx, do some fstatat-noent porting
Started on porting to the new `glnx_fstatat_allow_noent()`.  The
usage varies a lot and it felt easy to screw up, so I'm just
starting by doing a few of them.

Update submodule: libglnx

Closes: #978
Approved by: jlebon
2017-09-07 22:30:34 +00:00
Colin Walters
79086f8c3d core: Use a single tmpdir/ostreetxn for relabeling
I'd like to cut down on the number of tempdirs we make; when
debugging with gdb I often use e.g. `r` to restart forcibly, and
that leaves the dirs around.

Previously we'd make a tmpdir per relabeled package; this hoists it into a
well-known "relabel/" subdir of the per-core tmpdir we make. (A next step here
is to dedup the core tmpdir with the tmpdir that the upgrader makes)

Further while I'm here, let's use a single OSTree transaction for all of the
relabeling; the rationale here is that it's better to do fewer `syncfs()`
operations.

Closes: #970
Approved by: jlebon
2017-09-07 19:04:08 +00:00
Colin Walters
b5c98ec4f1 daemon: Add a policy for reload-config
This fixes `rpm-ostree reload` as root, and supports configuring
it to be enabled for other users as well.  This was overlooked
in the polkit work originally.

Closes: https://github.com/projectatomic/rpm-ostree/issues/976

Closes: #977
Approved by: jlebon
2017-09-07 15:11:00 +00:00
Jonathan Lebon
b0c75ae900 vmcheck: use --selinux-policy when committing
Let's make SELinux and atomic-host-tests happy with trees we concoct
ourselves by using the new --selinux-policy. (Specifically, we want our
sync'ed binaries to have install_exec_t).

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Jonathan Lebon
50ecc04cb6 overlay.sh: drop hack for backup passwd files
We shouldn't need this anymore.

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Jonathan Lebon
b22d7d764d vmcheck: rename build.sh to install.sh
I had meant for this to be in the other PR#968. I originally did both
`make` and `make install` there, but now it only does `make install`, so
let's just rename it to make that more obvious.

Closes: #953
Approved by: cgwalters
2017-09-07 14:41:19 +00:00
Colin Walters
c093a587a2 tests: Add a test case for lua scripts
This is currently unsupported, let's add a test case. Prep for support for
script overrides.

Closes: #973
Approved by: jlebon
2017-09-07 14:09:47 +00:00
Colin Walters
f69717bf69 libpriv+daemon: Use autocleanup to abort ostree txn
This allows porting one function each in the unpacker and sysroot to new style.
There were also two cases in the core where we were missing an
`abort_transaction()` invocation.

libostree version in: https://github.com/ostreedev/ostree/pull/1132
Prep for: https://github.com/projectatomic/rpm-ostree/pull/970

Closes: #972
Approved by: jlebon
2017-09-07 13:50:19 +00:00
Colin Walters
20abc85243 unpacker: Translate /boot → /usr/lib/ostree-boot
At one point `rpm-ostree install libvirt` dragged in libguestfs which in turn
brought in `syslinux-extlinux-nonlinux` which has files in `/boot/extlinux`,
which we rejected.  (That dependency chain appears to have been fixed currently)

For the general case, this is just a partial fix in that we haven't nailed down
the semantics of how updates for `/boot` work.  But in this particular case,
we'll just break libguestfs' `extlinux` verb, which I'm OK with.

Another case is `fwupdate-efi` - we require manual intervention to copy the
data into `/boot` after installing the package.

This is also preparation for [unified core](https://github.com/projectatomic/rpm-ostree/issues/729)
in that we now ensure imported kernels don't end up in `/boot` unless
explicitly configured.

Closes: https://github.com/projectatomic/rpm-ostree/issues/853

Closes: #969
Approved by: jlebon
2017-09-07 13:30:36 +00:00
Ruixin Bao
d6b3217a86 lib/core: Detect file name conflict before rpmdb
This PR uses https://github.com/ostreedev/ostree/pull/1116
to ensure that when installing packages containing conflicting files, the
error will be detected at the ostree side, rather than when
writing the rpm database.

Closes: https://github.com/projectatomic/rpm-ostree/issues/365

Closes: #974
Approved by: cgwalters
2017-09-07 13:13:12 +00:00
Jonathan Lebon
48ddca9280 ci: workaround broken fedora:26 image
This is essentially the same workaround as
https://github.com/ostreedev/ostree/pull/1143.

See https://bugzilla.redhat.com/show_bug.cgi?id=1483553.

Closes: #975
Approved by: cgwalters
2017-09-07 12:27:54 +00:00
Jonathan Lebon
7fbf9f32d4 hacking: make it easier to use a custom tree
Let's make using a custom install tree easier and document the process.
We split out the insttree step into `build.sh` so that we no longer have
to `flock(1)` around it, and also share between `overlay.sh` and
`sync.sh`.

Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Jonathan Lebon
5294c1ea69 scripts: squash a -Wmaybe-uninitialized
Closes: #968
Approved by: cgwalters
2017-09-01 19:58:55 +00:00
Colin Walters
a78a88844c libpriv/scripts: Use gperf comments, not C comments
I noticed our C-style comments were actually showing up in the generated hash
table. Consistently use gperf's `#`, and also move all comments to the line
above the value for more readability.

Closes: #967
Approved by: jlebon
2017-09-01 17:58:51 +00:00
Colin Walters
213a656ddf postprocess: Unlink our treecompose-post out of the final /bin
Another variant of <https://github.com/projectatomic/rpm-ostree/pull/822> 😭

Closes: #964
Approved by: jlebon
2017-08-31 03:06:11 +00:00
Colin Walters
3047513509 core: Run %post before %posttrans
While working on unified core and the Fedora Atomic Host content set, I hit a
dependency between `docker.posttrans` which tries to read `/etc/os-release`, and
`fedora-release-atomichost.post` which creates that symlink.

It seems best practice to me to run `%post`s strictly before
`%posttrans`; we're not likely to do parallelization anytime
soon anyways.

While here I cleaned things up by having an enum for the script kind,
rather than multiple functions, otherwise we would have had another
wrapper in core.c.

Closes: #963
Approved by: jlebon
2017-08-30 15:50:09 +00:00
Colin Walters
4cbdcf93c8 daemon/syscore: Add some error prefixing to cleanup paths
Would likely have helped clarify this issue where an ostree repo txn failed
post-bootloader txn: <https://github.com/projectatomic/rpm-ostree/issues/958>

Closes: #960
Approved by: jlebon
2017-08-29 13:24:06 +00:00
Jonathan Lebon
44ea9f35a1 ci: also compile tests
This should make sure we catch test errors such as
https://github.com/GNOME/libglnx/pull/76 when bumping submodules (until
we eventually get a PR tester on libglnx).

Requires: https://github.com/GNOME/libglnx/pull/76

Update submodule: libglnx

Closes: #954
Approved by: cgwalters
2017-08-29 13:07:21 +00:00
Colin Walters
8a87887988 libpriv/postprocess: Port add-files to fd-relative
This is the last bit that used a `yumroot` variable.

Closes: #950
Approved by: jlebon
2017-08-28 13:46:44 +00:00
Colin Walters
d469323693 libpriv/passwd: Fully complete fd-relative porting
I initially thought I wouldn't need this but it turns out it makes things nicer
unified core. In the legacy path we'll need to write
`etc/passwd`, but now we need `usr/etc/passwd` in the unified path, and
we can start to match up with the other code paths which are fd-relative.

Closes: #950
Approved by: jlebon
2017-08-28 13:46:44 +00:00
Colin Walters
3f5cc629dc lib/unpacker: Add a SKIP_EXTRANEOUS flag, use it for filesystem+rootfiles
Prep for unified core. For packages we know ship stuff we want to skip, enable
this flag.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
acbf00724d lib/unpacker: Drop ostree convention flag, always enable it
The core use case always enabled this; we exposed it on the command line, but
honestly I think it's a lot simpler if our imports always enable this. If we
need to *undo* it (for example, generating a "traditional feeling" OCI/Docker
image a la <https://github.com/projectatomic/rpm-ostree/pull/367>), we can
easily do that as postprocessing afterwards.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Colin Walters
5633424628 lib/unpacker: Fix theoretical error handling case
We should be using `g_propagate_error()` to ensure we handle the case where the
caller specifies `NULL` for the error.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Jonathan Lebon
e605386818 daemon: make use of the new deployment flags
Now that we have the semantics we need in libostree, let's just use that
and drop the logic here.

Closes: #942
Approved by: cgwalters
2017-08-25 18:59:24 +00:00
Jonathan Lebon
2f6f8a910d libvm: set up ControlPath socket in /var/tmp
It turns out there's a much smaller limit than PATH_MAX for Unix
sockets. On Linux, it's 108 characters. It took me some time to figure
out why `vmcheck` would sometimes fail depending on where the src
directory is and how ${topsrcdir} is defined. Let's just make things
safer by just using /var/tmp.

Closes: #949
Approved by: cgwalters
2017-08-24 22:12:17 +00:00
Jonathan Lebon
b05b9679f9 scripts: fix argument ordering
Just noticed this while trying out the unified core work. We were
passing arguments in the wrong order (or alternatively, receiving them
in the wrong order).

Closes: #952
Approved by: cgwalters
2017-08-24 21:48:14 +00:00
Jonathan Lebon
f5bcd62364 ci: run the improved-sanity-test
Start running the improved-sanity-test from atomic-host-tests to make
sure the system works. This is an easy way to get PR-level comprehensive
integration tests for free.

But note that we *don't* mark it as required since the tests are not
stored here and it can happen that they need to be adjusted for new
rpm-ostree behaviours. In this way, this added check also allows us to
give a heads-up that breaking changes are coming.

Closes: #899
Approved by: cgwalters
2017-08-24 20:43:16 +00:00
Jonathan Lebon
c7837d003a Makefile-tests.am: drop vmshell target
I actually stopped using this a long time ago. Looking at it now, it
doesn't make much sense in multi-host situations. Let's just nix it.

Closes: #899
Approved by: cgwalters
2017-08-24 20:43:16 +00:00
Jonathan Lebon
a1b5109e8b docs/manual: be more explicit about bootloader order
Make it more clear that bootloader order implies what the default
deployment is.

Closes: #947
Approved by: cgwalters
2017-08-24 20:13:05 +00:00
Colin Walters
00231446b3 app: Clarify message on repeated Ctrl-C
While we have cancellation issues¹, let's help the
user understand that we heard them the first time.

¹ <https://github.com/projectatomic/rpm-ostree/issues/897>

Closes: #948
Approved by: jlebon
2017-08-24 17:30:10 +00:00
Colin Walters
06f3a1811a lib/passwd: Port passwd checking to fd-relative
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
a4ca8c8888 app/compose: Drop GFile usage from package install path
Prep for unified core.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
addc53a707 tree-wide: Fix extra colon in error prefixes
This is a trap when doing style porting; we added a rule to ostree's `maint.mk`
to catch it but not here. Not sure I want to add `maint.mk` here yet.

Note that in newer libglnx `glnx_opendirat()` does error prefixing, so
drop the prefixing there.

Closes: #943
Approved by: jlebon
2017-08-24 15:13:30 +00:00
Colin Walters
278ed216d0 tree-wide: A bit more libglnx porting
Mostly for `glnx_renameat()`.

Closes: #941
Approved by: jlebon
2017-08-24 13:33:59 +00:00