Commit Graph

79 Commits

Author SHA1 Message Date
Colin Walters
9ab46f5221 Require ostree 2017.5
This is preparatory for future changes which will make use the newer sysroot
writing API. But in this change, we can drop all of our version ifdef'd code.

Closes: #740
Approved by: jlebon
2017-04-20 20:58:51 +00:00
Colin Walters
b060ecda21 unpacker: Also set flag to make failure to label fatal
There's no immediate need for this; was just a lead
I was chasing down while debugging
https://github.com/ostreedev/ostree/pull/775

Closes: #724
Approved by: jlebon
2017-04-05 13:19:39 +00:00
Jonathan Lebon
7b9f4870bd dracut: always specify kver
Otherwise dracut will just default to the running kernel, which will be
wrong if the pending deployment doesn't have the same kernel. Use --kver
rather than appending to be more explicit.

Closes: #722
Approved by: cgwalters
2017-04-04 18:47:31 +00:00
Colin Walters
c1872bc29e postprocess: Tell ostree to error out on missing labels
See https://github.com/ostreedev/ostree/pull/768 - basically
`/proc` is unlabeled, and this gets annoying for livefs work
since it shows up as a diff.

Plus at some point in the future `ostree fsck` might actually learn to verify
deployment directories.

Closes: #714
Approved by: jlebon
2017-03-31 14:30:03 +00:00
Colin Walters
f71e1b8b4d all: Conditionally use new ostree_sepolicy_new_at()
If available, using the also-new `OSTREE_CHECK_VERSION`. I dropped the `path`
argument from one of the internal APIs since it made the code simpler, and every
caller except one was passing `.`.

Closes: #712
Approved by: jlebon
2017-03-30 13:06:48 +00:00
Colin Walters
a05e256177 compose: Delete /usr/etc/passwd- (and the other variants)
There's no point to shipping these backup files in the base tree. We already had
code to delete them for the package layering case where they caused active harm.
At the point we added that code we really should have *also* changed treecompose
to delete them. Better late than never.

The reason I'm doing this now is because having them in the base tree causes `ex
livefs` to spuriously think that layering a package that *doesn't* change `/etc`
as if it does, because the layering code deletes the backup files.

Closes: #693
Approved by: jlebon
2017-03-20 16:35:17 +00:00
Colin Walters
62a93c7850 postprocess: Handle f26 /etc/nsswitch.conf configuration
F26 put sss first, which broke our regexp. When we switch to sysusers, man it'll
be nice to dump ♲ this.

Closes: https://github.com/projectatomic/rpm-ostree/issues/685

Closes: #686
Approved by: jlebon
2017-03-16 18:39:38 +00:00
Jonathan Lebon
5c5ebc1b0a core: add support for installing from cache
We start by adding support in the core for installing packages strictly
from the cache repo. We fool the libdnf stack by re-exporting the header
as an RPM, and explicitly marking it for install. The treefile format
supports specifying the expected SHA-256 of the metadata header, in case
the cache for a specific NEVRA changed.

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Jonathan Lebon
6a5d4a2699 RpmOstreeOrigin: also cache initramfs args
Ideally, we want to parse out everything in the origin on init and then
cache the values to make it even easier to inspect it. We make this more
explicit by renaming the struct members to "cache_*" and also caching
the initramfs args. This doesn't actually save any allocations right
now (though it might in the future), but we do the same for everything
else, and it's nice to be consistent.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Colin Walters
cc20f32fb7 tree-wide: Use autoptr cleanup for HyQuery
This was a noticeable source of leaks.

Closes: #644
Approved by: jlebon
2017-02-24 18:55:36 +00:00
Chen Fan
7ee41f75b7 remove duplicate words in comment
Signed-off-by: Chen Fan <fan.chen@easystack.cn>

Closes: #635
Approved by: cgwalters
2017-02-21 14:47:59 +00:00
Jonathan Lebon
c756b26521 compose: fix mutate-os-release handling
- Let --add-metadata-string=version=val override any automatic version
  prefixing.
- Don't error out if mutate-os-release is given, but no new version is
  given by --add-metadata-string or automatic version prefixing.

Checking keys in parse_keyvalue_strings() is slightly hacky. I initially
wanted to just inspect the GVariantBuilder, but AFAICT, there is no way
to actually look up values from a builder (plus, we need that info early
to know whether automatic_version_prefix should itself inject in the
builder or not).

Closes: #603
Approved by: cgwalters
2017-02-01 22:25:54 +00:00
Colin Walters
cac4522e5b Support "system/regenerate-initramfs=true" flag in origin
Currently we push for a model where the initramfs is
generated (in non-hostonly mode), and merely replicated.

However, to support a few unfortunate corner cases like dm-multipath which wants
to inject a config file into the initramfs, we need to support regenerating it
client side too.

Down the line, we'll need this to support overriding the kernel too.

This changes things in the core to add the concept of an "empty"
`RpmOstreeContext`. I initially tried skipping it, but that was too much
duplication. We still want all of the core ostree-related logic that lives in
that code too.

The treespec bits barfed if the spec didn't have a `tree/packages` key. It was
simplest to change that to allow it - and because that was the only case where
we errored out in parsing, I dropped the error handling.

There was another place in the upgrader that now needed to be fixed to handle
transitioning from just regenerating initramfs to not.

Closes: #574
Approved by: jlebon
2017-01-20 16:17:51 +00:00
Colin Walters
f215acb64f libpriv: Move kernel finalization into a helper function
This will need to be callable outside of postprocess for client side initramfs
generation.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
4c98943a64 postprocess: Clean up kernel/initramfs checksum calculation
This avoids having an intermediate file path, and makes the code easier to
follow.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
16efa12ae0 libpriv: Factor out "find kernel/initramfs" code into kernel.[ch]
To support running dracut on the client side, the dracut code
needs this, and it makes more sense in kernel.[ch] anyways.

I chose to use a GVariant instead of making a custom structure to avoid having
to manage custom free funcs.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
e1cdfa1712 libpriv: Introduce an rpmostree-kernel.[ch] for dracut
Since the dracut run is now separated from the rest of postprocess, we change it
to use the bwrap API diretly, and this lets use the new _IMMUTABLE bwrap type.
This will make it easier to reuse for client-side initramfs regeneration.

Splitting this off makes it also easier to simplify the remaining mutable
usage in postprocess.c.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:26 +00:00
Colin Walters
f0ec738376 libpriv: Enhance internal bwrap API
Make it a real struct with methods.  This noticeably increases
the ergonomics and design of the API.

The main goal here is to introduce the enum which defines whether or not the
rootfs is mutable or not. We move the "rofiles" mode from the RPM script code
down into the bwrap layer, which will make it easier to reuse for treecompose.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:26 +00:00
Colin Walters
50ab2983ab postprocess: Use O_TMPFILE for dracut, merge reproducible bits
I was planning to do some further changes here, and I really don't like the
manual fork/exec stuff on in the --reproducible checks. Our subprocess code
should basically be all bwrap.  Synchronous code execution while not reading from the
pipe is a recipe for deadlocks.

What simplifies things a lot is to write to an `O_TMPFILE` fd (or a tempfile on
legacy kernels), and slightly extend our bwrap-executing code to support a child
setup function, so we can set the tmpfile fd to be stdout.

Now that we have a shell script wrapper we inject, it's trivial to reimplement
the "detect reproducibility" in shell script there, rather than C.

This doesn't matter much for treecompose today, but it will matter more when
we're supporting client side initramfs regeneration, since now the dracut
container can be fully immutable.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Jonathan Lebon
0e431a8f83 core: add RPMOSTREE_USE_CACHED_METADATA
Being able to just reuse metadata is especially helpful when trying to
debug things lower down that path, as well as cuts about 2 mins from the
compose test.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Colin Walters
3a409342e6 postprocessing: Add a g_prefix_error for kernel
Saw this un-prefixed error path in a build; having error prefixes makes for
easier debugging.

Closes: #554
Approved by: giuseppe
2016-12-20 16:24:25 +00:00
Colin Walters
69cace4ce8 [ASAN] Squash a few minor misc leaks
This is part of the leak set when doing a treecompose.  There's
some that go down into libdnf...will get to that over time.

Closes: #547
Approved by: jlebon
2016-12-08 16:27:24 +00:00
Colin Walters
a347ff5f69 postprocess: Port remaining libgsystem call sites
Except for `gs_file_get_path_cached()`.  We have to carry
the recursive hardlinking code, oh well.

Closes: #511
Approved by: jlebon
2016-11-08 14:38:54 +00:00
Colin Walters
5c8d80042c postprocess: More fd-relative-ification
I hit a bit of a wall here around `gs_shutil_cp_al_or_fallback()`.
I guess we'll have to reimplement it here.

In the meantime, this gets us closer.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
634a475ae4 postprocess: Drop a libgsystem use in add-files
I'm cowardly just using gs_file_get_path_cached() for now.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
e0357f41d1 postprocess: Port tmpfiles.d-copying code to fd-relative
Drops libgsystem use.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
68898217ed postprocess: Drop use of gs_file_rename(), use fd-relative
Another libgsystem use gone.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
7ee4edf8e3 tree-wide: Remove more unused variables
Now that the other PRs are merged, kill of the rest of the unused
variables.

Closes: #504
Approved by: jlebon
2016-10-28 18:25:33 +00:00
Colin Walters
5100506909 compose: Port a large chunk of postprocessing to fd-relative
Part of the de-libgsystem-ification porting.

Closes: #498
Approved by: jlebon
2016-10-27 14:57:43 +00:00
Colin Walters
1c2908cbef tree-wide: Port away from gs_transfer_out_value()
Using `g_steal_pointer()` is a little bit more typing, but less
magical which is good.

Closes: #498
Approved by: jlebon
2016-10-27 14:57:42 +00:00
Colin Walters
49fffad471 Run bwrap with fixed environment (PATH + LANG)
This is basically a re-implementation of
85c9dd5c07
for the same reasons: it makes things work for rpm-ostree running
inside NixOS.  But there are other cases where if e.g. an unprivileged
user runs rpm-ostree and may not have `/usr/sbin` in PATH, we still
want the container to pick it up, etc.

While changing things I nuked the hacky `RPMOSTREE_DEBUG_SCRIPT` env
var, we can re-add it later in a better way if needed.

Closes: #490
Approved by: jlebon
2016-10-17 20:29:35 +00:00
Patrick Uiterwijk
98332a3be4 compose: Add option to write commitid to file instead of updating ref
Closes: #492
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>

Closes: #494
Approved by: cgwalters
2016-10-17 20:02:55 +00:00
Colin Walters
4542778648 tree-wide: Port to g_file_enumerator_iterate()
Unlike OSTree, I'm going to assert we can take a hard dependency
on the version of GLib that has it.

Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
4ccd31fcb5 tree-wide: Port to libglnx for shutil_rm_rf
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
b47301b3e1 tree-wide: Port to libglnx autocleanup macros
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
a15364c185 compose: Respect absolute paths for postprocess-script again
Regressed from some code refactoring.

Closes: #491
Approved by: jlebon
2016-10-14 17:29:43 +00:00
Colin Walters
4a86bdd196 postprocess: Undo Fedora SELinux policy move
Basically, this brings us back to where we are with RHEL/CentOS 7.

For lots more information, see
https://bugzilla.redhat.com/show_bug.cgi?id=1290659

Closes: #479
Approved by: jlebon
2016-10-07 15:57:15 +00:00
Colin Walters
1281434cec postprocess: Print absolute path for ignored files in /var
For ease of debugging.  Maybe in the future we try to look them up in
the RPM database too.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Colin Walters
0cd584ccf6 postprocess: Silently clean up known-unnecessary files in /var
Per OSTree design, /var can start out empty.  However, our warning
spam here is annoying.  Let's first delete some known files -
obviously this won't be exhaustive, but it's way faster than trying to
fix all of this in the packages right now.

The major one is the SELinux policy, which resulted in a lot of spam.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Jonathan Lebon
0fe12fe6a3 mutate-os-release: skip VERSION_ID
I hit this with librepo subbing out the $releasever with e.g. 7.2016.1
when trying to pull various URLs. It should be enough for the user to
see the ostree version in VERSION and PRETTY_NAME. For applications,
there's OSTREE_VERSION if they need just that.

Closes: #433
Approved by: cgwalters
2016-08-17 20:45:29 +00:00
Colin Walters
551e4c91f9 postprocess: Switch to using bwrap for script execution
The previous commit https://github.com/projectatomic/rpm-ostree/pull/422
introduced a regression in the "outside of a container" path - we
get `EINVAL` trying to `mount("proc",...)` and honestly I'm not sure why.

We can either back up or plow forward, and it turns out to be
pretty straightforward to complete the port to using bwrap.

I extracted the bwrap-execution code out of the RPM script engine,
because the treecompose model is currently different (no hardlinks
yet).

NOTE: A *very* important side effect of this is that we now
require "privileged" containers on hosts without user namespaces,
and on userns hosts, require `CLONE_NEWUSER` to be exported to the
container host.

In general though, the previous path of blindly executing scripts as root
without e.g. `proc` mounted was just bad.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
3d1040a780 compose: Port bits of postprocessing to fd-relative
I was looking at starting to unify the treecompose path to the
"new world" infrastructure in package layering.  An initial
step here is to port to fd-relative, which cleans up the code.

Note this depends on a libglnx pull.

Closes: #422
Approved by: jlebon
2016-08-08 18:11:42 +00:00
Colin Walters
ff8e2992da libglnx porting: Switch to glnx_file_get_contents_utf8_at()
A small one, but a start.

Closes: #421
Approved by: jlebon
2016-08-08 18:07:46 +00:00
Colin Walters
39913a2c25 Update for libhif -> libdnf
Still not many PRs outstanding, so it's a good time to take this plunge.
Mostly automated via `sed`.

Closes: #416
Approved by: Conan-Kudo
2016-08-04 01:10:08 +00:00
Colin Walters
c4e98f3f47 compose: Print progress during ostree commit
Until we finally do the "store packages as ostree commits and union"
thing, doing commits can be slow, particularly into archive repos
where we pay lots of cost in gzip.

Let's show a progress bar.  The implementation here uses a background
thread which communicates with the "UI" via atomics.  The UI uses a
timer - and if stdout isn't a tty, we assume it's Jenkins or something
and dial updates back to every 5 seconds to avoid spamming output.

Closes: #409
Approved by: giuseppe
2016-08-01 15:55:12 +00:00
Colin Walters
563fcd5750 compose: Print stats at end of commit
`ostree commit --table-output` can already do this, and since we're
super noisy for `compose tree` right now, we might as well print this.
It's useful to see how many new objects were created.

Closes: #409
Approved by: giuseppe
2016-08-01 15:55:12 +00:00
Colin Walters
c9c1483f58 postprocess: Also pick up user.pax.flags
This is in use by [PaX](https://en.wikipedia.org/wiki/PaX); see also
the [Arch Linux wiki](https://wiki.archlinux.org/index.php/PaX).

Closes: #412

Closes: #414
Approved by: copumpkin
2016-07-29 18:39:26 +00:00
Jonathan Lebon
b3c3c029f0 compose: add "mutate-os-release" treefile key
Users/administrators can now enable the "mutate-os-release" string
key to tell rpm-ostree to modify /etc/os-release in the following ways:

- All occurrences of the key's value found in VERSION, VERSION_ID, and
  PRETTY_NAME will be replaced by the version string of new compose.
- A new OSTREE_VERSION key is appended containing the version string of
  the new compose.

This provides an easy way for clients and third-party applications to
find out what version they are currently running.

Also bump libglnx so that we can use the latest version of
glnx_file_replace_contents_at() which supports AT_FDCWD.

Closes: #410
Approved by: cgwalters
2016-07-28 00:29:41 +00:00
Colin Walters
cc2551dc06 Switch to glib cleanup macros
We don't have a lot of outstanding changes to the C code, so now seems
like a good time to do this.  I implemented this with some highly
sophisticated sed commands like:

```
find -name '*.c' | while read name; do sed -i -e 's,gs_unref_object \([A-Za-z]*\) \*,g_autoptr(\1),' ${name}; done
```

Part of dropping the dependency on libgsystem, same as what we're
doing in ostree.

Closes: #393
Approved by: jlebon
2016-07-15 13:08:58 +00:00
Jonathan Lebon
8016319bcb postprocess: fix typo in error msg
Closes: #381
Approved by: cgwalters
2016-07-07 21:54:57 +00:00