Commit Graph

150 Commits

Author SHA1 Message Date
Colin Walters
a347ff5f69 postprocess: Port remaining libgsystem call sites
Except for `gs_file_get_path_cached()`.  We have to carry
the recursive hardlinking code, oh well.

Closes: #511
Approved by: jlebon
2016-11-08 14:38:54 +00:00
Colin Walters
5c8d80042c postprocess: More fd-relative-ification
I hit a bit of a wall here around `gs_shutil_cp_al_or_fallback()`.
I guess we'll have to reimplement it here.

In the meantime, this gets us closer.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
634a475ae4 postprocess: Drop a libgsystem use in add-files
I'm cowardly just using gs_file_get_path_cached() for now.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
e0357f41d1 postprocess: Port tmpfiles.d-copying code to fd-relative
Drops libgsystem use.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
68898217ed postprocess: Drop use of gs_file_rename(), use fd-relative
Another libgsystem use gone.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
1e6f4d4f21 compose: Create rootdir more directly, use fd-relative
Mostly to drop a libgsystem use.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
7ee4edf8e3 tree-wide: Remove more unused variables
Now that the other PRs are merged, kill of the rest of the unused
variables.

Closes: #504
Approved by: jlebon
2016-10-28 18:25:33 +00:00
Colin Walters
aed6077e73 core: Fix lock directory initialization
Though in practice, we just aren't using this right now.

Closes: #504
Approved by: jlebon
2016-10-28 18:25:33 +00:00
Colin Walters
5100506909 compose: Port a large chunk of postprocessing to fd-relative
Part of the de-libgsystem-ification porting.

Closes: #498
Approved by: jlebon
2016-10-27 14:57:43 +00:00
Colin Walters
1c2908cbef tree-wide: Port away from gs_transfer_out_value()
Using `g_steal_pointer()` is a little bit more typing, but less
magical which is good.

Closes: #498
Approved by: jlebon
2016-10-27 14:57:42 +00:00
Colin Walters
94a461cb4e tree-wide: Remove some unused variables found by CLang
This isn't all of them, but I'm trying to avoid conflicting
with my other outstanding PR.

Closes: #501
Approved by: jlebon
2016-10-27 14:30:47 +00:00
Colin Walters
49fffad471 Run bwrap with fixed environment (PATH + LANG)
This is basically a re-implementation of
85c9dd5c07
for the same reasons: it makes things work for rpm-ostree running
inside NixOS.  But there are other cases where if e.g. an unprivileged
user runs rpm-ostree and may not have `/usr/sbin` in PATH, we still
want the container to pick it up, etc.

While changing things I nuked the hacky `RPMOSTREE_DEBUG_SCRIPT` env
var, we can re-add it later in a better way if needed.

Closes: #490
Approved by: jlebon
2016-10-17 20:29:35 +00:00
Patrick Uiterwijk
98332a3be4 compose: Add option to write commitid to file instead of updating ref
Closes: #492
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>

Closes: #494
Approved by: cgwalters
2016-10-17 20:02:55 +00:00
Colin Walters
4542778648 tree-wide: Port to g_file_enumerator_iterate()
Unlike OSTree, I'm going to assert we can take a hard dependency
on the version of GLib that has it.

Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
4ccd31fcb5 tree-wide: Port to libglnx for shutil_rm_rf
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
b47301b3e1 tree-wide: Port to libglnx autocleanup macros
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
a2ab0e4415 tree-wide: Port to glnx_set_error API
Away from libgsystem.

Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
a15364c185 compose: Respect absolute paths for postprocess-script again
Regressed from some code refactoring.

Closes: #491
Approved by: jlebon
2016-10-14 17:29:43 +00:00
Colin Walters
56f3f509a1 core: Canonicalize an epoch of zero to the empty string
We have two different codepaths for creating a cache branch header
string, one from libdnf, and one from librpm.  It turns out if
an RPM package explicitly specifies an `Epoch: 0` like various Fedora
perl subpackages do, these are different.

Explicitly convert `0:` to the empty string to make them match, and
hence installation will work.

Closes: https://github.com/projectatomic/rpm-ostree/issues/349

Closes: #482
Approved by: jlebon
2016-10-12 18:09:30 +00:00
Daniel Peebles
b32bc75b6c Get rid of unnecessary selinux.h include
Nothing in the file seems to use it. Doing this quickly via GitHub UI under assumption that the magic @walters bot will take care of running the full test suite, but otherwise I'll test it later when I get back to a machine I can use for that.
Closes: #481
Approved by: cgwalters
2016-10-12 16:25:45 +00:00
Colin Walters
4a86bdd196 postprocess: Undo Fedora SELinux policy move
Basically, this brings us back to where we are with RHEL/CentOS 7.

For lots more information, see
https://bugzilla.redhat.com/show_bug.cgi?id=1290659

Closes: #479
Approved by: jlebon
2016-10-07 15:57:15 +00:00
Jonathan Lebon
485d170c25 core: update some comments re. relabeling
Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
dbbe8d0687 core: Don't use parent commits for pkg cache
This makes the design fully coherent now - we don't try to store
multiple commits, we no longer reference them, and the cleanup does a
depth=0 pruning.

Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
f9f326f0f9 core: Don't try to walk up pkgcache parent chains
My desktop system was aborting on upgrade with a missing metadata
object, which turned out to be the fact that my previous changes to
the package layering code pruned with depth=0, but the commit it was
looking for had a parent commit that had been pruned.

Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
7dd28fa964 core: Fix erroneous ret = TRUE
I hit a pile of:
```
Oct 04 12:44:15 icarus.verbum.local rpm-ostreed[26257]: ostree_repo_resolve_partial_checksum: assertion 'error == NULL || *error == NULL' failed
```

Which turned out to be a missing metadata object (for some reason),
but this function's incorrect use of `ret = TRUE` caused the GError
to have already been set.

Fix this, and we change to "direct return" style which is more
readable.

Closes: #474
Approved by: jlebon
2016-10-04 17:55:10 +00:00
Colin Walters
1281434cec postprocess: Print absolute path for ignored files in /var
For ease of debugging.  Maybe in the future we try to look them up in
the RPM database too.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Colin Walters
0cd584ccf6 postprocess: Silently clean up known-unnecessary files in /var
Per OSTree design, /var can start out empty.  However, our warning
spam here is annoying.  Let's first delete some known files -
obviously this won't be exhaustive, but it's way faster than trying to
fix all of this in the packages right now.

The major one is the SELinux policy, which resulted in a lot of spam.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Colin Walters
0fd83d47b7 passwd: Also clone subuid/subgid files
These were added to shadow-utils in recent times (Fedora 25 e.g.)  and
like the other passwd files, shadow-utils opens them with `O_TRUNC`.

(At some point we should patch it to do create-new-then-replace internally)

Closes: https://github.com/projectatomic/rpm-ostree/issues/458

Closes: #459
Approved by: jlebon
2016-09-13 13:30:15 +00:00
Colin Walters
0649567ffc compose: Nuke mock workaround
It turns out it was buggy (for some reason `PS1` wasn't propagating),
and furthermore, things mostly work if one enables `--new-chroot` i.e.
`systemd-nspawn`, which is what Fedora is going to do, and everyone
else in the world uses Docker.

While we're here, tweak the error message to use `<>` around the URL
which makes it more easily clickable from terminals.

Closes: #449
Approved by: jlebon
2016-09-07 21:57:01 +00:00
Colin Walters
665083dff9 core: Define an autocleanup macro for Header
And switch a few things to `return FALSE`.  Preparatory work
for other cleanups.

Closes: #441
Approved by: jlebon
2016-08-31 20:15:14 +00:00
Colin Walters
525c605092 diff: Don't print changelogs by default
Communicating information is a hard problem, but basically there
are two things here.  First, the changelogs make the diff visual
spaghetti.  (If we were rendering to HTML with expanders or something,
it could work).

Second, I think RPM changelogs are legacy - tooling should pull from
git.

So far when doing release announcements I've been manually stripping
out the changelogs, embarassingly enough.  Time to fix that.

Anyways, we preserve the changelog-emitting code, it's just not the
default now.

Closes: #439
Approved by: jlebon
2016-08-30 20:13:35 +00:00
Colin Walters
436a8116f2 status: Sort the output package list
I am thinking of indenting into multiple lines if it gets long too,
but this is a small visual improvement.

Closes: #440
Approved by: jlebon
2016-08-30 20:09:24 +00:00
Colin Walters
35d84f7507 pkglayering: Commit in base repo, not pkgcache
Currently, we do the final commit into the pkgcache repo, then
pull it to the base.  The problem with this is that, combined
with the fact that we're not presently pruning the pkgcache repo,
we leak space.

In preparation for a cleaner fix for this, rework things so that the
core infra can know about *both* a base repo and a pkgcache repo.  If
they're separate (as is the case for rpm-ostree-on-host), whenever we
are doing layering, explicitly pull just the `.file` objects that are
referenced into the parent repo.  We do the final commit in the base
repo.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Jonathan Lebon
0fe12fe6a3 mutate-os-release: skip VERSION_ID
I hit this with librepo subbing out the $releasever with e.g. 7.2016.1
when trying to pull various URLs. It should be enough for the user to
see the ostree version in VERSION and PRETTY_NAME. For applications,
there's OSTREE_VERSION if they need just that.

Closes: #433
Approved by: cgwalters
2016-08-17 20:45:29 +00:00
Colin Walters
589a8e0974 core: Neuter systemctl during script execution
`systemctl start/stop/try-restart` are all useless for us in
scripts, since changes should only affect the *next* boot.

`systemctl enable` is also wrong - one should use presets instead.

Currently, systemd has code to detect whether it's inside
a chroot, which works for mock, but *not* for Docker or bubblewrap.
(We should teach systemd a nicer way to disable itself, but
 even if we did that we'd have to support old scripts)

So, this fixes layering `glusterfs` in CAHC.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
6900c616ec scripts: Execute %pre
Now that we handle `useradd`, which is most of what anyone wants
to do in `%pre`, let's turn it on.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
550adecb47 core: Support scripts which call useradd
When executing scripts, map `/usr/lib/passwd` temporarily back to
`/etc/passwd` from the POV of the script, then move it back.  This
allows client-side layered packages be merged with the base tree
version, while still being distinct from the `/etc/passwd`.

(In the future we'll likely rewrite all of this to use
`systemd-sysusers(8)`, but for now let's be incremental.)

Note in this commit it isn't very useful, because pretty much all RPMs
only call `useradd` in `%pre`, which is the next commit.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
dbd2c0ee7a unpacker: Synthesize tmpfiles.d for nonroot-owned /run and /var files
Because OSTree includes numeric uid/gid, but package layering (and
traditional full-client-side RPM) expects to allocate UIDs on the
machine, we need to handle non-root owned files specially.

For files in /run and /var, we can convert them to
`systemd-tmpfiles(8)` snippets which will be executed at the next
boot, after a uid/gid has been assigned.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
3ad4e6c72b bwrap/compose: Add a workaround for Fedora's use of rpm-ostree-in-mock
Decided to test this on Sunday evening.  Of course it was broken =(
(Actually I tested mock-in-Docker but it should be the same)

The core problem is that mock does `chroot()` without using `/`
as a mount point.  This breaks an assumption in bwrap that it is.
Now, in theory we could move this same logic down into bwrap to
work around this situation, but for now let's hack it here.

Mock is old, legacy container code that doesn't really do anything
in a modern way - in fact our goal should be to replace it
with a combination of rpm-ostree and bwrap.  So carrying this
hack here to get us to that future should be OK for now.

Closes: #431
Approved by: jlebon
2016-08-15 18:31:58 +00:00
Colin Walters
7455e26273 bwrap: Add a selftest
I want a better error message if the user happens to execute
inside e.g. a Docker container without sufficient privileges
for recursive containerization.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
551e4c91f9 postprocess: Switch to using bwrap for script execution
The previous commit https://github.com/projectatomic/rpm-ostree/pull/422
introduced a regression in the "outside of a container" path - we
get `EINVAL` trying to `mount("proc",...)` and honestly I'm not sure why.

We can either back up or plow forward, and it turns out to be
pretty straightforward to complete the port to using bwrap.

I extracted the bwrap-execution code out of the RPM script engine,
because the treecompose model is currently different (no hardlinks
yet).

NOTE: A *very* important side effect of this is that we now
require "privileged" containers on hosts without user namespaces,
and on userns hosts, require `CLONE_NEWUSER` to be exported to the
container host.

In general though, the previous path of blindly executing scripts as root
without e.g. `proc` mounted was just bad.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
88a130da09 lib: Extract bwrap-executing internal API
The treecompose code will learn how to use bwrap instead of
libcontainer in libglnx, since the latter is a buggy copy of a subset
of the former.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
a299083b09 core: Fix an error-handling logic error in relabel path
If next_dent returns `FALSE`, we need to also return `FALSE` from
the function.

I just noticed this as I happened to be reading the code randomly.

Closes: #426
Approved by: jlebon
2016-08-11 19:40:29 +00:00
Colin Walters
19490c593e Update to latest libdnf
I want the excludes= fix, and the memleak fixes are nice.

Closes: #425
Approved by: jlebon
2016-08-11 19:26:06 +00:00
Colin Walters
56a4f73901 scripts: Error out if we encounter lua
Otherwise we try to execute as shell script which obviously doesn't
work.  This was noticed when @dustymabe was trying to assemble
a Fedora container with `glibc-all-langpacks`.

Closes: #424
Approved by: jlebon
2016-08-11 15:32:49 +00:00
Colin Walters
3d1040a780 compose: Port bits of postprocessing to fd-relative
I was looking at starting to unify the treecompose path to the
"new world" infrastructure in package layering.  An initial
step here is to port to fd-relative, which cleans up the code.

Note this depends on a libglnx pull.

Closes: #422
Approved by: jlebon
2016-08-08 18:11:42 +00:00
Colin Walters
ff8e2992da libglnx porting: Switch to glnx_file_get_contents_utf8_at()
A small one, but a start.

Closes: #421
Approved by: jlebon
2016-08-08 18:07:46 +00:00
Colin Walters
a852ad3459 Fix accidental double __ in previous commit
My sed powers got overly enthusiastic.

Closes: #417
Approved by: giuseppe
2016-08-04 12:33:07 +00:00
Colin Walters
e18d43bad3 Port to new ostree_repo_checkout_at()
Squashes some deprecation warnings, and now with the latest master we
can stop setting `disable_fsync`.

Closes: #418
Approved by: giuseppe
2016-08-04 12:21:38 +00:00
Colin Walters
39913a2c25 Update for libhif -> libdnf
Still not many PRs outstanding, so it's a good time to take this plunge.
Mostly automated via `sed`.

Closes: #416
Approved by: Conan-Kudo
2016-08-04 01:10:08 +00:00