Commit Graph

1503 Commits

Author SHA1 Message Date
Jonathan Lebon
485d170c25 core: update some comments re. relabeling
Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
dbbe8d0687 core: Don't use parent commits for pkg cache
This makes the design fully coherent now - we don't try to store
multiple commits, we no longer reference them, and the cleanup does a
depth=0 pruning.

Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
f9f326f0f9 core: Don't try to walk up pkgcache parent chains
My desktop system was aborting on upgrade with a missing metadata
object, which turned out to be the fact that my previous changes to
the package layering code pruned with depth=0, but the commit it was
looking for had a parent commit that had been pruned.

Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
7dd28fa964 core: Fix erroneous ret = TRUE
I hit a pile of:
```
Oct 04 12:44:15 icarus.verbum.local rpm-ostreed[26257]: ostree_repo_resolve_partial_checksum: assertion 'error == NULL || *error == NULL' failed
```

Which turned out to be a missing metadata object (for some reason),
but this function's incorrect use of `ret = TRUE` caused the GError
to have already been set.

Fix this, and we change to "direct return" style which is more
readable.

Closes: #474
Approved by: jlebon
2016-10-04 17:55:10 +00:00
Colin Walters
4852543ecc compose: Support packages-$basearch
See
https://mail.gnome.org/archives/ostree-list/2016-April/msg00020.html

Closes: #305
Approved by: jlebon
2016-10-04 15:10:32 +00:00
Colin Walters
1281434cec postprocess: Print absolute path for ignored files in /var
For ease of debugging.  Maybe in the future we try to look them up in
the RPM database too.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Colin Walters
0cd584ccf6 postprocess: Silently clean up known-unnecessary files in /var
Per OSTree design, /var can start out empty.  However, our warning
spam here is annoying.  Let's first delete some known files -
obviously this won't be exhaustive, but it's way faster than trying to
fix all of this in the packages right now.

The major one is the SELinux policy, which resulted in a lot of spam.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Jonathan Lebon
80267a423a status: handle errors from json-glib
The API for json_generator_to_stream() says:

    Return value: %TRUE if the write operation was successful, and
    %FALSE on failure. In case of error, the #GError will be filled
    accordingly

When in fact, because it just gives back what g_output_stream_write(),

    1. it doesn't strictly return TRUE/FALSE, but a full-range uint, and
    2. it will return -1 (which is as good as anything > 0 for success
       checks) when an error occurs.

Although a simple <= 0 would fix it, let's just be extra strict and also
check for an error.

Closes: #468

Closes: #469
Approved by: miabbott
2016-09-26 14:49:51 +00:00
Jonathan Lebon
fb081ff10a docs: update link to sample and link to some dists
Closes: #466

Closes: #467
Approved by: miabbott
2016-09-22 21:47:26 +00:00
Colin Walters
63f0f5af47 packaging: Add a cccp.yml and tweak Dockerfile
Let's try out https://wiki.centos.org/ContainerPipeline

Having maintained Docker images for rpm-ostree seems kind of overdue.
(I didn't actually test the CP bits since I'm not sure how to do that)

Closes: #460
Approved by: jlebon
2016-09-13 13:35:12 +00:00
Colin Walters
0fd83d47b7 passwd: Also clone subuid/subgid files
These were added to shadow-utils in recent times (Fedora 25 e.g.)  and
like the other passwd files, shadow-utils opens them with `O_TRUNC`.

(At some point we should patch it to do create-new-then-replace internally)

Closes: https://github.com/projectatomic/rpm-ostree/issues/458

Closes: #459
Approved by: jlebon
2016-09-13 13:30:15 +00:00
Jonathan Lebon
037a310964 sysroot-upgrader: make pkgcache cleaner speak well
Just some polish. Noticed it while watching someone give a demo.

Closes: #457
Approved by: cgwalters
2016-09-09 20:36:20 +00:00
Colin Walters
65424b0e10 Release 2016.9
Closes: #453
Approved by: cgwalters
2016-09-09 00:10:10 +00:00
Colin Walters
70d7501e67 Fix regression from baseref cleanup
Commit 334cec56a0 had a fatal
logic error - I changed it to write a temporary ref for the final
commit, not the base.

I noticed this by getting `missing metadata object $base.commit` on my
desktop system.  Hooray for dogfooding git master.

Closes: #452
Approved by: jlebon
2016-09-08 17:51:26 +00:00
Colin Walters
9dda288f70 Release 2016.8
Closes: #451
Approved by: cgwalters
2016-09-08 01:19:10 +00:00
Colin Walters
80e616af7f Rename pkg-add -> install, move out of preview
I think these are suitable for wider consumption now, and we should
be able to support existing installs.

Rename the verb to just `install` since it's what every other package
manager uses.

Closes: #450
Approved by: jlebon
2016-09-07 22:11:33 +00:00
Colin Walters
0649567ffc compose: Nuke mock workaround
It turns out it was buggy (for some reason `PS1` wasn't propagating),
and furthermore, things mostly work if one enables `--new-chroot` i.e.
`systemd-nspawn`, which is what Fedora is going to do, and everyone
else in the world uses Docker.

While we're here, tweak the error message to use `<>` around the URL
which makes it more easily clickable from terminals.

Closes: #449
Approved by: jlebon
2016-09-07 21:57:01 +00:00
Colin Walters
9596bfcc42 daemon: Fix ConditionPathExists location
It needs to be under `[Unit]`.  I noticed the systemd warning
in my logs; not sure how this worked when I was testing it locally.

Closes: #446
Approved by: jlebon
2016-09-06 20:34:13 +00:00
Colin Walters
7145cbf7f3 daemon: Add ConditionPathExists=/ostree
Right now the daemon assumes the system is using ostree, but
for various reasons people can try to start it on non-ostree systems.

This is a simple fix to avoid crashing.  A better fix would
need to rework a lot of the code to return dummy/stub values but
that would be painful.  Maybe later.

https://bugzilla.redhat.com/show_bug.cgi?id=1372194

Closes: #445
Approved by: jlebon
2016-09-01 14:49:51 +00:00
Colin Walters
665083dff9 core: Define an autocleanup macro for Header
And switch a few things to `return FALSE`.  Preparatory work
for other cleanups.

Closes: #441
Approved by: jlebon
2016-08-31 20:15:14 +00:00
Colin Walters
525c605092 diff: Don't print changelogs by default
Communicating information is a hard problem, but basically there
are two things here.  First, the changelogs make the diff visual
spaghetti.  (If we were rendering to HTML with expanders or something,
it could work).

Second, I think RPM changelogs are legacy - tooling should pull from
git.

So far when doing release announcements I've been manually stripping
out the changelogs, embarassingly enough.  Time to fix that.

Anyways, we preserve the changelog-emitting code, it's just not the
default now.

Closes: #439
Approved by: jlebon
2016-08-30 20:13:35 +00:00
Colin Walters
436a8116f2 status: Sort the output package list
I am thinking of indenting into multiple lines if it gets long too,
but this is a small visual improvement.

Closes: #440
Approved by: jlebon
2016-08-30 20:09:24 +00:00
Colin Walters
363e213f18 upgrader: Avoid cleaning system repo twice
We need to manage our baselayer refs independently of the deployment
being written by libostree, so suppress the default clean; we'll do
it in rpm-ostree.

I noticed the `Freed objects:` message twice and always wondered why
until today.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Colin Walters
2bc8d7cccb upgrader: Prune pkgcache repo
Until now, we weren't pruning the pkgcache repo at all.  I ran out of
space in the root partition in my CAHC vagrant test box, so it's time
to fix this.

The basic algorithm is to walk over the full rpmdb contents of each
root, generate a set of "currently referenced" cached refs, then delete
any refs in the pkgcache repo which aren't included.  Then, do a prune
of the pkgcache repo.

While we're here, factor out a `sysroot_upgrader_cleanup()` function
which does all of the cleanup.  The idea is at some point we need to
introduce an `rpm-ostree cleanup` command or so which calls this, to
handle the case where the system is interrupted post-deploy but
pre-clean.

Closes: https://github.com/projectatomic/rpm-ostree/issues/428

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Colin Walters
334cec56a0 upgrader: Use a temporary baseref, rather than walking twice
In preparation for future changes here, rework things so
that we only walk the set of deployments once after the new
deployment has been generated.

To avoid a race where if we're interrupted after the deployment write
but before the regeneration, create a special `rpmostree/base/tmp` ref
that we delete after all the bases are regenerated.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Colin Walters
35d84f7507 pkglayering: Commit in base repo, not pkgcache
Currently, we do the final commit into the pkgcache repo, then
pull it to the base.  The problem with this is that, combined
with the fact that we're not presently pruning the pkgcache repo,
we leak space.

In preparation for a cleaner fix for this, rework things so that the
core infra can know about *both* a base repo and a pkgcache repo.  If
they're separate (as is the case for rpm-ostree-on-host), whenever we
are doing layering, explicitly pull just the `.file` objects that are
referenced into the parent repo.  We do the final commit in the base
repo.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Jonathan Lebon
accd2b9f6f vmcheck: add a test for RPM scripts
Thought it'd be fun to write a test for verifying proper handling of
scriptlets during package layering. There's obviously a lot more that
could go in here (patches welcome!), but it's a start.

Closes: #434
Approved by: cgwalters
2016-08-17 21:28:01 +00:00
Jonathan Lebon
83ee4ee638 testenv: add builddir to PATH
Closes: #434
Approved by: cgwalters
2016-08-17 21:28:01 +00:00
Jonathan Lebon
0fe12fe6a3 mutate-os-release: skip VERSION_ID
I hit this with librepo subbing out the $releasever with e.g. 7.2016.1
when trying to pull various URLs. It should be enough for the user to
see the ostree version in VERSION and PRETTY_NAME. For applications,
there's OSTREE_VERSION if they need just that.

Closes: #433
Approved by: cgwalters
2016-08-17 20:45:29 +00:00
Colin Walters
589a8e0974 core: Neuter systemctl during script execution
`systemctl start/stop/try-restart` are all useless for us in
scripts, since changes should only affect the *next* boot.

`systemctl enable` is also wrong - one should use presets instead.

Currently, systemd has code to detect whether it's inside
a chroot, which works for mock, but *not* for Docker or bubblewrap.
(We should teach systemd a nicer way to disable itself, but
 even if we did that we'd have to support old scripts)

So, this fixes layering `glusterfs` in CAHC.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
6900c616ec scripts: Execute %pre
Now that we handle `useradd`, which is most of what anyone wants
to do in `%pre`, let's turn it on.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
550adecb47 core: Support scripts which call useradd
When executing scripts, map `/usr/lib/passwd` temporarily back to
`/etc/passwd` from the POV of the script, then move it back.  This
allows client-side layered packages be merged with the base tree
version, while still being distinct from the `/etc/passwd`.

(In the future we'll likely rewrite all of this to use
`systemd-sysusers(8)`, but for now let's be incremental.)

Note in this commit it isn't very useful, because pretty much all RPMs
only call `useradd` in `%pre`, which is the next commit.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
dbd2c0ee7a unpacker: Synthesize tmpfiles.d for nonroot-owned /run and /var files
Because OSTree includes numeric uid/gid, but package layering (and
traditional full-client-side RPM) expects to allocate UIDs on the
machine, we need to handle non-root owned files specially.

For files in /run and /var, we can convert them to
`systemd-tmpfiles(8)` snippets which will be executed at the next
boot, after a uid/gid has been assigned.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
3ad4e6c72b bwrap/compose: Add a workaround for Fedora's use of rpm-ostree-in-mock
Decided to test this on Sunday evening.  Of course it was broken =(
(Actually I tested mock-in-Docker but it should be the same)

The core problem is that mock does `chroot()` without using `/`
as a mount point.  This breaks an assumption in bwrap that it is.
Now, in theory we could move this same logic down into bwrap to
work around this situation, but for now let's hack it here.

Mock is old, legacy container code that doesn't really do anything
in a modern way - in fact our goal should be to replace it
with a combination of rpm-ostree and bwrap.  So carrying this
hack here to get us to that future should be OK for now.

Closes: #431
Approved by: jlebon
2016-08-15 18:31:58 +00:00
Colin Walters
09c5f9a4fe Release 2016.7
Closes: #430
Approved by: cgwalters
2016-08-13 14:34:38 +00:00
Colin Walters
7455e26273 bwrap: Add a selftest
I want a better error message if the user happens to execute
inside e.g. a Docker container without sufficient privileges
for recursive containerization.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
551e4c91f9 postprocess: Switch to using bwrap for script execution
The previous commit https://github.com/projectatomic/rpm-ostree/pull/422
introduced a regression in the "outside of a container" path - we
get `EINVAL` trying to `mount("proc",...)` and honestly I'm not sure why.

We can either back up or plow forward, and it turns out to be
pretty straightforward to complete the port to using bwrap.

I extracted the bwrap-execution code out of the RPM script engine,
because the treecompose model is currently different (no hardlinks
yet).

NOTE: A *very* important side effect of this is that we now
require "privileged" containers on hosts without user namespaces,
and on userns hosts, require `CLONE_NEWUSER` to be exported to the
container host.

In general though, the previous path of blindly executing scripts as root
without e.g. `proc` mounted was just bad.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
88a130da09 lib: Extract bwrap-executing internal API
The treecompose code will learn how to use bwrap instead of
libcontainer in libglnx, since the latter is a buggy copy of a subset
of the former.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
873f4766d7 Release 2016.6
Closes: #427
Approved by: jlebon
2016-08-11 20:22:54 +00:00
Colin Walters
a299083b09 core: Fix an error-handling logic error in relabel path
If next_dent returns `FALSE`, we need to also return `FALSE` from
the function.

I just noticed this as I happened to be reading the code randomly.

Closes: #426
Approved by: jlebon
2016-08-11 19:40:29 +00:00
Colin Walters
19490c593e Update to latest libdnf
I want the excludes= fix, and the memleak fixes are nice.

Closes: #425
Approved by: jlebon
2016-08-11 19:26:06 +00:00
Colin Walters
56a4f73901 scripts: Error out if we encounter lua
Otherwise we try to execute as shell script which obviously doesn't
work.  This was noticed when @dustymabe was trying to assemble
a Fedora container with `glibc-all-langpacks`.

Closes: #424
Approved by: jlebon
2016-08-11 15:32:49 +00:00
Colin Walters
aa2295db26 compose: Add --cache-only
When trying to test treecomposes, it's annoying to hit the
metadata servers each time.  Add a `--cache-only` option which
is kind of like yum's `-C` option (but not quite because
if the metadata doesn't exist we will update it).

Closes: #423
Approved by: jlebon
2016-08-09 15:18:14 +00:00
Colin Walters
3d1040a780 compose: Port bits of postprocessing to fd-relative
I was looking at starting to unify the treecompose path to the
"new world" infrastructure in package layering.  An initial
step here is to port to fd-relative, which cleans up the code.

Note this depends on a libglnx pull.

Closes: #422
Approved by: jlebon
2016-08-08 18:11:42 +00:00
Colin Walters
ff8e2992da libglnx porting: Switch to glnx_file_get_contents_utf8_at()
A small one, but a start.

Closes: #421
Approved by: jlebon
2016-08-08 18:07:46 +00:00
Colin Walters
0c8f6b982e build: Import attributes.m4 copy from systemd, update warnings
This matches https://github.com/ostreedev/ostree/pull/431
I also merged in the `-Wint-conversion` here since we build fine
with it now.

Closes: #420
Approved by: jlebon
2016-08-08 14:26:28 +00:00
Colin Walters
dee758e111 gitmodules: Update for rename
rpmdistro-gitoverlay doesn't like when the submodule name differs
from its path, so let's make it happy.

Closes: #419
Approved by: giuseppe
2016-08-04 22:10:59 +00:00
Colin Walters
a852ad3459 Fix accidental double __ in previous commit
My sed powers got overly enthusiastic.

Closes: #417
Approved by: giuseppe
2016-08-04 12:33:07 +00:00
Colin Walters
e18d43bad3 Port to new ostree_repo_checkout_at()
Squashes some deprecation warnings, and now with the latest master we
can stop setting `disable_fsync`.

Closes: #418
Approved by: giuseppe
2016-08-04 12:21:38 +00:00
Colin Walters
39913a2c25 Update for libhif -> libdnf
Still not many PRs outstanding, so it's a good time to take this plunge.
Mostly automated via `sed`.

Closes: #416
Approved by: Conan-Kudo
2016-08-04 01:10:08 +00:00