Commit Graph

838 Commits

Author SHA1 Message Date
Jonathan Lebon
62c7242a98 upgrader: move origin mutation outside of the upgrader
This is the beginning of a move towards making the upgrader more
stateless and less hysteretic. Ideally, the only input required should
be an origin file. Users of the upgrader are in charge of modifying the
origin as needed. The main goal of this is to simplify the upgrader
model and a better separation of concerns.

This was already done for the initramfs-related entries. This patch
formalises this for most of the other inputs, except packages, which
will be done soon.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Jonathan Lebon
c437f90a83 upgrader: drop support for ignore scripts
This functionality is disabled right now, and if we want to re-enable
it, we'll want to do it in a way that ensures the ignore scripts are
permanent.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Jonathan Lebon
3858913c98 package_diff: make use of apply_revision_override
Dedup some more code. Might be worth it to just completely unify
package_diff with deploy at some point.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Chen Fan
7ee41f75b7 remove duplicate words in comment
Signed-off-by: Chen Fan <fan.chen@easystack.cn>

Closes: #635
Approved by: cgwalters
2017-02-21 14:47:59 +00:00
Colin Walters
279287d69d upgrader: Avoid "Freed 0 pkgcache branch" pluralization failure
Let's not hardcode a broken English-only version of `ngettext` here; rework the
message to avoid requiring pluralization.

Closes: #632
Approved by: jlebon
2017-02-17 20:57:59 +00:00
Colin Walters
f6842838d7 build: Deal with gperf 3.1 changing to size_t
https://bugzilla.redhat.com/show_bug.cgi?id=1424268

Closes: #631
Approved by: jlebon
2017-02-17 18:51:53 +00:00
Eduardo Mayorga
1f2170e10c libpriv: Reduce scope of variables
nchange_date, nchange_name, and nchange_text are declared in a larger
scope in rpmostree-rpm-util.c. Reduce the scope to satisfy cppcheck.

Closes: #628
Approved by: jlebon
2017-02-16 15:40:20 +00:00
Colin Walters
ddbaf19f1e importer: Error importing RPMs which install to /opt (outside of /usr)
See https://github.com/projectatomic/rpm-ostree/issues/233 - for RPMs which
place files in e.g. `/opt`, we have different behavior in the treecompose case
(silently drop it) versus package layering (does the wrong thing).

Since the unpacker right now is only used in the layering case, this just
ensures we'll get a consistent error there.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Colin Walters
325de9d898 unpacker: Check for filter errors while committing tmpfiles too
This masked an issue in a change I was working on in the filter.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Colin Walters
775c7819b7 daemon: Avoid erroring out on startup/status with origin unconfigured-state
As part of an earlier cleanup of origin parsing, we started checking
the origin `unconfigured-state` even just starting the daemon, which
is kind of bad.

It's tempting to flip the default for the parser so that we *only* check
unconfigured state if we go to upgrade, but let's not do that in this patch.

Closes: #626
Approved by: jlebon
2017-02-14 14:00:09 +00:00
Colin Walters
ae90a9d2b8 upgrader: Avoid GLib warning on shutdown after failed update
The hash tables are only initialized if the initable init completes, but it
won't if something goes wrong there. In this case, an origin
`unconfigured-state`.

Closes: #626
Approved by: jlebon
2017-02-14 14:00:09 +00:00
Jonathan Lebon
8cf071127e unpacker: support root-owned /var & /run files
I'm not sure why we weren't doing this before, but we need to also
support files in /var and /run that are owned by root.

Related: RHBZ#1421781

Closes: #622
Approved by: cgwalters
2017-02-13 16:49:59 +00:00
Colin Walters
a66d27230d Add a cleanup command and DBus API
We sometimes talk about using `ostree admin undeploy`, but that
doesn't know about the pkgcache, and hence space there leaks
until the next rpm-ostree operation.

Just for this, we need to expose a cleanup command (and API).  But
we also need to support cleaning:

 - repomd
 - downloads (repo/tmp)

So let's start implementing that.

Closes: #614
Approved by: jlebon
2017-02-13 15:28:05 +00:00
Jonathan Lebon
3c77b6e0a9 compose: fix bad umount
Closes: #602

Closes: #620
Approved by: cgwalters
2017-02-10 22:55:17 +00:00
Jonathan Lebon
dd4b553bbc ux: improve package diff outputs
This patch makes the diff outputs generated by deploy commands (like
upgrade/deploy/rebase) and db diff commands the same. They were both
tweaked in different ways. The former gained the ability to discern
between upgrades and downgrades, the latter gained the ability the print
package version transitions.

Of course, the best way to make them the same would be to make them use
the same code, though I'm not sure it's worth the pain to gain ratio at
this point...

Closes: #575

Closes: #619
Approved by: cgwalters
2017-02-10 22:08:22 +00:00
Jonathan Lebon
b0727d88fd start-daemon: make hidden
I debated just putting this in the supported list, but decided against
it in the end. This really should be something that happens
transparently, and if it doesn't then something else is probably wrong.

Closes: #617
Approved by: cgwalters
2017-02-10 20:11:27 +00:00
Jonathan Lebon
31c68b6edf compose: make CLI helper string more accurate
We no longer run yum.

Closes: #617
Approved by: cgwalters
2017-02-10 20:11:27 +00:00
Jonathan Lebon
ffed2a0c60 compose: remove unimplemented option
This doesn't look like it actually does anything... Though I haven't
checked the history to see if it used to and it somehow got nixed by
accident.

Closes: #615
Approved by: cgwalters
2017-02-10 19:28:41 +00:00
Jonathan Lebon
85ed0c4471 daemon: fix assertion error on exit
We weren't exiting the intended way before because we were tripping an
assertion:

  GLib-CRITICAL **: g_main_loop_quit: assertion 'loop != NULL' failed

This was because the global loop was being shadowed by a local variable.

Closes: #612
Approved by: cgwalters
2017-02-08 20:33:05 +00:00
Colin Walters
0a1289ae38 upgrader: Use "pending" deployment for origin by default
Until now, we always used the booted deployment, and would
garbage collect the "pending" deployment.  This is the
way OSTree was designed, but I think for rpm-ostree given
how mutable we are on the client side, there's a much stronger
argument for being more stateful too.

This is a relatively simple code change to split the "merge deployment"
concept into two.  There's now the "config merge deployment" and the
"origin merge deployment".

Basically, `rpm-ostree install foo; rpm-ostree install bar` will
now install both `foo` and `bar`.  But we will still use the booted
deployment for `/etc`.

Down the line, I think I'd like to drive into OSTree the concept of
a "staged" deployment, that has the hardlink checkout done, but doesn't
have the config merge.

But we don't need to change the OSTree core for this yet; we can
do it here in rpm-ostree, and this relatively simple code change
fixes many issues at once.

For example, `rpm-ostree upgrade && rpm-ostree install foo` now
does what you expect as well.

Obviously, we want to enable doing multiple things in *one* transaction,
and we're not far away, but I think this is also the right thing to do now.

I'm relatively confident it won't break anyone's workflow, as what
we did before wasn't generally that useful.  However, people will
need to learn to `ostree admin undeploy 0` if they *don't* want
this behavior.  (We need to have `rpm-ostree cleanup`).

Closes: https://github.com/projectatomic/rpm-ostree/issues/406

Closes: #611
Approved by: jlebon
2017-02-08 19:55:44 +00:00
Colin Walters
a52cb7d78e core: Add rpmostree.repo metadata to imported packages
I'm watching https://github.com/rpm-software-management/libdnf/pull/199 and I
really don't like it. We already have a place to put out-of-rpmdb metadata,
which is in the ostree commit for imported packages. No need to involve a
relational database for this (and further, one that would need to learn about
multiple ostrees).

We're not yet *using* this information in the UI, but we could; imagine
changing the `status` `Packages:` to show packages-per-repo or so.  We
could also expose an `rpm-ostree pkg-info foo`.

But for now, let's just start recording this.

Closes: #610
Approved by: jlebon
2017-02-08 17:52:16 +00:00
Colin Walters
ace223acf8 Add pending-base-commit to status
One thing that's very confusing about OSTree is there are two layers -
deployments and the refs/commits. If one does an `rpm-ostree upgrade`, but then
e.g. `ostree admin undeploy 0`, you still have the new revision in the repo.

We don't do a good job of displaying this state, or helping people clean
it up.

Down the line, I also want to better support something like `rpm-ostree pull` to
cache updates explicitly *without* deploying.

This commit just adds a bit of information to the status display. We might want
to have better formatting, but I think this an OK start.

Closes: #595
Approved by: jlebon
2017-02-08 13:59:48 +00:00
Jonathan Lebon
cdac757434 daemon: delete overrides if not doing a deploy
We seem to have lost this little guy during the latest unification work.

Closes: #608
Approved by: cgwalters
2017-02-07 22:43:32 +00:00
Colin Walters
5f7c5305ff Delete "compose sign"
We want people to use the libostree API for things like this. Further, the
`rpm-sign` tool that this calls is Red Hat internal, so it doesn't make sense to
have a public wrapper for it.

Closes: https://github.com/projectatomic/rpm-ostree/pull/152

Closes: #607
Approved by: jlebon
2017-02-07 21:43:22 +00:00
Colin Walters
687567d3ee Merge daemon binary into main binary
The actual problem I am trying to fix with this is fallout from the
introduction of `/usr/libexec/rpm-ostreed`, which required a SELinux
policy change.  Specifically for CentOS, the base policy is rev'd
slowly.

My hope was that by merging the daemon code back into `/usr/bin/rpm-ostree`
which is labeled `install_exec_t`, starting via systemd would do
the right thing.  It turns out that doesn't happen.

Now later, I'm picking this patch back up because I want to do multprocessing in
the daemon (and in the core), and it makes sense to share code between them,
because multiprocessing will need to go through a re-exec path.

Another benefit is we avoid duplicated text (libglnx, internal helpers) between
the two binaries.

Closes: #292
Approved by: jlebon
2017-02-07 16:07:09 +00:00
Colin Walters
7887c5a923 treecompose: Print enabled repositories and their timestamps
In particular, I want to start using the repo timestamp as a poor man's
versioning. Knowing when the last time a repo was updated is pretty important if
you're potentially expecting security updates for example.

(I plan to also do something like this on the client side, but let's do it here
 first since we already dump lots of crud to stdout. Client side would require
 structure/design/thought)

Closes: #604
Approved by: jlebon
2017-02-06 19:13:19 +00:00
Jonathan Lebon
566367ca3f upgrader: let ostree create the commit dir
When we checked out the base tree for package layering, we would create
the directory in which ostree did the checkout. This meant however that
ostree wouldn't apply xattrs on the root directory itself. This would
cause the directory to be mislabeled (as system_conf_t instead of
root_t), which in turn cause SELinux violations on reboot when systemd
tried to make the root mount shared.

This patch fixes this by first settling on a permanent directory in
which to do checkouts -- really, we'll never have multiple package
layering operations going on at the same time. Once we know that we have
a reserved path, we can safely let ostree create it for us with the
proper xattrs.

Resolves: RHBZ#1318547

Closes: #605
Approved by: cgwalters
2017-02-04 11:08:27 +00:00
Jonathan Lebon
c756b26521 compose: fix mutate-os-release handling
- Let --add-metadata-string=version=val override any automatic version
  prefixing.
- Don't error out if mutate-os-release is given, but no new version is
  given by --add-metadata-string or automatic version prefixing.

Checking keys in parse_keyvalue_strings() is slightly hacky. I initially
wanted to just inspect the GVariantBuilder, but AFAICT, there is no way
to actually look up values from a builder (plus, we need that info early
to know whether automatic_version_prefix should itself inject in the
builder or not).

Closes: #603
Approved by: cgwalters
2017-02-01 22:25:54 +00:00
Colin Walters
1b7e35abec upgrader: Fix use of abspath
In the previous commit I changed to abspath, since when reworking things the
relative path broke (I'm still not entirely sure why, but we were basically
lucky before). However I then later realized this wouldn't be right if
rpm-ostree was started with a `--sysroot` argument from Anaconda.

What we can do instead (at least until libdnf grows fd-relative APIs)
is to make it a fd-relative abspath.

Closes: #596
Approved by: jlebon
2017-01-28 22:48:08 +00:00
Colin Walters
7c7806f831 daemon: Fold pkgchange txn into deploy
The next step will be to design a DBus + commandline API for this. In the
meantime, there are some small subtle new features like honoring the dry-run
option for upgrade. I'm not testing that explicitly yet, but I think that's OK.

Closes: #593
Approved by: jlebon
2017-01-28 22:16:09 +00:00
Colin Walters
c7d44e67d1 daemon: Create a flags type for behaviors rather than 3 bools
3 bools in parameters is too many; time to do flags. This also will lead to
deduplicating with the pkg txn flag type when we merge that in. Also, we dedup
the gvariant parsing code.

Closes: #593
Approved by: jlebon
2017-01-28 22:16:09 +00:00
Colin Walters
d7e709b11c daemon/upgrader: Move repo member into object
Following on the previous commit, this drops the last bit of state that we
threaded through various calls. We only need one reference.

Closes: #590
Approved by: jlebon
2017-01-28 13:58:16 +00:00
Colin Walters
84343ae208 daemon/upgrader: Move some state into the object
I was going to do some work here, and adding another parameter to thread down
through these function calls got painful. We already have an object to store
state in, so do it. This makes us more consistent.

Closes: #590
Approved by: jlebon
2017-01-28 13:58:16 +00:00
Jonathan Lebon
e351cb39a1 main.c: check for root after parsing options
So that e.g `rpm-ostree compose tree --help` doesn't need root.

Closes: #592
Approved by: cgwalters
2017-01-26 15:32:27 +00:00
Colin Walters
74afb02a90 daemon: Fold rebase and deploy txn code together
More txn unification.

Closes: #589
Approved by: jlebon
2017-01-26 14:53:04 +00:00
Colin Walters
7ff54daa89 daemon: Fold rebase/upgrade txn code together
Upgrade is just a special case of rebase conceptually where:
  - We fetch the latest tip
  - We enable downgrade protection

This is part of a larger effort where I am aiming to have a *single* transaction
type for system mutation, so we can do both upgrades and package layering and
enable initramfs all in one go.

Closes: #589
Approved by: jlebon
2017-01-26 14:53:04 +00:00
Colin Walters
cac4522e5b Support "system/regenerate-initramfs=true" flag in origin
Currently we push for a model where the initramfs is
generated (in non-hostonly mode), and merely replicated.

However, to support a few unfortunate corner cases like dm-multipath which wants
to inject a config file into the initramfs, we need to support regenerating it
client side too.

Down the line, we'll need this to support overriding the kernel too.

This changes things in the core to add the concept of an "empty"
`RpmOstreeContext`. I initially tried skipping it, but that was too much
duplication. We still want all of the core ostree-related logic that lives in
that code too.

The treespec bits barfed if the spec didn't have a `tree/packages` key. It was
simplest to change that to allow it - and because that was the only case where
we errored out in parsing, I dropped the error handling.

There was another place in the upgrader that now needed to be fixed to handle
transitioning from just regenerating initramfs to not.

Closes: #574
Approved by: jlebon
2017-01-20 16:17:51 +00:00
Colin Walters
b099680d1d build: Capture git describe on build if available, add to --version
Many projects do this, and it really helps debugging to know the
exact hash.

(Of course this is broken in traditional rpm builds from a tarball,
 and rpmdistro-gitoverlay injects it into the Version field,
 but it will help me for vmcheck debugging)

Closes: #584
Approved by: jlebon
2017-01-19 19:49:58 +00:00
Colin Walters
02166aeecd Move SIGINT handling into transaction helper
Not being able to `Ctrl-C` treecompose kept irritating me and I
finally looked it into it.  I'd thought it was rpm or librepo's
fault, but nope, it's ours!

We had `SIG{INT,HUP,TERM}` handling globally, but unfortunately
right now some things in libdnf don't respect the cancellable
(It's hard to do without threading it down all the way into libcurl
 and rpm).

Really for treecompose we don't need a `SIGINT` (or other) handlers - we should
just take the default action of immediate process exit.

Now, for the command line dbus calls, *only* when we go to execute a txn should
we catch `SIGINT` so that we can forward it to the daemon to cancel there.

Closes: https://github.com/projectatomic/rpm-ostree/issues/489

Closes: #583
Approved by: jlebon
2017-01-19 16:56:48 +00:00
Colin Walters
30bed29230 daemon/upgrader: Rework layer tracking
Working on initramfs, I hit a subtle issue with the fact that
I was trying to "redeploy", but with the origin file changed
during the process.

Previously, it was a bit unclear which parts of the upgrader logic are operating
on the *new* origin versus the "original origin".

The package layering code in the upgrader explicitly carries a delta on top in
the "add/remove" hash sets, which means it isn't visible to
`rpmostree_origin_is_locally_assembled()`.

Whereas for initramfs, I set a new origin. This broke things since we were
expecting to find a parent commit, but the original origin wasn't locally
assembled.

When looking more at this, I realized there's a far simpler model -
rather than keeping track of commit + origin, and using the origin
to try to determine whether or not the commit is layered, we can
keep track of `base_revision` and `final_revision`, and the latter
is only set if we're doing layering.

The diff speaks for itself here - a lot of fragile logic looking at the origin
drops away.

The next step here is probably to drop away the package layering hash sets, but
I'm trying to not change everything at once.

Closes: #579
Approved by: jlebon
2017-01-18 20:05:19 +00:00
Colin Walters
59f34e8a03 tests: Disable ASAN leak checks
Right now we're far from leak free, and I want to use ASAN
as a sanity checker, not a leak checker.

Closes: #576
Approved by: jlebon
2017-01-18 16:52:46 +00:00
Colin Walters
99bca8ad3f Suppress uid == 0 check in unit tests
Our `make check` runs an unprivileged rpm-ostreed if run as non-root; this is a
feature. We didn't notice in the CI tests since those run as "docker-uid0". It
does break my local workflow though.

Closes: #573
Approved by: jlebon
2017-01-16 19:03:02 +00:00
Colin Walters
b3e16a3313 bwrap: Turn on all namespaces
The fact we weren't doing this is an oversight.  We should *really*
be using the PID namespace at a minimum, but I decided to just turn
them all on.

The one that seems most likely to potentially introduce a regression is turning
on the netns (i.e. disabling networking). But I can't really think of what we'd
be running in a script today that would break in practice.

Closes: #572
Approved by: jlebon
2017-01-16 14:40:47 +00:00
Colin Walters
f8edd59d5f daemon: Introduce an origin structure
In prep for adding "regenerates initramfs" to "is locally assembled",
we need more abstraction.  I previously introduced a helper function, but
I'd still need to touch every call site when changing that.

Rather than having each caller re-parse the origin, let's parse it once into a
structure, and then have error-free accessors for it (that also don't malloc).
This way when adding a new flag, I don't need to touch every call site.

Notes:

 - Some places in the code (like the deployments -> variant bits) tried to
   handle deployments without an origin gracefully.  That's no
   longer true.  I'm not sure how much we care - do we?
 - There are a few places where I changed `packagelist.len() > 0` to
   `is_locally_assembled()`.  I think this is right, but we'll need
   to be sure the "no packages to overlay, just initramfs" case
   works when that lands.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
f215acb64f libpriv: Move kernel finalization into a helper function
This will need to be callable outside of postprocess for client side initramfs
generation.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
4c98943a64 postprocess: Clean up kernel/initramfs checksum calculation
This avoids having an intermediate file path, and makes the code easier to
follow.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
16efa12ae0 libpriv: Factor out "find kernel/initramfs" code into kernel.[ch]
To support running dracut on the client side, the dracut code
needs this, and it makes more sense in kernel.[ch] anyways.

I chose to use a GVariant instead of making a custom structure to avoid having
to manage custom free funcs.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
4a511fa011 app: Check for root privileges where required early
Rather than sending a dbus message that gets denied, which
is ugly.

Closes: #565

Closes: #570
Approved by: jlebon
2017-01-13 19:46:24 +00:00
Colin Walters
e1cdfa1712 libpriv: Introduce an rpmostree-kernel.[ch] for dracut
Since the dracut run is now separated from the rest of postprocess, we change it
to use the bwrap API diretly, and this lets use the new _IMMUTABLE bwrap type.
This will make it easier to reuse for client-side initramfs regeneration.

Splitting this off makes it also easier to simplify the remaining mutable
usage in postprocess.c.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:26 +00:00
Colin Walters
f0ec738376 libpriv: Enhance internal bwrap API
Make it a real struct with methods.  This noticeably increases
the ergonomics and design of the API.

The main goal here is to introduce the enum which defines whether or not the
rootfs is mutable or not. We move the "rofiles" mode from the RPM script code
down into the bwrap layer, which will make it easier to reuse for treecompose.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:26 +00:00
Colin Walters
50ab2983ab postprocess: Use O_TMPFILE for dracut, merge reproducible bits
I was planning to do some further changes here, and I really don't like the
manual fork/exec stuff on in the --reproducible checks. Our subprocess code
should basically be all bwrap.  Synchronous code execution while not reading from the
pipe is a recipe for deadlocks.

What simplifies things a lot is to write to an `O_TMPFILE` fd (or a tempfile on
legacy kernels), and slightly extend our bwrap-executing code to support a child
setup function, so we can set the tmpfile fd to be stdout.

Now that we have a shell script wrapper we inject, it's trivial to reimplement
the "detect reproducibility" in shell script there, rather than C.

This doesn't matter much for treecompose today, but it will matter more when
we're supporting client side initramfs regeneration, since now the dracut
container can be fully immutable.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
7a421443f9 core: Split up overlay/commit API
In a future change I want to optionally rerun dracut between these. Keep the old
API as a trivial wrapper around the two.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
f766b59fbf daemon: Add a helper function for "is locally assembled"
This is all two callers need, and in the future I want to add
local initramfs generation.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
1c3b03af8d daemon: Log into the journal for txn failures
We should really be logging more into the journal in general,
but particularly this will make it easier to find errors
in the test suite.

Right now we only dump script stderr into the journal.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
0ed86d7475 daemon: Use autoptr for cancellable, initialize inline
This drops one line + whitespace line from each txn, and updates us to use
autoptr. Just a bit less duplication.

I noticed while doing this that not every txn initializes a cancellable, but not
shaving that yak right now.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
26909a44c1 daemon: Dedup transaction-merging code a bit
We had a lot of copy/paste of this, it's hard to entirely get rid of, but this
is at least smaller.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
petervo
4fc3589dc0 daemon: Really allow downgrades when pulling a rebase diff
Closes: #563
Approved by: jlebon
2017-01-10 14:09:38 +00:00
Jonathan Lebon
e32bbf16d1 passwd_prepare_rpm_layering: account for local entries
On Fedora 25, systemd adds a sysuser config file for multiple users. It
also explicitly creates those same users in its %pre, except for one:
systemd-coredump. This means that the tree's /usr/lib/passwd doesn't
contain systemd-coredump. Of course, on first boot, it gets created and
added to /etc/passwd.

During package layering, we map /usr/lib/passwd to the container's
/etc/passwd. If the %pre calls useradd/groupadd without passing an
explicit uid/gid, it's possible that the allocated id is already in use
by an entry in the deployment's /etc/{passwd,group} (such as
systemd-coredump, but the same holds for any manually-added entry).

We resolve this by taking the switcheroo a step further: we map
/usr/lib/passwd to /usr/etc/passwd, and then also map /etc/passwd to
/usr/lib/passwd. That way, useradd in %pre will account for already
allocated local uids and react accordingly.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
0bbba38727 core: add support for layering non-root files
We lift the restriction from the unpacker on non-root owners. The
unpacker now imports all files as root:root.

We split the running of rpm scripts between %pre and %post/posttrans as
two different stages. During assembly, we first run the %pre scripts,
collect users and groups, then chown as needed (and then finally run the
%post scripts).

Closes: #462

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
69e75debc2 rpmostree-passwd: make passwd/group parsers public
We're going to need to access these functions later.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
0e431a8f83 core: add RPMOSTREE_USE_CACHED_METADATA
Being able to just reuse metadata is especially helpful when trying to
debug things lower down that path, as well as cuts about 2 mins from the
compose test.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
b68209b6d4 daemon: start with one commit only when resolving versions
During a deploy operation, we would fetch commit objects from the remote
to resolve the version string. If gpg-verify was turned on, we would
fail to pull them if some of the commits were not signed. This is
because we pulled them in batches. We partially address this by only
fetching the HEAD commit on the first pass. This allows `upgrade`
operations to work just as well as `deploy` operations.

Though there is still an issue if we have to traverse farther back than
when signed commits become unsigned (unless they happen to fall on a
batch boundary). We leave that unsolved for now, since that would likely
require a more complex solution and it's not clear whether it's a real
world issue (signers can just retroactively sign commits).

Copy the gpghome from ostree so that we can test GPG-related cases in
our suite.

Closes: #527

Closes: #557
Approved by: cgwalters
2016-12-24 12:28:48 +00:00
Jonathan Lebon
e10c97007f rebase: add support for rebasing to a specific rev
Expand the available options in the Rebase() D-Bus method to also have a
"revision" key. Its value has the same semantics as the "revision" key
in the Deploy() method (e.g. the "revision=" and "version=" prefixes are
also supported). Also expand the rebase CLI to allow for specifying the
revision as an additional argument.

This allows users to rebase to a specific version or checksum, rather
than only to the latest. Conceptually, this is the equivalent of doing a
rebase followed by a deploy. I.e. we specify an override-commit in the
origin and expect the same behaviours that apply after a deploy to also
apply here.

Closes: #212

Closes: #555
Approved by: cgwalters
2016-12-21 20:00:43 +00:00
petervo
9a2788c7b6 daemon: Allow downgrades when pulling a rebase diff
Running the actual rebase already does this automatically.

Closes: #552
Approved by: jlebon
2016-12-20 20:04:25 +00:00
Colin Walters
3a409342e6 postprocessing: Add a g_prefix_error for kernel
Saw this un-prefixed error path in a build; having error prefixes makes for
easier debugging.

Closes: #554
Approved by: giuseppe
2016-12-20 16:24:25 +00:00
Colin Walters
69cace4ce8 [ASAN] Squash a few minor misc leaks
This is part of the leak set when doing a treecompose.  There's
some that go down into libdnf...will get to that over time.

Closes: #547
Approved by: jlebon
2016-12-08 16:27:24 +00:00
Colin Walters
6fb29775a3 [ASAN] libpriv: Squash misc small leaks
Nothing big here, but this seems to be the last of the ASAN hits
in the unit tests.

Closes: #539
Approved by: jlebon
2016-12-07 20:22:17 +00:00
Colin Walters
1209942fc2 [ASAN] unpacker: Clean up leak of rpmts
This was 800 bytes, which could add up.

Closes: #539
Approved by: jlebon
2016-12-07 20:22:17 +00:00
Colin Walters
977835fd41 unpacker: Fix logic error regression in previous commit
I spotted this warning go by in the build logs.  Let's make it
fatal from now on.

Closes: #536
Approved by: jlebon
2016-12-07 20:16:32 +00:00
Colin Walters
0f4ef0d5a1 [ASAN] unpacker: Fix leaks of header/fi
We weren't freeing the Header struct in finalize if we happened
to own it.  Also define and use a cleanup for `rpmfi`.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
12a6999d02 [ASAN] tree-wide: Squash a few string leaks
Just a few random string leaks.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
975c0703c3 [ASAN] tree-wide: Clean up gvariant-related leaks
I kept thinking `^a&s` was sufficient for allocation-free reading
of `as`, but it's not, we need to free the outer buffer.  Other
minor cases were using `s` instead of `&s`.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
7095c313d8 [ASAN] tree-wide: Clean up leaks of option context
Not all of these were leaks, but let's be consistent and use
`g_autoptr()` for `GOptionContext` everywhere.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
88b8f1a655 unpacker: Don't error out layering fcaps-only binaries
In f25 now, `kubernetes-master` is changed to be owned by `root:root`
but still has file capabilities.  Our "non-root owned" check was
bombing out because we need to explicitly ignore "root" ownership.

(If there were no fcaps, then there's no override to check)

Closes: #532
Approved by: jlebon
2016-12-05 19:35:07 +00:00
Colin Walters
7149e9fee5 unpacker: Reorganize code for non-root checks
No functional changes, but I find this more legible - fewer
interacting conditionals.

Closes: #532
Approved by: jlebon
2016-12-05 19:35:07 +00:00
Colin Walters
74c43a0209 core: Error out if we're trying to do an empty install set
This is a partial fix for
https://github.com/projectatomic/rpm-ostree/issues/529

We could drop into this codepath via multiple ways (containers,
etc.), so it makes sense to have a last ditch error here, even
if we should really give an error earlier.

Closes: #530
Approved by: jlebon
2016-12-01 21:15:38 +00:00
Colin Walters
351832f29e Fix install-langs support
We're looking at changing Atomic Host to use multiple locales (but not all);
See: https://bugzilla.redhat.com/show_bug.cgi?id=1186757

This revealed our `install-langs` support didn't really work.  Our
`treecompose-post.sh` was deleting the extraneous translations anyways,
which masked this.  And for other cases like workstations where
we drag along all the translations anyways, it was fine too.

There were two bugs:
 - In the keyfile spec it's `instlangs`
 - We were setting the macro a bit too late, it should be before
   `dnf_context_setup()`.

Closes: #525
Approved by: jlebon
2016-11-24 14:38:16 +00:00
Colin Walters
6074a6d728 daemon: Use new OSTree API to verify commits by remotes
Otherwise, we would show remotes using the new `gpgkeypath`
as unknown.

See: https://github.com/ostreedev/ostree/pull/575
See: https://github.com/ostreedev/ostree/issues/573
See: https://bugzilla.redhat.com/show_bug.cgi?id=1393545

Closes: #521
Approved by: jlebon
2016-11-17 16:32:11 +00:00
Colin Walters
ae108e08a5 Add rpmostree.clientlayer metadata to derived commits
In the future we may add more commands that take as input commit
IDs.  However, we really want to distinguish between server and
client generated commits, as some of these operations won't
make sense for derived commits.

This changes the API to have callers say which type of commit
they're generating, which also fixes a FIXME, and helps get us
a bit closer to the "unified core".

Closes: #520
Approved by: jlebon
2016-11-15 17:11:11 +00:00
Colin Walters
c8e7c63ab2 Final removal of libgsystem dependency
Just like ostree.  Now we can consider it dead.

Closes: #511
Approved by: jlebon
2016-11-08 14:38:54 +00:00
Colin Walters
90d52e3d5e libpriv: Pull in copy of gs_file_get_path_cached()
This is the same thing done in OSTree; we still have a lot of
references to it.  Maybe it should go in libglnx actually, since
flatpak uses it too.

Closes: #511
Approved by: jlebon
2016-11-08 14:38:54 +00:00
Colin Walters
a347ff5f69 postprocess: Port remaining libgsystem call sites
Except for `gs_file_get_path_cached()`.  We have to carry
the recursive hardlinking code, oh well.

Closes: #511
Approved by: jlebon
2016-11-08 14:38:54 +00:00
Colin Walters
5c8d80042c postprocess: More fd-relative-ification
I hit a bit of a wall here around `gs_shutil_cp_al_or_fallback()`.
I guess we'll have to reimplement it here.

In the meantime, this gets us closer.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
634a475ae4 postprocess: Drop a libgsystem use in add-files
I'm cowardly just using gs_file_get_path_cached() for now.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
e0357f41d1 postprocess: Port tmpfiles.d-copying code to fd-relative
Drops libgsystem use.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
68898217ed postprocess: Drop use of gs_file_rename(), use fd-relative
Another libgsystem use gone.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
1e6f4d4f21 compose: Create rootdir more directly, use fd-relative
Mostly to drop a libgsystem use.

Closes: #510
Approved by: jlebon
2016-11-04 14:59:27 +00:00
Colin Walters
d45c0253a0 compose: Honor --dry-run --touch-if-changed
Some people using Jenkins jobs and the like may want to separate
"trigger detection" from execution.  If combined with `--cachedir`,
it's reasonably efficient to separate the two.

Closes: #508
Approved by: jlebon
2016-11-02 16:54:29 +00:00
Jonathan Lebon
47ba21aca1 daemon: use refspec after pulling ancestry
Previously, in preparation for validating e.g. versions during 'deploy'
operations, we would pull the latest commit metadata. However, we would
then do resolve on the branch name only rather than the full refspec.
But this can sometimes give the wrong checksum. For example, if we have
multiple remotes holding the same branch name, ostree_repo_resolve_rev
will just start looking in each remote for the specified ref, and we may
thus end up with the checksum from the wrong remote.

Related: RHBZ#1390259

Closes: #507
Approved by: cgwalters
2016-10-31 16:45:08 +00:00
Jonathan Lebon
e2e7908d39 upgrader: small amendment to previous commit
I actually meant to push this during PR review so that it could be
squashed with 5cbc57b, but forgot.

Closes: #507
Approved by: cgwalters
2016-10-31 16:45:08 +00:00
Colin Walters
e4068c5094 tree-wide: Fix use of potentially uninitialized variables
These were caught by Clang.

Closes: #505
Approved by: jlebon
2016-10-28 20:28:35 +00:00
Colin Walters
55593b77c4 tree-wide: Fix use of incorrect flag type
This is clang's `-Wenum-conversion`, which I'd like to make fatal,
except libdnf trips it too.

Closes: #505
Approved by: jlebon
2016-10-28 20:28:35 +00:00
Colin Walters
7ee4edf8e3 tree-wide: Remove more unused variables
Now that the other PRs are merged, kill of the rest of the unused
variables.

Closes: #504
Approved by: jlebon
2016-10-28 18:25:33 +00:00
Colin Walters
aed6077e73 core: Fix lock directory initialization
Though in practice, we just aren't using this right now.

Closes: #504
Approved by: jlebon
2016-10-28 18:25:33 +00:00
Colin Walters
be7e29fb61 libglnx: Update to master (for -fsanitize=undefined fixes)
Closes: #504
Approved by: jlebon
2016-10-28 18:25:33 +00:00
Colin Walters
5100506909 compose: Port a large chunk of postprocessing to fd-relative
Part of the de-libgsystem-ification porting.

Closes: #498
Approved by: jlebon
2016-10-27 14:57:43 +00:00
Colin Walters
233f8c7602 tree-wide: Drop the last libgsystem autocleanup usage
Closes: #498
Approved by: jlebon
2016-10-27 14:57:43 +00:00
Colin Walters
1c2908cbef tree-wide: Port away from gs_transfer_out_value()
Using `g_steal_pointer()` is a little bit more typing, but less
magical which is good.

Closes: #498
Approved by: jlebon
2016-10-27 14:57:42 +00:00
Colin Walters
94a461cb4e tree-wide: Remove some unused variables found by CLang
This isn't all of them, but I'm trying to avoid conflicting
with my other outstanding PR.

Closes: #501
Approved by: jlebon
2016-10-27 14:30:47 +00:00
Colin Walters
a9ccf765c4 daemon: Fix use of a potentially uninitialized variable
If there's no origin, we'd be adding undefined stack memory for
the value of `gpg-enabled`.

Closes: #501
Approved by: jlebon
2016-10-27 14:30:47 +00:00
Colin Walters
328b85d396 Add infrastructure for alternative name (currently nts)
See https://github.com/projectatomic/rpm-ostree/issues/405

This patch adds an (off by default) `--enable-new-name` build option
which currently defaults to `nts`.  This is purely additive, and
the intention is that we'll support the rpm-ostree name in
perpetuity most likely.

At the moment, we add a new name for:
  - /usr/bin/$name
  - The systemd unit file

But we notably *don't* attempt to add a new name to the DBus API,
as it'd be a lot more invasive of a patch, and less payoff (it's
mostly just programs/scripts that interact with the DBus).

Closes: #497
Approved by: jlebon
2016-10-20 14:22:28 +00:00
Jonathan Lebon
0b6721a10a deploy: make sure commits are on the current branch
A funky behaviour of `rpm-ostree deploy` was that specifying a csum
directly allowed you to jump to any commit, regardless of whether that
commit exists on the current branch or not. We tighten that up here so
that we check that the checksum does exist on the current branch.

The previous behaviour can be useful of course, but we might want to
change how users access it so that we don't get inconsistencies such as
rpm-ostree status saying that we're sitting on a specific branch with a
specific commit which doesn't actually belong to that branch.

Closes: #495
Approved by: cgwalters
2016-10-20 14:06:58 +00:00
Colin Walters
5cbc57b426 upgrader: Use new tree for libdnf source root to major version rebasing
I'm trying to rebase my workstation across a major version, but
hit the issue that we were pointing libdnf at the current root,
and hence it inherited the major of 24, when my target base ref
was 25.  In trivial cases this may work, but I have various things
like Emacs layered that caused dependency resolution failure.

The obvious fix here is to point libdnf at the target base ref.
However, until now we'd been checking out a minimal copy of the tree
*before* doing package upgrade resolution.  I considered trying
to add `/usr/lib/os-release` to that set, but it doesn't seem
sustainable.

Particularly since I think the original rationale for this was that
the checkout was slow, but that was when we had `fsync()` on, which we
don't now.

Closes: https://github.com/projectatomic/rpm-ostree/issues/486

Closes: #496
Approved by: jlebon
2016-10-19 15:43:13 +00:00
Colin Walters
49fffad471 Run bwrap with fixed environment (PATH + LANG)
This is basically a re-implementation of
85c9dd5c07
for the same reasons: it makes things work for rpm-ostree running
inside NixOS.  But there are other cases where if e.g. an unprivileged
user runs rpm-ostree and may not have `/usr/sbin` in PATH, we still
want the container to pick it up, etc.

While changing things I nuked the hacky `RPMOSTREE_DEBUG_SCRIPT` env
var, we can re-add it later in a better way if needed.

Closes: #490
Approved by: jlebon
2016-10-17 20:29:35 +00:00
Patrick Uiterwijk
98332a3be4 compose: Add option to write commitid to file instead of updating ref
Closes: #492
Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>

Closes: #494
Approved by: cgwalters
2016-10-17 20:02:55 +00:00
Colin Walters
4542778648 tree-wide: Port to g_file_enumerator_iterate()
Unlike OSTree, I'm going to assert we can take a hard dependency
on the version of GLib that has it.

Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
afb7ad6bbe tree-wide: Port to g_spawn_sync instead of libgsystem subprocess API
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
4ccd31fcb5 tree-wide: Port to libglnx for shutil_rm_rf
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
b47301b3e1 tree-wide: Port to libglnx autocleanup macros
Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
c3789bcac9 tree-wide: Port away from gs_file_rename()
These use cases are better just invoking `unlink()`.

Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
a2ab0e4415 tree-wide: Port to glnx_set_error API
Away from libgsystem.

Closes: #483
Approved by: jlebon
2016-10-17 19:59:40 +00:00
Colin Walters
a15364c185 compose: Respect absolute paths for postprocess-script again
Regressed from some code refactoring.

Closes: #491
Approved by: jlebon
2016-10-14 17:29:43 +00:00
Colin Walters
56f3f509a1 core: Canonicalize an epoch of zero to the empty string
We have two different codepaths for creating a cache branch header
string, one from libdnf, and one from librpm.  It turns out if
an RPM package explicitly specifies an `Epoch: 0` like various Fedora
perl subpackages do, these are different.

Explicitly convert `0:` to the empty string to make them match, and
hence installation will work.

Closes: https://github.com/projectatomic/rpm-ostree/issues/349

Closes: #482
Approved by: jlebon
2016-10-12 18:09:30 +00:00
Daniel Peebles
b32bc75b6c Get rid of unnecessary selinux.h include
Nothing in the file seems to use it. Doing this quickly via GitHub UI under assumption that the magic @walters bot will take care of running the full test suite, but otherwise I'll test it later when I get back to a machine I can use for that.
Closes: #481
Approved by: cgwalters
2016-10-12 16:25:45 +00:00
Colin Walters
4a86bdd196 postprocess: Undo Fedora SELinux policy move
Basically, this brings us back to where we are with RHEL/CentOS 7.

For lots more information, see
https://bugzilla.redhat.com/show_bug.cgi?id=1290659

Closes: #479
Approved by: jlebon
2016-10-07 15:57:15 +00:00
Jonathan Lebon
485d170c25 core: update some comments re. relabeling
Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
dbbe8d0687 core: Don't use parent commits for pkg cache
This makes the design fully coherent now - we don't try to store
multiple commits, we no longer reference them, and the cleanup does a
depth=0 pruning.

Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
f9f326f0f9 core: Don't try to walk up pkgcache parent chains
My desktop system was aborting on upgrade with a missing metadata
object, which turned out to be the fact that my previous changes to
the package layering code pruned with depth=0, but the commit it was
looking for had a parent commit that had been pruned.

Closes: #475
Approved by: jlebon
2016-10-05 21:27:05 +00:00
Colin Walters
7dd28fa964 core: Fix erroneous ret = TRUE
I hit a pile of:
```
Oct 04 12:44:15 icarus.verbum.local rpm-ostreed[26257]: ostree_repo_resolve_partial_checksum: assertion 'error == NULL || *error == NULL' failed
```

Which turned out to be a missing metadata object (for some reason),
but this function's incorrect use of `ret = TRUE` caused the GError
to have already been set.

Fix this, and we change to "direct return" style which is more
readable.

Closes: #474
Approved by: jlebon
2016-10-04 17:55:10 +00:00
Colin Walters
4852543ecc compose: Support packages-$basearch
See
https://mail.gnome.org/archives/ostree-list/2016-April/msg00020.html

Closes: #305
Approved by: jlebon
2016-10-04 15:10:32 +00:00
Colin Walters
1281434cec postprocess: Print absolute path for ignored files in /var
For ease of debugging.  Maybe in the future we try to look them up in
the RPM database too.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Colin Walters
0cd584ccf6 postprocess: Silently clean up known-unnecessary files in /var
Per OSTree design, /var can start out empty.  However, our warning
spam here is annoying.  Let's first delete some known files -
obviously this won't be exhaustive, but it's way faster than trying to
fix all of this in the packages right now.

The major one is the SELinux policy, which resulted in a lot of spam.

Closes: #473
Approved by: jlebon
2016-10-04 13:28:11 +00:00
Jonathan Lebon
80267a423a status: handle errors from json-glib
The API for json_generator_to_stream() says:

    Return value: %TRUE if the write operation was successful, and
    %FALSE on failure. In case of error, the #GError will be filled
    accordingly

When in fact, because it just gives back what g_output_stream_write(),

    1. it doesn't strictly return TRUE/FALSE, but a full-range uint, and
    2. it will return -1 (which is as good as anything > 0 for success
       checks) when an error occurs.

Although a simple <= 0 would fix it, let's just be extra strict and also
check for an error.

Closes: #468

Closes: #469
Approved by: miabbott
2016-09-26 14:49:51 +00:00
Colin Walters
0fd83d47b7 passwd: Also clone subuid/subgid files
These were added to shadow-utils in recent times (Fedora 25 e.g.)  and
like the other passwd files, shadow-utils opens them with `O_TRUNC`.

(At some point we should patch it to do create-new-then-replace internally)

Closes: https://github.com/projectatomic/rpm-ostree/issues/458

Closes: #459
Approved by: jlebon
2016-09-13 13:30:15 +00:00
Jonathan Lebon
037a310964 sysroot-upgrader: make pkgcache cleaner speak well
Just some polish. Noticed it while watching someone give a demo.

Closes: #457
Approved by: cgwalters
2016-09-09 20:36:20 +00:00
Colin Walters
70d7501e67 Fix regression from baseref cleanup
Commit 334cec56a0 had a fatal
logic error - I changed it to write a temporary ref for the final
commit, not the base.

I noticed this by getting `missing metadata object $base.commit` on my
desktop system.  Hooray for dogfooding git master.

Closes: #452
Approved by: jlebon
2016-09-08 17:51:26 +00:00
Colin Walters
80e616af7f Rename pkg-add -> install, move out of preview
I think these are suitable for wider consumption now, and we should
be able to support existing installs.

Rename the verb to just `install` since it's what every other package
manager uses.

Closes: #450
Approved by: jlebon
2016-09-07 22:11:33 +00:00
Colin Walters
0649567ffc compose: Nuke mock workaround
It turns out it was buggy (for some reason `PS1` wasn't propagating),
and furthermore, things mostly work if one enables `--new-chroot` i.e.
`systemd-nspawn`, which is what Fedora is going to do, and everyone
else in the world uses Docker.

While we're here, tweak the error message to use `<>` around the URL
which makes it more easily clickable from terminals.

Closes: #449
Approved by: jlebon
2016-09-07 21:57:01 +00:00
Colin Walters
9596bfcc42 daemon: Fix ConditionPathExists location
It needs to be under `[Unit]`.  I noticed the systemd warning
in my logs; not sure how this worked when I was testing it locally.

Closes: #446
Approved by: jlebon
2016-09-06 20:34:13 +00:00
Colin Walters
7145cbf7f3 daemon: Add ConditionPathExists=/ostree
Right now the daemon assumes the system is using ostree, but
for various reasons people can try to start it on non-ostree systems.

This is a simple fix to avoid crashing.  A better fix would
need to rework a lot of the code to return dummy/stub values but
that would be painful.  Maybe later.

https://bugzilla.redhat.com/show_bug.cgi?id=1372194

Closes: #445
Approved by: jlebon
2016-09-01 14:49:51 +00:00
Colin Walters
665083dff9 core: Define an autocleanup macro for Header
And switch a few things to `return FALSE`.  Preparatory work
for other cleanups.

Closes: #441
Approved by: jlebon
2016-08-31 20:15:14 +00:00
Colin Walters
525c605092 diff: Don't print changelogs by default
Communicating information is a hard problem, but basically there
are two things here.  First, the changelogs make the diff visual
spaghetti.  (If we were rendering to HTML with expanders or something,
it could work).

Second, I think RPM changelogs are legacy - tooling should pull from
git.

So far when doing release announcements I've been manually stripping
out the changelogs, embarassingly enough.  Time to fix that.

Anyways, we preserve the changelog-emitting code, it's just not the
default now.

Closes: #439
Approved by: jlebon
2016-08-30 20:13:35 +00:00
Colin Walters
436a8116f2 status: Sort the output package list
I am thinking of indenting into multiple lines if it gets long too,
but this is a small visual improvement.

Closes: #440
Approved by: jlebon
2016-08-30 20:09:24 +00:00
Colin Walters
363e213f18 upgrader: Avoid cleaning system repo twice
We need to manage our baselayer refs independently of the deployment
being written by libostree, so suppress the default clean; we'll do
it in rpm-ostree.

I noticed the `Freed objects:` message twice and always wondered why
until today.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Colin Walters
2bc8d7cccb upgrader: Prune pkgcache repo
Until now, we weren't pruning the pkgcache repo at all.  I ran out of
space in the root partition in my CAHC vagrant test box, so it's time
to fix this.

The basic algorithm is to walk over the full rpmdb contents of each
root, generate a set of "currently referenced" cached refs, then delete
any refs in the pkgcache repo which aren't included.  Then, do a prune
of the pkgcache repo.

While we're here, factor out a `sysroot_upgrader_cleanup()` function
which does all of the cleanup.  The idea is at some point we need to
introduce an `rpm-ostree cleanup` command or so which calls this, to
handle the case where the system is interrupted post-deploy but
pre-clean.

Closes: https://github.com/projectatomic/rpm-ostree/issues/428

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Colin Walters
334cec56a0 upgrader: Use a temporary baseref, rather than walking twice
In preparation for future changes here, rework things so
that we only walk the set of deployments once after the new
deployment has been generated.

To avoid a race where if we're interrupted after the deployment write
but before the regeneration, create a special `rpmostree/base/tmp` ref
that we delete after all the bases are regenerated.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Colin Walters
35d84f7507 pkglayering: Commit in base repo, not pkgcache
Currently, we do the final commit into the pkgcache repo, then
pull it to the base.  The problem with this is that, combined
with the fact that we're not presently pruning the pkgcache repo,
we leak space.

In preparation for a cleaner fix for this, rework things so that the
core infra can know about *both* a base repo and a pkgcache repo.  If
they're separate (as is the case for rpm-ostree-on-host), whenever we
are doing layering, explicitly pull just the `.file` objects that are
referenced into the parent repo.  We do the final commit in the base
repo.

Closes: #437
Approved by: jlebon
2016-08-30 19:53:24 +00:00
Jonathan Lebon
0fe12fe6a3 mutate-os-release: skip VERSION_ID
I hit this with librepo subbing out the $releasever with e.g. 7.2016.1
when trying to pull various URLs. It should be enough for the user to
see the ostree version in VERSION and PRETTY_NAME. For applications,
there's OSTREE_VERSION if they need just that.

Closes: #433
Approved by: cgwalters
2016-08-17 20:45:29 +00:00
Colin Walters
589a8e0974 core: Neuter systemctl during script execution
`systemctl start/stop/try-restart` are all useless for us in
scripts, since changes should only affect the *next* boot.

`systemctl enable` is also wrong - one should use presets instead.

Currently, systemd has code to detect whether it's inside
a chroot, which works for mock, but *not* for Docker or bubblewrap.
(We should teach systemd a nicer way to disable itself, but
 even if we did that we'd have to support old scripts)

So, this fixes layering `glusterfs` in CAHC.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
6900c616ec scripts: Execute %pre
Now that we handle `useradd`, which is most of what anyone wants
to do in `%pre`, let's turn it on.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
550adecb47 core: Support scripts which call useradd
When executing scripts, map `/usr/lib/passwd` temporarily back to
`/etc/passwd` from the POV of the script, then move it back.  This
allows client-side layered packages be merged with the base tree
version, while still being distinct from the `/etc/passwd`.

(In the future we'll likely rewrite all of this to use
`systemd-sysusers(8)`, but for now let's be incremental.)

Note in this commit it isn't very useful, because pretty much all RPMs
only call `useradd` in `%pre`, which is the next commit.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
dbd2c0ee7a unpacker: Synthesize tmpfiles.d for nonroot-owned /run and /var files
Because OSTree includes numeric uid/gid, but package layering (and
traditional full-client-side RPM) expects to allocate UIDs on the
machine, we need to handle non-root owned files specially.

For files in /run and /var, we can convert them to
`systemd-tmpfiles(8)` snippets which will be executed at the next
boot, after a uid/gid has been assigned.

Closes: #432
Approved by: jlebon
2016-08-17 20:41:36 +00:00
Colin Walters
3ad4e6c72b bwrap/compose: Add a workaround for Fedora's use of rpm-ostree-in-mock
Decided to test this on Sunday evening.  Of course it was broken =(
(Actually I tested mock-in-Docker but it should be the same)

The core problem is that mock does `chroot()` without using `/`
as a mount point.  This breaks an assumption in bwrap that it is.
Now, in theory we could move this same logic down into bwrap to
work around this situation, but for now let's hack it here.

Mock is old, legacy container code that doesn't really do anything
in a modern way - in fact our goal should be to replace it
with a combination of rpm-ostree and bwrap.  So carrying this
hack here to get us to that future should be OK for now.

Closes: #431
Approved by: jlebon
2016-08-15 18:31:58 +00:00
Colin Walters
7455e26273 bwrap: Add a selftest
I want a better error message if the user happens to execute
inside e.g. a Docker container without sufficient privileges
for recursive containerization.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
551e4c91f9 postprocess: Switch to using bwrap for script execution
The previous commit https://github.com/projectatomic/rpm-ostree/pull/422
introduced a regression in the "outside of a container" path - we
get `EINVAL` trying to `mount("proc",...)` and honestly I'm not sure why.

We can either back up or plow forward, and it turns out to be
pretty straightforward to complete the port to using bwrap.

I extracted the bwrap-execution code out of the RPM script engine,
because the treecompose model is currently different (no hardlinks
yet).

NOTE: A *very* important side effect of this is that we now
require "privileged" containers on hosts without user namespaces,
and on userns hosts, require `CLONE_NEWUSER` to be exported to the
container host.

In general though, the previous path of blindly executing scripts as root
without e.g. `proc` mounted was just bad.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
88a130da09 lib: Extract bwrap-executing internal API
The treecompose code will learn how to use bwrap instead of
libcontainer in libglnx, since the latter is a buggy copy of a subset
of the former.

Closes: #429
Approved by: jlebon
2016-08-12 23:34:33 +00:00
Colin Walters
a299083b09 core: Fix an error-handling logic error in relabel path
If next_dent returns `FALSE`, we need to also return `FALSE` from
the function.

I just noticed this as I happened to be reading the code randomly.

Closes: #426
Approved by: jlebon
2016-08-11 19:40:29 +00:00
Colin Walters
19490c593e Update to latest libdnf
I want the excludes= fix, and the memleak fixes are nice.

Closes: #425
Approved by: jlebon
2016-08-11 19:26:06 +00:00
Colin Walters
56a4f73901 scripts: Error out if we encounter lua
Otherwise we try to execute as shell script which obviously doesn't
work.  This was noticed when @dustymabe was trying to assemble
a Fedora container with `glibc-all-langpacks`.

Closes: #424
Approved by: jlebon
2016-08-11 15:32:49 +00:00
Colin Walters
aa2295db26 compose: Add --cache-only
When trying to test treecomposes, it's annoying to hit the
metadata servers each time.  Add a `--cache-only` option which
is kind of like yum's `-C` option (but not quite because
if the metadata doesn't exist we will update it).

Closes: #423
Approved by: jlebon
2016-08-09 15:18:14 +00:00