Commit Graph

301 Commits

Author SHA1 Message Date
Colin Walters
2adc1cf246 core: Ignore overrides for nonexistent %ghost files in /etc
As seen in e.g. `ipa-client-common`.  We expect ghosts 👻 to not
exist.

Closes: https://github.com/projectatomic/rpm-ostree/issues/784

Closes: #785
Approved by: jlebon
2017-05-24 15:53:07 +00:00
Jonathan Lebon
174510fc5d core: allow no enabled repos for local pkgs
This is a valid case when layering local RPMs. Otherwise (e.g. if
specific non-local packages are requested), give the nicer error rather
than letting libdnf throw something less useful.

Closes: #780

Closes: #781
Approved by: cgwalters
2017-05-19 16:08:18 +00:00
Colin Walters
82fbb5342f postprocess: Some porting to new code style
Prep for future work.  No surprises here.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
024f953124 core: Log to journal rpm-md repo timestamps
This is part of: https://github.com/projectatomic/rpm-ostree/issues/774

Basically, it's very useful for debugging since one could get an out-of-date
mirror or the like.

Down the line we may more formally store this data in a permanently-persistent
way, but this is a simple extension of the journaling we do now.

Closes: #775
Approved by: jlebon
2017-05-12 18:09:02 +00:00
Jonathan Lebon
ab1176f682 transaction-types.c: convert to new style
Pretty straightforward stuff.

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
da4c25fea1 libglnx: bump and use new helpers
Update submodule: libglnx

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
3fae2eaf89 daemon/core: Fix leaks in metadata cleanup
I noticed I had `/tmp/rpmostree-metadata*` directories, and there
were two cases where we created those dirs.  Astonishingly,
*both* had bugs that prevented cleaning up.

Rename one of them so in the future it'll be clearer which is buggy (or both).

Closes: #772
Approved by: jlebon
2017-05-12 14:05:58 +00:00
Jonathan Lebon
96f75519f3 core: partial switch to new style
Switch a couple of core functions to the new style in preparation for
some other work there. This doesn't tackle the larger assemble()
function.

Another function which escaped for now is one that uses a tmpdir. Makes
me wonder if we should add a "GlnxTmpDir" type thing which simplifies
the mkdtempat/opendirat pattern and has a shutil_rmrf cleanup.

Closes: #771
Approved by: cgwalters
2017-05-10 20:46:43 +00:00
Jonathan Lebon
143b1a5bd5 rpm-util: use glib auto cleanups macros
Rather than defining our own cleanup functions. And it's easier on the
eyes!

Closes: #771
Approved by: cgwalters
2017-05-10 20:46:43 +00:00
Jonathan Lebon
8c3e8b24ad core: invalidate pkgcache based on RPM checksum
Now that the unpacker actually tracks what the RPM checksum was at
import time, we can make use of that to determine whether we should
download the newer RPM, even if it has the same NEVRA.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
eff00053ef unpacker: embed RPM checksum in metadata
In addition to the header checksum, we also want a checksum of the RPM
file itself. This will allow us to know right away whether an RPM with
the same NEVRA in the repos is actually the same one we already
imported.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
fa10f36fb6 core: write number of imported packages to journal
The actual number of imported packages is not printed right now on the
terminal. But it's useful information to have, so stick it in the
journal.

Also make both the ENABLED_REPOS and ENABLED_REPOS_SOLVABLES
comma-separated lists. This essentially makes them valid YAML/JSON,
which is a nice property to have for e.g. test/shell interactions.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Colin Walters
670b134cb2 postprocess: Convert a few funcs to new code style
Prep for more work here.

Closes: #766
Approved by: jlebon
2017-05-08 13:34:29 +00:00
Colin Walters
11309cab97 core: Ignore %pretrans
This was argued on list a while ago: http://lists.rpm.org/pipermail/rpm-ecosystem/2016-August/000391.html
Basically, every use of `%pretrans` is a workaround for some in-place
upgrade problem.  But we never do inplace updates, we always assemble
a new tree.  Hence, there is no point for us to ever execute these.

Let's stop erroring out on them. If there does turn out to be some problem with
a particular package (e.g. the `openjdk-copy-configs` or whatever), we would
likely end up needing to fix that package's `%post`, not start executing its
`%pretrans`.

Closes: #763
Approved by: jlebon
2017-05-01 18:07:06 +00:00
Colin Walters
081c9dc65b core: Journal selinux relabeling stats
An updated version of: https://github.com/projectatomic/rpm-ostree/pull/316
using the journal (and structured logging) instead.

I was just curious.  And yeah, in some quick testing via deploying an old
FAH `25.67`, there are zero relabels.

We should likely have a test that actually forces a relabel...would require
custom SELinux policy build?  Or hm, maybe we could override a file context.

Closes: #760
Approved by: jlebon
2017-04-28 21:42:39 +00:00
Jonathan Lebon
f58600a348 util: fix potential overflow in var substitution
When appending the trailing bytes, we were passing (p - s), but p by
definition is always NULL at that point. Chaos ensues.

The really evil part about this is that the len is passed signed and
glib treats negative values to mean it should lookup the length itself,
so this worked *most of the time*. Though I'm guessing if the address at
s is large enough, (p - s) can wrap around and become positive again,
thus causing a massive allocation. Anyway, I didn't actually check this
(esp. since the report was for ARM), but the patch is clearly right.

I think this may be the cause for
https://bugzilla.redhat.com/show_bug.cgi?id=1381357.

Closes: #756
Approved by: cgwalters
2017-04-27 20:55:52 +00:00
Colin Walters
6f3f20a900 utils: Mostly convert to new code style
I was looking at the varsubst code, doing a style conversion first.

Closes: #755
Approved by: jlebon
2017-04-27 18:57:10 +00:00
Colin Walters
7d98e3efc8 libpriv: Add a helper function to print pkg diff
I want to use this in livefs, where I'll end up doing some diff
computations on the server and am currently rendering text there.

It might also be a step towards using this in `db diff`.

Closes: #709
Approved by: jlebon
2017-04-26 14:26:00 +00:00
Colin Walters
6485a8b306 core: Port a few functions to new style
I was looking at livefs executing scripts, realized I may need to
do some refactoring here, and decided to do some style updates.

Also, we make use of the `g_autoptr()` for ostree types in a few
places.

Closes: #752
Approved by: jlebon
2017-04-26 13:26:49 +00:00
Colin Walters
c8546b62a3 core: Fix error handling in hardlink changes for directory iteration
This function wasn't like the others.  If an error happened during
directory iteration, we'd still return TRUE, but with an error set.

Noticed while I was going to do some code style porting.

Closes: #752
Approved by: jlebon
2017-04-26 13:26:49 +00:00
Colin Walters
06e4ba2602 scripts: Ignore openjdk pretrans
I honestly didn't really dive into this, but it looks like
this is yet another "hack upgrades in the yum case", which we
should be able to ignore since we always do reassembly.

I at least tested `java -version` works with this.

This is a band-aid for the bigger issue of:
https://github.com/projectatomic/rpm-ostree/issues/749

(Doing this one since an AtomicWS user reported it)

Closes: #750
Approved by: jlebon
2017-04-24 14:38:33 +00:00
Colin Walters
65cfd22314 libpriv: Delete some dead code
I happened to notice these functions are unused. Purge them from existence.

Closes: #746
Approved by: jlebon
2017-04-21 13:54:05 +00:00
Colin Walters
9ab46f5221 Require ostree 2017.5
This is preparatory for future changes which will make use the newer sysroot
writing API. But in this change, we can drop all of our version ifdef'd code.

Closes: #740
Approved by: jlebon
2017-04-20 20:58:51 +00:00
Colin Walters
0c99106a99 Update libdnf submodule
To pull in the GPG https:// key fix.

Closes: https://github.com/projectatomic/rpm-ostree/issues/715

Closes: #719
Approved by: jlebon
2017-04-13 15:15:03 +00:00
Jonathan Lebon
f13db897f6 core: handle enabled_metadata repos
In #728, we started querying enabled repos using get_n_solvables().
However, there are different kinds of enabled repos, and
`dnf_repo_get_enabled()` reflects that through the bitmask it returns:
  - DNF_REPO_ENABLED_NONE: repo disabled
  - DNF_REPO_ENABLED_PACKAGES: repo enabled for package installs
  - DNF_REPO_ENABLED_METADATA: repo enabled for metadata

We were treating it as a boolean, though really, we should only print
data about repos with ENABLED_PACKAGES on, which are the actual repos
libdnf can fetch packages from. Repos with only ENABLED_METADATA on are
not fetched by default, and thus will cause SIGSEGV when trying to
get_n_solvables().

I ran into this while trying to debug #720 on F25 AH, which has this
repo by default:

  [fedora-cisco-openh264]
  name=Fedora $releasever openh264 (From Cisco) - $basearch
  baseurl=https://codecs.fedoraproject.org/openh264/$releasever/$basearch/
  enabled=0
  enabled_metadata=1

Closes: #736
Approved by: cgwalters
2017-04-13 14:06:04 +00:00
Colin Walters
530ab23cb8 core: Add journal message for pkg state (repos, solvables)
I'm debugging a depsolve failure right now, and just like for
the treecompose case, it's useful to see the information about
enabled repositories.  (We should actually maintain this
better in the daemon, but this is a start)

Closes: #728
Approved by: jlebon
2017-04-11 19:24:33 +00:00
Colin Walters
b060ecda21 unpacker: Also set flag to make failure to label fatal
There's no immediate need for this; was just a lead
I was chasing down while debugging
https://github.com/ostreedev/ostree/pull/775

Closes: #724
Approved by: jlebon
2017-04-05 13:19:39 +00:00
Jonathan Lebon
d5eeea646a rpmostree-kernel.c: always get kver from /lib/modules
The kver that rpmostree_find_kernel() returned would include the csum
that ostree tacks on. This would then cause dracut to fail to find the
modules for the given kver.

The issue is that rpmostree_find_kernel() can be called for both the
ostree checkout case (pkglayering) and the yumroot case (treecompose).
Rather than using a heuristic to determine whether there is a trailing
csum we should trim, we instead always fetch the kver from the dirname
in /usr/lib/modules, which will work in both cases and is stronger.

This also fixes a few cosmetic issues. Notably the final initramfs and
vmlinuz filenames in /usr/lib/ostree-boot would have *two* chekcsums
tacked on, rather than just one.

Closes: #722
Approved by: cgwalters
2017-04-04 18:47:31 +00:00
Jonathan Lebon
7b9f4870bd dracut: always specify kver
Otherwise dracut will just default to the running kernel, which will be
wrong if the pending deployment doesn't have the same kernel. Use --kver
rather than appending to be more explicit.

Closes: #722
Approved by: cgwalters
2017-04-04 18:47:31 +00:00
Colin Walters
c1872bc29e postprocess: Tell ostree to error out on missing labels
See https://github.com/ostreedev/ostree/pull/768 - basically
`/proc` is unlabeled, and this gets annoying for livefs work
since it shows up as a diff.

Plus at some point in the future `ostree fsck` might actually learn to verify
deployment directories.

Closes: #714
Approved by: jlebon
2017-03-31 14:30:03 +00:00
Colin Walters
f71e1b8b4d all: Conditionally use new ostree_sepolicy_new_at()
If available, using the also-new `OSTREE_CHECK_VERSION`. I dropped the `path`
argument from one of the internal APIs since it made the code simpler, and every
caller except one was passing `.`.

Closes: #712
Approved by: jlebon
2017-03-30 13:06:48 +00:00
Colin Walters
2abb3a547a util: Stub out OSTREE_CHECK_VERSION if we don't have it
Prep for using it in other places.

Closes: #712
Approved by: jlebon
2017-03-30 13:06:48 +00:00
Colin Walters
1969b96f91 libpriv/origin: Convert to new code style
This is a small one, but I like the momentum. Also note my use of the `,`
operator here. A little unusual maybe but I think it makes sense.

Closes: #700
Approved by: jlebon
2017-03-22 20:42:38 +00:00
Jonathan Lebon
702ccadb64 core: convert some functions to new style
Closes: #699
Approved by: cgwalters
2017-03-22 17:07:10 +00:00
Colin Walters
8b6c55ca60 core: Also apply mode overrides to symlinks
I originally thought there was a bug here, but missed we were skipping
symlinks earlier above.  See [previous discussion][1].

Now, I'm not aware right now of something this patch actively fixes, but I think
it makes sense to be consistent in our handling of things here with respect to
symlinks.

1: 29dd1bd801..8158dcfb47 (r95017893)

Closes: #689
Approved by: jlebon
2017-03-21 21:35:37 +00:00
Colin Walters
a05e256177 compose: Delete /usr/etc/passwd- (and the other variants)
There's no point to shipping these backup files in the base tree. We already had
code to delete them for the package layering case where they caused active harm.
At the point we added that code we really should have *also* changed treecompose
to delete them. Better late than never.

The reason I'm doing this now is because having them in the base tree causes `ex
livefs` to spuriously think that layering a package that *doesn't* change `/etc`
as if it does, because the layering code deletes the backup files.

Closes: #693
Approved by: jlebon
2017-03-20 16:35:17 +00:00
Colin Walters
62a93c7850 postprocess: Handle f26 /etc/nsswitch.conf configuration
F26 put sss first, which broke our regexp. When we switch to sysusers, man it'll
be nice to dump ♲ this.

Closes: https://github.com/projectatomic/rpm-ostree/issues/685

Closes: #686
Approved by: jlebon
2017-03-16 18:39:38 +00:00
Colin Walters
05a59e5af5 core: Don't try to chmod() symlinks
I just noticed this while reading the code. Right now we're actually following
links (since `AT_SYMLINK_NOFOLLOW` isn't implemented), which could definitely
lead to bad things if e.g. one has a symlink pointing to a file which shouldn't
be executable or world-readable etc.

(Also I noticed our hardlink-breaking-only-if-regular logic is now broken;
 I added a TODO)

Closes: #677
Approved by: jlebon
2017-03-13 17:53:51 +00:00
Colin Walters
77fa10cfbe core: Skip (and warn on) rpmfi overrides outside of /{usr,etc,var}
I still need to query whether Fedora has anything like this; maybe `/boot`? But
things are not likely to work right now if we do, so let's ignore it.

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
ab7431c458 core: Apply rpmfi overrides for /etc to /usr/etc
This fixes installing `mock`, which for some reason has files in `/etc/mock`
owned by `root:mock`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/671

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
d8b96a8531 core: Include filename in error messages for rpmfi overrides
This gives me now:

```
error: While applying overrides for pkg mock: fstatat: etc/mock/custom-1-aarch64.cfg: No such file or directory
```

Which is definitely more useful for debugging 💣.

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Jonathan Lebon
81c43e81fb PkgChange: add support for layering local RPMs
Allow users to directly specify an RPM file on the command-line. The
"packages_added" array of the PkgChange() method can now contain
absolute paths to RPM files.

Grow the origin format to have a new "requested-local" key. This is
similar to the "requested" key, except that the packages are always
installed from cache. The "requested-local" array values also embed the
SHA-256 of the header we expect.

There is now a new "LocalPackages" line in the status. These packages
are a subset of the "packages" element (which are printed as
"LayeredPackages") and represent the packages that are explicitly marked
for installing from cache.

Interesting design choices/notes:
  - Just as before, even with foo-1.0-1.x86_64 installed from RPM, a
    user can still request "/usr/bin/foo": it will be made dormant. As
    soon as foo stops being explicitly layered from the RPM, it will try
    to fulfill the request by going to the repos. This allows users to
    "pin" a layered package to a certain RPM, and then unpin it.
  - The strings/NEVRAs in "requested" and "requested-local" are strictly
     distinct. This allows us to be able to tell what the user means
     exactly when they do "rpm-ostree uninstall".

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Jonathan Lebon
5c5ebc1b0a core: add support for installing from cache
We start by adding support in the core for installing packages strictly
from the cache repo. We fool the libdnf stack by re-exporting the header
as an RPM, and explicitly marking it for install. The treefile format
supports specifying the expected SHA-256 of the metadata header, in case
the cache for a specific NEVRA changed.

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Colin Walters
e86dc13c49 bwrap: Don't use --unshare-net in nspawn by default
This will fix rpm-ostree-in-mock-in-koji. The drawback is minor: post scripts
will have network access. But we're going to be testing the no-network case in
our Docker-based builds, so that's fine.

Closes: #672
Approved by: jlebon
2017-03-10 17:27:56 +00:00
Colin Walters
7b8e05ffc2 Allow and start using C99 declaration-after-statement
The equivalent of https://github.com/ostreedev/ostree/pull/718
but for this codebase.

I just picked one example at random, there's plenty of others, but I don't want
to do any kind of tree-wide conversion since we have lots of outstanding
patches.

Closes: #664
Approved by: jlebon
2017-03-06 23:16:30 +00:00
Colin Walters
79ee224709 core: Log commit stats into journal
It's useful to see how many new objects were added and how much storage they
use. Just part of a general trend where I'd like to make a lot more extensive
use of the journal, particularly with structured logging.

Closes: #661
Approved by: jlebon
2017-03-06 22:19:10 +00:00
Colin Walters
66425c3161 core: Do GPG verification before importing
While reading a recent conversation about GPG checking at treecompose
time, I had a sudden thought - were we actually doing verification
client side?  Turned out, we aren't.  That happens as part of
`dnf_transaction_commit()` which we don't use.

That function verifies every package at one go, but for us I think it's better
to do it before "importing". We shouldn't have untrusted bits that we've
unpacked (they might have suid binaries, for one thing).

This is an embarassing problem, but it's worth emphasizing that everyone should
be retrieving repodata at a minimum over TLS, which sets a baseline. On RHEL, we
already do pinned TLS, and there are discussions about extending that elsewhere.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1422157

Closes: #656
Approved by: jlebon
2017-03-06 15:19:43 +00:00
Colin Walters
3568ed9a83 core: Plug leak of pkgcache repo
This showed up as a number of duplicate fds to the pkgcache root in
`/proc/<pid>/fd` for the daemon.

Closes: #654
Approved by: jlebon
2017-03-03 13:38:40 +00:00
Jonathan Lebon
89693c0385 core: no longer embed treespec
In the case of client layering, we hackily use the treespec because
that's what the core understands (for now), but it really shouldn't be
part of the final commit, nor should we rely on it.

This patch starts the path towards moving us away from the treespec by
not embedding it in client layers, and instead directly inserting
layered packages under the "rpmostree.packages" key.

The SERVER_BASE case still embeds the treespec, since only the container
path uses it for now and it needs it.

Down the line, we'll want to make the treespec just one of the methods
by which we initialize RpmOstreeContext. But nothing stops us from
hiding that detail already.

Closes: #650
Approved by: cgwalters
2017-03-01 17:42:42 +00:00
Colin Walters
22048b25a7 scripts: Use tmpfs for /var/tmp, not the host's /tmp
We don't want to expose the host's `/tmp` since that means scripts could
potentially find things like the X11 socket or whatever.

To debug things better, add a quick bash script to run bwrap like the C code
does. Perhaps down the line we can add `rpm-ostree internals run-bwrap` or so.

Closes: #647
Approved by: jlebon
2017-02-28 23:37:15 +00:00
Jonathan Lebon
b364357f60 origin: fix indentation
Closes: #651
Approved by: cgwalters
2017-02-28 22:45:07 +00:00
Jonathan Lebon
051492b3e6 upgrader: switch to stateless model
This is the culmination of the work paved by the previous commits. We no
longer handle package set mutations in the upgrader, but rather let
clients modify the origin as needed for that. As such, the upgrader is
now completely memoryless. We no longer try to error out if a package is
already in the base and we no longer drop out packages previously
layered packages that are now in the base. *Which* packages actually get
layered is determined during deployment and not coded in the origin.

At an API level, the deployment variant returned (and thus what gets
printed by `status --json`) now has both "packages" and
"requested-packages". The former retains its original semantic: it's the
set of packages that are *actually* layered. The latter contains
everything from the origin, which includes the former set + dormant
packages.

Accordingly, the output of `rpm-ostree status` also now distinguishes
between "RequestedPackages" and "LayeredPackages". Extra: we now quote
provides that contain spaces.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Jonathan Lebon
6a5d4a2699 RpmOstreeOrigin: also cache initramfs args
Ideally, we want to parse out everything in the origin on init and then
cache the values to make it even easier to inspect it. We make this more
explicit by renaming the struct members to "cache_*" and also caching
the initramfs args. This doesn't actually save any allocations right
now (though it might in the future), but we do the same for everything
else, and it's nice to be consistent.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Jonathan Lebon
d0c8dddf99 RpmOstreeOrigin: remove is_locally_assembled()
There are multiple reasons for this. The first is that with the
introduction of "dormant" packages, it becomes impossible from simply
looking at a deployment's origin whether it was layered or not. We
*could* make the upgrader turn around and insert back the "real" final
packages that were layered. But we're really trying to reinforce the
concept that the origin is mutable and represents an input for the
upgrader. Derived data shouldn't live there.

Whether a deployment is actually layered or not instead is determined by
the deployment commit itself. The embedded treespec records the exact
packages that were layered, which we will later use to enhance status
output.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Colin Walters
229fdfa7f9 scripts: Bind rather than symlink /usr/etc → /etc
This fixes resolution of relative symlinks, which fixes installation
of `firewalld` in Fedora 25.

Closes: https://github.com/projectatomic/rpm-ostree/issues/638

Closes: #640
Approved by: jlebon
2017-02-24 20:43:05 +00:00
Colin Walters
cc20f32fb7 tree-wide: Use autoptr cleanup for HyQuery
This was a noticeable source of leaks.

Closes: #644
Approved by: jlebon
2017-02-24 18:55:36 +00:00
Jonathan Lebon
c0b4fbc0f5 util: drop custom set_prefix_error_from_errno
We can use libglnx functions for this now.

Closes: #643
Approved by: cgwalters
2017-02-23 20:57:51 +00:00
Jonathan Lebon
b7613dcc81 RpmOstreeOrigin: don't error on unconfigured state
This is a follow-up to 775c781 (#626). Really, RpmOstreeOrigin shouldn't
concern itself with whether the origin is unconfigured. Its main goal
should be to parse it out and make it easy for users to modify it. That
sort of business logic lives in the upgrader, which *should* be
concerned if the origin in unconfigured.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Jonathan Lebon
62c7242a98 upgrader: move origin mutation outside of the upgrader
This is the beginning of a move towards making the upgrader more
stateless and less hysteretic. Ideally, the only input required should
be an origin file. Users of the upgrader are in charge of modifying the
origin as needed. The main goal of this is to simplify the upgrader
model and a better separation of concerns.

This was already done for the initramfs-related entries. This patch
formalises this for most of the other inputs, except packages, which
will be done soon.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Chen Fan
7ee41f75b7 remove duplicate words in comment
Signed-off-by: Chen Fan <fan.chen@easystack.cn>

Closes: #635
Approved by: cgwalters
2017-02-21 14:47:59 +00:00
Colin Walters
f6842838d7 build: Deal with gperf 3.1 changing to size_t
https://bugzilla.redhat.com/show_bug.cgi?id=1424268

Closes: #631
Approved by: jlebon
2017-02-17 18:51:53 +00:00
Eduardo Mayorga
1f2170e10c libpriv: Reduce scope of variables
nchange_date, nchange_name, and nchange_text are declared in a larger
scope in rpmostree-rpm-util.c. Reduce the scope to satisfy cppcheck.

Closes: #628
Approved by: jlebon
2017-02-16 15:40:20 +00:00
Colin Walters
ddbaf19f1e importer: Error importing RPMs which install to /opt (outside of /usr)
See https://github.com/projectatomic/rpm-ostree/issues/233 - for RPMs which
place files in e.g. `/opt`, we have different behavior in the treecompose case
(silently drop it) versus package layering (does the wrong thing).

Since the unpacker right now is only used in the layering case, this just
ensures we'll get a consistent error there.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Colin Walters
325de9d898 unpacker: Check for filter errors while committing tmpfiles too
This masked an issue in a change I was working on in the filter.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Jonathan Lebon
8cf071127e unpacker: support root-owned /var & /run files
I'm not sure why we weren't doing this before, but we need to also
support files in /var and /run that are owned by root.

Related: RHBZ#1421781

Closes: #622
Approved by: cgwalters
2017-02-13 16:49:59 +00:00
Colin Walters
a66d27230d Add a cleanup command and DBus API
We sometimes talk about using `ostree admin undeploy`, but that
doesn't know about the pkgcache, and hence space there leaks
until the next rpm-ostree operation.

Just for this, we need to expose a cleanup command (and API).  But
we also need to support cleaning:

 - repomd
 - downloads (repo/tmp)

So let's start implementing that.

Closes: #614
Approved by: jlebon
2017-02-13 15:28:05 +00:00
Jonathan Lebon
dd4b553bbc ux: improve package diff outputs
This patch makes the diff outputs generated by deploy commands (like
upgrade/deploy/rebase) and db diff commands the same. They were both
tweaked in different ways. The former gained the ability to discern
between upgrades and downgrades, the latter gained the ability the print
package version transitions.

Of course, the best way to make them the same would be to make them use
the same code, though I'm not sure it's worth the pain to gain ratio at
this point...

Closes: #575

Closes: #619
Approved by: cgwalters
2017-02-10 22:08:22 +00:00
Colin Walters
a52cb7d78e core: Add rpmostree.repo metadata to imported packages
I'm watching https://github.com/rpm-software-management/libdnf/pull/199 and I
really don't like it. We already have a place to put out-of-rpmdb metadata,
which is in the ostree commit for imported packages. No need to involve a
relational database for this (and further, one that would need to learn about
multiple ostrees).

We're not yet *using* this information in the UI, but we could; imagine
changing the `status` `Packages:` to show packages-per-repo or so.  We
could also expose an `rpm-ostree pkg-info foo`.

But for now, let's just start recording this.

Closes: #610
Approved by: jlebon
2017-02-08 17:52:16 +00:00
Jonathan Lebon
c756b26521 compose: fix mutate-os-release handling
- Let --add-metadata-string=version=val override any automatic version
  prefixing.
- Don't error out if mutate-os-release is given, but no new version is
  given by --add-metadata-string or automatic version prefixing.

Checking keys in parse_keyvalue_strings() is slightly hacky. I initially
wanted to just inspect the GVariantBuilder, but AFAICT, there is no way
to actually look up values from a builder (plus, we need that info early
to know whether automatic_version_prefix should itself inject in the
builder or not).

Closes: #603
Approved by: cgwalters
2017-02-01 22:25:54 +00:00
Colin Walters
cac4522e5b Support "system/regenerate-initramfs=true" flag in origin
Currently we push for a model where the initramfs is
generated (in non-hostonly mode), and merely replicated.

However, to support a few unfortunate corner cases like dm-multipath which wants
to inject a config file into the initramfs, we need to support regenerating it
client side too.

Down the line, we'll need this to support overriding the kernel too.

This changes things in the core to add the concept of an "empty"
`RpmOstreeContext`. I initially tried skipping it, but that was too much
duplication. We still want all of the core ostree-related logic that lives in
that code too.

The treespec bits barfed if the spec didn't have a `tree/packages` key. It was
simplest to change that to allow it - and because that was the only case where
we errored out in parsing, I dropped the error handling.

There was another place in the upgrader that now needed to be fixed to handle
transitioning from just regenerating initramfs to not.

Closes: #574
Approved by: jlebon
2017-01-20 16:17:51 +00:00
Colin Walters
30bed29230 daemon/upgrader: Rework layer tracking
Working on initramfs, I hit a subtle issue with the fact that
I was trying to "redeploy", but with the origin file changed
during the process.

Previously, it was a bit unclear which parts of the upgrader logic are operating
on the *new* origin versus the "original origin".

The package layering code in the upgrader explicitly carries a delta on top in
the "add/remove" hash sets, which means it isn't visible to
`rpmostree_origin_is_locally_assembled()`.

Whereas for initramfs, I set a new origin. This broke things since we were
expecting to find a parent commit, but the original origin wasn't locally
assembled.

When looking more at this, I realized there's a far simpler model -
rather than keeping track of commit + origin, and using the origin
to try to determine whether or not the commit is layered, we can
keep track of `base_revision` and `final_revision`, and the latter
is only set if we're doing layering.

The diff speaks for itself here - a lot of fragile logic looking at the origin
drops away.

The next step here is probably to drop away the package layering hash sets, but
I'm trying to not change everything at once.

Closes: #579
Approved by: jlebon
2017-01-18 20:05:19 +00:00
Colin Walters
b3e16a3313 bwrap: Turn on all namespaces
The fact we weren't doing this is an oversight.  We should *really*
be using the PID namespace at a minimum, but I decided to just turn
them all on.

The one that seems most likely to potentially introduce a regression is turning
on the netns (i.e. disabling networking). But I can't really think of what we'd
be running in a script today that would break in practice.

Closes: #572
Approved by: jlebon
2017-01-16 14:40:47 +00:00
Colin Walters
f8edd59d5f daemon: Introduce an origin structure
In prep for adding "regenerates initramfs" to "is locally assembled",
we need more abstraction.  I previously introduced a helper function, but
I'd still need to touch every call site when changing that.

Rather than having each caller re-parse the origin, let's parse it once into a
structure, and then have error-free accessors for it (that also don't malloc).
This way when adding a new flag, I don't need to touch every call site.

Notes:

 - Some places in the code (like the deployments -> variant bits) tried to
   handle deployments without an origin gracefully.  That's no
   longer true.  I'm not sure how much we care - do we?
 - There are a few places where I changed `packagelist.len() > 0` to
   `is_locally_assembled()`.  I think this is right, but we'll need
   to be sure the "no packages to overlay, just initramfs" case
   works when that lands.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
f215acb64f libpriv: Move kernel finalization into a helper function
This will need to be callable outside of postprocess for client side initramfs
generation.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
4c98943a64 postprocess: Clean up kernel/initramfs checksum calculation
This avoids having an intermediate file path, and makes the code easier to
follow.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
16efa12ae0 libpriv: Factor out "find kernel/initramfs" code into kernel.[ch]
To support running dracut on the client side, the dracut code
needs this, and it makes more sense in kernel.[ch] anyways.

I chose to use a GVariant instead of making a custom structure to avoid having
to manage custom free funcs.

Closes: #566
Approved by: jlebon
2017-01-13 20:08:24 +00:00
Colin Walters
e1cdfa1712 libpriv: Introduce an rpmostree-kernel.[ch] for dracut
Since the dracut run is now separated from the rest of postprocess, we change it
to use the bwrap API diretly, and this lets use the new _IMMUTABLE bwrap type.
This will make it easier to reuse for client-side initramfs regeneration.

Splitting this off makes it also easier to simplify the remaining mutable
usage in postprocess.c.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:26 +00:00
Colin Walters
f0ec738376 libpriv: Enhance internal bwrap API
Make it a real struct with methods.  This noticeably increases
the ergonomics and design of the API.

The main goal here is to introduce the enum which defines whether or not the
rootfs is mutable or not. We move the "rofiles" mode from the RPM script code
down into the bwrap layer, which will make it easier to reuse for treecompose.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:26 +00:00
Colin Walters
50ab2983ab postprocess: Use O_TMPFILE for dracut, merge reproducible bits
I was planning to do some further changes here, and I really don't like the
manual fork/exec stuff on in the --reproducible checks. Our subprocess code
should basically be all bwrap.  Synchronous code execution while not reading from the
pipe is a recipe for deadlocks.

What simplifies things a lot is to write to an `O_TMPFILE` fd (or a tempfile on
legacy kernels), and slightly extend our bwrap-executing code to support a child
setup function, so we can set the tmpfile fd to be stdout.

Now that we have a shell script wrapper we inject, it's trivial to reimplement
the "detect reproducibility" in shell script there, rather than C.

This doesn't matter much for treecompose today, but it will matter more when
we're supporting client side initramfs regeneration, since now the dracut
container can be fully immutable.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
7a421443f9 core: Split up overlay/commit API
In a future change I want to optionally rerun dracut between these. Keep the old
API as a trivial wrapper around the two.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Colin Walters
f766b59fbf daemon: Add a helper function for "is locally assembled"
This is all two callers need, and in the future I want to add
local initramfs generation.

Closes: #560
Approved by: jlebon
2017-01-11 18:39:25 +00:00
Jonathan Lebon
e32bbf16d1 passwd_prepare_rpm_layering: account for local entries
On Fedora 25, systemd adds a sysuser config file for multiple users. It
also explicitly creates those same users in its %pre, except for one:
systemd-coredump. This means that the tree's /usr/lib/passwd doesn't
contain systemd-coredump. Of course, on first boot, it gets created and
added to /etc/passwd.

During package layering, we map /usr/lib/passwd to the container's
/etc/passwd. If the %pre calls useradd/groupadd without passing an
explicit uid/gid, it's possible that the allocated id is already in use
by an entry in the deployment's /etc/{passwd,group} (such as
systemd-coredump, but the same holds for any manually-added entry).

We resolve this by taking the switcheroo a step further: we map
/usr/lib/passwd to /usr/etc/passwd, and then also map /etc/passwd to
/usr/lib/passwd. That way, useradd in %pre will account for already
allocated local uids and react accordingly.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
0bbba38727 core: add support for layering non-root files
We lift the restriction from the unpacker on non-root owners. The
unpacker now imports all files as root:root.

We split the running of rpm scripts between %pre and %post/posttrans as
two different stages. During assembly, we first run the %pre scripts,
collect users and groups, then chown as needed (and then finally run the
%post scripts).

Closes: #462

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
69e75debc2 rpmostree-passwd: make passwd/group parsers public
We're going to need to access these functions later.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Jonathan Lebon
0e431a8f83 core: add RPMOSTREE_USE_CACHED_METADATA
Being able to just reuse metadata is especially helpful when trying to
debug things lower down that path, as well as cuts about 2 mins from the
compose test.

Closes: #561
Approved by: cgwalters
2017-01-08 21:05:06 +00:00
Colin Walters
3a409342e6 postprocessing: Add a g_prefix_error for kernel
Saw this un-prefixed error path in a build; having error prefixes makes for
easier debugging.

Closes: #554
Approved by: giuseppe
2016-12-20 16:24:25 +00:00
Colin Walters
69cace4ce8 [ASAN] Squash a few minor misc leaks
This is part of the leak set when doing a treecompose.  There's
some that go down into libdnf...will get to that over time.

Closes: #547
Approved by: jlebon
2016-12-08 16:27:24 +00:00
Colin Walters
6fb29775a3 [ASAN] libpriv: Squash misc small leaks
Nothing big here, but this seems to be the last of the ASAN hits
in the unit tests.

Closes: #539
Approved by: jlebon
2016-12-07 20:22:17 +00:00
Colin Walters
1209942fc2 [ASAN] unpacker: Clean up leak of rpmts
This was 800 bytes, which could add up.

Closes: #539
Approved by: jlebon
2016-12-07 20:22:17 +00:00
Colin Walters
977835fd41 unpacker: Fix logic error regression in previous commit
I spotted this warning go by in the build logs.  Let's make it
fatal from now on.

Closes: #536
Approved by: jlebon
2016-12-07 20:16:32 +00:00
Colin Walters
0f4ef0d5a1 [ASAN] unpacker: Fix leaks of header/fi
We weren't freeing the Header struct in finalize if we happened
to own it.  Also define and use a cleanup for `rpmfi`.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
12a6999d02 [ASAN] tree-wide: Squash a few string leaks
Just a few random string leaks.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
975c0703c3 [ASAN] tree-wide: Clean up gvariant-related leaks
I kept thinking `^a&s` was sufficient for allocation-free reading
of `as`, but it's not, we need to free the outer buffer.  Other
minor cases were using `s` instead of `&s`.

Closes: #533
Approved by: jlebon
2016-12-06 16:47:26 +00:00
Colin Walters
88b8f1a655 unpacker: Don't error out layering fcaps-only binaries
In f25 now, `kubernetes-master` is changed to be owned by `root:root`
but still has file capabilities.  Our "non-root owned" check was
bombing out because we need to explicitly ignore "root" ownership.

(If there were no fcaps, then there's no override to check)

Closes: #532
Approved by: jlebon
2016-12-05 19:35:07 +00:00
Colin Walters
7149e9fee5 unpacker: Reorganize code for non-root checks
No functional changes, but I find this more legible - fewer
interacting conditionals.

Closes: #532
Approved by: jlebon
2016-12-05 19:35:07 +00:00
Colin Walters
74c43a0209 core: Error out if we're trying to do an empty install set
This is a partial fix for
https://github.com/projectatomic/rpm-ostree/issues/529

We could drop into this codepath via multiple ways (containers,
etc.), so it makes sense to have a last ditch error here, even
if we should really give an error earlier.

Closes: #530
Approved by: jlebon
2016-12-01 21:15:38 +00:00
Colin Walters
351832f29e Fix install-langs support
We're looking at changing Atomic Host to use multiple locales (but not all);
See: https://bugzilla.redhat.com/show_bug.cgi?id=1186757

This revealed our `install-langs` support didn't really work.  Our
`treecompose-post.sh` was deleting the extraneous translations anyways,
which masked this.  And for other cases like workstations where
we drag along all the translations anyways, it was fine too.

There were two bugs:
 - In the keyfile spec it's `instlangs`
 - We were setting the macro a bit too late, it should be before
   `dnf_context_setup()`.

Closes: #525
Approved by: jlebon
2016-11-24 14:38:16 +00:00
Colin Walters
ae108e08a5 Add rpmostree.clientlayer metadata to derived commits
In the future we may add more commands that take as input commit
IDs.  However, we really want to distinguish between server and
client generated commits, as some of these operations won't
make sense for derived commits.

This changes the API to have callers say which type of commit
they're generating, which also fixes a FIXME, and helps get us
a bit closer to the "unified core".

Closes: #520
Approved by: jlebon
2016-11-15 17:11:11 +00:00
Colin Walters
c8e7c63ab2 Final removal of libgsystem dependency
Just like ostree.  Now we can consider it dead.

Closes: #511
Approved by: jlebon
2016-11-08 14:38:54 +00:00