Commit Graph

1398 Commits

Author SHA1 Message Date
Colin Walters
2ada15aff3 libtest: Fix bugs in assert_file_has_content()
First, we need to preserve whitespace in the regexps.  Second,
in the error path, all of our variables were wrong.

Commit 376a2cc3f5 is an excellent
poster child for how many bugs one can introduce in a single commit
for a ~5 line function.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
69d8bfa042 compose-tests: Consolidate some misc options into test-misc-tweaks.sh
I was about to add another one of these but it feels like a bit
overkill to run through a recompose for trivial tweaks like turning off docs.

We can do a compose with multiple options at once and test the result as a unit,
at least for the smaller/less invasive options.

This change is prep for adding a switch to do `/tmp` as a regular dir.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Colin Walters
82fbb5342f postprocess: Some porting to new code style
Prep for future work.  No surprises here.

Closes: #777
Approved by: jlebon
2017-05-18 13:43:15 +00:00
Jonathan Lebon
68c8d5e039 docs: update linked project names and minor tweaks
Just a few nits I noticed while preparing some slides.

Closes: #776
Approved by: cgwalters
2017-05-15 15:47:39 +00:00
Colin Walters
024f953124 core: Log to journal rpm-md repo timestamps
This is part of: https://github.com/projectatomic/rpm-ostree/issues/774

Basically, it's very useful for debugging since one could get an out-of-date
mirror or the like.

Down the line we may more formally store this data in a permanently-persistent
way, but this is a simple extension of the journaling we do now.

Closes: #775
Approved by: jlebon
2017-05-12 18:09:02 +00:00
Jonathan Lebon
63d648b9aa f25-build-check: install git early
Install `git` before `build-check.sh`, so that we can run the cheaper
`ci-commitmessage-submodules.sh` test first.

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
ab1176f682 transaction-types.c: convert to new style
Pretty straightforward stuff.

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
1105ef872d pkg-builtins: rename internal functions
The pkg-add/remove names are legacy now.

s/pkg_add/install/
s/pkg_remove/uninstall/

This is mostly for cosmetics.

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
da4c25fea1 libglnx: bump and use new helpers
Update submodule: libglnx

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Jonathan Lebon
ba35102408 import ci/ci-commitmessage-submodules.sh from ostree
This would be useful here as well. Added small tweaks that I intend to
upstream (using realpath and --no-merges).

Closes: #773
Approved by: cgwalters
2017-05-12 16:32:49 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
376a2cc3f5 libtest: Support matching multiple regexps in files
For a future patch.

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
3fae2eaf89 daemon/core: Fix leaks in metadata cleanup
I noticed I had `/tmp/rpmostree-metadata*` directories, and there
were two cases where we created those dirs.  Astonishingly,
*both* had bugs that prevented cleaning up.

Rename one of them so in the future it'll be clearer which is buggy (or both).

Closes: #772
Approved by: jlebon
2017-05-12 14:05:58 +00:00
Jonathan Lebon
96f75519f3 core: partial switch to new style
Switch a couple of core functions to the new style in preparation for
some other work there. This doesn't tackle the larger assemble()
function.

Another function which escaped for now is one that uses a tmpdir. Makes
me wonder if we should add a "GlnxTmpDir" type thing which simplifies
the mkdtempat/opendirat pattern and has a shutil_rmrf cleanup.

Closes: #771
Approved by: cgwalters
2017-05-10 20:46:43 +00:00
Jonathan Lebon
143b1a5bd5 rpm-util: use glib auto cleanups macros
Rather than defining our own cleanup functions. And it's easier on the
eyes!

Closes: #771
Approved by: cgwalters
2017-05-10 20:46:43 +00:00
Jonathan Lebon
27c525b706 vmcheck: stronger post-test cleanup
Now that rpm-ostree operations are cumulative, we need to make sure we
also clean up pending deployments. Also clean up the rollback for
consistency.

We also strengthen our ref handling. Always nuke the vmcheck ref (which
might not even exist anymore if a test rebased without --skip-purge) and
recreate it from the original ref.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
caafaf6c1b vmcheck: add test for pkgcache chksum invalidation
Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
8c3e8b24ad core: invalidate pkgcache based on RPM checksum
Now that the unpacker actually tracks what the RPM checksum was at
import time, we can make use of that to determine whether we should
download the newer RPM, even if it has the same NEVRA.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
eff00053ef unpacker: embed RPM checksum in metadata
In addition to the header checksum, we also want a checksum of the RPM
file itself. This will allow us to know right away whether an RPM with
the same NEVRA in the repos is actually the same one we already
imported.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Jonathan Lebon
fa10f36fb6 core: write number of imported packages to journal
The actual number of imported packages is not printed right now on the
terminal. But it's useful information to have, so stick it in the
journal.

Also make both the ENABLED_REPOS and ENABLED_REPOS_SOLVABLES
comma-separated lists. This essentially makes them valid YAML/JSON,
which is a nice property to have for e.g. test/shell interactions.

Closes: #769
Approved by: cgwalters
2017-05-09 18:53:49 +00:00
Colin Walters
e66ea0865a libtest: Adapt xattr disabling for newer ostree
Matches the test changes in e8efd1c8dc

Closes: #770
Approved by: jlebon
2017-05-09 13:32:25 +00:00
Colin Walters
f81626d359 ci: Ensure fahc builds win over base
Right now due to rdgo using `git describe` we have `v2017.4.65`, which
is lower than `v2017.5`.

Closes: #770
Approved by: jlebon
2017-05-09 13:32:25 +00:00
Jonathan Lebon
04adff5d75 rpmostreed-os.c: remove unused variable
Not sure why the clang tester didn't pick this up.

Closes: #768
Approved by: cgwalters
2017-05-08 21:11:46 +00:00
Colin Walters
b7cf58efc1 daemon: Rework API for rollback deployment lookup
Before this commit, in the case where we have 3 deployments, the rollback code
would pick the merge/pending deployment. This is understandable since that logic
predates us potentially creating 3 deployments.

For livefs, I want rollback to actually go to the rollback, not the pending.

Add an API to find the pending/rollback, and put it in syscore, since we have
other similar functions to this there. (Perhaps in the future we'll figure out a
way to dedup the deployment walking/filtering logic a bit).

Port the two callers to it.

Closes: #767
Approved by: jlebon
2017-05-08 14:33:11 +00:00
Colin Walters
3f3090ee7e daemon: Reimplement ClearRollbackTarget in terms of cleanup txn
I didn't realize we had this dbus method.  It was added in the initial
drop of the DBus API, and AFAICS nothing uses it; it's not exposed
via the cmdline, and a quick `git grep` in Cockpit shows it's
not used there either.

It's tempting to delete it.

However, before we take that step, let's reimplement it in terms of the new,
shiny, more powerful "cleanup" txn.

The particular reason I'm doing this is I want to change how we handle
rollbacks.

Closes: #767
Approved by: jlebon
2017-05-08 14:33:11 +00:00
Colin Walters
e7a16c561b os: Use correct completion for cleanup txn
Spotted while doing further work.

Closes: #767
Approved by: jlebon
2017-05-08 14:33:11 +00:00
Colin Walters
670b134cb2 postprocess: Convert a few funcs to new code style
Prep for more work here.

Closes: #766
Approved by: jlebon
2017-05-08 13:34:29 +00:00
Jonathan Lebon
752368eb5b rebase: allow rebasing to a local branch
This is a follow-up to commit 77acf62. There, we added support for
rebasing from a local branch to another local branch. But in testing,
it's also really useful to be able to rebase from a remote-based refspec
to a local branch. We allow this here by slightly expanding the syntax
of allowed refspecs.

Now, we can use rpm-ostree all the time rather than fallback to `ostree
admin deploy`, which isn't pkg-aware.

Closes: #764
Approved by: cgwalters
2017-05-05 21:01:26 +00:00
Jonathan Lebon
a54efdff8a rpmostreed-utils.c: switch to new style
Since we're going to do some work there. Also fix not always translating
the error from ostree_parse_refspec.

Closes: #764
Approved by: cgwalters
2017-05-05 21:01:26 +00:00
Jonathan Lebon
3f5f749f82 libbuild.sh: pass arguments to make
Otherwise, e.g. `make check` && `make install` won't actually do
anything.

Closes: #765
Approved by: cgwalters
2017-05-05 20:23:22 +00:00
Colin Walters
a284b64479 ci: Use FAHC for build container
Conceptually: we're going to move rpm-ostree and ostree at the same
cadence most of the time; for both releases *and* for git master.
The problem so far has been the latter part.  Reusing FAHC
for the build gets us half of the problem.

The other trick I realized we can do - just pull ostree out from the build
container. This avoids fetching it from the internet, and makes my workflow for
hacking on both nicer - I just `sudo make install` in my build container for
ostree.

It's tempting to make the whole thing symmetric and require `sudo make install`
for rpm-ostree and not do the insttree thing but...perhaps after.

Closes: #758
Approved by: jlebon
2017-05-01 19:10:05 +00:00
Colin Walters
11309cab97 core: Ignore %pretrans
This was argued on list a while ago: http://lists.rpm.org/pipermail/rpm-ecosystem/2016-August/000391.html
Basically, every use of `%pretrans` is a workaround for some in-place
upgrade problem.  But we never do inplace updates, we always assemble
a new tree.  Hence, there is no point for us to ever execute these.

Let's stop erroring out on them. If there does turn out to be some problem with
a particular package (e.g. the `openjdk-copy-configs` or whatever), we would
likely end up needing to fix that package's `%post`, not start executing its
`%pretrans`.

Closes: #763
Approved by: jlebon
2017-05-01 18:07:06 +00:00
Colin Walters
081c9dc65b core: Journal selinux relabeling stats
An updated version of: https://github.com/projectatomic/rpm-ostree/pull/316
using the journal (and structured logging) instead.

I was just curious.  And yeah, in some quick testing via deploying an old
FAH `25.67`, there are zero relabels.

We should likely have a test that actually forces a relabel...would require
custom SELinux policy build?  Or hm, maybe we could override a file context.

Closes: #760
Approved by: jlebon
2017-04-28 21:42:39 +00:00
Colin Walters
4bbea19a31 ci: Delete dockerfile, move to common scripts, consolidate gcc/clang
The Dockerfile is problematic since we can't update it atomically. I also really
dislike reliance on the Hub. Further, I think rather than caching our build deps
as built containers, we should be caching RPMs in CI. And we should be using
rpm-ostree at some point to assemble filesystem trees faster.

Also, consolidate the clang to be serial with gcc, since while we lose a tiny
bit of parallelism, it's not really worth its own context right now.

Closes: #759
Approved by: jlebon
2017-04-28 21:17:18 +00:00
Colin Walters
23a9e7e1bc Release 2017.5
A very minor release, but better than backporting more patches. This rolls in
the `RegisterClient` fix as non-root, plus the arm32 crasher.

The syscore refactoring is the only risky part, but I'm pretty confident in it.

Closes: #757
Approved by: jlebon
2017-04-28 13:24:12 +00:00
Jonathan Lebon
f58600a348 util: fix potential overflow in var substitution
When appending the trailing bytes, we were passing (p - s), but p by
definition is always NULL at that point. Chaos ensues.

The really evil part about this is that the len is passed signed and
glib treats negative values to mean it should lookup the length itself,
so this worked *most of the time*. Though I'm guessing if the address at
s is large enough, (p - s) can wrap around and become positive again,
thus causing a massive allocation. Anyway, I didn't actually check this
(esp. since the report was for ARM), but the patch is clearly right.

I think this may be the cause for
https://bugzilla.redhat.com/show_bug.cgi?id=1381357.

Closes: #756
Approved by: cgwalters
2017-04-27 20:55:52 +00:00
Colin Walters
e4b5e77e64 tests: Add unit tests for varsubst
Trying to debug https://bugzilla.redhat.com/show_bug.cgi?id=1381357

I don't see the problem yet, but let's exercise the code a bit.

Closes: #755
Approved by: jlebon
2017-04-27 18:57:10 +00:00
Colin Walters
6f3f20a900 utils: Mostly convert to new code style
I was looking at the varsubst code, doing a style conversion first.

Closes: #755
Approved by: jlebon
2017-04-27 18:57:10 +00:00
Colin Walters
7d98e3efc8 libpriv: Add a helper function to print pkg diff
I want to use this in livefs, where I'll end up doing some diff
computations on the server and am currently rendering text there.

It might also be a step towards using this in `db diff`.

Closes: #709
Approved by: jlebon
2017-04-26 14:26:00 +00:00
Colin Walters
999e0034ef daemon/cleanup: Make use of new libostree API to fix TODO
We now avoid doing cleanup twice.

Closes: #709
Approved by: jlebon
2017-04-26 14:26:00 +00:00
Colin Walters
9260d828a8 daemon: Move deployment cleanup logic into core
While nothing else besides cleanup right now would call this directly, the code
should live close to the other very similar function:
`rpmostree_syscore_add_deployment()`.

Closes: #709
Approved by: jlebon
2017-04-26 14:25:59 +00:00
Colin Walters
386ac7537b daemon/core: Move some deployment logic here, also add bump_mtime()
This is prep for livefs.  We need to tweak the logic from what
core libostree has in `ostree_sysroot_simple_write_deployment()`,
and while we could land improved logic there, I think it makes
sense to carry this here until we're confident enough in the logic
to make it ABI.

This does depend on a [new libostree API](https://github.com/ostreedev/ostree/pull/745)
that allows writing deployments without doing cleanup.

The `bump_mtime()` bit is also prep for livefs, carrying in this patch to avoid
splitting things too much.

Closes: #709
Approved by: jlebon
2017-04-26 14:25:59 +00:00
Colin Walters
6485a8b306 core: Port a few functions to new style
I was looking at livefs executing scripts, realized I may need to
do some refactoring here, and decided to do some style updates.

Also, we make use of the `g_autoptr()` for ostree types in a few
places.

Closes: #752
Approved by: jlebon
2017-04-26 13:26:49 +00:00
Colin Walters
c8546b62a3 core: Fix error handling in hardlink changes for directory iteration
This function wasn't like the others.  If an error happened during
directory iteration, we'd still return TRUE, but with an error set.

Noticed while I was going to do some code style porting.

Closes: #752
Approved by: jlebon
2017-04-26 13:26:49 +00:00
Colin Walters
c87151b910 app: Prefix error when trying to connect to the bus
I typed `rpm-ostree status` in a container, and today we print:
`error: Could not connect: No such file or directory` which isn't
quite informative enough for me.

Closes: #751
Approved by: jlebon
2017-04-24 20:12:40 +00:00
Colin Walters
06e4ba2602 scripts: Ignore openjdk pretrans
I honestly didn't really dive into this, but it looks like
this is yet another "hack upgrades in the yum case", which we
should be able to ignore since we always do reassembly.

I at least tested `java -version` works with this.

This is a band-aid for the bigger issue of:
https://github.com/projectatomic/rpm-ostree/issues/749

(Doing this one since an AtomicWS user reported it)

Closes: #750
Approved by: jlebon
2017-04-24 14:38:33 +00:00
Colin Walters
65cfd22314 libpriv: Delete some dead code
I happened to notice these functions are unused. Purge them from existence.

Closes: #746
Approved by: jlebon
2017-04-21 13:54:05 +00:00
Colin Walters
a180a35a48 status: Do show Commit with --verbose, don't show StateRoot without it
`Commit` is useful in cases where one wants to e.g. run `rpm-ostree db diff`.
`StateRoot` however is going to be in the vast majority of cases identical
across deployments.

Closes: #743
Approved by: jlebon
2017-04-21 13:31:21 +00:00
Colin Walters
b2f9c5ed82 status: Rename OSName → StateRoot in the human-readable text
See: https://github.com/ostreedev/ostree/pull/794

I originally tried to do this ~a year ago in:
https://github.com/projectatomic/rpm-ostree/pull/295

But Micah pushed back at the time.  I'd still like to do this
though, since I personally find `OSName` *more* confusing
than `StateRoot`.

One specific rationale I have for making this change is that
I rebased a home server from CentOSAH to FAH - and I feel
it's just saner to see `StateRoot: centos-atomic-host` instead
of `OSName: centos-atomic-host` underneath the fedora refspec.

Closes: #743
Approved by: jlebon
2017-04-21 13:31:21 +00:00
Colin Walters
9ab46f5221 Require ostree 2017.5
This is preparatory for future changes which will make use the newer sysroot
writing API. But in this change, we can drop all of our version ifdef'd code.

Closes: #740
Approved by: jlebon
2017-04-20 20:58:51 +00:00