IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Add a new `install/uninstall --idempotent` option to make it easier to
interact with the CLI through scripts. E.g. one doesn't have to check
first if a request has already been installed/uninstalled.
Closes: #1467Closes: #1478
Approved by: cgwalters
This makes it easier for scripts to determine whether there is a pending
deployment instead of using `--json/--jsonpath`.
Closes: #1478
Approved by: cgwalters
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.
We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.
Closes: #1446Closes: #1455
Approved by: cgwalters
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.
Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM). This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.
There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.
This builds on prior work in
27bd7b97bb from #1396 .
Closes: #1406
Approved by: jlebon
This test was hard to adapt for staging. I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs. Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults. So I just deleted those bits.
Closes: #1453
Approved by: jlebon
Noticed this while looking at the logs for #1432.
Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.
I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...
Closes: #1450
Approved by: cgwalters
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units. For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.
(I briefly looked at implementing the `Trigger` line from `systemctl status`
but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)
Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.
Closes: #1432
Approved by: jlebon
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system. It's tempting to do a `reset` before each one
and reboot but this makes things work.
There's some additional assertions here as I went through and
was debugging.
Prep for making staging the default.
Closes: #1438
Approved by: jlebon
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.
I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?
Closes: #1387Closes: #1419
Approved by: cgwalters
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.
Closes: #1428
Approved by: cgwalters
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.
rpm-ostree uninstall --all -install foobar
to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.
Closes: #1424
Approved by: cgwalters
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).
The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.
This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem
Closes: https://github.com/projectatomic/rpm-ostree/issues/1392Closes: #1420
Approved by: cgwalters
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).
Closes: #1408
Approved by: cgwalters
Doing `override reset --all` may be a no-op if there are no overrides
currently applied. But we were trying to always print the tree diff
under the assumption that all override invocations have to result in a
new deployment.
Fix this by using the same default deployment check that `upgrade` does
before printing.
Closes: #1407
Approved by: cgwalters
This fixes a painful UX issue where one must use the full NEVRA when
uninstalling a locally layered RPM. Now, one can specify either the
NEVRA or the package name only. Though we still try to interpret the
request first as a NEVRA or a capability.
Closes: #1386Closes: #1400
Approved by: cgwalters
There are a few scenarios today where one might deliver content
to a machine via an external transport. For example, take the
scenario of a single server updated via USB drive. While we
can provide a refspec...what should the remote be? (This gets
into ostree collections). There's nothing really that can
happen when typing `rpm-ostree upgrade` unless the USB stick
is plugged in. That type of scenario should be emphasized
by pinning the commit - the machine is updated via an external
script.
Another case: we're experimenting embedding OSTree commits inside OCI
containers. Here again since rpm-ostree can't understand how to
pull content from containers, it's saner to drop the refspec
bits, and pin to a commit.
Further enhancements will follow to make the admin experience more
obvious.
Closes: #1396
Approved by: jlebon
For now all this switch does is turn off the scary warning. We also
tweak the warning to make it clear that this will be required in a
future release.
Closes: #1378
Approved by: cgwalters
I was honestly surprised to see this test was running all this time for
CentOS. As cool as it is, this test is really more meant for Fedora.
Closes: #1382
Approved by: jlebon
We were trying to test that doing `override remove` on a layered package
errored out. But the test was bogus since `foo` is a base package here.
Let's just use a brand new package that will actually be layered.
Closes: #1382
Approved by: jlebon
Just part of the continuing breakdown of our testsuites into smaller
pieces for more efficient allocation.
Will add a test case to the `layering-basic-2.sh` in a follow-up commit.
Closes: #1382
Approved by: jlebon
With the new support for pinning deployments, we need to also update
rpm-ostree to clean up the transient state as is now done in the ostree
sysroot upgrader.
This addresses that issue as well as tries to be a little cleaner in how
we clean up other transient state. Notably, we add a new helper function
to `RpmOstreeOrigin` to do this for us and use it in the upgrader. In
other cases, we do want this transient information since it allows us to
describe the deployment.
Closes: https://github.com/ostreedev/ostree/issues/1595Closes: #1372
Approved by: cgwalters
This makes the logs a bit more useful, but the ultimate goal
here is to write the originating client `id` to the cached update
data, so users know that e.g. `gnome-software` triggered it.
Closes: #1368
Approved by: jlebon
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.
I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.
This was all quite straightforward, just plumbing through lots
of layers.
Closes: #1321
Approved by: jlebon
The common case is having layered packages and no overrides; seeing
`Applying 8 overlays` then `Building filesystem [0/8]` is redundant.
Tweak the progress to avoid the double message. Also change the terminology
to clarify that each item is a package.
Also update libglnx to pull in progress bar newline fix.
Update submodule: libglnx
Closes: #1363
Approved by: jlebon
Follow-up from #1344.
In the case where a cached update is created from an `upgrade` operation
(and soon, "stage" auto-update policy runs), we can just print the diff
and advisory info together with the pending deployment. This makes the
output look much more natural.
Closes: #1350
Approved by: cgwalters
This now works after https://github.com/ostreedev/ostree/pull/1570
However, I was hitting an issue with the temp httpd running the
tests serially, let's split them up.
Really the only sane thing is to run each of these tests in a fresh
VM; we'll do the VM-in-container pattern here hopefully soon.
Closes: #1355
Approved by: jlebon
Now that infrastructure for this has landed in libostree,
let's make it easy for people to opt-in to testing it. This is a distinct first
step for adding it as an update policy.
Closes: #1352
Approved by: jlebon
Right now, cached updates generated during "check" policy runs are
completely decoupled from upgrade operations. This can lead
to the surprising situation where the "Available update" is *older* than
a freshly deployed pending tree with `rpm-ostree upgrade`.
We should just generate a cached update after upgrade operations. This
is also prep for staged deployments, where we'll want to do this as
well.
Note that we write out the cached update here even if automatic updates
are turned off since it's essentially free. I've been thinking about
always displaying that information after an `rpm-ostree upgrade` in
`status`. Though not sure if we should keep it in a separate "Available
update" section, or somehow morph it as part of the pending deployment
output.
Closes: #1344
Approved by: cgwalters
Previously we merged: #1228 AKA 12dc565b00
My recollection is that was working on it the background, while doing
something else, and I clearly didn't get to the point of testing it "for real".
There are many interlocking issues here to make this work. For example,
the "remove RPM" logic needs special handling for the kernel, because
we also inject content into `/usr/lib/ostree-boot` and also generate
the initramfs, etc.
The architecture I chose is to have the core *detect* when a kernel
is changed, and also call into the kernel processing code when removing
a kernel package. But the logic for doing kernel reinstallation client-side
is best alongside the initramfs generation logic which already existed
in the sysroot upgrader.
I extended the test suite to cover what was failing before, and I
tested this interactively. But I'm uncertain about adding a test
for actually *booting* into the GA kernel as it's quite possible
some bits in userspace rely on a newer kernel. Fixing this properly
really wants some infrastructure to better "re-version" an existing
package without changing its content.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1334Closes: #1346
Approved by: jlebon
Now we stop running rpm-ostreed as non-root, which is going to be
a maintenance pain going forward. If we do introduce non-VM based
tests I think we should look to doing in-container testing.
Closes: #1339
Approved by: cgwalters
Building on:
- 9cbec27d4c
- e7a42f70a9
I was looking at a rpm-ostree run that imports a variety of rpmmd-repos,
and information about the source repositories is really useful for determining
the up-to-dateness. We've been capturing this data for a while, it's
about time we started showing it somewhere.
This does make `status --verbose` notably more verbose, but eh, that
seems fine for now.
See also https://github.com/projectatomic/rpm-ostree/issues/774Closes: #1345
Approved by: jlebon
It's very normal for base packages to own directories with
dependent packages installing files there. Doing an rm-rf
for directories was just wrong. Concretely this fixes
an `override replace ./systemd-*.rpm`.
librpm is also pretty conservative here (for good reason)
and just ignores `ENOTEMPTY`, so let's match that.
I opted to split things up so we remove not-directories
in a first pass, then remove all directories we can in the
second. This should maximize our chances of removing what we can
in a scenario where e.g. two co-dependent packages install files
to a directory one of them owns.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1340Closes: #1341
Approved by: jlebon
Our test suite originated when package layering was still being
developed, but now that that's mature, the logic where layering
tests are distinct makes less sense.
The `basic` test had grown to really be a collection of many
miscellaneous things. Let's make that more explicit. Further,
let's avoid having each test suite grow too large; when a single
test fails we don't have an easy way to rerun just that test,
so a crude way to have faster local iteration is to split into groups.
My plan is to reintroduce a `basic` test that covers the basics
of all functionality - update, deploy, layering, etc. The advanced/corner
cases of layering like the `rm -rf /` test would still live in a
`test-layering.sh` or so.
Closes: #1336
Approved by: jlebon
The unit tests run an rpm-ostree daemon as non-root, which worked
surprisingly well for quite a while. But it started failing when
working on a patch which adds caching that writes to `/var`.
Since we have the vmcheck system now, let's switch over to that.
This PR moves the random "basic" tests we'd accumulated like
one to verify `StateRoot` is only in `status --verbose`, but not
the tests for the `rebase` command etc.
Closes: #1336
Approved by: jlebon
We should be checking that there are no requested base replacements
here, not removals. This was likely a copy-paste slip-up from copying
the override-remove tests. I still kept those checks for sanity, and
just tacked on the correct ones.
Closes: #1323
Approved by: cgwalters
This fixes a small regression from #852 which prevented inactive
overrides to be reset. Which is funny, because that's exactly the most
likely time when you would want to reset an override.
Basically, the `!is_layered --> no overrides` doesn't make sense for
inactive overrides. I suspect most people worked around this by just
using `reset --all`.
Closes: #1323
Approved by: cgwalters
I was trying a `--ex-unified-core` compose of FAW, and things fell over
on `urw-base35-fonts` which does a dance of setting a stamp file in
`%post` and checking it in `%posttrans`.
This whole pattern should be considered deprecated by file triggers. But let's
support it for now.
Note there's a lot of parameter passing as we need a single directory which is
held across multiple script invocations.
Closes: #1319
Approved by: jlebon
Specifically in this case, this allows us to close a race condition
during `upgrade --check` where the `CachedUpdate` property might not
have been updated yet when we read it after finishing the transaction.
Closes: #1311
Approved by: cgwalters
In this PR: https://github.com/projectatomic/rpm-ostree/pull/1309
I was hitting race conditions running `ostree admin pin` then
`rpm-ostree cleanup` as it was possible that the daemon hadn't handled
the inotify on the sysroot and reloaded the deployment state before
the txn request came in.
Close this race by doing an implicit `reload` before starting a txn.
This is a pretty efficient operation because for the sysroot we're
just doing a `stat()` and comparing mtime.
Implementation wise, change the external API to drop the "did change"
boolean as nothing outside of the `sysroot.c` file used it.
A followup to this would be changing the `status` CLI to call a
(new) DBus API like `RequestReload` that at least did the sysroot
reload if the daemon was otherwise idle or so? And it'd be available
to unprivileged users.
Closes: #1311
Approved by: cgwalters
Moving to caching the GVariant to disk rather than during RPMOSTreeOS
reloads broke the legacy `DownloadUpdateRpmDiff` path because it relied
on the implied recalculations that occur on reloads to update
`CachedUpdate`.
This patch series was initially going to be about just migrating all the
legacy APIs to make use of the new metadata, which would have fixed this
properly. But we first need some real coverage in that aread, which is
very poor right now. I'd like to investigate integrating the Cockpit
tests (at least the ostree-specific parts) into our CI to remedy this.
Anyways, for now at least, let's fix Cockpit.
Closes: #1300Closes: #1303
Approved by: cgwalters
This patch teaches `db list` to also use the pkglist metadata when it
can, just like we did for `db diff`. To spell it out: this then allows
`db list` to work on commits for which we only have the commit object.
I went for the surgical incision here and didn't try to support
invocations which use fnmatch patterns for now. Definitely possible,
though it didn't feel like it was worth the effort given that the common
case is just a raw `db list` (I'd wager most people are probably
hard-wired to pipe to `grep` anyway for filtering).
Also fix the usage string, which had the arguments flipped.
Closes: #1299
Approved by: cgwalters
Support a `--base` switch to perform the diff on the base layer if
deployment checksum was locally composed. This is useful to filter out
from the diff changes due to layered packages.
Closes: #1299
Approved by: cgwalters
This came up in a few places recently; it happens for RHEL in some
cases, and in general we don't want to completely fail the daemon
start if someone messes up their remote config.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1301Closes: #1302
Approved by: jlebon
This adds a shell primitive to make it easy to execute a playbook
task list.
The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.
Closes: #1297
Approved by: jlebon
This is a logical followup to:
- "core: Add rpmostree.repo metadata to imported packages" AKA a52cb7d78e
- "compose: Add rpmostree.rpmmd-repo metadata to commits by default" AKA e7a42f70a9
Basically I'd like to display this in `status` at least, though how we render it
gets...interesting when there is layering in play. For now though let's just
capture the data the same way we do server side.
Theoretically we could reconstruct this data by walking all of the pkgrefs in
the tree but...ugh. It's just a lot simpler to have it in this form too just
like the server-side path.
Closes: #1296
Approved by: jlebon
Very often when I have a pending deployment, I just want to find out
what the diff is before rebooting. We do print it after whatever
transaction laid down the new deployment, though we may not reboot right
away. It seems like a common enough operation to warrant some shortcuts.
This makes `db diff` more useful by automatically subbing in the booted
deployment if less arguments are given:
rpm-ostree db diff
<diff booted and pending deployment>
rpm-ostree db diff $csum
<diff booted and $csum>
rpm-ostree db diff $csum1 $csum2
<as before, diff $csum1 and $csum2>
As before, we never try to even load the sysroot in the last invocation,
so it still works equally well on non-OSTree systems/unprivileged.
Closes: #1294
Approved by: cgwalters
Implemented in libostree in https://github.com/ostreedev/ostree/pull/1464
Let's display it - wrapping the command will come later.
I also just noticed `rpmostree_syscore_filter_deployments()` at least is
going to have to learn about pinning; will need to improve the test suite
around this too.
Closes: #1292
Approved by: jlebon
We had no coverage for `--check` and `--preview`. Now that they use the
more efficient auto-updates API, it should be fine to document/recommend
to users. Let's add some coverage in the existing auto-updates tests to
make sure we don't regress on them e.g. only working when auto-update
"check" mode is on.
Closes: #1268
Approved by: cgwalters
Rather than recalculating `cached-update` as part of transaction
cleanups and RpmostreedOS internal reloads, write it directly to a file
from `deploy_transaction_execute`. This gives two major benefits:
1. Auto-updates now has virtually zero impact to daemon startup time.
2. We get to directly use the `DnfSack` created during metadata refresh
rather than reconstructing it later on. This greatly simplifies code.
This makes use of new APIs in libdnf to skip filelists and load
updateinfo metadata right from the start.
Closes: #1268
Approved by: cgwalters
This renames the remaining C files, tests, etc. There are only
a few hits for `jigdo` left; changing them would be a format break,
so let's wait to do that until we need to.
Closes: #1279
Approved by: jlebon
We're continuing an incremental renaming process; previously we changed
the most user-visible strings. Now we're doing some internal variables,
and notably the cached refs and the origin files - the latter set is
things that end up on disk.
This leaves the biggest items; renaming APIs, files, and tests.
Closes: #1276
Approved by: jlebon
Let's try to match expectations a bit from the dnf/yum world and
describe e.g. a `noarch` package become archful as an upgrade rather
than a removal and an addition. This was originally implicitly supported
(before PR #1230) by the fact that we didn't compare arches at all (and
in fact, arches don't even show up in a `db diff` output for modified
packages).
We bring this back here, but only in the simple case that it's a single
package. We still don't try to do any fancy handling for packages of the
same name.
Closes: #1272Closes: #1274
Approved by: cgwalters
This is an essential functionality rather than a nicety. Some
replacements can *only* be done without conflicts if we can remove
packages at the same time.
I do like that this has to be done explicitly, though OTOH, I can
definitely see folks wanting an `--allow-removals` type of switch in the
future.
Closes: #1255Closes: #1257
Approved by: cgwalters
Changes in a server-side tree can cause the need for clients to import different
objects from packages. For example, turning on documentation. Another more
subtle case is where an object might "move" from package A to B by being deleted
from A - then the jigdo build process will pick the B version.
We need a "cache validation key"; a way for the server to tell the client that
the objects it should import from the package have changed. Initially I was
thinking of using the libostree "content hash" but that would be awkward as we'd
have to do an import on the server side too.
After more consideration I realized a simple *count* of the number of objects
actually works, because (as I note in a comment) changing a file in the tree
will result in it ending up in the jigdoRPM (and count as a deletion). And
obviously adding or removing objects changes the count too.
In fact we could have done this *without* breaking the format by just having the
client start recording the number of xattr entries, but this adds greater
flexibility down the line since we can in theory change how we do cache
invalidation if we *really* need to (but at the cost of triggering clients to
redownload packages).
Note the client logic got moved around as now we need to parse all the xattrs
before we decide what packages to download.
My test case here is turning on docs - I noticed this actually affects *every*
package which was surprising to me; I expected at least some packages wouldn't
have docs. I'll double check this.
It'd be good to have a "moving object" case too which I may look at.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1197Closes: #1256
Approved by: jlebon
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`. However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.
This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo ♲📦:
https://github.com/projectatomic/rpm-ostree/issues/1081#issuecomment-348540604
Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.
(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.
Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.
Closes: #1240
Approved by: jlebon
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.
This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.
Closes: #1249
Approved by: cgwalters
Fix logic to make sure we check if the refspec is of type `ostree://`
even when it's explicitly specified. Also fix `Deploy` in the case where
we didn't just `Download` the RPM diff by adding a new @checksum
parameter to the higher-level API.
Finally, add a basic test for the `GetCached*RpmDiff` APIs so we have at
least *some* coverage. This is also good prep for making sure we don't
break anything when we convert those APIs to use the more efficient
pkglist metadata. The tests completely ignore the `DownloadRpmDiff`
paths for now though.
Closes: #1250Closes: #1253
Approved by: cgwalters
This fleshes out an important piece of the story, showing that
we can support history versioning the same way that we did with
ostree.
Also it's very useful for testing; I'm going to extend the suite after this to
deploy the previous version, clean everything up, then upgrade and verify we
only download changed RPMs.
Closes: #1232
Approved by: jlebon
The use case here is to run our tests against the shipped tree;
this could be used by a-h-t for example:
https://github.com/projectatomic/atomic-host-tests/issues/74
I tried this with just `TESTS=basic` and it failed for `usroverlay`. So we'd
have to start adding feature detection to the test suite to make this truly
useful, but let's at least start with the basic bits now to play with it.
Closes: #1251
Approved by: jlebon
The goal here is to support `override replace kernel.x86_64.rpm`.
There's a whole lot of logic in libdnf to support having multiple
kernel packages installed. AIUI, that was implemented because:
1) The kernel is the biggest source of regressions
2) It'd be quite noticeable if all of your kernel modules were deleted
Of course point 2) applies to a lot of userspace too...that's something
rpm-ostree fixes of course.
Anyways, in some testing all we need to do really is just turn that
logic off unconditionally.
Closes: https://github.com/projectatomic/rpm-ostree/issues/946Closes: #1228
Approved by: jlebon
While debugging the recent `BDB0087 DB_RUNRECOVERY` issues that cropped
up recently, I came upon the fact that we're leaving leftover rpmdb
files in the rootfs on client-side assemblies. Let's clean those up too.
We do this in the `commit` phase, *after* the final time we access the
rpmdb to generate the pkglist.
Closes: #1247
Approved by: cgwalters
The way we import packages in jigdo mode is different from package layering; we
may only import a subset of files for example. In general, we need to treat
jigdo differently.
Related: https://github.com/projectatomic/rpm-ostree/issues/1197Closes: #1238
Approved by: jlebon
I saw kalev's slides reference `rpm-ostree unlock`; this patch makes it exist.
In general, people have a hard time (understandably) grasping the distinction
between ostree and rpm-ostree; along with the goal of making ostree really
"libostree", let's start wrapping more commands where it makes sense.
I also took this opportunity to have a more descriptive name; it's important
to note that it *doesn't* overlay `/etc`, `/var`, or `/boot` for example.
Closes: #1233
Approved by: jlebon
Not all files from an RPM are necessarily removed during pkg erasure.
For example, files which are shared between pkgs shouldn't be deleted.
Similarly, not all files in an RPM are necessarily copied during pkg
installs. This is the case for multilib handling, which is a mess in its
own right. But such is the cost of trying to replace major parts of a
long-standing foundational project like RPM.
This patch adds some smarts to the way we do overlays and overrides to
handle these cases by calculating beforehand which files we *should't*
checkout/delete.
Closes: #1217Closes: #1145Closes: #1227
Approved by: cgwalters
Our complicated heuristics for handling multiple packages of the same
name comes back to bite us. In the multilib case, we can have packages
of the same NEVR, but different arch, sitting in the same tree.
Previously, even if the arch was different, we would still mark it as an
upgrade or downgrade. But that complicates things in the case of
multiple packages of the same name in the same tree.
We greatly simplify things here by making the diff algorithm dumber. We
now only consider a package as "modified" (i.e. upgraded/downgraded) if
it has the same NA (but different EVR). This makes handling multilib
cases natural and seems worth it overall vs trying to handle the odd
e.g. noarch <--> archful pkg transitions that could happen.
Closes: #1230
Approved by: cgwalters
This is an initial drop of support for:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`. We also
then support `rpm-ostree upgrade` from that.
There's a lot that could be improved here; the test coverage is relatively
minimal. A blocking issue there is having a realistic jigdo setup, and that's
going to require changing how we do testing. For now, this means that if we want
to e.g. change the format we'll have to temporarily disable this test, get the
format change in, update FAHC, then re-enable the test.
Closes: #1166
Approved by: jlebon
I'm working on supporting `override replace ./kernel-*.rpm`:
https://github.com/projectatomic/rpm-ostree/issues/946
But after battling the "installonly" logic in libdnf, I was confused why we
still had the files in `/usr/lib/modules`. It turned out to be because we only
remove files in `/usr`, but the code didn't handle UsrMove compat links.
There are a variety of approaches to fix this. Obviously a lot
of things get nicer in jigdo mode as we do UsrMove canonicalization
on import, and we could change this code to walk the imported pkg
ostree ref.
Another approach would be to walk the initial symlink, and check whether or not
it's the same as `/usr` (i.e. let the kernel do it).
For now though, what I chose to do was to scan the rootfs and find the UsrMove
compat links (i.e. we avoid hardcoding them again here). This is
fewer syscalls than the above and works well in practice.
Closes: #1226
Approved by: jlebon
This is what a lot of other tools do. It can get very verbose, with a
potentially huge amount of output if things are trickling in. This way
we're at least more friendly to someone running `cmd: rpm-ostree upgrade`
via Ansible or equivalent.
The slight hack here is that we *do* output `100%` on non-ttys to ensure we
print the result of the task.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1183Closes: #1225
Approved by: jlebon
This patch introduces a new `AutomaticUpdatePolicy` configuration. This
was a long time coming for rpm-ostree, given that its update model makes
it extremely apt for such a feature.
The config supports a `check` mode, which should be very useful to
Atomic Workstation users, as well as a `reboot` mode, which could be
used in its present form in simple single node Atomic Host situations.
There is still a lot of work to be done, including integrating
advisories, and supporting a `deploy` mode. This feature hopefully will
be leveraged as well by higher-level projects like GNOME Software and
Cockpit.
Closes: #1147
Approved by: cgwalters
It obviously shouldn't block the ability to install, and anyways
the single use in `ksh` is not compelling. If someone comes up with
one we can revisit supporting this.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1216Closes: #1218
Approved by: jlebon
For CI purposes at least, these features are already part of FAH and the
latest CAH smoketested. Anyway, we're already hacking the latest ostree
into place for `--keep-metadata`. If it already supports the latter,
then clearly it'll support those.
Closes: #1215
Approved by: cgwalters
I spent some time trying to figure out why on CentOS my changes to
`/etc/rpm-ostreed.conf` didn't seem to have any effect. It turns out
that for some reason I didn't spend too much time looking into, the
autoconf in CentOS defaults to `${prefix}/etc` when `--prefix` is given.
This is also why I was under the erroneous assumption that D-Bus config
files were in `/usr/etc` in CentOS. This patch fixes that as well.
Closes: #1215
Approved by: cgwalters
Let's standardize on the default config file when running tests. We copy
the original out of the way and install the default one so tests can do
whatever they want with it.
This also strengthens the post-test cleanup to make sure we rebase back
to the local vmcheck branch, in case we're somehow on a different branch
with the exact same commit.
Closes: #1212
Approved by: cgwalters
One issue right now is that repeated applications of `make vmoverlay`
will result in losing the source title.
Let's use the new `--keep-metadata` in subsequent `make vmoverlay`
invocations to make sure it gets transferred over correctly.
Closes: #1205
Approved by: cgwalters
This makes it much easier to know how old the base we're working from is
and whether it's time to upgrade before hacking some more.
Closes: #1205
Approved by: cgwalters
This is essentially a revert of #1190. I initially changed from storing
the epoch, version, and release separately to together to avoid endian
issues with epoch. This works around that instead by just encoding it as
a string.
As is done in our custom NEVRA printer (which before #1190 we used
here), we make sure that epochs of "0" are not printed as part of the
`nevra` or `evr` members.
Closes: #1198
Approved by: cgwalters
Basically we're choosing the libdnf behavior of omitting `Epoch: 0` which IMO is
the sane thing to do even though in *theory* they're different RPMs but
who would ever ship both without incrementing release/etc?
More information in: https://github.com/projectatomic/rpm-ostree/pull/187Closes: #1198
Approved by: cgwalters
This fixes a large swath of compatibility issues, for the same reasons as
overlayfs makes a lot of things Just Work. The ugly part of course is
doing hidden copyups inside the filesystem.
We've gone quite a long time with the "pure rofiles" mode, and have made changes
to various bits of userspace to be compatible with it. But what finally made me
give up on that is glibc's locale-archive; there's a patch for it that
is stalled, but even if it was applied we would still need to work with
older glibc.
This issue comes to the fore in unified core 🌐 mode, as without this
we won't get a correct locale archive.
Closes: #1171
Approved by: jlebon
This is the first step towards unifying how we introspect packages from
a specific commit. We currently do this in three ways: libdnf, librpm,
and now `rpmostree.rpmdb.pkglist`. I'd like to get to a point where we
only have `rpmostree.rpmdb.pkglist` and libdnf, the latter only when
more complex queries are required.
This patch teaches the `db diff` command to make use of the new db diff
API so that it can work even on metadata-only commits. This is relevant
for use cases mentioned in #558.
I didn't get rid of the `rpmhdrs_diff` functions right now because of
the `--changelogs` option: libdnf currently does not expose this, so we
fall back to the previous API in that case. OTOH, I wonder how much it's
actually used in the wild; maybe we could just nix it?
Closes: #1162
Approved by: cgwalters
The new `rpmdb.pkglist` metadata is a cheap way of retrieving the set of
packages in a commit. I'd like to make use of it as much as possible vs.
checking out the rpmdb and setting up a DnfSack.
Of course, in the case of layered commits, it doesn't matter *as* much,
because a layered commit being present in the repo should mean that a
deployment is currently using it, and we should learn to reuse the rpmdb
checkout of that deployment. Though keeping it consistent across both
server and client commits makes implementing `OstreeDeployment`-agnostic
things like `db diff` more efficient too. I also plan to use this in the
upcoming auto-update code.
Closes: #1158
Approved by: cgwalters
This rolls up several libglnx changes: https://github.com/GNOME/libglnx/pull/101
Now of course things are trickier here because we have an internal
abstraction over directly emitting to a console versus sending the
result over DBus. Further complicating things is that some things
call into libdnf and thus *require* use of `DnfState` which does
not give us the "n items" information, versus other parts which
we implement and can do what we want.
Even *further* complicating things is that we have to take care around non-CLI
callers like Cockpit; so I didn't try to pass the "n items" over DBus, rather
just reimplemented the "insert into text" that libglnx is doing.
Anyways overall this looks better IMO for all cases.
Update submodule: libglnx
Closes: #1143
Approved by: jlebon
This function is now basically reimplementing all of the intelligence
we have in libostree today. It wouldn't have worked before we
fixed https://github.com/ostreedev/ostree/issues/1165 but now
that's done, conceptually we just need to call
`ostree_repo_commit_modifier_set_sepolicy()`. However, I had
to drop the "number of files changed" since currently the libostree
API doesn't support that.
Also, in order to detect the case that content changes at all (so
we still have some useful information in the journal), implement
a "content hash" check.
See also https://github.com/projectatomic/rpm-ostree/pull/1123
(And note we *don't* use `DEVINO_CANONICAL` here)
Closes: #1138
Approved by: jlebon
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).
Closes: #1019
Approved by: jlebon
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).
Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).
There are two reasons to do this:
- We want "offline" updates by default; updates shouldn't affect the
running system. If we prepare the new root in the background, a
%post shouldn't restart a service for example. We already "handle"
this by making `systemctl` a symlink to `/bin/true`, but this approach
also shuts off `%post`s that do e.g. `insmod`.
- Protection against accidental system damage
Closes: #1099
Approved by: jlebon
We originally needed the pkgcache to be a separate repo due to ostree's
overzealous pruning policies. The idea was to maintain multiple commits
in each pkg branch for different SELinux policies. In practice, there's
not much use in maintaining old copies and it's just easier to always
relabel on the fly. So then, the need for a separate repo completely
melts away.
This helps simplify the mental model a bit and allows us to avoid subtle
issues like #1047. Note however that the core is still capable of
handling split repos for the `--ex-unified-core` compose use case. Once
that and the jigdo work are a bit more settled, we can have a clearer
picture of how to simplify the core further.
The tricky bit is migrating the cache. When deploying, we check if a
pkgcache repo exists and migrate its refs if so. We then leave behind a
symlink to the system repo to remain compatible with older rpm-ostrees.
Closes: #1055
Approved by: cgwalters
It took me way, way, way too long to debug that my dev container somehow missing
`ostree-grub2` caused the script to fail early, but we'd still continue trying
to sync, leading to us not actually changing the installed `rpm-ostree` binary,
leading to test suite failures.
Also add the chmod hammer at the top so we can delete even if we fail after
doing an rsync.
Closes: #1115
Approved by: jlebon
We have a *lot* of experimental functionality. I think the
`override` bits are fleshed out enough now that we can lift
the `ex` designation. For example, jlebon fixed SELinux
labeling in the presence of override-replace.
Closes: #1089
Approved by: jlebon
Refined a bit for the previous written kargs test.( mainly for
checking kargs after rollback )
Added tests for import-proc-cmdline and deploy-index option
Closes: #1013
Approved by: cgwalters
Refactor the similar code portion from delete option into
a new function, and reuse it in the replace logic.
Add the replace functionality to allow kernel arguments
replacement. Now support two different types of replacement:
1: replace by key=value when there is only one key existing
e.g we have 'hi=hello' as our one of the arguments (hi only has
one value associated with it) , and 'rpm-ostree ex kargs --replace hi=new'
will replace the value and change hi=hello into hi=new.
2: replace by key=value=new_value for all other replacements
( this will work for swapping single value pair too !)
Some tests for rpm-ostree ex kargs --replace are added
for future regression.
Closes: #1013
Approved by: cgwalters
Tests are introduced in this commit to test the basic
functionality for rpm-ostree ex kargs --append,
and rpm-ostree ex kargs --delete.
Those tests are added for future regression.
Closes: #1013
Approved by: cgwalters
Related to: https://github.com/projectatomic/rpm-ostree/issues/774
We aren't yet trying to render this in any way on the client side, but let's
start capturing the timestamp data now so that we can make use of it later
in e.g. `rpm-ostree status -v`.
Closes: #1072
Approved by: jlebon
Depends: https://github.com/ostreedev/ostree/pull/1296
As I mention in the commit there, I see two uses for this in rpm-ostree; first
in our test suite, and second for OCI-built image imports.
I also took a step further here and inject an `original-origin` metadata
key, though we aren't actually using that yet. The problem I'm trying
to solve there is that repeated `make vmoverlay` starts chaining things up,
but that gets very confusing. I think we should always have `vmoverlay` unwind
back to the base ref. (Or at least do that by default)
Closes: #1069
Approved by: jlebon
In another PR I did the manual bridging of commit metadata to deployment
property, but that's annoying. Let's just bridge all commit metadata.
Closes: #1069
Approved by: jlebon
To complement the new `--cache-only` option, add a `--download-only`
option. This does exactly what it says: we download the ostree, download
and import packages, but don't actually commit & deploy. This can be
used to effectively prime a follow-up `--cache-only` operation that can
be done during a more convenient/safer maintenance window.
I debated naming the two options `--pull-only` and `--deploy-only` like
the ostree equivalents. Though "pull" felt like the wrong word given
that it's associated more with ostree pulling but rpm-ostree also
downloads & imports RPMs. As for `--deploy-only` vs `--cache-only`, it
seems like `--cache-only` is a more accurate description of the
functionality (i.e. rather than describing an action, it describes a
mode). I also considered `--no-download` to make the synergy with
`--download-only` more obvious. Maybe that's better? Naming is hard...
Closes: #713Closes: #1049
Approved by: cgwalters
Now that we have a strong notion of `cache-only` mode, make use of it
when performing an `uninstall` or `ex override remove/reset`.
Closes: #944Closes: #1049
Approved by: cgwalters
As Colin mentioned in #1035, the new `--cache-only` implemented only the
rpmmd half of the story. Here we complete that story by also ensuring
that when in cache-only mode, we don't download new ostree data nor new
packages. We try to complete the requested operation with what we have.
To do this, we add support for the same `SYNTHETIC` pull that was added
in ostree[1] so that we don't actually pull, but still perform timestamp
checking.
On the pkgcache side, we disable all remote repos and instead insert all
our cached RPMs into the `DnfSack`. Care is taken to still perform
SHA256 verification for local pkg installs/replacements.
[1] https://github.com/ostreedev/ostree/pull/642Closes: #687Closes: #1049
Approved by: cgwalters
The new idle exit behaviour is nice, but it makes debugging it harder
because you have to be fast enough to attach or place your breakpoints
and trigger it before it auto-exits. Add a compile-time flag that
developers can easily turn on to disable the auto-exit behaviour.
Closes: #1052
Approved by: cgwalters
Just taking what I learned from #1035 and applying it here. What's nice
about this is that there's no cleanup needed. Once the process is killed
(or worst case, we reboot the VM), there's no traces left at all.
Also added a few extra "ok" outputs.
Closes: #1043
Approved by: cgwalters
I've been lazy about actually using using rsync instead of scp when
copying new RPMs over to the VM. We do this here. Also make
`vm_send_test_repo` take a mode argument that allows callers to
completely skip the sending of the repo file itself. This will be needed
for the `makecache` test, in which we *don't* want the repo to be local.
It looks cleaner anyway for the gpgcheck use case as well.
Closes: #1035
Approved by: cgwalters
Prep for implementing `rpm-ostree cancel`, but this works with the way we handle
`Ctrl-C` interactively on a client as well. Being able to cancel a script
execution is quite nice; some of them are expensive, and having one loop forever
has been known to happen.
Closes: #1025
Approved by: jlebon
There's a lot of paths in the core related to SELinux policy changes and
relabeling packages. We currently have no test coverage for them. We add
support in the test libraries here to build such packages.
We also add a test that checks both that we correctly relabel RPMs when
the policy changes and that we handle layered packages that install
SELinux packages properly.
Closes: #999
Approved by: cgwalters
Rather than just letting the scriptlets inherit the daemon's
stdout/stderr, redirect their outputs so that we can set a customized
identifier to make it easier to distinguish from the daemon output.
Also print out the `journalctl` command needed so that users can
investigate the output themselves.
Closes: #998
Approved by: cgwalters
I suspect a common pattern with local replacement overrides is to
simultaneously replace a group of packages that depend on each other in
one shot, as is the case with docker, docker-common, and
docker-rhel-push-plugin currently in Fedora Atomic Host. In such cases,
we can print a cleaner diff in the status to make it easier to grok.
Before:
ReplacedBasePackages: strace 4.18-1.fc26 -> 4.19-1.fc26, docker-common 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, docker 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, docker-rhel-push-plugin 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26
After:
ReplacedBasePackages: docker-common docker docker-rhel-push-plugin 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, strace 4.18-1.fc26 -> 4.19-1.fc26
Closes: #1004
Approved by: cgwalters
We were directly bind mounting the checked out `/usr/etc` onto `/etc`
which was exposing us to corruption from scriptlets. Since we already
have an rofiles-fuse mount for `/usr`, let's just re-use its `etc/`
subdir and bind mount that instead.
Closes: #1003
Approved by: cgwalters
We don't need those in the tree, so let's nuke them. This also fixes
subtle compatibility issues between hardlinks and lock files (see #999).
Closes: #1002
Approved by: cgwalters
This PR uses https://github.com/ostreedev/ostree/pull/1156
to ensure that when installing packages containing files with
exact same content, the files will be merged.
When installing packages containing conflicting files,
the error will still be detected at the ostree side.
The checkout overwrite option at rpm-ostree side is also
modified to accomodate the changes made in ostree side.
A test is added for regression
Commands like `upgrade` and `deploy` need to know if a new deployment
was actually laid down so that it may print a pkg diff if so. This is
implemented by listening for changes to the DefaultDeployment D-Bus
property. D-Bus emits a signal when the deployment variant changes
value.
However, in #595, with the introduction of `pending-*` related keys, the
deployment variant no longer represents data solely tied to that
specific deployment. In this case, because `deploy` operations currently
set the ref to the resolved checksum, it can happen that deploying the
same base commit when the current refspec *isn't* pointing to that base
commit will result in the `pending-*` keys dropping out and a default
deployment change notification going out.
In this patch, we strengthen how we determine whether a new deployment
was laid down by actually looking at the deployment id, rather than just
assuming that a change to the property implies a new deployment.
Closes: #981Closes: #984
Approved by: cgwalters
Also something I noticed while working on #981. When sitting on a livefs
commit, once a user does `rpm-ostree cleanup --pending --rollback`, it's
impossible to redeploy the same booted commit. Let's allow users to do
this.
Closes: #984
Approved by: cgwalters
Today in Fedora the `glibc-all-langpacks.posttrans` is implemented
in lua, for no good reason. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1367585
Since that's stalled out, let's add support for overrides. This
is obviously a much bigger step with more long term maintenance
implications over our current "ignore scripts" list. But we can't
block either.
This is needed for unified core work:
https://github.com/projectatomic/rpm-ostree/issues/729
(We also override `fedora-release-atomichost` but I'll likely
submit a patch for that upstream)
Closes: #980
Approved by: jlebon
I was linking to this code from elsewhere and noticed that
for our hardlink breaks we were not using fd-relative even
though we can. Down the line if we fork librpm into a separate
process and do e.g. `--dbpath=.` it'll do it too.
(Side note, I verified that commenting out the hardlink breaking
here was caught by the `ostree fsck` I added to the test suite)
Closes: #979
Approved by: jlebon
This fixes `rpm-ostree reload` as root, and supports configuring
it to be enabled for other users as well. This was overlooked
in the polkit work originally.
Closes: https://github.com/projectatomic/rpm-ostree/issues/976Closes: #977
Approved by: jlebon
Let's make SELinux and atomic-host-tests happy with trees we concoct
ourselves by using the new --selinux-policy. (Specifically, we want our
sync'ed binaries to have install_exec_t).
Closes: #953
Approved by: cgwalters
I had meant for this to be in the other PR#968. I originally did both
`make` and `make install` there, but now it only does `make install`, so
let's just rename it to make that more obvious.
Closes: #953
Approved by: cgwalters
At one point `rpm-ostree install libvirt` dragged in libguestfs which in turn
brought in `syslinux-extlinux-nonlinux` which has files in `/boot/extlinux`,
which we rejected. (That dependency chain appears to have been fixed currently)
For the general case, this is just a partial fix in that we haven't nailed down
the semantics of how updates for `/boot` work. But in this particular case,
we'll just break libguestfs' `extlinux` verb, which I'm OK with.
Another case is `fwupdate-efi` - we require manual intervention to copy the
data into `/boot` after installing the package.
This is also preparation for [unified core](https://github.com/projectatomic/rpm-ostree/issues/729)
in that we now ensure imported kernels don't end up in `/boot` unless
explicitly configured.
Closes: https://github.com/projectatomic/rpm-ostree/issues/853Closes: #969
Approved by: jlebon
