IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This adds a shell primitive to make it easy to execute a playbook
task list.
The big picture idea is to sync with https://github.com/ostreedev/ostree/pull/1462
and rewrite some of the libvm shell stuff as playbooks, allowing easier
code sharing with a-h-t and just in general being a better library for
talking ssh and executing commnads.
Closes: #1297
Approved by: jlebon
This is a logical followup to:
- "core: Add rpmostree.repo metadata to imported packages" AKA a52cb7d78e
- "compose: Add rpmostree.rpmmd-repo metadata to commits by default" AKA e7a42f70a9
Basically I'd like to display this in `status` at least, though how we render it
gets...interesting when there is layering in play. For now though let's just
capture the data the same way we do server side.
Theoretically we could reconstruct this data by walking all of the pkgrefs in
the tree but...ugh. It's just a lot simpler to have it in this form too just
like the server-side path.
Closes: #1296
Approved by: jlebon
Very often when I have a pending deployment, I just want to find out
what the diff is before rebooting. We do print it after whatever
transaction laid down the new deployment, though we may not reboot right
away. It seems like a common enough operation to warrant some shortcuts.
This makes `db diff` more useful by automatically subbing in the booted
deployment if less arguments are given:
rpm-ostree db diff
<diff booted and pending deployment>
rpm-ostree db diff $csum
<diff booted and $csum>
rpm-ostree db diff $csum1 $csum2
<as before, diff $csum1 and $csum2>
As before, we never try to even load the sysroot in the last invocation,
so it still works equally well on non-OSTree systems/unprivileged.
Closes: #1294
Approved by: cgwalters
Implemented in libostree in https://github.com/ostreedev/ostree/pull/1464
Let's display it - wrapping the command will come later.
I also just noticed `rpmostree_syscore_filter_deployments()` at least is
going to have to learn about pinning; will need to improve the test suite
around this too.
Closes: #1292
Approved by: jlebon
We had no coverage for `--check` and `--preview`. Now that they use the
more efficient auto-updates API, it should be fine to document/recommend
to users. Let's add some coverage in the existing auto-updates tests to
make sure we don't regress on them e.g. only working when auto-update
"check" mode is on.
Closes: #1268
Approved by: cgwalters
Rather than recalculating `cached-update` as part of transaction
cleanups and RpmostreedOS internal reloads, write it directly to a file
from `deploy_transaction_execute`. This gives two major benefits:
1. Auto-updates now has virtually zero impact to daemon startup time.
2. We get to directly use the `DnfSack` created during metadata refresh
rather than reconstructing it later on. This greatly simplifies code.
This makes use of new APIs in libdnf to skip filelists and load
updateinfo metadata right from the start.
Closes: #1268
Approved by: cgwalters
We would error out when trying to start the transient httpd service if
it already exists, e.g. from a previous test.
Depending on how we exit, the `vm_stop_httpd` trap for the previous test
might not have been able to kick in. I think this happens when we exit
using `fatal`, which just does an `exit 1`. It's not strictly an error,
so doesn't trip the `ERR` handler.
Let's just go the extra mile and explicitly delete transient services if
they already exist.
Closes: #1284
Approved by: cgwalters
This renames the remaining C files, tests, etc. There are only
a few hits for `jigdo` left; changing them would be a format break,
so let's wait to do that until we need to.
Closes: #1279
Approved by: jlebon
We're continuing an incremental renaming process; previously we changed
the most user-visible strings. Now we're doing some internal variables,
and notably the cached refs and the origin files - the latter set is
things that end up on disk.
This leaves the biggest items; renaming APIs, files, and tests.
Closes: #1276
Approved by: jlebon
Let's try to match expectations a bit from the dnf/yum world and
describe e.g. a `noarch` package become archful as an upgrade rather
than a removal and an addition. This was originally implicitly supported
(before PR #1230) by the fact that we didn't compare arches at all (and
in fact, arches don't even show up in a `db diff` output for modified
packages).
We bring this back here, but only in the simple case that it's a single
package. We still don't try to do any fancy handling for packages of the
same name.
Closes: #1272Closes: #1274
Approved by: cgwalters
I noticed that Ubuntu also uses the original "jigdo", so let's start
pulling off the band-aid here and do a mass rename.
For this first pass I'm focusing on CLI entrypoints and docs, as that's what
people are going to see; renaming all of the internal C functions, structure
variables etc. can come later.
Closes: #1269
Approved by: jlebon
This is an essential functionality rather than a nicety. Some
replacements can *only* be done without conflicts if we can remove
packages at the same time.
I do like that this has to be done explicitly, though OTOH, I can
definitely see folks wanting an `--allow-removals` type of switch in the
future.
Closes: #1255Closes: #1257
Approved by: cgwalters
Changes in a server-side tree can cause the need for clients to import different
objects from packages. For example, turning on documentation. Another more
subtle case is where an object might "move" from package A to B by being deleted
from A - then the jigdo build process will pick the B version.
We need a "cache validation key"; a way for the server to tell the client that
the objects it should import from the package have changed. Initially I was
thinking of using the libostree "content hash" but that would be awkward as we'd
have to do an import on the server side too.
After more consideration I realized a simple *count* of the number of objects
actually works, because (as I note in a comment) changing a file in the tree
will result in it ending up in the jigdoRPM (and count as a deletion). And
obviously adding or removing objects changes the count too.
In fact we could have done this *without* breaking the format by just having the
client start recording the number of xattr entries, but this adds greater
flexibility down the line since we can in theory change how we do cache
invalidation if we *really* need to (but at the cost of triggering clients to
redownload packages).
Note the client logic got moved around as now we need to parse all the xattrs
before we decide what packages to download.
My test case here is turning on docs - I noticed this actually affects *every*
package which was surprising to me; I expected at least some packages wouldn't
have docs. I'll double check this.
It'd be good to have a "moving object" case too which I may look at.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1197Closes: #1256
Approved by: jlebon
Since we changed things to have `jigdoSet = pkgSet`, we can just require exactly
`${npkgs}` here on import, which is what we found from `db list`.
Closes: #1256
Approved by: jlebon
What's happened up till now is supporting `rojig://` in the same way as
`ostree://`. However, part of the high level goal here is to reduce
the need for system administrators to understand ostree.
This patch set starts to introduce some of the ideas for client-side
changes as part of jigdo ♲📦:
https://github.com/projectatomic/rpm-ostree/issues/1081#issuecomment-348540604
Concretely, we start using `${repo}:${nevra}` instead of `rojig://`.
(v2): Keep `Version` (plus timestamp) as a split out field for maximum visual aid.
Also, let's be opinionated here and entirely drop the `Commit` checksum by
default. I believe the Cockpit guys were right here - versions are for humans.
The fact that we have a checksum is powerful; and we still show it with `status
-v`. The way I think of it is: the checksum shows we're really an image system.
But we don't need to show it by default.
Closes: #1240
Approved by: jlebon
Pick up security advisories when checking for pending updates and
include them in the `cached-update` property. On the client-side,
display them in the output of `status`.
This was part of the original vision for how useful a smart `check` mode
could be. It directly impacts how one manages their individual system
(e.g. when to reboot), and paves the way for integration into
higher-level apps that act at the cluster level.
Closes: #1249
Approved by: cgwalters
Fix logic to make sure we check if the refspec is of type `ostree://`
even when it's explicitly specified. Also fix `Deploy` in the case where
we didn't just `Download` the RPM diff by adding a new @checksum
parameter to the higher-level API.
Finally, add a basic test for the `GetCached*RpmDiff` APIs so we have at
least *some* coverage. This is also good prep for making sure we don't
break anything when we convert those APIs to use the more efficient
pkglist metadata. The tests completely ignore the `DownloadRpmDiff`
paths for now though.
Closes: #1250Closes: #1253
Approved by: cgwalters
This fleshes out an important piece of the story, showing that
we can support history versioning the same way that we did with
ostree.
Also it's very useful for testing; I'm going to extend the suite after this to
deploy the previous version, clean everything up, then upgrade and verify we
only download changed RPMs.
Closes: #1232
Approved by: jlebon
The use case here is to run our tests against the shipped tree;
this could be used by a-h-t for example:
https://github.com/projectatomic/atomic-host-tests/issues/74
I tried this with just `TESTS=basic` and it failed for `usroverlay`. So we'd
have to start adding feature detection to the test suite to make this truly
useful, but let's at least start with the basic bits now to play with it.
Closes: #1251
Approved by: jlebon
The goal here is to support `override replace kernel.x86_64.rpm`.
There's a whole lot of logic in libdnf to support having multiple
kernel packages installed. AIUI, that was implemented because:
1) The kernel is the biggest source of regressions
2) It'd be quite noticeable if all of your kernel modules were deleted
Of course point 2) applies to a lot of userspace too...that's something
rpm-ostree fixes of course.
Anyways, in some testing all we need to do really is just turn that
logic off unconditionally.
Closes: https://github.com/projectatomic/rpm-ostree/issues/946Closes: #1228
Approved by: jlebon
While debugging the recent `BDB0087 DB_RUNRECOVERY` issues that cropped
up recently, I came upon the fact that we're leaving leftover rpmdb
files in the rootfs on client-side assemblies. Let's clean those up too.
We do this in the `commit` phase, *after* the final time we access the
rpmdb to generate the pkglist.
Closes: #1247
Approved by: cgwalters
The way we import packages in jigdo mode is different from package layering; we
may only import a subset of files for example. In general, we need to treat
jigdo differently.
Related: https://github.com/projectatomic/rpm-ostree/issues/1197Closes: #1238
Approved by: jlebon
I *think* this is why our tests started failing recently. It seems somehow very
unlikely to me though that we'd somehow managed to avoid `f` in the boot
checksums until now, but without doing some math...it seems plausible.
Closes: #1243
Approved by: jlebon
I saw kalev's slides reference `rpm-ostree unlock`; this patch makes it exist.
In general, people have a hard time (understandably) grasping the distinction
between ostree and rpm-ostree; along with the goal of making ostree really
"libostree", let's start wrapping more commands where it makes sense.
I also took this opportunity to have a more descriptive name; it's important
to note that it *doesn't* overlay `/etc`, `/var`, or `/boot` for example.
Closes: #1233
Approved by: jlebon
Not all files from an RPM are necessarily removed during pkg erasure.
For example, files which are shared between pkgs shouldn't be deleted.
Similarly, not all files in an RPM are necessarily copied during pkg
installs. This is the case for multilib handling, which is a mess in its
own right. But such is the cost of trying to replace major parts of a
long-standing foundational project like RPM.
This patch adds some smarts to the way we do overlays and overrides to
handle these cases by calculating beforehand which files we *should't*
checkout/delete.
Closes: #1217Closes: #1145Closes: #1227
Approved by: cgwalters
Our complicated heuristics for handling multiple packages of the same
name comes back to bite us. In the multilib case, we can have packages
of the same NEVR, but different arch, sitting in the same tree.
Previously, even if the arch was different, we would still mark it as an
upgrade or downgrade. But that complicates things in the case of
multiple packages of the same name in the same tree.
We greatly simplify things here by making the diff algorithm dumber. We
now only consider a package as "modified" (i.e. upgraded/downgraded) if
it has the same NA (but different EVR). This makes handling multilib
cases natural and seems worth it overall vs trying to handle the odd
e.g. noarch <--> archful pkg transitions that could happen.
Closes: #1230
Approved by: cgwalters
This is an initial drop of support for:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`. We also
then support `rpm-ostree upgrade` from that.
There's a lot that could be improved here; the test coverage is relatively
minimal. A blocking issue there is having a realistic jigdo setup, and that's
going to require changing how we do testing. For now, this means that if we want
to e.g. change the format we'll have to temporarily disable this test, get the
format change in, update FAHC, then re-enable the test.
Closes: #1166
Approved by: jlebon
I'm working on supporting `override replace ./kernel-*.rpm`:
https://github.com/projectatomic/rpm-ostree/issues/946
But after battling the "installonly" logic in libdnf, I was confused why we
still had the files in `/usr/lib/modules`. It turned out to be because we only
remove files in `/usr`, but the code didn't handle UsrMove compat links.
There are a variety of approaches to fix this. Obviously a lot
of things get nicer in jigdo mode as we do UsrMove canonicalization
on import, and we could change this code to walk the imported pkg
ostree ref.
Another approach would be to walk the initial symlink, and check whether or not
it's the same as `/usr` (i.e. let the kernel do it).
For now though, what I chose to do was to scan the rootfs and find the UsrMove
compat links (i.e. we avoid hardcoding them again here). This is
fewer syscalls than the above and works well in practice.
Closes: #1226
Approved by: jlebon
This is what a lot of other tools do. It can get very verbose, with a
potentially huge amount of output if things are trickling in. This way
we're at least more friendly to someone running `cmd: rpm-ostree upgrade`
via Ansible or equivalent.
The slight hack here is that we *do* output `100%` on non-ttys to ensure we
print the result of the task.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1183Closes: #1225
Approved by: jlebon
This patch introduces a new `AutomaticUpdatePolicy` configuration. This
was a long time coming for rpm-ostree, given that its update model makes
it extremely apt for such a feature.
The config supports a `check` mode, which should be very useful to
Atomic Workstation users, as well as a `reboot` mode, which could be
used in its present form in simple single node Atomic Host situations.
There is still a lot of work to be done, including integrating
advisories, and supporting a `deploy` mode. This feature hopefully will
be leveraged as well by higher-level projects like GNOME Software and
Cockpit.
Closes: #1147
Approved by: cgwalters
In #875 AKA b46fc35901 we
added support for the `releasever` option in treefiles. I am
pretty sure it worked at the time...but I didn't add tests.
Either it never worked or some refactoring broke it. The whole chain of
`GKeyFile` → `GVariant` is so confusing. Anyways fix it by copying the string.
Now let's use it by default in the compose tests, and while we're here bump
those to F27.
I'm doing this patch now as I was playing with doing a compose from
the `/usr/share/rpm-ostree/treefile.json` and wanted to use the stock
`.repo` files.
Closes: #1220
Approved by: jlebon
It obviously shouldn't block the ability to install, and anyways
the single use in `ksh` is not compelling. If someone comes up with
one we can revisit supporting this.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1216Closes: #1218
Approved by: jlebon
When I tried to use my WIP client patches to do:
`rpm-ostree rebase rojig://fahc:fedora-atomic-host`,
I got a missing file object which turned out to
be the client importing the i686 RPMs.
This was passing in the test suite because we don't mirror i686 of course, but
on the client side right now we end up using all enabled repos, and since Fedora
is multiarch, the behavior is going to be...not predictable.
Thinking a bit about on this problem I actually happened to recall
the RPM `%{_isa}` macro which is used in Fedora in various places;
for example to "arch bind" `-devel` packages to their base. See
for example [this case](33c7dc02bc/f/ostree.spec (_79)) in libostree.
As I noted at first, the core problem here is that the "final"
RPM architecture field is not symmetric in any way with the definition
of that `%{_isa}` macro. See:
d9d47e0114/installplatform (L25)
The *third* solution I ended up on here is to iterate over the
`Provides` on the server side and we take the first thing
that matches `Provides: %{name}(whatever)`.
I briefly thought about trying to somehow drive into libsolv the
logic to prefer the jigdoRPM's native architecture...IIRC yum did
something like that in the past but it was never done in libsolv?
Anyways the dependencies here are now more correct, so other tools
will also handle it.
Closes: #1213
Approved by: jlebon
For CI purposes at least, these features are already part of FAH and the
latest CAH smoketested. Anyway, we're already hacking the latest ostree
into place for `--keep-metadata`. If it already supports the latter,
then clearly it'll support those.
Closes: #1215
Approved by: cgwalters
This reverts commit 1ef259ed76.
Basically, for historical POSIX compatibility, `errexit` doesn't work at
all inside functions called from an if-statement/boolean context. This
is something I had already learned (and forgotten) when hacking on PAPR.
There are ways around this, but it's just not worth the added complex
shell goop for what it gives us.
http://mywiki.wooledge.org/BashFAQ/105Closes: #1215
Approved by: cgwalters
I spent some time trying to figure out why on CentOS my changes to
`/etc/rpm-ostreed.conf` didn't seem to have any effect. It turns out
that for some reason I didn't spend too much time looking into, the
autoconf in CentOS defaults to `${prefix}/etc` when `--prefix` is given.
This is also why I was under the erroneous assumption that D-Bus config
files were in `/usr/etc` in CentOS. This patch fixes that as well.
Closes: #1215
Approved by: cgwalters
Let's standardize on the default config file when running tests. We copy
the original out of the way and install the default one so tests can do
whatever they want with it.
This also strengthens the post-test cleanup to make sure we rebase back
to the local vmcheck branch, in case we're somehow on a different branch
with the exact same commit.
Closes: #1212
Approved by: cgwalters
The output from `rpmbuild` makes test results harder to comb through
when debugging. Let's just dump all that to file and only output it if
something goes wrong.
Closes: #1212
Approved by: cgwalters
This fixes an asymmetry we introduced when rendering using the `ostree://`
prefix; we now support parsing it as well. We reject the `rojig://` prefix
explicitly for now.
On the implementation side...things are messy. My thought is that we do our best
to canonicalize at the entrypoints for both the client side as well as the
server side. However, for backwards compatibility we can't go all the way to
writing out `ostree://` to disk in the origin files.
I'm actually uncertain if Cockpit will deal with this...need to test that.
Closes: #1210
Approved by: jlebon
One issue right now is that repeated applications of `make vmoverlay`
will result in losing the source title.
Let's use the new `--keep-metadata` in subsequent `make vmoverlay`
invocations to make sure it gets transferred over correctly.
Closes: #1205
Approved by: cgwalters
This makes it much easier to know how old the base we're working from is
and whether it's time to upgrade before hacking some more.
Closes: #1205
Approved by: cgwalters
This is essentially a revert of #1190. I initially changed from storing
the epoch, version, and release separately to together to avoid endian
issues with epoch. This works around that instead by just encoding it as
a string.
As is done in our custom NEVRA printer (which before #1190 we used
here), we make sure that epochs of "0" are not printed as part of the
`nevra` or `evr` members.
Closes: #1198
Approved by: cgwalters
Basically we're choosing the libdnf behavior of omitting `Epoch: 0` which IMO is
the sane thing to do even though in *theory* they're different RPMs but
who would ever ship both without incrementing release/etc?
More information in: https://github.com/projectatomic/rpm-ostree/pull/187Closes: #1198
Approved by: cgwalters
