IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This reverts commit 1ef259ed76.
Basically, for historical POSIX compatibility, `errexit` doesn't work at
all inside functions called from an if-statement/boolean context. This
is something I had already learned (and forgotten) when hacking on PAPR.
There are ways around this, but it's just not worth the added complex
shell goop for what it gives us.
http://mywiki.wooledge.org/BashFAQ/105Closes: #1215
Approved by: cgwalters
I spent some time trying to figure out why on CentOS my changes to
`/etc/rpm-ostreed.conf` didn't seem to have any effect. It turns out
that for some reason I didn't spend too much time looking into, the
autoconf in CentOS defaults to `${prefix}/etc` when `--prefix` is given.
This is also why I was under the erroneous assumption that D-Bus config
files were in `/usr/etc` in CentOS. This patch fixes that as well.
Closes: #1215
Approved by: cgwalters
Let's standardize on the default config file when running tests. We copy
the original out of the way and install the default one so tests can do
whatever they want with it.
This also strengthens the post-test cleanup to make sure we rebase back
to the local vmcheck branch, in case we're somehow on a different branch
with the exact same commit.
Closes: #1212
Approved by: cgwalters
The output from `rpmbuild` makes test results harder to comb through
when debugging. Let's just dump all that to file and only output it if
something goes wrong.
Closes: #1212
Approved by: cgwalters
This fixes an asymmetry we introduced when rendering using the `ostree://`
prefix; we now support parsing it as well. We reject the `rojig://` prefix
explicitly for now.
On the implementation side...things are messy. My thought is that we do our best
to canonicalize at the entrypoints for both the client side as well as the
server side. However, for backwards compatibility we can't go all the way to
writing out `ostree://` to disk in the origin files.
I'm actually uncertain if Cockpit will deal with this...need to test that.
Closes: #1210
Approved by: jlebon
One issue right now is that repeated applications of `make vmoverlay`
will result in losing the source title.
Let's use the new `--keep-metadata` in subsequent `make vmoverlay`
invocations to make sure it gets transferred over correctly.
Closes: #1205
Approved by: cgwalters
This makes it much easier to know how old the base we're working from is
and whether it's time to upgrade before hacking some more.
Closes: #1205
Approved by: cgwalters
This is essentially a revert of #1190. I initially changed from storing
the epoch, version, and release separately to together to avoid endian
issues with epoch. This works around that instead by just encoding it as
a string.
As is done in our custom NEVRA printer (which before #1190 we used
here), we make sure that epochs of "0" are not printed as part of the
`nevra` or `evr` members.
Closes: #1198
Approved by: cgwalters
Basically we're choosing the libdnf behavior of omitting `Epoch: 0` which IMO is
the sane thing to do even though in *theory* they're different RPMs but
who would ever ship both without incrementing release/etc?
More information in: https://github.com/projectatomic/rpm-ostree/pull/187Closes: #1198
Approved by: cgwalters
`grep` supports checking multiple fixed strings separated by newlines,
but it's mostly just easier to pass them as separate arguments, so let's
support that. This is now at parity with the similar
`assert_file_has_content`.
Will upstream this to ostree as well once reviewed.
Closes: #1200
Approved by: cgwalters
Following up on `--ex-jigdo-output-rpm`, add support for writing the entire set
to an output directory. This is intended for use cases like FAHC, where we're
generating data outside of the upstream Fedora infrastructure. Further, we want
to support having our own history stored reliably, even if upstream prunes RPMs.
Now, this can be interesting even for upstreams like Fedora, as it naturally
captures just the subset of RPMs; doing full history support for that would
likely be a lot more palatable than for Everything.
Closes: #1165
Approved by: jlebon
Part of the goal of jigdo ♲📦 is to support organizations switching to *only*
providing RPMs. An intermediate step there is to "lock" the repo and jigdo
together; we don't want to update the ref if building the jigdoRPM fails.
Add an option to perform `rpm-ostree compose tree` and `rpm-ostree ex
commit2jigdo` together; notably we generate a commit, but only update the ref
once the jigdoRPM is built.
Closes: #1165
Approved by: jlebon
- Actually use separate `${test_tmpdir}` for test setup (closes a race)
- Merge stdout/stderr (more readable)
- Ensure logs are renamed to `.txt` even on failure
- Use `--progress` for some feedback
- Use `-j +1` so that even on unicore machines we get at least 2
jobs (and in general NCPUS+1)
Closes: #1188
Approved by: jlebon
Basically the `rpmostree_context_relabel()` call we had in the treecompose path
for unified core didn't actually have any effect as the core code did a relabel
and unset the array.
I think this may actually be a regression from: https://github.com/projectatomic/rpm-ostree/pull/1137
though I didn't verify.
Anyways looking at this, the code is a lot simpler if we change the API so that
the "normal" relabeling is folded into `rpmostree_context_assemble()`. Then we
change the public relabel API to be "force relabel" which we use in the unified
core 🌐 treecompose path.
This shrinks the jigdoRPM for FAH from 90MB to 68MB.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1172Closes: #1173
Approved by: jlebon
This fixes a large swath of compatibility issues, for the same reasons as
overlayfs makes a lot of things Just Work. The ugly part of course is
doing hidden copyups inside the filesystem.
We've gone quite a long time with the "pure rofiles" mode, and have made changes
to various bits of userspace to be compatible with it. But what finally made me
give up on that is glibc's locale-archive; there's a patch for it that
is stalled, but even if it was applied we would still need to work with
older glibc.
This issue comes to the fore in unified core 🌐 mode, as without this
we won't get a correct locale archive.
Closes: #1171
Approved by: jlebon
Over a year later, the "opening the host rpmdb" bug is fixed,
so we can do composes in parallel ∥, hooray!
I'm dusting this off since we were running into CI (PAPR) timeouts
when I was adding more to the compose tests.
Closes: #545
Approved by: jlebon
This is the first step towards unifying how we introspect packages from
a specific commit. We currently do this in three ways: libdnf, librpm,
and now `rpmostree.rpmdb.pkglist`. I'd like to get to a point where we
only have `rpmostree.rpmdb.pkglist` and libdnf, the latter only when
more complex queries are required.
This patch teaches the `db diff` command to make use of the new db diff
API so that it can work even on metadata-only commits. This is relevant
for use cases mentioned in #558.
I didn't get rid of the `rpmhdrs_diff` functions right now because of
the `--changelogs` option: libdnf currently does not expose this, so we
fall back to the previous API in that case. OTOH, I wonder how much it's
actually used in the wild; maybe we could just nix it?
Closes: #1162
Approved by: cgwalters
This is used in critical paths like pungi, so let's be sure it works;
the semantics are a bit subtle as it overrides setting the ref.
Closes: #1161
Approved by: jlebon
To make the new pkglist metadata even more usable, let's insert it
sorted. This ensures that we can bsearch the GVariant on the
client-side.
Enhance the bsearch utility function we have to deal with duplicate key
names. Although this is not the case today in rpm-ostree-managed streams
I know, some packages are allowed to have multiple versions installed,
so let's make sure we handle that deterministically by always returning
the first (earliest) version of the package.
Closes: #1158
Approved by: cgwalters
The new `rpmdb.pkglist` metadata is a cheap way of retrieving the set of
packages in a commit. I'd like to make use of it as much as possible vs.
checking out the rpmdb and setting up a DnfSack.
Of course, in the case of layered commits, it doesn't matter *as* much,
because a layered commit being present in the repo should mean that a
deployment is currently using it, and we should learn to reuse the rpmdb
checkout of that deployment. Though keeping it consistent across both
server and client commits makes implementing `OstreeDeployment`-agnostic
things like `db diff` more efficient too. I also plan to use this in the
upcoming auto-update code.
Closes: #1158
Approved by: cgwalters
This fixes another thing broken with `compose --ex-unified-core`;
for e.g. `/usr/bin/ping` from `iputils`, the classic example of a filecaps
binary.
As I'm writing this commit message I realize it will actually also
take effect for package layering unnecessarily; we'll pointlessly
break the hardlink. But eh, it doesn't matter right now, we can
optimize that later.
Closes: #1151
Approved by: jlebon
Rather than entirely symlinking `systemctl` → `/bin/true`, in order
to e.g. have NetworkManager be enabled, we need to process presets.
This is one of the things that's breaking FAHC where I did a
`--ex-unified-core` deployment.
(Actually it's a bit tempting to run a mass preset pass at the end,
but for now let's do this)
Implementation note: this is our first use of GResources, which
is a handy way to embed data into our final binary.
Closes: https://github.com/projectatomic/rpm-ostree/issues/550Closes: #1148
Approved by: jlebon
This rolls up several libglnx changes: https://github.com/GNOME/libglnx/pull/101
Now of course things are trickier here because we have an internal
abstraction over directly emitting to a console versus sending the
result over DBus. Further complicating things is that some things
call into libdnf and thus *require* use of `DnfState` which does
not give us the "n items" information, versus other parts which
we implement and can do what we want.
Even *further* complicating things is that we have to take care around non-CLI
callers like Cockpit; so I didn't try to pass the "n items" over DBus, rather
just reimplemented the "insert into text" that libglnx is doing.
Anyways overall this looks better IMO for all cases.
Update submodule: libglnx
Closes: #1143
Approved by: jlebon
This is prep for a rework of
https://github.com/projectatomic/rpm-ostree/pull/621
For a no-op `rpm-ostree upgrade` (i.e. no updates available), as long as
layering is enabled, we pay the cost of checking out the base tree, *mostly*
only to get the base rpmdb.
This is prep for fixing that down the line by knowing we always have the "base"
tree's rpmdb checked out. Then in the layering case we only modify
`/usr/share/rpm` (eventually that will point to `/usr/lib/sysimage/rpm`).
Teaching `rpmostree-core.c` about this can follow on later.
Closes: #1142
Approved by: jlebon
Having the "jigdo set" in repodata makes it so we can parallel download the
jigdo RPM with the set. However for now, I kept the jigdo set in the jigdoRPM,
since that way it'll be covered by the signature.
Also, this changes the way we inject metadata to use a magic comment string,
since trying to pass a gigantic macro to `rpmbuild` via its argv didn't work out
so well (it looks like rpmbuild eats newlines). This approach is more robust.
Closes: https://github.com/projectatomic/rpm-ostree/issues/1132Closes: #1140
Approved by: jlebon
With unified core 🌐 we are *relying* on the devino cache
for correctness when using `bare-user` repos. Otherwise lots
of bad things will happen as we won't hit the happy path from
[this libostree PR](https://github.com/ostreedev/ostree/pull/1297)
(I should probably add an assertion there that we aren't trying to commit
`user.ostreemeta`).
It looks like I had this working in some of the old unified core WIP patches,
but it was lost when rebasing 🏄.
We noticed this when I was trying to deploy jigdo in FAHC and the system
wouldn't boot as various things rely on those suid transitions.
Closes: #1139
Approved by: jlebon
We don't want to have to download all of `/usr/share/rpm` just to get
the list of packages used to compose the tree. This is fundamental
information that needs to be easier to discover. So let's stick it right
in the commit metadata. There's various use cases for this information,
including easily checking for and displaying updates and a pkglist-aware
version of `ostree log`.
Closes: #1134
Approved by: cgwalters
This function is now basically reimplementing all of the intelligence
we have in libostree today. It wouldn't have worked before we
fixed https://github.com/ostreedev/ostree/issues/1165 but now
that's done, conceptually we just need to call
`ostree_repo_commit_modifier_set_sepolicy()`. However, I had
to drop the "number of files changed" since currently the libostree
API doesn't support that.
Also, in order to detect the case that content changes at all (so
we still have some useful information in the journal), implement
a "content hash" check.
See also https://github.com/projectatomic/rpm-ostree/pull/1123
(And note we *don't* use `DEVINO_CANONICAL` here)
Closes: #1138
Approved by: jlebon
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).
Closes: #1019
Approved by: jlebon
Now that the importer *only* imports into OSTree repos, let's
clean up the API so that the `OstreeRepo` and `OstreeSePolicy`
are passed as constructor args.
Also rework things so there's only one constructor API that
steals the fd.
This is prep for adding another async import API.
Closes: #1124
Approved by: jlebon
We are going to want versioning on the jigdo RPMs, since it's
highly likely things change.
This is done via new magic '-D rpmostree_jigdo_meta` macro, which we can also
use for other things down the line.
Closes: #1135
Approved by: jlebon
Let's "repo bind" the OIRPM by default; this makes the rpm-md repo feel a lot
more like an ostree remote, and IMO is just a really good idea in general to
increase predictabilty.
Closes: #1130
Approved by: jlebon
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).
Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).
There are two reasons to do this:
- We want "offline" updates by default; updates shouldn't affect the
running system. If we prepare the new root in the background, a
%post shouldn't restart a service for example. We already "handle"
this by making `systemctl` a symlink to `/bin/true`, but this approach
also shuts off `%post`s that do e.g. `insmod`.
- Protection against accidental system damage
Closes: #1099
Approved by: jlebon
Tracking issue: https://github.com/projectatomic/rpm-ostree/issues/1081
To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.
This is a lot like [Jigdo](http://atterer.org/jigdo/)
Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.
If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.
Closes: #1103
Approved by: jlebon
`test -n` would always be true. Also while I have the patient open
let's make writing the file atomic. Maybe someday I'll be motivated
enough to write an `O_TMPFILE` patch for bash.
Closes: #1103
Approved by: jlebon
When we added the `--ex-unified-core` option our caching story got
very messy because the non-unified core caches RPMs, but unified
does ostree repo caching.
For jigdo, we want the RPMs. Fix this by mirroring the RPMs using
`--download-only` and pointing the tests consistently at that.
Closes: #1122
Approved by: jlebon
We originally needed the pkgcache to be a separate repo due to ostree's
overzealous pruning policies. The idea was to maintain multiple commits
in each pkg branch for different SELinux policies. In practice, there's
not much use in maintaining old copies and it's just easier to always
relabel on the fly. So then, the need for a separate repo completely
melts away.
This helps simplify the mental model a bit and allows us to avoid subtle
issues like #1047. Note however that the core is still capable of
handling split repos for the `--ex-unified-core` compose use case. Once
that and the jigdo work are a bit more settled, we can have a clearer
picture of how to simplify the core further.
The tricky bit is migrating the cache. When deploying, we check if a
pkgcache repo exists and migrate its refs if so. We then leave behind a
symlink to the system repo to remain compatible with older rpm-ostrees.
Closes: #1055
Approved by: cgwalters
Right now each ostree txn incurs a `syncfs()`; see
https://github.com/ostreedev/ostree/issues/1184
And before this patch, we were doing a txn per package import.
We can really do better in libostree - we'll fix that, but in the short term
let's use a bigger txn for every package. However, the obvious change here of
simply hoisting up the txn is that on failure for imports, we'd discard all
downloaded packages. We fix that by changing the auto txn API to have
a `commit_on_failure` boolean, and use it in cases where we're doing
imports.
This is prep work for jigdo, where we'll be using the import path all the time.
My bigger plan is to do multithreaded imports.
Closes: #1116
Approved by: jlebon
It took me way, way, way too long to debug that my dev container somehow missing
`ostree-grub2` caused the script to fail early, but we'd still continue trying
to sync, leading to us not actually changing the installed `rpm-ostree` binary,
leading to test suite failures.
Also add the chmod hammer at the top so we can delete even if we fail after
doing an rsync.
Closes: #1115
Approved by: jlebon
In the jigdo path we don't actually want to import the OIRPM literally
into ostree. I considered adding jigdo logic into `rpmostree-unpacker.c`
but it'd be a mess as the functionality is quite logically separate
from importing.
So split off an `unpacker-core.c` file which has the bare libarchive+RPM
helpers, and rename `RpmOstreeUnpacker` to `RpmOstreeImporter`.
Closes: #1110
Approved by: jlebon
We removed this in review, but I rediscovered why I added it. We
fail the `g_assert (sepolicy_matches)` if we already had packages
done with the final label when we go to reuse the cache.
(Basically, if we use the cache multiple times it's hard to avoid
relabeling all the time which is unfortunate...gets back a bit to
a way to annotate pkgcache commits as supporting multiple policies)
Closes: #1109
Approved by: jlebon
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).
And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet. For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)
Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.
Closes: #940
Approved by: jlebon
In an unprivileged case, we can't do this on the real filesystem. For
`ex container`, we want to completely ignore uid/gid. I added a test
installing `httpd` which failed previously.
TODO: For non-root `--ex-unified-core` we need to do it as a commit modifier.
Closes: #940
Approved by: jlebon
More work for unified core; this was one of the biggest parts of the `ostree
diff` when comparing a unified core treecompose of Fedora Atomic Host versus
the libdnf path.
This is also the first case where we're teaching the pkgcache to redownload
based on state.
Closes: #1105
Approved by: jlebon
I was trying to figure out why:
rpm-ostree compose tree --repo repo/ manifest.json
would result in:
error: opendir(manifest.json): No such file or directory
It turned out to be because we had `--repo` in *both* the `install`
options and the `commit` options. This makes sense since both these
subcommands need to be given a repo. However, in the `tree` case, we
were adding both arrays, which meant we inherited two `GOptionEntry`s
for `--repo`. This confused glib2 which consumed not one but two CLI
arguments when looking for the argument associated with `--repo`.
Our CI didn't notice this because it uses the `--repo=foo` notation,
which doesn't throw off glib2.
Fix this by factoring out the `--repo` option into a separate array so
that in the `tree` case, it only gets added once. Exercise the fix in CI
by using the two argument notation for `--repo`.
Closes: #1101
Approved by: cgwalters
We have a *lot* of experimental functionality. I think the
`override` bits are fleshed out enough now that we can lift
the `ex` designation. For example, jlebon fixed SELinux
labeling in the presence of override-replace.
Closes: #1089
Approved by: jlebon
Added unit tests for rpm-ostree ex kargs --delete,
--append and --replace.
Also exposed two getter functions for kargs table
and array so people can retrieve information from
kargs.
Also includes a minor fix for a bug caught by the unit
test.
Closes: #1013
Approved by: cgwalters
Refined a bit for the previous written kargs test.( mainly for
checking kargs after rollback )
Added tests for import-proc-cmdline and deploy-index option
Closes: #1013
Approved by: cgwalters
Refactor the similar code portion from delete option into
a new function, and reuse it in the replace logic.
Add the replace functionality to allow kernel arguments
replacement. Now support two different types of replacement:
1: replace by key=value when there is only one key existing
e.g we have 'hi=hello' as our one of the arguments (hi only has
one value associated with it) , and 'rpm-ostree ex kargs --replace hi=new'
will replace the value and change hi=hello into hi=new.
2: replace by key=value=new_value for all other replacements
( this will work for swapping single value pair too !)
Some tests for rpm-ostree ex kargs --replace are added
for future regression.
Closes: #1013
Approved by: cgwalters
Tests are introduced in this commit to test the basic
functionality for rpm-ostree ex kargs --append,
and rpm-ostree ex kargs --delete.
Those tests are added for future regression.
Closes: #1013
Approved by: cgwalters
This is a revisit of a PR for client-side layering: https://github.com/projectatomic/rpm-ostree/pull/1072
Here though we're doing this by default for server-side composes.
There are a few reasons to do this; first, I'm seeing an issue
in some of our Jenkins jobs for Fedora that hit "mirror roulette"
and end up creating commits that "revert" to older versions temporarily.
While I've [certainly pitched](https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/IMPE6KCRBHCEJH5VBE6ZFIRLPAD743JT/) this as a feature, I think
we really want something like `--force-older-timestamp` - basically
error out if the timestamps on one or more input repos were older.
Not doing that in this patch, but it paves the way to do so.
Second, I'd like to use this data in the `ostree.source-title`
metadata key down the line. Something like:
`└ rpmmd: fedora-26 (20170310), fedora-26-updates (20171101)`
(This could be a lot nicer if we drive versioning in to the rpm-md repo info,
and e.g. there's some friendly "week number" style versioning for the updates
repo now that it's batched...for now we have timestamps)
For CentOS/RHELAH this gets interesting and potentially more verbose,
to the point where we may want to render it more explicitly.
But anyways, let's do this now, as it will be useful even without
an explicit rendering, since users can do e.g. `ostree show` on
a base commit hash to dump the data.
I had a concern that some users may not want to emit this metadata;
they can currently do `--add-metadata-string rpmostree.rpmmd-repos ''`
and that will "win".
Closes: #1079
Approved by: jlebon
Related to: https://github.com/projectatomic/rpm-ostree/issues/49
We want to support "name binding" per client system, rather than
having a hardcoded mapping in our tree. Currently if e.g. a new
daemon is added as a dependency (or as part of e.g. systemd) it's
easy to silently miss it.
This is prep for doing that binding client side consistently, which is what we
do with package layering.
Closes: #1077
Approved by: jlebon
We won't have done the postprocessing, so `/usr/lib/passwd` won't exist. Trying
to use `compose install` with current fedora-atomic failed (I *really* should
have tested that at least manually with the final patchset). Add `check-passwd`
to the test suite so this gets coverage too.
Closes: #1076
Approved by: jlebon
Related to: https://github.com/projectatomic/rpm-ostree/issues/774
We aren't yet trying to render this in any way on the client side, but let's
start capturing the timestamp data now so that we can make use of it later
in e.g. `rpm-ostree status -v`.
Closes: #1072
Approved by: jlebon
Right now `rpm-ostree compose tree` is very prescriptive about how things work.
Trying to add anything that isn't an RPM is absolutely fighting the system. Our
postprocessing system *enforces* no network access (good for reproducibilty, but
still prescriptive).
There's really a logical split between three phases:
- install: "build a rootfs that installs packages"
- postprocess: "run magical ostree postprocessing like kernel"
- commit: "commit result to ostree"
So there are two high level flows I'd like to enable here. First is to allow
people to do *arbitrary* postprocessing between `install` and `commit`. For
example, run Ansible and change `/etc`. This path basically is like what we have
today with `postprocess-script.sh`, except the builder can do anything they want
with network access enabled.
Going much farther, this helps us support a "build with Dockerfile" style flow.
We can then provide tooling to extract the container image, and combine
`postprocess` and `commit`.
Or completely the other way - if for example someone wants to use `rpm-ostree
compose install`, they could tar up the result as a Docker/OCI image. That's now
easier; an advantage of this flow over e.g. `yum --installroot` is the "change
detection" code we have.
Related issues/PRs:
- https://github.com/projectatomic/rpm-ostree/pull/96
- https://github.com/projectatomic/rpm-ostree/issues/471
One disadvantage of this approach right now is that if one *does* go for
the split approach, we lose the "input hash" metadata for example. And
down the line, I'd like to add even more metadata, like the input rpm repos,
which could also be rendered on the client side.
But, I think we can address that later by e.g. caching the metadata in a file in
the install root and picking it back up or something.
Closes: #1039
Approved by: jlebon
Depends: https://github.com/ostreedev/ostree/pull/1296
As I mention in the commit there, I see two uses for this in rpm-ostree; first
in our test suite, and second for OCI-built image imports.
I also took a step further here and inject an `original-origin` metadata
key, though we aren't actually using that yet. The problem I'm trying
to solve there is that repeated `make vmoverlay` starts chaining things up,
but that gets very confusing. I think we should always have `vmoverlay` unwind
back to the base ref. (Or at least do that by default)
Closes: #1069
Approved by: jlebon
In another PR I did the manual bridging of commit metadata to deployment
property, but that's annoying. Let's just bridge all commit metadata.
Closes: #1069
Approved by: jlebon
Switching to the `_CONSUME` flag revealed an "oh god how did I write that"
bug in the previous patch in https://github.com/projectatomic/rpm-ostree/pull/1046
AKA commit: 334f0b89be
The way that actually fixed the bug before was because we were using
hardlink checkouts, and we were operating outside an `rofiles-fuse`
context, we simply directly changed the on-disk object mode.
But with the `_CONSUME` flag we started deleting the files as we write,
meaning that stopped working.
I *initially* wrote a patch to do the same split "prepare/processing/commit"
flow that treecompose and package layering do, but that can't really fix this
bug - we need to do it on import.
So do the chmod on import and drop the postprocessing bits.
Closes: #1067
Approved by: jlebon
To complement the new `--cache-only` option, add a `--download-only`
option. This does exactly what it says: we download the ostree, download
and import packages, but don't actually commit & deploy. This can be
used to effectively prime a follow-up `--cache-only` operation that can
be done during a more convenient/safer maintenance window.
I debated naming the two options `--pull-only` and `--deploy-only` like
the ostree equivalents. Though "pull" felt like the wrong word given
that it's associated more with ostree pulling but rpm-ostree also
downloads & imports RPMs. As for `--deploy-only` vs `--cache-only`, it
seems like `--cache-only` is a more accurate description of the
functionality (i.e. rather than describing an action, it describes a
mode). I also considered `--no-download` to make the synergy with
`--download-only` more obvious. Maybe that's better? Naming is hard...
Closes: #713Closes: #1049
Approved by: cgwalters
Now that we have a strong notion of `cache-only` mode, make use of it
when performing an `uninstall` or `ex override remove/reset`.
Closes: #944Closes: #1049
Approved by: cgwalters
As Colin mentioned in #1035, the new `--cache-only` implemented only the
rpmmd half of the story. Here we complete that story by also ensuring
that when in cache-only mode, we don't download new ostree data nor new
packages. We try to complete the requested operation with what we have.
To do this, we add support for the same `SYNTHETIC` pull that was added
in ostree[1] so that we don't actually pull, but still perform timestamp
checking.
On the pkgcache side, we disable all remote repos and instead insert all
our cached RPMs into the `DnfSack`. Care is taken to still perform
SHA256 verification for local pkg installs/replacements.
[1] https://github.com/ostreedev/ostree/pull/642Closes: #687Closes: #1049
Approved by: cgwalters
The new idle exit behaviour is nice, but it makes debugging it harder
because you have to be fast enough to attach or place your breakpoints
and trigger it before it auto-exits. Add a compile-time flag that
developers can easily turn on to disable the auto-exit behaviour.
Closes: #1052
Approved by: cgwalters
For the `ex container` case, there's no security issues here; one shouldn't be
doing user management in these roots at all.
This is for work on exporting `ex container` roots to OCI as non-root. Without
this fix, libostree just tries to `openat()` the object for export to tar, and
fails.
See also https://github.com/projectatomic/rpm-ostree/issues/1045Closes: #1046
Approved by: jlebon
Just taking what I learned from #1035 and applying it here. What's nice
about this is that there's no cleanup needed. Once the process is killed
(or worst case, we reboot the VM), there's no traces left at all.
Also added a few extra "ok" outputs.
Closes: #1043
Approved by: cgwalters
I've been lazy about actually using using rsync instead of scp when
copying new RPMs over to the VM. We do this here. Also make
`vm_send_test_repo` take a mode argument that allows callers to
completely skip the sending of the repo file itself. This will be needed
for the `makecache` test, in which we *don't* want the repo to be local.
It looks cleaner anyway for the gpgcheck use case as well.
Closes: #1035
Approved by: cgwalters
Prep for implementing `rpm-ostree cancel`, but this works with the way we handle
`Ctrl-C` interactively on a client as well. Being able to cancel a script
execution is quite nice; some of them are expensive, and having one loop forever
has been known to happen.
Closes: #1025
Approved by: jlebon
There's a lot of paths in the core related to SELinux policy changes and
relabeling packages. We currently have no test coverage for them. We add
support in the test libraries here to build such packages.
We also add a test that checks both that we correctly relabel RPMs when
the policy changes and that we handle layered packages that install
SELinux packages properly.
Closes: #999
Approved by: cgwalters
We pretty standardized on insttree/ being the installation tree, and
we don't actually support $VMCHECK_INSTTREE anymore.
Closes: #1032
Approved by: cgwalters
We have some unit-style tests that run `ex container`, but
they aren't "real"; they don't use scripts for example. Let's
add tests for this similar to `tests/compose`.
We use a 26 base, but the target repos need to be 27
to pick up the fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1478172
Add some bits to share infra between `tests/compose` and `tests/ex-container`;
basically handling the rpmmd repos. I tweaked things to be more streamlined
there between the `.papr.yml` and the test script.
Right now this is just one test for `bash`, but lays some of the infrastructure
for doing more. One thing that we need to do to improve more here is to better
cache RPMs, a bit like the compose tests do.
Closes: #1024
Approved by: jlebon
Sometimes it's useful to have access to the additional files when running
the post script, so this re-orders the compose process to copy the
additional files in before the post script runs
Signed-off-by: Jonathan Dieter <jdieter@lesbg.com>
Closes: #997
Approved by: jlebon
Even though it's really `/usr/etc`. This is for greater consistency with
`postprocess-script` where it appears as `/etc`.
Closes: #997
Approved by: jlebon
Rather than just letting the scriptlets inherit the daemon's
stdout/stderr, redirect their outputs so that we can set a customized
identifier to make it easier to distinguish from the daemon output.
Also print out the `journalctl` command needed so that users can
investigate the output themselves.
Closes: #998
Approved by: cgwalters
I suspect a common pattern with local replacement overrides is to
simultaneously replace a group of packages that depend on each other in
one shot, as is the case with docker, docker-common, and
docker-rhel-push-plugin currently in Fedora Atomic Host. In such cases,
we can print a cleaner diff in the status to make it easier to grok.
Before:
ReplacedBasePackages: strace 4.18-1.fc26 -> 4.19-1.fc26, docker-common 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, docker 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, docker-rhel-push-plugin 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26
After:
ReplacedBasePackages: docker-common docker docker-rhel-push-plugin 2:1.13.1-22.gitb5e3294.fc26 -> 2:1.13.1-21.git27e468e.fc26, strace 4.18-1.fc26 -> 4.19-1.fc26
Closes: #1004
Approved by: cgwalters
The comment here was wrong; we don't rely on `O_APPEND` here for package
layering since we convert on import. I noticed this while I was doing
a grep for `O_APPEND` in the codebase as part of unified core work.
Fix this by converting to `O_TMPFILE`+`GLNX_LINK_TMPFILE_NOREPLACE`.
Prep for unified core.
Closes: #1009
Approved by: jlebon
This is the rpm-ostree equivalent of
<47b4dd1b38>
Unfortunately, introspection uses `dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.
Prep for syncing PAPR config with ostree.
Closes: #1000
Approved by: jlebon
We were directly bind mounting the checked out `/usr/etc` onto `/etc`
which was exposing us to corruption from scriptlets. Since we already
have an rofiles-fuse mount for `/usr`, let's just re-use its `etc/`
subdir and bind mount that instead.
Closes: #1003
Approved by: cgwalters
We don't need those in the tree, so let's nuke them. This also fixes
subtle compatibility issues between hardlinks and lock files (see #999).
Closes: #1002
Approved by: cgwalters
This PR uses https://github.com/ostreedev/ostree/pull/1156
to ensure that when installing packages containing files with
exact same content, the files will be merged.
When installing packages containing conflicting files,
the error will still be detected at the ostree side.
The checkout overwrite option at rpm-ostree side is also
modified to accomodate the changes made in ostree side.
A test is added for regression
Commands like `upgrade` and `deploy` need to know if a new deployment
was actually laid down so that it may print a pkg diff if so. This is
implemented by listening for changes to the DefaultDeployment D-Bus
property. D-Bus emits a signal when the deployment variant changes
value.
However, in #595, with the introduction of `pending-*` related keys, the
deployment variant no longer represents data solely tied to that
specific deployment. In this case, because `deploy` operations currently
set the ref to the resolved checksum, it can happen that deploying the
same base commit when the current refspec *isn't* pointing to that base
commit will result in the `pending-*` keys dropping out and a default
deployment change notification going out.
In this patch, we strengthen how we determine whether a new deployment
was laid down by actually looking at the deployment id, rather than just
assuming that a change to the property implies a new deployment.
Closes: #981Closes: #984
Approved by: cgwalters
