IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This is a revisit of a PR for client-side layering: https://github.com/projectatomic/rpm-ostree/pull/1072
Here though we're doing this by default for server-side composes.
There are a few reasons to do this; first, I'm seeing an issue
in some of our Jenkins jobs for Fedora that hit "mirror roulette"
and end up creating commits that "revert" to older versions temporarily.
While I've [certainly pitched](https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/IMPE6KCRBHCEJH5VBE6ZFIRLPAD743JT/) this as a feature, I think
we really want something like `--force-older-timestamp` - basically
error out if the timestamps on one or more input repos were older.
Not doing that in this patch, but it paves the way to do so.
Second, I'd like to use this data in the `ostree.source-title`
metadata key down the line. Something like:
`└ rpmmd: fedora-26 (20170310), fedora-26-updates (20171101)`
(This could be a lot nicer if we drive versioning in to the rpm-md repo info,
and e.g. there's some friendly "week number" style versioning for the updates
repo now that it's batched...for now we have timestamps)
For CentOS/RHELAH this gets interesting and potentially more verbose,
to the point where we may want to render it more explicitly.
But anyways, let's do this now, as it will be useful even without
an explicit rendering, since users can do e.g. `ostree show` on
a base commit hash to dump the data.
I had a concern that some users may not want to emit this metadata;
they can currently do `--add-metadata-string rpmostree.rpmmd-repos ''`
and that will "win".
Closes: #1079
Approved by: jlebon
Now that there's a new libostree out, let's cut one here too, since we have a
fair bit accumulated, and I think it'd be good to have the oustanding kargs PR
to soak in master for a bit after landing.
Closes: #1080
Approved by: jlebon
Related to: https://github.com/projectatomic/rpm-ostree/issues/49
We want to support "name binding" per client system, rather than
having a hardcoded mapping in our tree. Currently if e.g. a new
daemon is added as a dependency (or as part of e.g. systemd) it's
easy to silently miss it.
This is prep for doing that binding client side consistently, which is what we
do with package layering.
Closes: #1077
Approved by: jlebon
We won't have done the postprocessing, so `/usr/lib/passwd` won't exist. Trying
to use `compose install` with current fedora-atomic failed (I *really* should
have tested that at least manually with the final patchset). Add `check-passwd`
to the test suite so this gets coverage too.
Closes: #1076
Approved by: jlebon
This is a cleanup I was going to do before, but had dropped. It's a general
followup to the `compose` rework; there's not much point to having a high level
wrapper, given that we generally need to do some postprocessing. Inline it into
the `ex container` code, which makes the core more "core".
Closes: #1074
Approved by: jlebon
I got a critical from an nonexistent `DESTDIR` in `rpm-ostree compose install`.
(I'm probably going to do a patch to add `rpmostree_command_invocation_set_exit_code()`
so the "exit 77" case can use that and everything else can `s/return EXIT_FAILURE/return FALSE/`
and we'll be free of these bugs)
Closes: #1075
Approved by: jlebon
Related to: https://github.com/projectatomic/rpm-ostree/issues/774
We aren't yet trying to render this in any way on the client side, but let's
start capturing the timestamp data now so that we can make use of it later
in e.g. `rpm-ostree status -v`.
Closes: #1072
Approved by: jlebon
Right now `rpm-ostree compose tree` is very prescriptive about how things work.
Trying to add anything that isn't an RPM is absolutely fighting the system. Our
postprocessing system *enforces* no network access (good for reproducibilty, but
still prescriptive).
There's really a logical split between three phases:
- install: "build a rootfs that installs packages"
- postprocess: "run magical ostree postprocessing like kernel"
- commit: "commit result to ostree"
So there are two high level flows I'd like to enable here. First is to allow
people to do *arbitrary* postprocessing between `install` and `commit`. For
example, run Ansible and change `/etc`. This path basically is like what we have
today with `postprocess-script.sh`, except the builder can do anything they want
with network access enabled.
Going much farther, this helps us support a "build with Dockerfile" style flow.
We can then provide tooling to extract the container image, and combine
`postprocess` and `commit`.
Or completely the other way - if for example someone wants to use `rpm-ostree
compose install`, they could tar up the result as a Docker/OCI image. That's now
easier; an advantage of this flow over e.g. `yum --installroot` is the "change
detection" code we have.
Related issues/PRs:
- https://github.com/projectatomic/rpm-ostree/pull/96
- https://github.com/projectatomic/rpm-ostree/issues/471
One disadvantage of this approach right now is that if one *does* go for
the split approach, we lose the "input hash" metadata for example. And
down the line, I'd like to add even more metadata, like the input rpm repos,
which could also be rendered on the client side.
But, I think we can address that later by e.g. caching the metadata in a file in
the install root and picking it back up or something.
Closes: #1039
Approved by: jlebon
Depends: https://github.com/ostreedev/ostree/pull/1296
As I mention in the commit there, I see two uses for this in rpm-ostree; first
in our test suite, and second for OCI-built image imports.
I also took a step further here and inject an `original-origin` metadata
key, though we aren't actually using that yet. The problem I'm trying
to solve there is that repeated `make vmoverlay` starts chaining things up,
but that gets very confusing. I think we should always have `vmoverlay` unwind
back to the base ref. (Or at least do that by default)
Closes: #1069
Approved by: jlebon
In another PR I did the manual bridging of commit metadata to deployment
property, but that's annoying. Let's just bridge all commit metadata.
Closes: #1069
Approved by: jlebon
Prep for split compose. The current Docker/OCI base images we
built at least for Fedora don't have this move done, so let's
ensure our postprocessing does it.
Closes: #1070
Approved by: jlebon
Prep for split compose. The Fedora base container image already empties
`/etc/machine-id`; let's handle that case rather than throwing an error. This
makes it easier to run our postprocessing on an image built on top of that base
image.
Closes: #1070
Approved by: jlebon
This is prep for split-compose. We have some options in the
treefile, like `boot_location` and `tmp-is-dir` etc. While those
are useful options, I don't want to force everyone using
`rpm-ostree compose postprocess` to write a treefile.
Change the code then to accept a `NULL` treefile to mean
"use the defaults".
Closes: #1070
Approved by: jlebon
Fixes a fsck issue that I saw while submitting another patch.
This is the second time in the last week I've fixed an issue like this; will
probably do a libostree patch to just turn it on by default for
`bare-user-only`, but let's fix this bug anyways.
Closes: #1071
Approved by: peterbaouoft
Switching to the `_CONSUME` flag revealed an "oh god how did I write that"
bug in the previous patch in https://github.com/projectatomic/rpm-ostree/pull/1046
AKA commit: 334f0b89be
The way that actually fixed the bug before was because we were using
hardlink checkouts, and we were operating outside an `rofiles-fuse`
context, we simply directly changed the on-disk object mode.
But with the `_CONSUME` flag we started deleting the files as we write,
meaning that stopped working.
I *initially* wrote a patch to do the same split "prepare/processing/commit"
flow that treecompose and package layering do, but that can't really fix this
bug - we need to do it on import.
So do the chmod on import and drop the postprocessing bits.
Closes: #1067
Approved by: jlebon
Now that libglnx allows it, we can skip declaring a `struct stat` we
don't actually need just to check if a file exists.
Closes: #1066
Approved by: cgwalters
This is used for both `ex container` and client-side operations, but not most
uses of `compose tree` (which should go into `bare-user` or `archive`).
We'll circle back and do `bare-user` for `_CONSUME` in libostree though
soon.
Closes: #1065
Approved by: jlebon
Just noticed this FIXME while working on split compose. Use
the `#define` we have, and inline the duplicated function which
boils down to `g_variant_lookup()` - I simply changed `&s` to `s`
to inline the `strdup`.
Closes: #1064
Approved by: jlebon
An interesting behaviour arises from the new `--cache-only` option. We
would essentially consider two goals with the same exact pkglist as
different depending on whether a package came from a yum repo or our
pkgcache repo. This meant that `rpm-ostree upgrade` followed by
`rpm-ostree upgrade -C` was never detected as a no-op and always
resulted in a relayer.
This was due to the fact that the checksum libsolv calculated was based
on our chopped off pkgcache meta-RPMs. This of course won't match the
checksum from the repodata, which is of the whole RPM.
Thankfully, we already save the repodata checksum when unpacking RPMs.
This patch simply detects the case where we're trying to checksum a pkg
originating from the pkgcache, and reads in the saved repodata checksum
rather than the wrong libsolv-derived one.
Closes: #1049
Approved by: cgwalters
To complement the new `--cache-only` option, add a `--download-only`
option. This does exactly what it says: we download the ostree, download
and import packages, but don't actually commit & deploy. This can be
used to effectively prime a follow-up `--cache-only` operation that can
be done during a more convenient/safer maintenance window.
I debated naming the two options `--pull-only` and `--deploy-only` like
the ostree equivalents. Though "pull" felt like the wrong word given
that it's associated more with ostree pulling but rpm-ostree also
downloads & imports RPMs. As for `--deploy-only` vs `--cache-only`, it
seems like `--cache-only` is a more accurate description of the
functionality (i.e. rather than describing an action, it describes a
mode). I also considered `--no-download` to make the synergy with
`--download-only` more obvious. Maybe that's better? Naming is hard...
Closes: #713Closes: #1049
Approved by: cgwalters
Now that we have a strong notion of `cache-only` mode, make use of it
when performing an `uninstall` or `ex override remove/reset`.
Closes: #944Closes: #1049
Approved by: cgwalters
As Colin mentioned in #1035, the new `--cache-only` implemented only the
rpmmd half of the story. Here we complete that story by also ensuring
that when in cache-only mode, we don't download new ostree data nor new
packages. We try to complete the requested operation with what we have.
To do this, we add support for the same `SYNTHETIC` pull that was added
in ostree[1] so that we don't actually pull, but still perform timestamp
checking.
On the pkgcache side, we disable all remote repos and instead insert all
our cached RPMs into the `DnfSack`. Care is taken to still perform
SHA256 verification for local pkg installs/replacements.
[1] https://github.com/ostreedev/ostree/pull/642Closes: #687Closes: #1049
Approved by: cgwalters
Minor detail. We still need to support the `--pretty` flag, but it
doesn't do anything anymore, so let's just hide it to clean up the help
output.
Closes: #1060
Approved by: cgwalters
Caught by bumping libglnx and its more widespread checking for `EBADF`. We
should probably add a `glnx_fdopen()` that steals ownership of the fd.
Closes: #1056
Approved by: jlebon
Prep for [compose split](https://github.com/projectatomic/rpm-ostree/pull/1039).
The next step here will be to basically have `postprocess_final()` + `ostree commit`
be `rpm-ostree compose commit`.
This reworks the order for a few things - we move `/usr` first right after
initializing the rootfs, which works now because from previous commits `/etc`
is really `/usr/etc`.
We also change things so that we grab `/boot` and `/var` from the src root,
then `postprocess_final()` handles them in place.
One detail here is that I had to do the `rpmostree_rootfs_prepare_links()`
after the `convert_var_to_tmpfiles_d()` since it writes into var; before
this worked because we were using different roots.
Closes: #1054
Approved by: jlebon
When calculating the goal checksum, we only accounted for package
installs. But with override support, there are now other package actions
we need to pick up on.
In practice, the fact that we also checksum the treespec made this hard
to actually cause an issue. Although, since the actual final goal is
also dependent on the underlying rpmdb, I can imagine e.g. a replacement
override be considered a `DNF_PACKAGE_INFO_UPDATE` for one rpmdb appear
as a `DNF_PACKAGE_INFO_DOWNGRADE` for another.
While we're here, do some light code style porting and streamline the
checksumming process to avoid creating a separate `GPtrArray`.
Closes: #1053
Approved by: cgwalters
The `--cache-only` option was added to the generic `option_entries`
array, which is shared by both `install` and `uninstall`. But really, it
only makes sense for `install`. Move it to the `install` specific array
instead.
I also renamed the option entry arrays to make more sense. Before, the
`install_option_entries` referred to the additional options that the
`uninstall` command took. Now the names refer to the command to which
they're associated.
Closes: #1053
Approved by: cgwalters
The new idle exit behaviour is nice, but it makes debugging it harder
because you have to be fast enough to attach or place your breakpoints
and trigger it before it auto-exits. Add a compile-time flag that
developers can easily turn on to disable the auto-exit behaviour.
Closes: #1052
Approved by: cgwalters
This moves the JSON parsing and reorders a few things, and
building on the previous commits, adds a constructor function.
Prep for [compose commit](https://github.com/projectatomic/rpm-ostree/pull/1039)
Closes: #1050
Approved by: jlebon
Prep for [rpmostree compose commit](https://github.com/projectatomic/rpm-ostree/pull/1039).
We still want to write the metadata in that path, so having the data
in the struct will help make things common between the two.
Closes: #1050
Approved by: jlebon
For the `ex container` case, there's no security issues here; one shouldn't be
doing user management in these roots at all.
This is for work on exporting `ex container` roots to OCI as non-root. Without
this fix, libostree just tries to `openat()` the object for export to tar, and
fails.
See also https://github.com/projectatomic/rpm-ostree/issues/1045Closes: #1046
Approved by: jlebon
In prep for potentially changing it:
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html
Of course actually doing a transition would be harder than this, as we'd need to
add a compat symlink, and even that wouldn't quite be enough as e.g. the
"preview" code would need to learn how to follow the symlink (or just try both
locations).
In practice I think we'd need to land the code to handle both locations, let
that trickle out for e.g. 3 months, then make the switch in our treecomposes.
But, might as well make this change now; using a `#define` makes it slightly
easier to find places that need changing later.
Closes: #1048
Approved by: jlebon
There's a lot going on in the final postprocessing; and some
potentially tricky dependencies. But we can cleanly move up
the "init rootfs" bits, which helps make the rest of the function
closer to the "final postprocessing" we need for
[split compose](https://github.com/projectatomic/rpm-ostree/pull/1039).
Closes: #1044
Approved by: jlebon
It's really way saner if we create the target root dir in the
compose high level. That way everything relating to actual file names
(and the concept of the "workdir") rather than the "library".
This is prep for [compose split](https://github.com/projectatomic/rpm-ostree/pull/1039)
in that it will be easier to tease out an API for less "opinionated"
postprocessing that just works on `target_rootfs_dfd`.
Closes: #1044
Approved by: jlebon
Just taking what I learned from #1035 and applying it here. What's nice
about this is that there's no cleanup needed. Once the process is killed
(or worst case, we reboot the VM), there's no traces left at all.
Also added a few extra "ok" outputs.
Closes: #1043
Approved by: cgwalters
Using an autoptr for the strbuf not only fixes a memleak in the error path, it's
a bit more efficient since we can just pass `buf->len` rather than running
`strlen()`.
Closes: #1042
Approved by: jlebon
This is prep for compose splitting:
https://github.com/projectatomic/rpm-ostree/issues/471
We'll need to run some postprocessing types in both phases, and it's a lot
easier to reason about if the postprocessing is "idempotent", i.e. it has no
effect a second time.
Closes: #1042
Approved by: jlebon
I've been lazy about actually using using rsync instead of scp when
copying new RPMs over to the VM. We do this here. Also make
`vm_send_test_repo` take a mode argument that allows callers to
completely skip the sending of the repo file itself. This will be needed
for the `makecache` test, in which we *don't* want the repo to be local.
It looks cleaner anyway for the gpgcheck use case as well.
Closes: #1035
Approved by: cgwalters
This is the equivalent version of `yum/dnf -C`. It goes together with
the new `makecache` command to allow completely asynchronous cache
update and usage.
Closes: #1035
Approved by: cgwalters
