IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
I believe this is a leftover vestige, and it was adding confusion when I was
debugging `rpmostree-core.c` async ops and cancellation.
Now the only cancellables in the daemon are created by transaction ops.
Closes: #1137
Approved by: jlebon
In the whole libdnf/C++ discussion I experimented with trying to build
rpm-ostree as C++. There's a whole ton of stuff there. I'm going to punt for
now, but let's land this one change so some progress was made.
Closes: #1141
Approved by: jlebon
With unified core 🌐 we are *relying* on the devino cache
for correctness when using `bare-user` repos. Otherwise lots
of bad things will happen as we won't hit the happy path from
[this libostree PR](https://github.com/ostreedev/ostree/pull/1297)
(I should probably add an assertion there that we aren't trying to commit
`user.ostreemeta`).
It looks like I had this working in some of the old unified core WIP patches,
but it was lost when rebasing 🏄.
We noticed this when I was trying to deploy jigdo in FAHC and the system
wouldn't boot as various things rely on those suid transitions.
Closes: #1139
Approved by: jlebon
We don't want to have to download all of `/usr/share/rpm` just to get
the list of packages used to compose the tree. This is fundamental
information that needs to be easier to discover. So let's stick it right
in the commit metadata. There's various use cases for this information,
including easily checking for and displaying updates and a pkglist-aware
version of `ostree log`.
Closes: #1134
Approved by: cgwalters
Basically, it doesn't make sense for the caller to only want the
pkglist, but not the refsack because the former has a more limited
lifetime than the latter. Check for that to make sure nobody falls in
this trap like I did.
Closes: #1134
Approved by: cgwalters
This function is now basically reimplementing all of the intelligence
we have in libostree today. It wouldn't have worked before we
fixed https://github.com/ostreedev/ostree/issues/1165 but now
that's done, conceptually we just need to call
`ostree_repo_commit_modifier_set_sepolicy()`. However, I had
to drop the "number of files changed" since currently the libostree
API doesn't support that.
Also, in order to detect the case that content changes at all (so
we still have some useful information in the journal), implement
a "content hash" check.
See also https://github.com/projectatomic/rpm-ostree/pull/1123
(And note we *don't* use `DEVINO_CANONICAL` here)
Closes: #1138
Approved by: jlebon
In preparation for multithreading relabeling; the problem I hit badly was the
`dnf_package_get_nevra()` isn't safe[1]. Since we were only using that for error
prefixing, move the error prefixing to the (currently non-MT) caller.
[1] Write it in C/C++, where multithreading is doomed to endlessly debug
data races?
Write it in Python, where MT isn't useful at all?
Write it in Golang, where you face deep hostility to in-process C/C++ libs
or really any other language?
Or write it in Rust, where we spend a lot of time fighting the borrow checker
and (to some degree) debugging the FFI glue to C/C++?
Closes: #1138
Approved by: jlebon
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).
Closes: #1019
Approved by: jlebon
Depends: https://github.com/ostreedev/ostree/pull/1358
For jigdo ♲📦 in order to get true image speed like libostree has we need to
interleave and parallelize downloading and importing.
The messy part about this is having sync API do the "invoke and wait on various
async tasks" pattern. It's the same thing in `ostree_repo_pull_with_options()`.
Importing is pretty dramatically faster with this, I can only imagine the speed
win if we actually interleaved with downloads. However doing that requires
libdnf/librepo work.
Closes: #1124
Approved by: jlebon
Now that the importer *only* imports into OSTree repos, let's
clean up the API so that the `OstreeRepo` and `OstreeSePolicy`
are passed as constructor args.
Also rework things so there's only one constructor API that
steals the fd.
This is prep for adding another async import API.
Closes: #1124
Approved by: jlebon
We are going to want versioning on the jigdo RPMs, since it's
highly likely things change.
This is done via new magic '-D rpmostree_jigdo_meta` macro, which we can also
use for other things down the line.
Closes: #1135
Approved by: jlebon
I have hit multiple times the bug with `bare-user` repos:
https://github.com/ostreedev/ostree/pull/1297
(Need to rebuild my dev container rather than relying on an updated
`sudo make install` from ostree git)
Anyways if things are out of sync, this bug would manifest as the
main repo wouldn't have the object.
Closes: #1135
Approved by: jlebon
Let's "repo bind" the OIRPM by default; this makes the rpm-md repo feel a lot
more like an ostree remote, and IMO is just a really good idea in general to
increase predictabilty.
Closes: #1130
Approved by: jlebon
Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0).
Change our bwrap invocations drop truly dangerous capabilities like
`cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the
popularity of Docker, we can be pretty sure that most RPM scripts should have
adapted to this (although a problematic area here is that traditional librpm
doesn't actually error out if scripts fail).
There are two reasons to do this:
- We want "offline" updates by default; updates shouldn't affect the
running system. If we prepare the new root in the background, a
%post shouldn't restart a service for example. We already "handle"
this by making `systemctl` a symlink to `/bin/true`, but this approach
also shuts off `%post`s that do e.g. `insmod`.
- Protection against accidental system damage
Closes: #1099
Approved by: jlebon
There's a lot more details in the libostree PR:
https://github.com/ostreedev/ostree/pull/1357
Basically loading the xattrs is slow; let's only do it if we need to, and "need
to" is defined by "SELinux policy changed". On my test F27AH VM, the difference
between a stat() + hash table lookup versus the full xattr load on my test case
of rpm-ostree install ./tree-1.7.0-10.fc27.x86_64.rpm is absolutely dramatic;
consistently on the order of 10s without this support, and <1s with (800ms).
Closes: #1123
Approved by: jlebon
In unified core mode, this avoids an intense spam of errors from `cp`
because `tmpfs` doesn't support the `user.` xattr namespace, and
since [this dracut commit](61c761bc2c)
dracut tries to copy all xattrs, which was just done for IMA.
There's no point to having the SELinux labels or other xattrs
in the initramfs.
The real fix here is dracut should learn to *only* copy the IMA
xattrs, or even better disable IMA enforcement for the dracut
run or something.
Closes: #1126
Approved by: jlebon
Tracking issue: https://github.com/projectatomic/rpm-ostree/issues/1081
To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.
This is a lot like [Jigdo](http://atterer.org/jigdo/)
Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.
If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.
Closes: #1103
Approved by: jlebon
With jigdo this is a very useful as one needs the ability to bind together the
RPMs and commits.
And for the compose tests this is a more reliable way to do caching anyways.
Closes: #1122
Approved by: jlebon
We originally needed the pkgcache to be a separate repo due to ostree's
overzealous pruning policies. The idea was to maintain multiple commits
in each pkg branch for different SELinux policies. In practice, there's
not much use in maintaining old copies and it's just easier to always
relabel on the fly. So then, the need for a separate repo completely
melts away.
This helps simplify the mental model a bit and allows us to avoid subtle
issues like #1047. Note however that the core is still capable of
handling split repos for the `--ex-unified-core` compose use case. Once
that and the jigdo work are a bit more settled, we can have a clearer
picture of how to simplify the core further.
The tricky bit is migrating the cache. When deploying, we check if a
pkgcache repo exists and migrate its refs if so. We then leave behind a
symlink to the system repo to remain compatible with older rpm-ostrees.
Closes: #1055
Approved by: cgwalters
Instead of just giving just the package that would be replaced, also
print out the package that replaces it, as well as the repo it comes
from.
Closes: #1113Closes: #1120Closes: #1121
Approved by: cgwalters
Noticed this while working on diff output somewhere else. I was
initially thinking of making use of this there too, but it's not quite
the right fit and would require a lot of churn.
Closes: #1121
Approved by: cgwalters
Rather than trying to be smart and re-use the same `GPtrArray`, let's
just segregate each check into its own scope to make it easier to
follow.
Diff best viewed with whitespace detection on.
Closes: #1121
Approved by: cgwalters
When writing this code, I made the false assumption that the nevra
string lives as long as the pool does, i.e. as long as we have a
reference to its `DnfSack`.
In fact, they have undefined lifetimes. Notably any place in which one
calls `dnf_package_get_nevra` a lot may result in the invalidation of
previously returned nevras.
This patch ensures that we copy the string in the few places where we
are susceptible to this.
There is a related libdnf patch[1] which tightens the definition here so
that we can assume the string at least lives as long as its
`DnfPackage`. It turns out that the callsites addressed in this patch
are also those in which we would break that assumption. IOW, this patch
is needed regardless of how [1] goes.
[1] https://github.com/rpm-software-management/libdnf/pull/388Closes: #1119
Approved by: cgwalters
Right now each ostree txn incurs a `syncfs()`; see
https://github.com/ostreedev/ostree/issues/1184
And before this patch, we were doing a txn per package import.
We can really do better in libostree - we'll fix that, but in the short term
let's use a bigger txn for every package. However, the obvious change here of
simply hoisting up the txn is that on failure for imports, we'd discard all
downloaded packages. We fix that by changing the auto txn API to have
a `commit_on_failure` boolean, and use it in cases where we're doing
imports.
This is prep work for jigdo, where we'll be using the import path all the time.
My bigger plan is to do multithreaded imports.
Closes: #1116
Approved by: jlebon
I think the `ex container` path supercedes this; it was really just a demo, and
having it around is annoying since I want to change the importer API and I have
to change it here too.
Closes: #1116
Approved by: jlebon
I was playing with `--download-only` a bit with an eye to
having something like this be used by Cockpit/gnome-software instead
of what it's doing now, but a problem is that at the moment we
don't have a way to reflect the "changed" state back to clients.
This is a first step towards that by simply printing a different
message.
I think really to make all of this work more nicely though, including
supporting e.g. rpm database diffs, we are going to have to instead
work on the [pending deployment](https://github.com/ostreedev/ostree/issues/545)
path. That way we'll have done the depsolve, stored repo timestamps
etc.; we'll be able to accurately show what *did* change rather than
try to recreate what will happen on the next `rpm-ostree upgrade --cache-only`.
Closes: #1118
Approved by: jlebon
Add editor option for kernel arguments. The logic
was taken from the ostree-commit editor.
The user now can edit the kernel arguments from an editor.
The synchronization related issue is resolved by reloading the
sysroot and see if the sysroot has been changed since the call
of the editor prompt.
Closes: #1097
Approved by: jlebon
This is for: https://pagure.io/atomic-wg/issue/387
Right now the way libostree stages objects into `${repo}/tmp` is basically an
anti-pattern for (possibly concurrent) operations on NFS. Having multiple
processes try to clean the tmpdir invites races, and there's really no reason to
"stage" all of the content.
(Unfortunately unless NFS supports `O_TMPFILE` we still need temp files,
but that's a separate issue)
In this patch we auto-detect NFS which should make the Fedora pungi runs "just
work", but I also added an environment variable to opt-in.
Closes: #1111
Approved by: jlebon
In the jigdo path we don't actually want to import the OIRPM literally
into ostree. I considered adding jigdo logic into `rpmostree-unpacker.c`
but it'd be a mess as the functionality is quite logically separate
from importing.
So split off an `unpacker-core.c` file which has the bare libarchive+RPM
helpers, and rename `RpmOstreeUnpacker` to `RpmOstreeImporter`.
Closes: #1110
Approved by: jlebon
We removed this in review, but I rediscovered why I added it. We
fail the `g_assert (sepolicy_matches)` if we already had packages
done with the final label when we go to reuse the cache.
(Basically, if we use the cache multiple times it's hard to avoid
relabeling all the time which is unfortunate...gets back a bit to
a way to annotate pkgcache commits as supporting multiple policies)
Closes: #1109
Approved by: jlebon
The "--ex" prefix here means it's an experimental option. A tremendous change
here is that start to support non-uid 0, but there are various things to fix there;
the unpacker for example needs to learn to set imported objects fully based
on the rpmfi information (i.e. default to uid 0, since libarchive gives the
current uid by default).
And even when run as uid 0, there are some bugs, though I'm not sure
of any showstoppers yet. For example, dracut's `dracut-install` calls
`cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs
from a checkout (it shouldn't be copying that anyways...)
Nevertheless, the infrastructure behind this really helps (is almost a hard
requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081).
Which is really only true due to SELinux - we need to import the packages,
then generate the final tree to get the final policy, then use that policy
to relabel all of the packages.
Closes: #940
Approved by: jlebon
In an unprivileged case, we can't do this on the real filesystem. For
`ex container`, we want to completely ignore uid/gid. I added a test
installing `httpd` which failed previously.
TODO: For non-root `--ex-unified-core` we need to do it as a commit modifier.
Closes: #940
Approved by: jlebon
It seems that libarchive ends up returning `getuid()` actually,
possibly because the cpio doesn't actually have ownership information?
Anyways, what we really want here is to set `0/0`, which is what
happens for the `ex container` path via
`OSTREE_REPO_COMMIT_MODIFIER_FLAGS_CANONICAL_PERMISSIONS`.
Prep for unified core 🌐.
Closes: #940
Approved by: jlebon
Not immediate prep for anything, I'd just been looking at this
code recently.
(Also a bit of `glnx_throw_errno` porting)
Closes: #1107
Approved by: jlebon
More work for unified core; this was one of the biggest parts of the `ostree
diff` when comparing a unified core treecompose of Fedora Atomic Host versus
the libdnf path.
This is also the first case where we're teaching the pkgcache to redownload
based on state.
Closes: #1105
Approved by: jlebon
Previously in 4a86bdd196 "postprocess: Undo Fedora SELinux policy move"
we fixed things for SELinux policy to move it out of /var and back
into /etc.
But for unified core where we're importing the RPM, the default "/var →
tmpfiles.d" kicks in before this can run. So teach the unpacker to do it too.
This is even less nice than what we had in the postprocessing code, as there we
went to some effort to handle non-targeted policy (e.g. MLS). That said I am not
super concerned about that right now...we can always deal with MLS or other
policies later. I'd be more inclined really to try to take the issue of moving
back to /etc (or implementing /usr with /etc overrides) to SELinux upstream.
Closes: #1105
Approved by: jlebon
For unified core work, we'll be caching the "has docs" flag in the pkgcache
commit. So let's load the commit object earlier rather than having each function
load it.
Closes: #1104
Approved by: jlebon
Avoid the `goto` by initializing the output variables directly early, so we can
do a full new style consistently. Prep for unified core work.
Closes: #1104
Approved by: jlebon
Prep for unified core 🌐. First, right now `shadow-utils` always opens those with
`O_RDWR`, so let's break those links even if we don't have `usr/lib/passwd`.
Also, while not strictly requried, let's also handle the `merge_passwd_dir`
being `NULL`, as that happened when I was trying a variant of this patch.
Closes: #1098
Approved by: jlebon
Prep for unified core 🌐. Here we don't have `/usr/lib/passwd` at the start, but
we still want to load the `usr/etc` copies for processing rpmfi overrides.
Closes: #1098
Approved by: jlebon
Prep for unified core 🌐. On the treecompose path we inject
`/usr/etc/passwd`, and trying to overlay the setup package on top
with the default `UNION_IDENTICAL` fails.
This is obviously a hack; but I can't think of a way to do this significantly
better. (We could do a DB query for the owner of `/etc/passwd`, but eh.)
Closes: #1098
Approved by: jlebon
I was trying to figure out why:
rpm-ostree compose tree --repo repo/ manifest.json
would result in:
error: opendir(manifest.json): No such file or directory
It turned out to be because we had `--repo` in *both* the `install`
options and the `commit` options. This makes sense since both these
subcommands need to be given a repo. However, in the `tree` case, we
were adding both arrays, which meant we inherited two `GOptionEntry`s
for `--repo`. This confused glib2 which consumed not one but two CLI
arguments when looking for the argument associated with `--repo`.
Our CI didn't notice this because it uses the `--repo=foo` notation,
which doesn't throw off glib2.
Fix this by factoring out the `--repo` option into a separate array so
that in the `tree` case, it only gets added once. Exercise the fix in CI
by using the two argument notation for `--repo`.
Closes: #1101
Approved by: cgwalters
