IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This is the equivalent of the 'removefrom' verb in Lorax's templating.
It's a lot more robust than a generic "rm-rf" type thing, because most
often you only want to remove files from particular packages.
It seems clearer to me if all code which is operating on the content
(such as enabling systemd units) is under -postprocess. The "compose
tree" code should be tying everything together and calling out to
sub-components.
This is prep for adding more postprocessing code.
We might as well do what systemd does and have a big header which
defines all of them, to more conveniently share them for libraries
that don't include them (like hawkey/librepo, as well as things that
libgsystem doesn't yet cover).
For some reason we ended up defining our own macro to define cleanup
functions, when libgsystem has a usable one. Switch to that, fewer
lines of code, more consistency.
I was looking again at using hawkey/librepo, and realized just how
much I'd have to fight all of these libraries to avoid affecting
the running system.
What we really want to do with librepo/hawkey is run them effectively
unprivileged, and to hide the system's RPM database from them. This
is a baby step towards that, by confining our existing yum.
- /usr, /etc, and /var/lib/rpm are mounted read-only
- yum is now run under CLONE_NEWPID, to avoid stray %post scripts
affecting system processes
This is taking us closer to deeper integration in the treecompose side
with RPM instead of forking out to things.
It works except...we end up with the dreaded __db.001, .dbenv.lock
files =/ Best option would be to teach RPM how to open a database
really read-only. Failing that, could use the immutable bit?
These lock and dbenv files are created by RPM when it accesses the
database read-write, but we only want read-only. Sadly that turns out
to be very, very hard to plumb through all the layers in
hawkey/libsolv etc.
So let's just forcibily remove them after postprocessing.
Since we're only quering the database once (and at least
one of the locations is going to be read-only), don't
create the cache.
Noted while just reading the code while working on something else.
I'd been resisting this for a long time - I really wanted the tree to
be a reflection of the packages; and not go down the path of
"forking".
The lorax model has shown that while you can definitely get a large
space reduction that way, you're going to be perpetually chasing
changes in the packages. No matter how good your templates are.
Furthermore, lorax is just to generate the installer - it's an OS that
runs a single app. Whereas here we're generating the target runtime
system; we can't add bugs.
Nevertheless, reality is that sometimes it's just too hard to change
the input package set - there's a risk of breaking things. Namely,
we're introducing a new update system here, but obviously there's a
previous one: yum. Upstream packages keep growing a dependency on it.
Note this patch allows *not* removing all files from the package,
because it's possible that other things (e.g. subscription-manager)
import it as a library.
So in the meantime while we're iterating on this, let's support:
"remove-files": ["usr/bin/yum"],
The code is generic beyond yum for obvious reasons, but I don't
think we should use it for a lot more than that.
Having content in /boot in OSTree was always ugly, because we ended up
mounting over it in the deployment location at boot.
This was even worse in the anaconda rpmostreepayload code, because of
the juggling of the mount point that needed to take place.
Trying to add a GRUB2 backend to OSTree is what finally forced this
change. Now, we put kernels (in the tree) by default in *both* /boot
and /usr/lib/ostree-boot.
OSTree itself knows to look in both locations. Anaconda is going to
just hard require trees with the new location though.
This would probably be best if we invoked systemctl in the compose
tooling, but at the moment we don't have any execution of target code
on the host. It's fine to assume that it's in /usr/lib.
We can revisit this if we start doing chrooted/containerized
execution.
"atomic rebase" is mostly a copy of "ostree admin switch", so let's
also pick up the changes in ostree admin switch for the new
unconfigured state flag.
This allows a user to "atomic rebase" on an unconfigured system.
Related: #31
For the test suite, we're doing an upgrade offline of another OS root.
What we should maybe do is print the package diff from the merge
deployment, if not the booted one.
However, I think this is OK for now.
Some downstreams want the ability to separate the compose tooling from
the client, for e.g. support reasons.
This approach supports generating a tarball without the source for the
compose command, and requires specifying a config option to disable
it.
Since the treefile format now supports includes, we would need to
either include the whole chain, or just the expanded portion we use in
the compose. This patch does the latter.
This should allow a client to take the same treefile and generate a
similar tree (if they want to reproduce with the same RPMs, those can
be extracted from the RPM database inside the tree).
This improves on the check-diff option by only downloading the
/usr/share/rpm directory to do a package diff. This prevents downloading
the whole deployment and the necessity to do a cleanup later.
As a developer, a workflow I have for testing things is to create
an RPM, toss it into a local yum repository, then do a compose.
However at the moment to add the local overrides I have to edit the
treefile, which is annoying. Let's add a commandline override for
this.
Note this also deletes the old "repos_data" code which was not being
used.
In 827e711 we stopped running two yum transactions. This means the
code that detects if the repodir exists in the yum installroot will
always return false and the code is dead.
These match OSTree. There are a variety of use cases here. One is
for test suites; we can stand up a temporary sysroot directory, and
operate on content inside there.
Another is doing virtual machine upgrades offline from a host system,
or upgrading a different OS.
The duplication here is a bit unfortunate; if we add a lot more
commands we should revisit this and perhaps have a common option
group.
I'm working on a patch to add --sysroot support, and for testing
OSTree stuff I often work from a "traditional" host workstation, and
then I have OSTree content in /.
This makes "atomic status" just work for that; I simply have no booted
deployment.
API probably needs to move to ostree library. Allows us to get a "list"
of REFSPECs, Eg.
"<newest>..<oldest>"
...as a REFSPEC will turn into a list of all the commits, inclusive.
It currently has the following sub-commands:
diff COMMIT COMMIT
for rpmtree diff.
list [prefix...] COMMIT...
for "yum list" like command.
version COMMIT...
for "yum version" like command.
...bunch of FIXME's, UI output isn't great, needs docs.
We also don't use the same code as the treediff on upgrade atm.
I had an epiphany today while working on
https://bugzilla.redhat.com/show_bug.cgi?id=1098304 - I realized that
I can just do an install, and then copy over everything except the
root entries from /etc/passwd into /usr/lib/passwd.
No need for a patched shadow-utils. No need to modify the
/etc/nsswitch.conf before doing the install root. It totally works.
I have no idea why I originally overcomplicated this.
The thing that sucks a bit about this code is that I have to drop to
the FILE * APIs so that I can use the glibc APIs for processing
group/shadow.
Also, the way I deduplicated the code paths for processing
passwd/group is crappy, but I think it's better than duplicating them
(as systemd-sysusers does).
The good: We don't need a two-step RPM transaction, we don't need
a patch for shadow-utils, it's just saner
The bad: Code is not the most beautiful? Not really bad.
The ugly: I didn't think of this in the first place and spent
months beating my head against the wall of shadow-utils...
Per http://fedoraproject.org/wiki/Features/RemoveSETUID
many programs (famously /usr/bin/ping) switched to using
"file capabilities" instead of setuid.
rpm-ostree had code to specially handle SELinux labels, but was
discarding other xattrs. Let's just whitelist reading in
security.capability.
Closes#14
They get deleted anyways since we blow away the tmpdir, but we expect
people to be using a local HTTP proxy. We stopped trying to cache
packages internally as that caused conflicts with the yum lock with
concurrently executing composes.
After a discussion on what the status command should offer a
sysadmin, we decided to try a column output format similar to that
of systemctl or docker images. This will print the timestamp,
a truncated checksum, the osname, and the refspec of each
deployment. It also includes a --pretty option to print
the information formatted in rows with the full checksum and a
title to specify the default and the rollback target.
This includes several changes to format the code to follow GNU Coding Standards as per
the restof rpm-ostree. This includes fixing indentations, adding spaces between
identifers and parentheses, replacing "goto print" with an else structure, and
reformatting print statements into complete sentences for internationalization.
This allows administrators to configure between deployments and easily see which deployment
they are booted into as well as indicating which is chronologically most recent. This makes
the process more user-friendly, rather than requiring the user to remember which deployment
checksum corresponds to the most recent upgrade.
Currently the systemd RPM ships with default.target ->
graphical.target, which is either itself changed by Anaconda (via
parsing /etc/sysconfig/desktop, which...anyways let's stop here).
Or anaconda might set it directly to multi-user.target.
For rpm-ostree, we perform some minimal level of "preconfiguration"
per tree, so they are directly usable without an intervening
installer.
As an example for fedora-atomic/base/core, we just want
multi-user.target. Thus, this patch provides the treefile author a
declarative mechanism to set it.
Currently on an Atomic compose, I'm seeing abrtd trying to write to
/usr/share/rpm/.dbenv.lock, which is denied by policy because it's
usr_t. There are multiple ways to address this, but there's no good
reason to leave the lock files and __db* files around.
rpm appears to operate correctly without them if calling process
merely gets EROFS.
This is exactly the code from "ostree admin switch", except it's
called "rebase" because in the future it will also carry along any
locally layered packages.
And do the same for "sign". This way we can have the compose server
utilities cleanly separated from what most people will see, which is
the client side tools.
The way this works is still fairly naive in that we hash in two
inputs:
1) The treefile JSON
2) The result of rpm -qa
If both of those are a hit, we reuse the existing commit.
Otherwise it just doesn't work. Note this hack is only necessary for
the case where some package pulls in e.g. fedora-release in the
initial transaction set when we're trying to install nss-altfiles.
This will move to a separate repository. This allows a clearer
separation between the core tool (which is shipped on client systems
too), and the compose infrastructure.
Furthermore, I want to make the autobuilder a Docker container.
It's a lot clearer if the inputs, outputs, and cache state are cleanly
separated. At least the "lorax" tool relies on a local HTTP cache
instead of keeping around the yum repos - let's do the same.
This commit causes treecompose to require a --repo argument, and it
also gains an optional --proxy argument.
This is just the start of moving more functionality into rpm-ostree,
with the goal of more tightly binding the rpm and ostree worldviews.
For exmaple, supporting package installation on top, showing the
package-level diff between trees, etc.
The web page previously bounced out to the "fedora-atomic" component -
but we really want a bit of web UI that's shared between products.
Like how Koji is a generic frontend.
In particular, this now comes with a start of a generic "repoweb".
For convenience, allow users to create an "overrides/" directory which
is a repo. If it exists, the builder automatically adds it.
(However, we should likely disallow distributing these builds on
the network)
We use the new unified OSTree API (OstreeSePolicy) to perform
labeling, rather than having our own here.
Also create a new rpm-ostree-relabeling-helper that is run to label
any leftover files such as /etc/fstab that we create offline, and also
to relabel the entire disk.
Unfortunately when composing from RHEL7 a current rawhide tree, the
PCRE version mismatch causes none of the regexps to match...
Work around this by temporarily moving the .bin files out of the way.
This is really a hack though.
The real fix is to change libselinux to check pcre_version().
These differ each run, polluting upgrades. James says we don't need
them anyways.postprocess: Remove var_uuid files from yumdb
These differ each run, polluting upgrades. James says we don't need
them anyways.
This is somewhat similar to what we've been doing with Continuous; we
take the manifest.json, and turn it into a "snapshot". Except here
there is a notion of inheritance.
This gets stored into the tree as /usr/share/rpm-ostree/treefile.json.
Additionally, it goes into the autobuilder directory in
products-built.json. Though really we should split up that file,
since it will be kind of...large.
It turns out OS X has bzip2 but not xz installed by default, so let's
use that instead. QCOW2 stays as xz since we can assume GNU/Linux
users have it better.
We really want people using these trees to know what went in it. If
the tree doesn't actually have /usr/bin/rpm inside it, then do an
"rpm -qa | sort" and dump that into: /usr/share/rpm-manifest.txt
This will help our story with GPL compliance too.
OSTree really wants /var to come empty; it's not going to be practical
to change every RPM right now though. So, let's dynamically turn the
content from /var into systemd-tmpfiles.d snippets.
This allows us to skip the whole install process if the
RPMs haven't changed, which is a rather large win.
Part of this commit involves some code to attempt to talk to "yum
shell" for dependency resolution, in an attempt to avoid depsolving
twice, which just isn't really going to work, since "yum shell" isn't
an API as it turns out.
The only other real alternative is doing something PackageKit-like,
which is ~4000 lines of very nontrivial Python.
There are two major reasons:
1) I want to do things like process SELinux labels here, and that
type of thing is best done in C.
2) There are presently 3 languages in this code, and this takes us
down to just two.
At the moment, a lot of RPM content has files whose content derives
from timestamps (like .pyc files). We really don't need to do another
commit to the repository if it's just these files which changed.
Fix this by caching the package list at the rpm-ostree level, and
don't commit if it hasn't changed.
We're moving more towards the autobuilder model; this allows us to
keep all of the data in one place. Important for the
cloud.fedoraproject.org instance since /mnt/ostree is the large
filesystem.
The two transaction thing is bad for multiple reasons, but what bit me
specifically was something in the group transaction pulling in
'kernel-debug', and then installing 'kernel' later.
