IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Bodhi points rpm-ostree at the "gold" Fedora repo via `file:///`, and
libhif is brokenly checking the mtime on `file://` repos.
Work around that here by just ignoring cache ages, because at present
we don't actually cache really - we drop the RPMs in the tempdir.
(Long term having actual caching of the RPMs would be nice, but
we can revisit this when we get there)
Closes#156
Someday we'll enhance RPM; see
https://github.com/rpm-software-management/rpm/issues/8
But anyways right now at least a few people have tried,
and we get all the way past downloading packages and then bomb
out when doing the actual transaction with a useless error
message.
So let's be up front about this.
This change causes the subcommands to be sorted alphabetically when
'rpm-ostree' is invoked without an argument or with '--help'. This
matches the behavior of the 'ostree' command.
This API was added to libhif a while ago. I'm adding this now because
libhif was changed recently to write the dnf copy of the yumdb, and we
didn't have code to remove it explicitly.
It's better not to write it in the first place.
We noticed that tree composes failed with a missing `atomic` package,
when really what happend is Fedora 22 was released and the repo
metadata moved from `development/` to `released/`.
See https://github.com/hughsie/libhif/pull/47
- Can also give you a file descriptor
- Takes a constant string as input, returning a mutated string as a
separate variable which means that one can check whether the variable
is `NULL` to know whether or not one needs to `rm -rf` it on error
paths.
The refsack code was using the latter, and it stood out. Now that
we're making use of explicit export markers, there's no need to uglify
internal APIs with a leading `_`.
While `rpm-util.c` may not best describe this, it's where most
of this code is ending up. Let's further centralize things.
We more consistently return an `RpmOstreeRefSack` instead of a
`HySack`, where the former supports refcounting and knows how to clean
up its temporary directory if it was allocated from a commit.
And now, finally the actual goal is achieved. \o/
Only one code path dealing with extracting the rpm database from an
OSTree commit.
An astute reader would notice that the `root` member of the struct was
actually only necessary as of a few commits ago. But said astute
reader would also realize it's kind of late in the evening and not
worth rebasing it to where it would properly go.
hawkey and libsolv are both patched to look in `usr/share/rpm` if the
db can't be found in `/var/lib/rpm`. However, librpm itself isn't.
One *can* override it with a macro...which is process global. Yuck.
Needs fixing.
Anyways, we can just make a symlink. That's a lot easier than writing
a patch for librpm and waiting a billion years to be able to use it
everywhere we care about.
This will help unify the librpm tempdir code with the hawkey tempdir
code.
As far as I can tell, this is basically a way to specify the temporary
directory. That significantly complicates the code as it now
needs to keep track of whether or not it owns the temporary directory.
This hinders unifying this code with the hawkey query path.
Because of this, and since I'm not aware of a use case for specifying
this tempdir, let's remove it.
It was only used to access the yumdb, which we don't use because:
- It badly exacerbates the OSTree one-HTTP-request-per-object issue
- We're assembling multiple repos on the server side, so things like
who took the action aren't relevant.
But the reason I did this patch at the moment is because I want to
unify the code that's creating tempdirs from commits so we can feed
real files to librpm.
For a future patch, I want to add an API to get an rpmts for a commit,
instead of a hawkey Sack, because libsolv doesn't expose some
optimized queries that we can get by just going directly to librpm,
such as package file owners.
We had `src/lib` having its own little private library; I wanted to
use some of it inside `src/libpriv`, so let's consistently have all
private utility code in `src/libpriv`.
Closes: https://github.com/projectatomic/rpm-ostree/pull/147
Equivalent to hy_package_cmp(), but works for comparing packages from
different memory pools.
This a stop gap measure in light of
https://github.com/rpm-software-management/hawkey/pull/90
If and when that pull request gets merged, then this function could at
some later point simply call hy_package_cmp().
This is a step forward to deduplicating; the client tooling now calls
into the public API for diffs, rather than using the older internal
function.
Note: this patch also links the client against the public library.
This is a *third* implementation of rpm database diffs in the code,
but it is now a public introspectable shared library API.
Further commits will change the command line tools to use this, and
then after that we'll further deduplicate the `db diff` from this
code.
On the plus side, we share some code between the library and the
binary now. On the downside, because `librpmostreepriv.la` is a
noinst library, its code text is duplicated between the shared library
and binary, at least until we either:
- Have the binary solely use the public shared library (like ostree does)
- Install `librpmostreepriv.so` to e.g. `/usr/lib64/rpm-ostree/librpmostreepriv.so`
without the headers being public
We presently have 3 internal code paths that are doing rpmdb
inspection. This conversion to fd-relative for one of them is a
generic cleanup preparatory to de-duplicating.
Note this bumps libglnx to include
381ca54ee3
The file is automatically populated by systemd when it is empty.
Apparently it doesn't work when the file is missing (as of
systemd-219-9.fc22).
https://bugzilla.redhat.com/show_bug.cgi?id=1198700
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
In pretty mode (--pretty), print signatures for each listed deployment.
Otherwise, just print signatures for the booted deployment at the end to
preserve the tabular formatting of the deployment list.
The `QueryResult` class ended up being too awkward; having NEVRA
strings meant for example that clients would have to parse them. It
would be harder to present something like the current `rpm-ostree
upgrade` package diff output.
Now...I debated quite a while before doing this patch. The thing
that's really awful about creating this library is there are *SO MANY*
layers. rpm-ostree → libhif → hawkey → libsolv → librpm. It's enough
to make one question whether one is actually accomplishing anything or
just contributing to a collective insanity...
Let's pretend for now it's the former.
Closes: https://github.com/projectatomic/rpm-ostree/pull/136
This will help build release engineering and other types of tools;
for example, rather than parsing the output of `db diff`, one
should be able to call an API.
Initially, this adds the generic infrastructure for a public shared
library, with a new function call to do the equivalent of `rpm -qa` on
a particular OSTree commit.
Closes: https://github.com/projectatomic/rpm-ostree/issues/117
Closes: https://github.com/projectatomic/rpm-ostree/pull/124
It breaks at least `/etc/resolv.conf` inside Docker. The right thing
here is to run all things involving networking (librepo) as an
unprivileged process with different constraints than the rpm installs.
We currently have an internal-only library, but the sources for it are
in the same dir as the app. For future work on a public shared
library, we'll need a clearer source structure.
Start by just renaming the app files into `src/app/`, and the internal
private library into `src/libpriv/`, with the appropriate
`Makefile.am` changes.
Closes: https://github.com/projectatomic/rpm-ostree/pull/123
The high level goal is to deprecate libgsystem. I was trying to share
code between ostree/rpm-ostree, but it was too painful to commit to
forver frozen ABI for new utility APIs.
The git submodule approach will much more easily allow breaking
API/ABI, and iterate on APIs until they either land in GLib or not.
Note that libglnx will not use GFile*, so a full port to it will
involve also not using that. Thus, it will be necessarily
incremental; in the meantime we'll link to both libgsystem and
libglnx.
This is a convenient way for external tools to integrate rpm-ostree
better into automation. With the caching, it's now cheaper to
just rerun rpm-ostree if you know an input yum repository changed,
or just on a timer.
Then one can use this flag to determine whether or not to launch any
further processes after rpm-ostree, such as disk image generation,
test suites, etc.
With yum, we would have had to make a custom Python app to cleanly
separate the fetch metadata/depsolve phases from installation.
Now that libhif/hawkey gives us that, make use of it by exiting after
depsolve if the previous compose has the same package set, and the
treefile is the same. This saves a fairly substantial amount of time
and I/O, and makes it much more palatable to simply run the compose
tool on demand in response to say repo regeneration notifications.
A further important note; --cachedir is no longer used; we store the
inputhash in the OSTree commit metadata itself.
There are a lot of advantages to this. See the linked issue
for more details, but briefly:
- Lays the groundwork for package layering
- Better caching (exit compose after depsolve if no changes)
- Better error handling
- Potential for unprivileged package downloads
- Potential to better containerize installs
TODO:
- langs handling
- progress output
Closes: https://github.com/projectatomic/rpm-ostree/issues/53
Maybe we should make this into an explicit error, but anyways
I somehow ended up with an empty /usr/etc/passwd in the tree
contents, I think due to bugs in earlier work there.
This causes fmemopen to return EINVAL, which errors out the
compose. Let's stumble forwards here.
Now that I think about it, it might be a valid case to have an
existant but empty /usr/etc/passwd in the tree - when we migrate to
systemd-sysusers, I think we'll want an empty file there by default.
Due to an intersection of #79 and #69, we ended up continually
accumulating copies in /usr/lib/{passwd,group}. The fix here is to
deduplicate when constructing the temporary /etc/passwd that the RPM
install will operate on.
Closes: https://github.com/projectatomic/rpm-ostree/issues/92
Pure code motion; no functional changes. Trying to get all of the
passwd/group code in the same place so I can fix bugs in the
interaction between them more easily.
I'm planning to replace the caching code with something that inspects
the previous commit rather than a lookaside cache, so raise this code
up to a higher level.
I swear I tested this, but anyways
https://github.com/projectatomic/rpm-ostree/pull/79
wasn't quite right. We need to look at /usr/etc/{passwd,group}
for previous data.
We happily noticed there was no /etc/passwd in the tree, then
proceeded to do the merge and split again, with the result
of an empty /usr/etc/passwd in the new tree.
That in turn resulted in an empty /etc/passwd in an installed system,
i.e. with no "root" user, with obvious bad consequences, namely in my
case crashing Anaconda.
(Yes, I will write a testsuite for this)
This matches recent work in OSTree to use *at() - it's faster and less
prone to error. In the case of directories which are mutable by
processes in different security domains, it's more secure too. (That's
not the case here though).
I sometimes run "rpm-ostree compose tree" directly, mainly so I can
use gdb and/or nonstandard options. In this case I don't get
version numbers injected.
That happens to trigger a bug in this code.
The checking code from #56 landed, and started triggering for me on
the `dockerroot` user. It's nice to know it works. Then the issue
is... "what now"?
It turns out in the case of `dockerroot` it's actually unused, so we
could fix this by deleting it. But in general we need to support
dynamic uids/gids/. And we can't yet take a hard dep on #49.
So this patch changes things so we take a copy of the passwd/group
data from the previous commit. Any users subsequently added in the
*new* commit will be additive.
Closes: https://github.com/projectatomic/rpm-ostree/issues/78
Remove redundant function _rpmostree_pull_progress().
Bumped ostree requirement to 2014.13, but this isn't quite right because
we actually need (unreleased) 2014.14. Post-release version bumps would
be useful here.
Verify uid/gid on files, directories and symlinks
Just output a msg when user/group is removed with no files
json-parsing: Add functions for strictly dealing with ints
passwd/json: Add simple scripts to convert passwd/group files to json data
docs: Check-passwd/groups and ignore-remove-users/groups JSON config. entries
It is confusing if you pull from an unversioned tree and suddenly
status loses versions. This comes at a cost of increased horizontal
space.
Closes#73
From #74
Letting GErrors pile up is wrong; either exit the function or clear the
GError (perhaps with a console warning).
In this case we tolerate missing database files (because users are free
to delete them), and build the checksum from whatever data is available.
Seems weird, but that's how I'm told it's meant to work. Bail out on
any other type of error.
Otherwise, upgraded systems which have modified /etc/group (by e.g.
adding a human user), will actually see the group drop out with bad
consequences.
It's harmless to have it in both, /etc will override /usr.
Fixes#67
We're building generic initramfs images on the server side, but dracut
has logic to pick up some things from the host, like filesystems.
In the absence of host-specific initramfs images, it needs to be up to
the generating system what kernel modules end up in the initramfs.
Provide a generic option to passthrough dracut arguments.
As a followup to renaming the "rpm" command to "db", split the "db"
subcommands into separate source files in the style of "ostree admin"
and "rpm-ostree compose".
Also create rpmostree-rpm-util.[ch] as a place for common rpm-related
functions needed by the "db" subcommands.
No intentional functional changes here, just a bunch of copy-n-paste
and minor cleanup.
Eliminates some confusion between "rpm-ostree rpm" (or "atomic rpm")
commands versus actual "rpm" commands.
The "rpm" subcommand is retained as a hidden alias for the "db"
subcommand for backward-compatibility. It is not listed in --help
output.
Fixes#22
Starting with pkg_yumdb_strdup(), the GError parameter is unused and
therefore (arguably) the GCancellable parameter is also not needed.
Remove them both, and clean up other functions that now have unused
parameters as a result.
Note that none of the callers of these functions were checking for
errors anyway.
Similar to the previous commit, but the "rpm" command needed some extra
attention.
I stopped short of splitting the subcommands into separate files (like
for "compose"), but refactored the parsing as though they were separate.
Refactor command-line parsing to better utilize GOptionContext. This
eliminates most of the manual parsing and global options are now shown
in the help output.
Some of the changes here are not strictly necessary for rpm-ostree,
but are done for consistency with ostree's command-line parsing.
The "rpm" subcommand needs some extra attention, so that's been split
into a separate commit.
=/
Originally it was "container_disabled" but the double negatives
started being awkward, I missed converting this negation.
This should really make us work again on RHEL6.
The current motivation for this is that
https://github.com/fedora-infra/fedmsg-atomic-composer
started using mock --new-chroot (which uses systemd-nspawn) to run
rpm-ostree, which in turn uses systemd-nspawn to run the post script.
Now systemd-nspawn is not really nestable (it wants to link up
journald, resolv.conf handling, etc).
First, dropping nspawn and going to raw containers fixes the nesting
problem.
Second, we don't need all the features of systemd-nspawn. We are ok
with log messages going to stdout, and we don't use networking, so no
resolv.conf is needed.
Third, this sets a bit of a stage for more sandboxing internally when
run on real systems. I already have a prototype branch which runs
librepo as an unprivileged user, that could be combined with this for
even stronger security.
Why not use systemd? Well...I'm still debating that. But the core
problem is systemd isn't a library in the C sense - to use its
sandboxing features we have to use unit files. It's harder to have a
daemon that looks like a single service from a management perspective,
but uses sandboxing internally.
"rpm-ostree compose {tree,sign} --help" acted as if no subcommand was
given. But the compose subcommands each have a good number of options
so it's particularly important to list them.
Currently adding human users to a system group such as 'wheel' does
not work with shadow-utils as it exists now. This is admittedly
a hack; basically we single out "wheel" as going in /etc/group, via:
"etc-group-members": ["wheel"],
A more comprehensive solution to this will be:
https://github.com/projectatomic/rpm-ostree/issues/49
Originally I was trying to have all of the treefile parsing
in treecompose, and then call other functions as API. But that
turns out to be unrealistic. We'll need finer grained control
over this in the future.
Therefore, let's move the boot-location handling down, in preparation
for further commits which parse the treefile in the commit phase.
This is the equivalent of the 'removefrom' verb in Lorax's templating.
It's a lot more robust than a generic "rm-rf" type thing, because most
often you only want to remove files from particular packages.
It seems clearer to me if all code which is operating on the content
(such as enabling systemd units) is under -postprocess. The "compose
tree" code should be tying everything together and calling out to
sub-components.
This is prep for adding more postprocessing code.
We might as well do what systemd does and have a big header which
defines all of them, to more conveniently share them for libraries
that don't include them (like hawkey/librepo, as well as things that
libgsystem doesn't yet cover).
For some reason we ended up defining our own macro to define cleanup
functions, when libgsystem has a usable one. Switch to that, fewer
lines of code, more consistency.
I was looking again at using hawkey/librepo, and realized just how
much I'd have to fight all of these libraries to avoid affecting
the running system.
What we really want to do with librepo/hawkey is run them effectively
unprivileged, and to hide the system's RPM database from them. This
is a baby step towards that, by confining our existing yum.
- /usr, /etc, and /var/lib/rpm are mounted read-only
- yum is now run under CLONE_NEWPID, to avoid stray %post scripts
affecting system processes
This is taking us closer to deeper integration in the treecompose side
with RPM instead of forking out to things.
It works except...we end up with the dreaded __db.001, .dbenv.lock
files =/ Best option would be to teach RPM how to open a database
really read-only. Failing that, could use the immutable bit?
These lock and dbenv files are created by RPM when it accesses the
database read-write, but we only want read-only. Sadly that turns out
to be very, very hard to plumb through all the layers in
hawkey/libsolv etc.
So let's just forcibily remove them after postprocessing.
Since we're only quering the database once (and at least
one of the locations is going to be read-only), don't
create the cache.
Noted while just reading the code while working on something else.
I'd been resisting this for a long time - I really wanted the tree to
be a reflection of the packages; and not go down the path of
"forking".
The lorax model has shown that while you can definitely get a large
space reduction that way, you're going to be perpetually chasing
changes in the packages. No matter how good your templates are.
Furthermore, lorax is just to generate the installer - it's an OS that
runs a single app. Whereas here we're generating the target runtime
system; we can't add bugs.
Nevertheless, reality is that sometimes it's just too hard to change
the input package set - there's a risk of breaking things. Namely,
we're introducing a new update system here, but obviously there's a
previous one: yum. Upstream packages keep growing a dependency on it.
Note this patch allows *not* removing all files from the package,
because it's possible that other things (e.g. subscription-manager)
import it as a library.
So in the meantime while we're iterating on this, let's support:
"remove-files": ["usr/bin/yum"],
The code is generic beyond yum for obvious reasons, but I don't
think we should use it for a lot more than that.
Having content in /boot in OSTree was always ugly, because we ended up
mounting over it in the deployment location at boot.
This was even worse in the anaconda rpmostreepayload code, because of
the juggling of the mount point that needed to take place.
Trying to add a GRUB2 backend to OSTree is what finally forced this
change. Now, we put kernels (in the tree) by default in *both* /boot
and /usr/lib/ostree-boot.
OSTree itself knows to look in both locations. Anaconda is going to
just hard require trees with the new location though.
This would probably be best if we invoked systemctl in the compose
tooling, but at the moment we don't have any execution of target code
on the host. It's fine to assume that it's in /usr/lib.
We can revisit this if we start doing chrooted/containerized
execution.
"atomic rebase" is mostly a copy of "ostree admin switch", so let's
also pick up the changes in ostree admin switch for the new
unconfigured state flag.
This allows a user to "atomic rebase" on an unconfigured system.
Related: #31
For the test suite, we're doing an upgrade offline of another OS root.
What we should maybe do is print the package diff from the merge
deployment, if not the booted one.
However, I think this is OK for now.
Some downstreams want the ability to separate the compose tooling from
the client, for e.g. support reasons.
This approach supports generating a tarball without the source for the
compose command, and requires specifying a config option to disable
it.
Since the treefile format now supports includes, we would need to
either include the whole chain, or just the expanded portion we use in
the compose. This patch does the latter.
This should allow a client to take the same treefile and generate a
similar tree (if they want to reproduce with the same RPMs, those can
be extracted from the RPM database inside the tree).
This improves on the check-diff option by only downloading the
/usr/share/rpm directory to do a package diff. This prevents downloading
the whole deployment and the necessity to do a cleanup later.
As a developer, a workflow I have for testing things is to create
an RPM, toss it into a local yum repository, then do a compose.
However at the moment to add the local overrides I have to edit the
treefile, which is annoying. Let's add a commandline override for
this.
Note this also deletes the old "repos_data" code which was not being
used.
In 827e711 we stopped running two yum transactions. This means the
code that detects if the repodir exists in the yum installroot will
always return false and the code is dead.
These match OSTree. There are a variety of use cases here. One is
for test suites; we can stand up a temporary sysroot directory, and
operate on content inside there.
Another is doing virtual machine upgrades offline from a host system,
or upgrading a different OS.
The duplication here is a bit unfortunate; if we add a lot more
commands we should revisit this and perhaps have a common option
group.
I'm working on a patch to add --sysroot support, and for testing
OSTree stuff I often work from a "traditional" host workstation, and
then I have OSTree content in /.
This makes "atomic status" just work for that; I simply have no booted
deployment.
API probably needs to move to ostree library. Allows us to get a "list"
of REFSPECs, Eg.
"<newest>..<oldest>"
...as a REFSPEC will turn into a list of all the commits, inclusive.
It currently has the following sub-commands:
diff COMMIT COMMIT
for rpmtree diff.
list [prefix...] COMMIT...
for "yum list" like command.
version COMMIT...
for "yum version" like command.
...bunch of FIXME's, UI output isn't great, needs docs.
We also don't use the same code as the treediff on upgrade atm.
I had an epiphany today while working on
https://bugzilla.redhat.com/show_bug.cgi?id=1098304 - I realized that
I can just do an install, and then copy over everything except the
root entries from /etc/passwd into /usr/lib/passwd.
No need for a patched shadow-utils. No need to modify the
/etc/nsswitch.conf before doing the install root. It totally works.
I have no idea why I originally overcomplicated this.
The thing that sucks a bit about this code is that I have to drop to
the FILE * APIs so that I can use the glibc APIs for processing
group/shadow.
Also, the way I deduplicated the code paths for processing
passwd/group is crappy, but I think it's better than duplicating them
(as systemd-sysusers does).
The good: We don't need a two-step RPM transaction, we don't need
a patch for shadow-utils, it's just saner
The bad: Code is not the most beautiful? Not really bad.
The ugly: I didn't think of this in the first place and spent
months beating my head against the wall of shadow-utils...
Per http://fedoraproject.org/wiki/Features/RemoveSETUID
many programs (famously /usr/bin/ping) switched to using
"file capabilities" instead of setuid.
rpm-ostree had code to specially handle SELinux labels, but was
discarding other xattrs. Let's just whitelist reading in
security.capability.
Closes#14
They get deleted anyways since we blow away the tmpdir, but we expect
people to be using a local HTTP proxy. We stopped trying to cache
packages internally as that caused conflicts with the yum lock with
concurrently executing composes.
After a discussion on what the status command should offer a
sysadmin, we decided to try a column output format similar to that
of systemctl or docker images. This will print the timestamp,
a truncated checksum, the osname, and the refspec of each
deployment. It also includes a --pretty option to print
the information formatted in rows with the full checksum and a
title to specify the default and the rollback target.
This includes several changes to format the code to follow GNU Coding Standards as per
the restof rpm-ostree. This includes fixing indentations, adding spaces between
identifers and parentheses, replacing "goto print" with an else structure, and
reformatting print statements into complete sentences for internationalization.
This allows administrators to configure between deployments and easily see which deployment
they are booted into as well as indicating which is chronologically most recent. This makes
the process more user-friendly, rather than requiring the user to remember which deployment
checksum corresponds to the most recent upgrade.
Currently the systemd RPM ships with default.target ->
graphical.target, which is either itself changed by Anaconda (via
parsing /etc/sysconfig/desktop, which...anyways let's stop here).
Or anaconda might set it directly to multi-user.target.
For rpm-ostree, we perform some minimal level of "preconfiguration"
per tree, so they are directly usable without an intervening
installer.
As an example for fedora-atomic/base/core, we just want
multi-user.target. Thus, this patch provides the treefile author a
declarative mechanism to set it.
Currently on an Atomic compose, I'm seeing abrtd trying to write to
/usr/share/rpm/.dbenv.lock, which is denied by policy because it's
usr_t. There are multiple ways to address this, but there's no good
reason to leave the lock files and __db* files around.
rpm appears to operate correctly without them if calling process
merely gets EROFS.
This is exactly the code from "ostree admin switch", except it's
called "rebase" because in the future it will also carry along any
locally layered packages.
And do the same for "sign". This way we can have the compose server
utilities cleanly separated from what most people will see, which is
the client side tools.
