Commit Graph

21 Commits

Author SHA1 Message Date
Colin Walters
4ecce5884d treecompose: Replace shell callout for package caching with builtin C
This is taking us closer to deeper integration in the treecompose side
with RPM instead of forking out to things.

It works except...we end up with the dreaded __db.001, .dbenv.lock
files =/ Best option would be to teach RPM how to open a database
really read-only.  Failing that, could use the immutable bit?
2014-11-03 16:42:16 -05:00
James Antill
ee1cd0d898 Workaround false GCC warning for uninit variable 2014-10-24 01:34:17 -04:00
Colin Walters
e6c42cb884 compose tree: Add --output-repodata-dir
This could be used by higher level tools like fedmsg-atomic-composer
that want to merge the yum repodata used for input with the tree
content.
2014-10-23 17:14:14 -04:00
Colin Walters
161324d73b compose: Add "remove-files" verb
I'd been resisting this for a long time - I really wanted the tree to
be a reflection of the packages; and not go down the path of
"forking".

The lorax model has shown that while you can definitely get a large
space reduction that way, you're going to be perpetually chasing
changes in the packages.  No matter how good your templates are.

Furthermore, lorax is just to generate the installer - it's an OS that
runs a single app.  Whereas here we're generating the target runtime
system; we can't add bugs.

Nevertheless, reality is that sometimes it's just too hard to change
the input package set - there's a risk of breaking things.  Namely,
we're introducing a new update system here, but obviously there's a
previous one: yum.  Upstream packages keep growing a dependency on it.

Note this patch allows *not* removing all files from the package,
because it's possible that other things (e.g. subscription-manager)
import it as a library.

So in the meantime while we're iterating on this, let's support:

  "remove-files": ["usr/bin/yum"],

The code is generic beyond yum for obvious reasons, but I don't
think we should use it for a lot more than that.
2014-10-23 11:42:52 -04:00
James Antill
5cd4a1d39b treecompose: Add --add-metadata-string argument for versioning
This will be used by rpm-ostree-toolbox to inject version numbers for
the trees.
2014-10-16 19:10:42 -04:00
Colin Walters
15ecaacd36 compose: Support 'boot_location' to facilitate GRUB2
Having content in /boot in OSTree was always ugly, because we ended up
mounting over it in the deployment location at boot.

This was even worse in the anaconda rpmostreepayload code, because of
the juggling of the mount point that needed to take place.

Trying to add a GRUB2 backend to OSTree is what finally forced this
change.  Now, we put kernels (in the tree) by default in *both* /boot
and /usr/lib/ostree-boot.

OSTree itself knows to look in both locations.  Anaconda is going to
just hard require trees with the new location though.
2014-10-15 22:10:15 -04:00
Colin Walters
03c368fec4 compose: Fix setting of default.target
This would probably be best if we invoked systemctl in the compose
tooling, but at the moment we don't have any execution of target code
on the host.  It's fine to assume that it's in /usr/lib.

We can revisit this if we start doing chrooted/containerized
execution.
2014-10-14 08:28:17 -04:00
Colin Walters
d3babd8a93 compose: Serialize treefile early
This way we don't emit a critical if we exit due to having already
cached content.
2014-09-29 16:25:36 -04:00
Colin Walters
297dbb0c73 compose: Put the expanded treefile content in /usr/share/rpm-ostree/treefile.json
Since the treefile format now supports includes, we would need to
either include the whole chain, or just the expanded portion we use in
the compose.  This patch does the latter.

This should allow a client to take the same treefile and generate a
similar tree (if they want to reproduce with the same RPMs, those can
be extracted from the RPM database inside the tree).
2014-09-07 12:48:05 -04:00
Colin Walters
e17419b978 compose: Add a --add-override-pkg-repo argument
As a developer, a workflow I have for testing things is to create
an RPM, toss it into a local yum repository, then do a compose.

However at the moment to add the local overrides I have to edit the
treefile, which is annoying.  Let's add a commandline override for
this.

Note this also deletes the old "repos_data" code which was not being
used.
2014-07-28 17:16:09 -04:00
Dusty Mabe
48e8c126f2 compose: Delete dead code that copies aside yum repos
In 827e711 we stopped running two yum transactions. This means the
code that detects if the repodir exists in the yum installroot will
always return false and the code is dead.
2014-07-28 13:55:17 -04:00
Colin Walters
827e711eb7 compose: Migrate content of /etc/{passwd,group} to /usr/lib more sanely
I had an epiphany today while working on
https://bugzilla.redhat.com/show_bug.cgi?id=1098304 - I realized that
I can just do an install, and then copy over everything except the
root entries from /etc/passwd into /usr/lib/passwd.

No need for a patched shadow-utils.  No need to modify the
/etc/nsswitch.conf before doing the install root.  It totally works.
I have no idea why I originally overcomplicated this.

The thing that sucks a bit about this code is that I have to drop to
the FILE * APIs so that I can use the glibc APIs for processing
group/shadow.

Also, the way I deduplicated the code paths for processing
passwd/group is crappy, but I think it's better than duplicating them
(as systemd-sysusers does).

The good: We don't need a two-step RPM transaction, we don't need
          a patch for shadow-utils, it's just saner
The bad: Code is not the most beautiful?  Not really bad.
The ugly: I didn't think of this in the first place and spent
          months beating my head against the wall of shadow-utils...
2014-07-10 18:50:54 -04:00
Colin Walters
aa5ae877de compose: Set SHADOW_USE_USRLIB
See https://bugzilla.redhat.com/show_bug.cgi?id=1098304
2014-07-09 17:29:28 -04:00
Colin Walters
19d093a496 compose: Add --workdir-tmpfs
This ~halves the write traffic to my SSD when doing local composes.
My laptop has 12GiB of RAM, plenty for this.
2014-07-02 08:43:59 -04:00
Colin Walters
d63bc65a06 compose: Tell yum not to cache packages locally
They get deleted anyways since we blow away the tmpdir, but we expect
people to be using a local HTTP proxy.  We stopped trying to cache
packages internally as that caused conflicts with the yum lock with
concurrently executing composes.
2014-06-19 11:57:33 -04:00
Colin Walters
227137ed41 compose: Add support for "default_target"
Currently the systemd RPM ships with default.target ->
graphical.target, which is either itself changed by Anaconda (via
parsing /etc/sysconfig/desktop, which...anyways let's stop here).

Or anaconda might set it directly to multi-user.target.

For rpm-ostree, we perform some minimal level of "preconfiguration"
per tree, so they are directly usable without an intervening
installer.

As an example for fedora-atomic/base/core, we just want
multi-user.target.  Thus, this patch provides the treefile author a
declarative mechanism to set it.
2014-06-17 13:49:15 -04:00
Colin Walters
c151b56595 compose tree: Cleanup workdir always
Particularly if we hit the cached case, but we might as well always do
it.
2014-06-11 07:27:31 -04:00
Colin Walters
87a00befe2 compose-tree: Auto-create cache directory
It's just more friendly.
2014-06-06 18:25:08 -04:00
Colin Walters
1613435f7d tree compose: Delete .dbenv.lock and __db.* files from /usr/share/rpm
Currently on an Atomic compose, I'm seeing abrtd trying to write to
/usr/share/rpm/.dbenv.lock, which is denied by policy because it's
usr_t.  There are multiple ways to address this, but there's no good
reason to leave the lock files and __db* files around.

rpm appears to operate correctly without them if calling process
merely gets EROFS.
2014-06-06 16:28:19 -04:00
Colin Walters
7ba1a9b46f compose-tree: Fix location of rpmdb querying for caching
We were just repeatedly hitting the cache...oops.

Also add an assertion that the returned data isn't empty.
2014-05-29 14:57:55 -04:00
Colin Walters
7c71fbc077 Add new "compose" builtin, rename "treecompose" -> "compose tree"
And do the same for "sign".  This way we can have the compose server
utilities cleanly separated from what most people will see, which is
the client side tools.
2014-05-26 15:05:08 -04:00