Commit Graph

83 Commits

Author SHA1 Message Date
Colin Walters
95b423afe9 Add cancel verb and DBus API
Right now the fact that one can only cancel via `Ctrl-C` of an existing client
process is rather frustrating if for example one's ssh connection to a machine
drops. Now, upon reconnecting, one can easily `rpm-ostree cancel` a hung update
or whatever rather than doing the more forcible `systemctl stop rpm-ostreed`
(which is safe of course, unless livefs is involved).

Closes: #1019
Approved by: jlebon
2017-12-07 20:12:52 +00:00
Colin Walters
694b798c73 Introduce experimental "rpm-ostree jigdo"
Tracking issue: https://github.com/projectatomic/rpm-ostree/issues/1081

To briefly recap: Let's experiment with doing ostree-in-RPM, basically the
"compose" process injects additional data (SELinux labels for example) in an
"ostree image" RPM, like `fedora-atomic-host-27.8-1.x86_64.rpm`. That "ostree
image" RPM will contain the OSTree commit+metadata, and tell us what RPMs we
need need to download. For updates, like `yum update` we only download changed
RPMs, plus the new "oirpm". But SELinux labeling, depsolving, etc. are still
done server side, and we still have a reliable OSTree commit checksum.

This is a lot like [Jigdo](http://atterer.org/jigdo/)

Here we fully demonstrate the concept working end-to-end; we use the
"traditional" `compose tree` to commit a bunch of RPMs to an OSTree repo, which
has a checksum, version etc. Then the new `ex commit2jigdo` generates the
"oirpm". This is the "server side" operation. Next simulating the client side,
`jigdo2commit` takes the OIRPM and uses it and downloads the "jigdo set" RPMs,
fully regenerating *bit for bit* the final OSTree commit.

If you want to play with this, I'd take a look at the `test-jigdo.sh`; from
there you can find other useful bits like the example `fedora-atomic-host.spec`
file (though the canonical copy of this will likely land in the
[fedora-atomic](http://pagure.io/fedora-atomic) manifest git repo.

Closes: #1103
Approved by: jlebon
2017-12-04 14:24:53 +00:00
Colin Walters
a822722a5b ex-unpack: Delete
I think the `ex container` path supercedes this; it was really just a demo, and
having it around is annoying since I want to change the importer API and I have
to change it here too.

Closes: #1116
Approved by: jlebon
2017-11-29 16:48:19 +00:00
Ruixin Bao
fa593afd89 app/daemon: add groundwork for rpm-ostree kargs command
This is initial groundwork for https://github.com/projectatomic/rpm-ostree/issues/594.

This commit sets up most of the required
front end logic( arg parsing, transaction handling), and will
be used in the following commits.

There is nothing really fancy in this commit, as most of the code
shares the similar style between other dbus related commands.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Jonathan Lebon
8285ef2a15 app: add 'refresh-md' command
This is essentially the `dnf/yum makecache` equivalent for rpm-ostree.
To complete the picture, this goes hand in hand with the `-C`
equivalent, which is added in the next patch.

Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Jonathan Lebon
577aef98e2 app: spawn polkit authentication agent
Spawn pkttyagent when trying to call a method that may require
authentication to give users a chance to provide auth right from the
terminal.

Since we're now relying on polkit for authorizing most of the OS
interface methods, let's drop the root check on those.

Closes: #894
Approved by: cgwalters
2017-07-27 17:10:41 +00:00
Jonathan Lebon
cde3295e26 app: add experimental support for pkg removals
This is one more step towards making rpm-ostree more powerful in its
quest to be the ultimate *hybrid* image/package system. Package layering
allows us to add packages on top of the base package set received from
the content provider. However, we're not able to remove or replace
packages in the base set itself.

This patch introduces a new `override` command, which is for now nested
under the experimental `ex` command. The `override` command will allow
users to modify the base package set itself. The first implemented
subcommands are `remove` and `reset`.

A stub has been provided for the more useful `replace` subcommand,
though much of the needed logic for that operation are implemented in
this patch as part of the `remove` subcommand.

Part of: https://github.com/projectatomic/rpm-ostree/issues/485

Closes: #797
Approved by: cgwalters
2017-06-05 20:48:50 +00:00
Colin Walters
95e9aa4284 Introduce ex livefs
There are a few different use cases here. First, for layering new packages,
there's no good reason for us to force a reboot. Second, we want some support
for cherry-picking security updates and allowing admins to restart services.  Finally,
at some point we should offer support for entirely replacing the running tree
if that's what the user wants.

Until now we've been very conservative, but there's a spectrum here. In
particular, this patch changes things so we push a rollback before we start
doing anything live. I think in practice, many use cases would be totally fine
with doing most changes live, and falling back to the rollback if something went
wrong.

This initial code drop *only* supports live layering of new packages.  However,
a lot of the base infrastructure is laid for future work.

For now, this will be classified as an experimental feature, hence `ex livefs`.

Part of: https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #652
Approved by: jlebon
2017-05-12 15:00:59 +00:00
Colin Walters
8dfed46e26 app: Rename internals → ex
The goal is to consolidate our "experimental" functionality under one
subcommand.  This makes it easier to determine when things "graduate"
to permanent-stability status under the main command line.

Closes: https://github.com/projectatomic/rpm-ostree/issues/682

Closes: #683
Approved by: jlebon
2017-03-16 19:57:14 +00:00
Colin Walters
b034381506 daemon: Implement "reload"
There are two main issues right now; first, we don't pick up manual changes to
`.origin` files, which occurs when one needs to sed it to remove `unconfigured`
for example. Second, we need to reload changes to the remotes.

Closes: #598
Approved by: jlebon
2017-02-24 21:36:45 +00:00
Colin Walters
a66d27230d Add a cleanup command and DBus API
We sometimes talk about using `ostree admin undeploy`, but that
doesn't know about the pkgcache, and hence space there leaks
until the next rpm-ostree operation.

Just for this, we need to expose a cleanup command (and API).  But
we also need to support cleaning:

 - repomd
 - downloads (repo/tmp)

So let's start implementing that.

Closes: #614
Approved by: jlebon
2017-02-13 15:28:05 +00:00
Jonathan Lebon
b0727d88fd start-daemon: make hidden
I debated just putting this in the supported list, but decided against
it in the end. This really should be something that happens
transparently, and if it doesn't then something else is probably wrong.

Closes: #617
Approved by: cgwalters
2017-02-10 20:11:27 +00:00
Colin Walters
5f7c5305ff Delete "compose sign"
We want people to use the libostree API for things like this. Further, the
`rpm-sign` tool that this calls is Red Hat internal, so it doesn't make sense to
have a public wrapper for it.

Closes: https://github.com/projectatomic/rpm-ostree/pull/152

Closes: #607
Approved by: jlebon
2017-02-07 21:43:22 +00:00
Colin Walters
687567d3ee Merge daemon binary into main binary
The actual problem I am trying to fix with this is fallout from the
introduction of `/usr/libexec/rpm-ostreed`, which required a SELinux
policy change.  Specifically for CentOS, the base policy is rev'd
slowly.

My hope was that by merging the daemon code back into `/usr/bin/rpm-ostree`
which is labeled `install_exec_t`, starting via systemd would do
the right thing.  It turns out that doesn't happen.

Now later, I'm picking this patch back up because I want to do multprocessing in
the daemon (and in the core), and it makes sense to share code between them,
because multiprocessing will need to go through a re-exec path.

Another benefit is we avoid duplicated text (libglnx, internal helpers) between
the two binaries.

Closes: #292
Approved by: jlebon
2017-02-07 16:07:09 +00:00
Colin Walters
cac4522e5b Support "system/regenerate-initramfs=true" flag in origin
Currently we push for a model where the initramfs is
generated (in non-hostonly mode), and merely replicated.

However, to support a few unfortunate corner cases like dm-multipath which wants
to inject a config file into the initramfs, we need to support regenerating it
client side too.

Down the line, we'll need this to support overriding the kernel too.

This changes things in the core to add the concept of an "empty"
`RpmOstreeContext`. I initially tried skipping it, but that was too much
duplication. We still want all of the core ostree-related logic that lives in
that code too.

The treespec bits barfed if the spec didn't have a `tree/packages` key. It was
simplest to change that to allow it - and because that was the only case where
we errored out in parsing, I dropped the error handling.

There was another place in the upgrader that now needed to be fixed to handle
transitioning from just regenerating initramfs to not.

Closes: #574
Approved by: jlebon
2017-01-20 16:17:51 +00:00
Colin Walters
328b85d396 Add infrastructure for alternative name (currently nts)
See https://github.com/projectatomic/rpm-ostree/issues/405

This patch adds an (off by default) `--enable-new-name` build option
which currently defaults to `nts`.  This is purely additive, and
the intention is that we'll support the rpm-ostree name in
perpetuity most likely.

At the moment, we add a new name for:
  - /usr/bin/$name
  - The systemd unit file

But we notably *don't* attempt to add a new name to the DBus API,
as it'd be a lot more invasive of a patch, and less payoff (it's
mostly just programs/scripts that interact with the DBus).

Closes: #497
Approved by: jlebon
2016-10-20 14:22:28 +00:00
Colin Walters
125c482b1d Switch to using libhif as a git submodule
So I was trying to hack on my host's copy of rpm-ostree inside a pet
docker container, but ran into a conflict with libhif since dnf uses
it.  I think we basically need to *always* build the bundled path,
rather than what I'm doing with CAHC and FADC where it's built as a
regular RPM.

It's not really sustainable right now for us to have both bundled and
not-bundled build paths - and we need to support co-installation with
dnf.

Another major issue is that we want to version lock with libhif -
right now our CI and both CAHC/FADC track libhif master, but that
means everything breaks if libhif breaks and we don't immediately
port.

git submodules solve all of these problems - the same as we're doing
with libglnx.

libglnx is *designed* for use as a git submodule, where as libhif
needs to support being both bundled and not-bundled.  So we end up
with some hacks on our side, but I think it's all not too bad.  I've
marked build rules with `# bundled libhif` so we know where to find
them later when libhif is stable.

Closes: #357
Approved by: jlebon
2016-06-30 14:27:55 +00:00
Colin Walters
9d473df11e build: Remove --enable-usrbin-atomic
We haven't used this in a long time now that the atomic command
exists.

Closes: #361
Approved by: jlebon
2016-06-29 16:44:08 +00:00
Colin Walters
1616ca5f07 app: Rename pkg-delete -> pkg-remove and unify codebase
I've found it's a lot less code to have multiple builtins share live
in the same `.c` file where they can share things like options.

Also, rename `pkg-delete` -> `pkg-remove` since the canonical antonym
of `add` is `remove`.

Closes: #345
Approved by: jlebon
2016-06-23 13:37:07 +00:00
Jonathan Lebon
3073a42868 tmpfiles.d: rename and re-order
According to tmpfiles.d(5), files should follow the convention
<package>.conf or <package>-<part>.conf. So we rename
tmpfiles-ostree-integration.conf to rpm-ostree-0-integration.conf.

The 0 index is so that the autovar conf created by postprocess is
sourced *after* this one, so that `integration.conf` has higher
precedence if there are duplicate entries.

Closes: #325
Approved by: cgwalters
2016-06-15 20:41:45 +00:00
Jonathan Lebon
ead1ecdd23 package layering: major rework
- Move the package layering logic away from pkg-add and into the
  upgrader
- Add pkg-delete
- Add dry-run option

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Colin Walters
beb026f701 pkg-add: New builtin to layer additional packages
This builds upon the earlier prototype in
https://github.com/cgwalters/atomic-pkglayer

The `.origin` file says for a replicated installation:

    [origin]
    refspec=local:rhel-atomic-host/7/x86_64/standard

If you then run `rpm-ostree pkg-add strace`, it will result in a new tree with:

    [origin]
    baserefspec=local:rhel-atomic-host/7/x86_64/standard

    [packages]
    requested=strace;

Work still remaining here is to teach `rpm-ostree status` and
`rpm-ostree upgrade` about this.

Closes: #289
Approved by: cgwalters
2016-05-27 22:18:44 +00:00
Colin Walters
8de4f9be27 Add a container builtin
This is just a tech demo.  Example usage:

```
mkdir -p ~/.cache/rpmostree-containers
cd ~/.cache/rpmostree-containers
rpm-ostree container init
cp /etc/yum.repos.d/CentOS-Core.repo rpmmd.repos.d
rpm-ostree container assemble bash
rpm-ostree container assemble httpd
```
2016-02-22 14:27:45 -05:00
Colin Walters
ec4387afba internals: New unpack verb
This is part of taking over from librpm.  The most important high
level goal is fully unprivilged operation.

Right now we're basically starting to do what
http://libguestfs.org/supermin.1.html does, except in C, and
faster.

There's no reason that `compose tree` should require privileges.
However right now, things like `%post` scripts will want to run in the
target root - so we'd have to require `linux-user-chroot`.

Regardless of unprivileged operation though, another major thing we
can do is use our control over the unpacking process to do a lot more
sophisticated caching.  We can build up a precise mapping of (rpm
ENVR, file path, selinux label) -> object and avoid rechecksumming
each time.

And even for files that aren't known, we can parallelize commit with
unpacking, etc.  (Ok assuming treecompose-post won't mutate anything).
2016-01-12 10:22:43 -05:00
Colin Walters
b456badba3 Add testing-only internals subcommand
I'd like to experiment with different things that end up
reusing chunks of the rpm-ostree internals, such as libhif, the
helpers we already have around RPM, etc.

In this particular case I'm experimenting with unpacking/committing
RPM packages as non-root.  Eventually most of this should end up as
internal private shared library, but it's convenient to have an
ABI-unstable and hidden "internals" command to run things directly.

This commit though just adds the scaffolding for "internals".
2016-01-11 23:06:27 -05:00
Matthew Barnes
56d28a76f3 app: Add "deploy" command
Command-line interface for new D-Bus "Deploy" methods.
2015-11-06 09:10:48 -05:00
petervo
f525730187 daemon: Make parts of cli use the daemon 2015-09-09 22:00:04 -04:00
petervo
c0d15a66ad daemon: Start of work on daemon 2015-09-09 22:00:04 -04:00
Colin Walters
e414be8ae2 Move diff printing code into client
This is a step forward to deduplicating; the client tooling now calls
into the public API for diffs, rather than using the older internal
function.

Note: this patch also links the client against the public library.
2015-04-23 16:30:18 -04:00
Colin Walters
7b45f13a62 Link lib/ against libpriv/, use it to share sack loading function
On the plus side, we share some code between the library and the
binary now.  On the downside, because `librpmostreepriv.la` is a
noinst library, its code text is duplicated between the shared library
and binary, at least until we either:

 - Have the binary solely use the public shared library (like ostree does)
 - Install `librpmostreepriv.so` to e.g. `/usr/lib64/rpm-ostree/librpmostreepriv.so`
   without the headers being public
2015-04-23 16:30:17 -04:00
Colin Walters
a8a2049443 build: Split up src into app/ and libpriv/
We currently have an internal-only library, but the sources for it are
in the same dir as the app.  For future work on a public shared
library, we'll need a clearer source structure.

Start by just renaming the app files into `src/app/`, and the internal
private library into `src/libpriv/`, with the appropriate
`Makefile.am` changes.

Closes: https://github.com/projectatomic/rpm-ostree/pull/123
2015-04-08 16:17:06 -04:00
Colin Walters
e79c73de20 build: Remove last vestiges of hawkey/libsolv compat
We can assume the versions we care about have propagated everywhere
that supports the libhif version we support.
2015-03-20 14:55:32 -04:00
Colin Walters
4b19dd1266 Move the little "libcontainer" code into libglnx
I want to share it with ostree, and potentially xdg-app.
2015-03-04 18:29:55 -05:00
Colin Walters
392acdf963 Move console code to libglnx
The main goal is to be able to share this with OSTree.
2015-02-17 16:54:54 -05:00
Colin Walters
3190eff276 Start using libglnx
The high level goal is to deprecate libgsystem.  I was trying to share
code between ostree/rpm-ostree, but it was too painful to commit to
forver frozen ABI for new utility APIs.

The git submodule approach will much more easily allow breaking
API/ABI, and iterate on APIs until they either land in GLib or not.

Note that libglnx will not use GFile*, so a full port to it will
involve also not using that.  Thus, it will be necessarily
incremental; in the meantime we'll link to both libgsystem and
libglnx.
2015-02-15 14:01:13 -05:00
Colin Walters
172ff5712b [libhif] Port to new hif_error_set_from_hawkey()
And drop our internal copy.
2015-02-02 16:39:26 -05:00
Colin Walters
7e0a55cd64 [libhif]: Add initial progress reporting
This requires open-coding the 3 steps that hif_context_run() was
doing.  Which is fine, because ultimately we want to have more control
over that.
2015-02-02 16:39:26 -05:00
Colin Walters
4511a1d8ac consoleprogress: New class for writing to the console
Some influence from both systemd and packagekit-glib2.
2015-02-02 16:39:25 -05:00
Matthew Barnes
31cd8297e7 Merge pull request #77 from mbarnes/pull-progress
Use ostree_repo_pull_default_console_progress_changed()
2014-12-18 23:02:26 -05:00
Matthew Barnes
7b36814441 Use ostree_repo_pull_default_console_progress_changed()
Remove redundant function _rpmostree_pull_progress().

Bumped ostree requirement to 2014.13, but this isn't quite right because
we actually need (unreleased) 2014.14.  Post-release version bumps would
be useful here.
2014-12-18 21:37:22 -05:00
James Antill
855ab9ad01 compose: Add check-passwd/group JSON options, fails compose if uids/gids change
Verify uid/gid on files, directories and symlinks
Just output a msg when user/group is removed with no files

json-parsing: Add functions for strictly dealing with ints
passwd/json: Add simple scripts to convert passwd/group files to json data

docs: Check-passwd/groups and ignore-remove-users/groups JSON config. entries
2014-12-18 16:59:33 -05:00
Matthew Barnes
b8c26805d3 Split up rpmostree-builtin-db.c
As a followup to renaming the "rpm" command to "db", split the "db"
subcommands into separate source files in the style of "ostree admin"
and "rpm-ostree compose".

Also create rpmostree-rpm-util.[ch] as a place for common rpm-related
functions needed by the "db" subcommands.

No intentional functional changes here, just a bunch of copy-n-paste
and minor cleanup.
2014-12-01 20:03:12 -05:00
Matthew Barnes
8336e504e1 Rename "rpm" command to "db"
Eliminates some confusion between "rpm-ostree rpm" (or "atomic rpm")
commands versus actual "rpm" commands.

The "rpm" subcommand is retained as a hidden alias for the "db"
subcommand for backward-compatibility.  It is not listed in --help
output.

Fixes #22
2014-12-01 20:03:11 -05:00
Colin Walters
901917ff85 compose: Introduce a little 'libcontainer', use it for the post script
The current motivation for this is that

https://github.com/fedora-infra/fedmsg-atomic-composer

started using mock --new-chroot (which uses systemd-nspawn) to run
rpm-ostree, which in turn uses systemd-nspawn to run the post script.
Now systemd-nspawn is not really nestable (it wants to link up
journald, resolv.conf handling, etc).

First, dropping nspawn and going to raw containers fixes the nesting
problem.

Second, we don't need all the features of systemd-nspawn.  We are ok
with log messages going to stdout, and we don't use networking, so no
resolv.conf is needed.

Third, this sets a bit of a stage for more sandboxing internally when
run on real systems.  I already have a prototype branch which runs
librepo as an unprivileged user, that could be combined with this for
even stronger security.

Why not use systemd?  Well...I'm still debating that.  But the core
problem is systemd isn't a library in the C sense - to use its
sandboxing features we have to use unit files.  It's harder to have a
daemon that looks like a single service from a management perspective,
but uses sandboxing internally.
2014-11-21 13:16:49 -05:00
Colin Walters
cb5681a4f4 Delete rpmqa-sorted-and-clean
This is no longer needed now that we use hawkey for this.
2014-11-17 17:46:40 -05:00
Colin Walters
421fb94ab2 compose: Split out JSON helpers into separate file
In preparation for using some of them in the postprocessing code too.

Yeah, parsing JSON in C sucks =( Oh well.
2014-11-13 15:08:16 -05:00
Colin Walters
571fa8ddec Centralize cleanup function definitions
We might as well do what systemd does and have a big header which
defines all of them, to more conveniently share them for libraries
that don't include them (like hawkey/librepo, as well as things that
libgsystem doesn't yet cover).
2014-11-11 18:40:58 -05:00
Colin Walters
4c405bf62f compose: Protect the system during treecompose using container APIs
I was looking again at using hawkey/librepo, and realized just how
much I'd have to fight all of these libraries to avoid affecting
the running system.

What we really want to do with librepo/hawkey is run them effectively
unprivileged, and to hide the system's RPM database from them.  This
is a baby step towards that, by confining our existing yum.

- /usr, /etc, and /var/lib/rpm are mounted read-only
- yum is now run under CLONE_NEWPID, to avoid stray %post scripts
  affecting system processes
2014-11-10 12:13:44 -05:00
Colin Walters
4ecce5884d treecompose: Replace shell callout for package caching with builtin C
This is taking us closer to deeper integration in the treecompose side
with RPM instead of forking out to things.

It works except...we end up with the dreaded __db.001, .dbenv.lock
files =/ Best option would be to teach RPM how to open a database
really read-only.  Failing that, could use the immutable bit?
2014-11-03 16:42:16 -05:00
Colin Walters
0f16e2e252 Add --disable-compose-tooling build option
Some downstreams want the ability to separate the compose tooling from
the client, for e.g. support reasons.

This approach supports generating a tarball without the source for the
compose command, and requires specifying a config option to disable
it.
2014-09-16 21:45:30 -04:00