Commit Graph

318 Commits

Author SHA1 Message Date
Colin Walters
7ea2a5afce (refactor) Move pkgcache helper function to sysroot-core
Trying to drain the `util.c` file; this one lives more logically
there.

Closes: #1086
Approved by: jlebon
2017-11-08 03:11:36 +00:00
Colin Walters
8b4c5e0901 tree-wide: Use glnx_close_fd()
It's cleaner, and we get `EBADF` checks, etc.

Closes: #1085
Approved by: jlebon
2017-11-08 02:50:12 +00:00
Colin Walters
a42c45f726 lib/core: Synthesize tmpdir for commit if no base provided
The `ex container` path was already doing this - lower the logic
into the core so I can use it for the jigdo path.  I think it's just
clearer as well even for the sysroot pkglayering path.

Closes: #1083
Approved by: jlebon
2017-11-08 02:12:52 +00:00
Colin Walters
170f74c282 lib/core: Add API to set devino cache
Rather than passing it to the assemble functions.

Prep for jigdo work.  I'm trying to move more logic
from the system upgrader path to the core, since some of
it is needed for jigdo and "unified core".

Closes: #1083
Approved by: jlebon
2017-11-08 02:12:52 +00:00
Ruixin Bao
27de772c4f libpriv/vmcheck: support replace option for karg command
Refactor the similar code portion from delete option into
a new function, and reuse it in the replace logic.

Add the replace functionality to allow kernel arguments
replacement. Now support two different types of replacement:

1: replace by key=value when there is only one key existing
e.g we have 'hi=hello' as our one of the arguments (hi only has
one value associated with it) , and 'rpm-ostree ex kargs --replace hi=new'
will replace the value and change hi=hello into hi=new.

2: replace by key=value=new_value for all other replacements
( this will work for swapping single value pair too !)

Some tests for rpm-ostree ex kargs --replace are added
for future regression.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Ruixin Bao
09cf438f04 vmcheck: Add tests for rpm-ostree ex kargs delete and append
Tests are introduced in this commit to test the basic
functionality for rpm-ostree ex kargs --append,
and rpm-ostree ex kargs --delete.

Those tests are added for future regression.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Ruixin Bao
50863c370d daemon: add handlation for kargs append
API functions from ostree-kernel-args.c
are copied to libpriv. The append functionality
reuses  _ostree_kernel_args_append_argv() for
collecting added kernel arguments.

Also added handlation in rpm-ostree upgrader
to allow deployments happen with kernel arguments.

Now, the user is able to add kernel arguments via
'rpm-ostree ex kargs --append key=value'
or 'rpm-ostree ex kargs --append key' if they
want to have an empty value with key.

The user is also able to display the current
kernel arguments via 'rpm-ostree ex kargs'

In addition, this functionality will create a pending deployment,
and will update the conf file in /boot/loader/entries/ostree-$osname-0.conf upon success.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Ruixin Bao
fa593afd89 app/daemon: add groundwork for rpm-ostree kargs command
This is initial groundwork for https://github.com/projectatomic/rpm-ostree/issues/594.

This commit sets up most of the required
front end logic( arg parsing, transaction handling), and will
be used in the following commits.

There is nothing really fancy in this commit, as most of the code
shares the similar style between other dbus related commands.

Closes: #1013
Approved by: cgwalters
2017-11-06 18:49:36 +00:00
Colin Walters
9ec45d3c89 daemon: Add commit metadata to deployment properties
In another PR I did the manual bridging of commit metadata to deployment
property, but that's annoying. Let's just bridge all commit metadata.

Closes: #1069
Approved by: jlebon
2017-10-25 03:37:05 +00:00
Jonathan Lebon
c107a05b8e tree: pass NULL to glnx_fstatat_allow_noent when needed
Now that libglnx allows it, we can skip declaring a `struct stat` we
don't actually need just to check if a file exists.

Closes: #1066
Approved by: cgwalters
2017-10-17 20:48:13 +00:00
Colin Walters
94527ab69c tree-wide: Use glnx_autofd
It's a better name.  Already ported ostree.

Closes: #1063
Approved by: jlebon
2017-10-17 16:01:34 +00:00
Jonathan Lebon
4d1b5b4636 libpriv/core: use cached repodata checksum for goal
An interesting behaviour arises from the new `--cache-only` option. We
would essentially consider two goals with the same exact pkglist as
different depending on whether a package came from a yum repo or our
pkgcache repo. This meant that `rpm-ostree upgrade` followed by
`rpm-ostree upgrade -C` was never detected as a no-op and always
resulted in a relayer.

This was due to the fact that the checksum libsolv calculated was based
on our chopped off pkgcache meta-RPMs. This of course won't match the
checksum from the repodata, which is of the whole RPM.

Thankfully, we already save the repodata checksum when unpacking RPMs.
This patch simply detects the case where we're trying to checksum a pkg
originating from the pkgcache, and reads in the saved repodata checksum
rather than the wrong libsolv-derived one.

Closes: #1049
Approved by: cgwalters
2017-10-16 17:49:51 +00:00
Jonathan Lebon
e49f7cdd81 app: add --download-only option
To complement the new `--cache-only` option, add a `--download-only`
option. This does exactly what it says: we download the ostree, download
and import packages, but don't actually commit & deploy. This can be
used to effectively prime a follow-up `--cache-only` operation that can
be done during a more convenient/safer maintenance window.

I debated naming the two options `--pull-only` and `--deploy-only` like
the ostree equivalents. Though "pull" felt like the wrong word given
that it's associated more with ostree pulling but rpm-ostree also
downloads & imports RPMs. As for `--deploy-only` vs `--cache-only`, it
seems like `--cache-only` is a more accurate description of the
functionality (i.e. rather than describing an action, it describes a
mode). I also considered `--no-download` to make the synergy with
`--download-only` more obvious. Maybe that's better? Naming is hard...

Closes: #713

Closes: #1049
Approved by: cgwalters
2017-10-16 17:49:51 +00:00
Jonathan Lebon
b811eb61c0 app: support full offline operations with --cache-only
As Colin mentioned in #1035, the new `--cache-only` implemented only the
rpmmd half of the story. Here we complete that story by also ensuring
that when in cache-only mode, we don't download new ostree data nor new
packages. We try to complete the requested operation with what we have.

To do this, we add support for the same `SYNTHETIC` pull that was added
in ostree[1] so that we don't actually pull, but still perform timestamp
checking.

On the pkgcache side, we disable all remote repos and instead insert all
our cached RPMs into the `DnfSack`. Care is taken to still perform
SHA256 verification for local pkg installs/replacements.

[1] https://github.com/ostreedev/ostree/pull/642

Closes: #687

Closes: #1049
Approved by: cgwalters
2017-10-16 17:49:51 +00:00
Colin Walters
aea688fa01 Update libglnx
I want to use `glnx_tmpdir_unset()` in the `compose commit` PR.

Update submodule: libglnx

Closes: #1056
Approved by: jlebon
2017-10-16 13:22:30 +00:00
Jonathan Lebon
dbceb2aa51 daemon/deploy: remove duplicate call to set_origin
We already set the origin unconditionally right after this block. No
point in `dup`'ing twice.

Closes: #1053
Approved by: cgwalters
2017-10-13 15:00:25 +00:00
Jonathan Lebon
74eaa2ba59 daemon: add RPMOSTREE_DEBUG_DISABLE_DAEMON_IDLE_EXIT
The new idle exit behaviour is nice, but it makes debugging it harder
because you have to be fast enough to attach or place your breakpoints
and trigger it before it auto-exits. Add a compile-time flag that
developers can easily turn on to disable the auto-exit behaviour.

Closes: #1052
Approved by: cgwalters
2017-10-12 15:35:44 +00:00
Colin Walters
4a4f9952c2 tree-wide: Use a #define for /usr/share/rpm location
In prep for potentially changing it:
http://lists.rpm.org/pipermail/rpm-maint/2017-October/006681.html

Of course actually doing a transition would be harder than this, as we'd need to
add a compat symlink, and even that wouldn't quite be enough as e.g. the
"preview" code would need to learn how to follow the symlink (or just try both
locations).

In practice I think we'd need to land the code to handle both locations, let
that trickle out for e.g. 3 months, then make the switch in our treecomposes.

But, might as well make this change now; using a `#define` makes it slightly
easier to find places that need changing later.

Closes: #1048
Approved by: jlebon
2017-10-10 19:59:21 +00:00
Jonathan Lebon
4442a0c362 app: add -C flag to always use cached metadata
This is the equivalent version of `yum/dnf -C`. It goes together with
the new `makecache` command to allow completely asynchronous cache
update and usage.

Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Jonathan Lebon
8285ef2a15 app: add 'refresh-md' command
This is essentially the `dnf/yum makecache` equivalent for rpm-ostree.
To complete the picture, this goes hand in hand with the `-C`
equivalent, which is added in the next patch.

Closes: #1035
Approved by: cgwalters
2017-10-06 18:20:59 +00:00
Jonathan Lebon
8fae56d120 libpriv/core: teach core to use deployment for yum repos
Add a function in the core that knows how to take a deployment and point
its passwd dir and libdnf's repos dir to it. We call it the
"configuration deployment", because those are configuration files which
should be inherited from the config merge, not the origin merge.

Prep for `makecache`.

Closes: #1038
Approved by: cgwalters
2017-10-05 18:00:54 +00:00
Colin Walters
95227f079a WIP: Implement ex livefs --replace
Lots of tradeoffs in this.  See the comments in the code for
more info.  WIP for now.

Closes: #1028
Approved by: jlebon
2017-10-05 13:19:20 +00:00
Jonathan Lebon
4501791dc2 daemon: fix uint negative comparison
Closes: #1036
Approved by: cgwalters
2017-10-04 21:10:55 +00:00
Colin Walters
cafe89cab7 daemon: Exit on idle after ~10-15 seconds
Now that we have the ability to both track clients and our active transaction,
and the `RegisterClient` call acts "atomically", let's start doing exit-on-idle
and return the RAM to the people.

Closes: #606
Approved by: jlebon
2017-10-03 17:56:21 +00:00
Colin Walters
6ad113ca46 daemon: Some misc new style porting
Was just reading the initramfs bits while talking to rubao about kargs work, and
noticed we had a lot of "old style" declarations.

Closes: #1020
Approved by: jlebon
2017-09-28 19:29:31 +00:00
Colin Walters
13f307da36 livefs: Update /usr/lib/{passwd,group}, run systemd-tmpfiles
This gets us to supporting live installs of `httpd` and many
other services.  I also tried `postgresql-server` for example.

Closes: https://github.com/projectatomic/rpm-ostree/issues/962

Closes: #996
Approved by: jlebon
2017-09-25 15:10:15 +00:00
Colin Walters
cc4844ff02 daemon/upgrader: Clear core ctx after committing
Amazingly we had 3 variations of this bug in the different
layers above using the `RpmOstreeContext` API (in compose, container,
and layering).

We fixed the first two already, this fixes the last one.  We
get a warning if librpm holds a ref to a deleted rpmdb.

Closes: https://github.com/projectatomic/rpm-ostree/issues/987

Closes: #994
Approved by: jlebon
2017-09-15 14:17:59 +00:00
Jonathan Lebon
f62f6b7086 livefs: tweak warning re. /etc
Be more explicit about what the warning means.

Closes: #988
Approved by: dustymabe
2017-09-13 13:51:06 +00:00
Jonathan Lebon
2a5ecf7a3e livefs: add warning that /etc won't be preserved
Since the whole premise of livefs is that you stay in your deployment
just a little longer, it becomes much more likely for people to run into
https://github.com/projectatomic/rpm-ostree/issues/40 (myself included).

Printing such a notice was discussed in the initial livefs design
discussions: https://github.com/projectatomic/rpm-ostree/issues/639.

Closes: #986
Approved by: cgwalters
2017-09-12 19:51:48 +00:00
Jonathan Lebon
077d7c1a9e daemon/deploy: allow redeploying livefs booted commit
Also something I noticed while working on #981. When sitting on a livefs
commit, once a user does `rpm-ostree cleanup --pending --rollback`, it's
impossible to redeploy the same booted commit. Let's allow users to do
this.

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Jonathan Lebon
c08ca8f922 daemon/upgrader: make use of override-commit-ids
Currently, when setting the `override-commit` key in the origin, the
upgrader pulls that commit checksum directly and then updates the
refspec to point to it. This behaviour was inherited from its ostree
version; at the time it was implemented, the pull code didn't support
passing a specific commit for a given refspec. However, we now have
the override-commit-ids option, which will make libostree update the ref
for us.

We change the code here to make use of it and simplify the function.
This also fixes the corner case of local branches: we shouldn't change
the ref if we're on a local branch. This is actually what drove me to
this patch as I was debugging #981.

(Aside: I'm still not convinced updating the refspec is always the
correct thing to do even in the remote case, though it's a bit messy to
disentangle).

Closes: #984
Approved by: cgwalters
2017-09-12 17:23:25 +00:00
Colin Walters
f113fc5e27 Rework treecompose kernel processing
Prep for changing `boot_location: new` to use `/usr/lib/ostree-boot`
and `/usr/lib/modules`.  Rework our kernel postprocessing
so that we unify the `boot_location` handling with initramfs generation.

Instead of doing the initramfs first in postprocessing, we do it nearly last,
after e.g. `etc` is renamed to `usr/etc`. This has some consequences, such as
the fact that `run_bwrap_mutably()` is now called in both situations. In
general, our handling of `etc` is inconsistent, although understandably so.

As part of this, I finally got around to implementing the bit from
https://github.com/systemd/systemd/pull/4174 however suboptimal it is; need the
unified core so we can cleanly ignore the posttrans like we do others.  We
intentionally keep the file around in the generated tree so that installing a
kernel RPM per client doesn't try to do any of this either.

This all gets folded together so that the logic for handling the bootloader gets
simpler - in the Fedora case, we now know to find kernels in `/usr/lib/modules`
and can ignore `/boot`.

Closes: #959
Approved by: jlebon
2017-09-12 14:26:44 +00:00
Colin Walters
b5c98ec4f1 daemon: Add a policy for reload-config
This fixes `rpm-ostree reload` as root, and supports configuring
it to be enabled for other users as well.  This was overlooked
in the polkit work originally.

Closes: https://github.com/projectatomic/rpm-ostree/issues/976

Closes: #977
Approved by: jlebon
2017-09-07 15:11:00 +00:00
Colin Walters
f69717bf69 libpriv+daemon: Use autocleanup to abort ostree txn
This allows porting one function each in the unpacker and sysroot to new style.
There were also two cases in the core where we were missing an
`abort_transaction()` invocation.

libostree version in: https://github.com/ostreedev/ostree/pull/1132
Prep for: https://github.com/projectatomic/rpm-ostree/pull/970

Closes: #972
Approved by: jlebon
2017-09-07 13:50:19 +00:00
Colin Walters
4cbdcf93c8 daemon/syscore: Add some error prefixing to cleanup paths
Would likely have helped clarify this issue where an ostree repo txn failed
post-bootloader txn: <https://github.com/projectatomic/rpm-ostree/issues/958>

Closes: #960
Approved by: jlebon
2017-08-29 13:24:06 +00:00
Colin Walters
acbf00724d lib/unpacker: Drop ostree convention flag, always enable it
The core use case always enabled this; we exposed it on the command line, but
honestly I think it's a lot simpler if our imports always enable this. If we
need to *undo* it (for example, generating a "traditional feeling" OCI/Docker
image a la <https://github.com/projectatomic/rpm-ostree/pull/367>), we can
easily do that as postprocessing afterwards.

Closes: #951
Approved by: jlebon
2017-08-25 19:25:29 +00:00
Jonathan Lebon
e605386818 daemon: make use of the new deployment flags
Now that we have the semantics we need in libostree, let's just use that
and drop the logic here.

Closes: #942
Approved by: cgwalters
2017-08-25 18:59:24 +00:00
Colin Walters
3840fad28e tree-wide: Fully purge the rest of "noscripts" support
It was partially removed in 325ee354e9
from PR #873 - this removes all of the rest of the layers in
the daemon and core for this.

Note that while working on this I found it astoundingly confusing that
`rpmts_add_install()` and `add_install()` had separate orderings for `gboolean
noscripts, is_install`. That's fixed by having the latter lose the `noscripts`
flag entirely, but while we're here change `rpmts_add_install()` to use flags
for clarity.

Closes: #938
Approved by: jlebon
2017-08-23 16:31:27 +00:00
Colin Walters
0428275a3c tree-wide: Port to new libglnx tmpdir API
Lots of cleanups, and this also allows us to fully port to new style in several
places.

We can drop our own `rpmostree_mkdtemp()`.

Update submodule: libglnx

Closes: #928
Approved by: jlebon
2017-08-22 17:21:37 +00:00
Colin Walters
82f95e7761 daemon/upgrader: Use new libostree timestamp-check option
Since we have a copy of this libostree code, pick up the new
changes from <https://github.com/ostreedev/ostree/pull/1055>.

Note the added test doesn't really test our logic since
we're only doing local pulls, but at least we have something.

Closes: #932
Approved by: jlebon
2017-08-22 16:08:21 +00:00
Colin Walters
f51611dc74 tree-wide: Some misc libglnx porting
Port away from `glnx_set_error_from_errno` to prefixing errors or libglnx
wrappers as appropriate.

Closes: #926
Approved by: jlebon
2017-08-18 17:38:49 +00:00
Colin Walters
fe1be77025 sysroot: Prefix error when loading deployment with index
This would have helped me debug <https://github.com/projectatomic/rpm-ostree/issues/923>.

Closes: #924
Approved by: jlebon
2017-08-17 16:53:48 +00:00
Jonathan Lebon
ef889b0b3a daemon: use new journal-msg signal
Make use of the new journal-msg signal that OstreeSysroot emits. Since
we now also only rely on the output API to print messages, this allows
us to drop the outrageous hacks we had to do to watch our own stdout,
part of which Coverity didn't like.

Coverity CID: 163694

Closes: #919
Approved by: cgwalters
2017-08-16 16:05:06 +00:00
Jonathan Lebon
1fb9dc57c9 daemon: ensure we consistently use output API
There were places where we were relying on the stdout self read hack to
relay messages to clients. Fix those by explicitly using the output API
in all spots where the daemon may pass. Really, we should exclusively be
using the output API in the daemon and libpriv trees, though many other
instances exist for now.

Closes: #919
Approved by: cgwalters
2017-08-16 16:05:06 +00:00
Jonathan Lebon
7b774cee7e output: add support for messages
This is already supported by the D-Bus API, though not in our output
wrapper.

Closes: #919
Approved by: cgwalters
2017-08-16 16:05:06 +00:00
Colin Walters
b6705f3feb daemon: Check for updated rpms when upgrading
This closes a longstanding bug - since package layering first
landed, we only checked for newer RPMs if the base tree changed.
In some scenarios like RHELAH, this doesn't matter much by default
since they move at the same cadence.  Except if you use EPEL for example.
In Fedora, today the FAH releases are async of the rpm-md repos, and
there's also COPR which can update more than once a day even.

We should check for both update sources. Luckily we'd already introduced logic
for this in the treecompose case (checksumming the depsolved package sack). We
just need to start using it for client side assembly too.

Closes: https://github.com/projectatomic/rpm-ostree/issues/391

Closes: #911
Approved by: jlebon
2017-08-16 15:33:11 +00:00
Jonathan Lebon
7553b0000b livefs: comment out dead code
The `replacing` var was already checked to be FALSE higher up.

Coverity CID: 163689

Closes: #918
Approved by: cgwalters
2017-08-15 20:26:14 +00:00
Colin Walters
ee1613e4bf livefs: Stop changing immutable bit on deployment roots
This is no longer necessary since we switched to tracking livefs state in the
origin, and we don't support changing content outside of `/usr` right now
via livefs.

Just noticed this while looking at the livefs code.

Closes: #912
Approved by: jlebon
2017-08-10 13:58:41 +00:00
Jonathan Lebon
dc24dd3105 rollback: allow users to undo a rollback
The new API to find pending and rollback deployments do so relative to
the booted deployment. This caused an interesting behaviour: the first
time a user uses "rpm-ostree rollback", it would (as expected) move the
previous deployment first. but the second call to "rpm-ostree rollback"
would fail since there were now no more rollback deployments.

We fine tune the logic here to allow this, as well as the more general
case of putting the booted deployment back on top.

This fixes a subtle regression from b7cf58e
(https://github.com/projectatomic/rpm-ostree/pull/767).

Closes: https://github.com/projectatomic/rpm-ostree/issues/906

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00
Jonathan Lebon
d9c1e4a5d3 daemon: use upstreamed function to find rollback
Now that the query_deployment() function was upstreamed in ostree (see
https://github.com/ostreedev/ostree/pull/897), let's just use it from
there and drop our copy here.

Closes: #907
Approved by: cgwalters
2017-08-03 14:43:16 +00:00