Commit Graph

775 Commits

Author SHA1 Message Date
Colin Walters
05a59e5af5 core: Don't try to chmod() symlinks
I just noticed this while reading the code. Right now we're actually following
links (since `AT_SYMLINK_NOFOLLOW` isn't implemented), which could definitely
lead to bad things if e.g. one has a symlink pointing to a file which shouldn't
be executable or world-readable etc.

(Also I noticed our hardlink-breaking-only-if-regular logic is now broken;
 I added a TODO)

Closes: #677
Approved by: jlebon
2017-03-13 17:53:51 +00:00
Colin Walters
77fa10cfbe core: Skip (and warn on) rpmfi overrides outside of /{usr,etc,var}
I still need to query whether Fedora has anything like this; maybe `/boot`? But
things are not likely to work right now if we do, so let's ignore it.

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
ab7431c458 core: Apply rpmfi overrides for /etc to /usr/etc
This fixes installing `mock`, which for some reason has files in `/etc/mock`
owned by `root:mock`.

Closes: https://github.com/projectatomic/rpm-ostree/issues/671

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
d8b96a8531 core: Include filename in error messages for rpmfi overrides
This gives me now:

```
error: While applying overrides for pkg mock: fstatat: etc/mock/custom-1-aarch64.cfg: No such file or directory
```

Which is definitely more useful for debugging 💣.

Closes: #677
Approved by: jlebon
2017-03-13 17:53:50 +00:00
Colin Walters
62320b1920 rebase: Update man page for syntax, print if just changing remote
The rationale for just printing if changing remote is I think
it's really only that version that feels "magical".

Closes: https://github.com/projectatomic/rpm-ostree/issues/569

Closes: #678
Approved by: jlebon
2017-03-13 14:45:30 +00:00
Jonathan Lebon
81c43e81fb PkgChange: add support for layering local RPMs
Allow users to directly specify an RPM file on the command-line. The
"packages_added" array of the PkgChange() method can now contain
absolute paths to RPM files.

Grow the origin format to have a new "requested-local" key. This is
similar to the "requested" key, except that the packages are always
installed from cache. The "requested-local" array values also embed the
SHA-256 of the header we expect.

There is now a new "LocalPackages" line in the status. These packages
are a subset of the "packages" element (which are printed as
"LayeredPackages") and represent the packages that are explicitly marked
for installing from cache.

Interesting design choices/notes:
  - Just as before, even with foo-1.0-1.x86_64 installed from RPM, a
    user can still request "/usr/bin/foo": it will be made dormant. As
    soon as foo stops being explicitly layered from the RPM, it will try
    to fulfill the request by going to the repos. This allows users to
    "pin" a layered package to a certain RPM, and then unpin it.
  - The strings/NEVRAs in "requested" and "requested-local" are strictly
     distinct. This allows us to be able to tell what the user means
     exactly when they do "rpm-ostree uninstall".

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Jonathan Lebon
5c5ebc1b0a core: add support for installing from cache
We start by adding support in the core for installing packages strictly
from the cache repo. We fool the libdnf stack by re-exporting the header
as an RPM, and explicitly marking it for install. The treefile format
supports specifying the expected SHA-256 of the metadata header, in case
the cache for a specific NEVRA changed.

Closes: #657
Approved by: cgwalters
2017-03-10 18:35:11 +00:00
Colin Walters
e86dc13c49 bwrap: Don't use --unshare-net in nspawn by default
This will fix rpm-ostree-in-mock-in-koji. The drawback is minor: post scripts
will have network access. But we're going to be testing the no-network case in
our Docker-based builds, so that's fine.

Closes: #672
Approved by: jlebon
2017-03-10 17:27:56 +00:00
Jonathan Lebon
49cbdb739a status: always include the packages entries
Pull #646 introduced a subtle regression: we went from always including
a "packages" entry to only including it if there are packages present.
Albeit it's easy to guard against, though to be nice, let's make it
easier for consumers by always including it.

Reported-by: Micah Abbott <miabbott@redhat.com>

Closes: #670
Approved by: cgwalters
2017-03-10 14:54:45 +00:00
Colin Walters
14a9135ea0 main: Output YAML for --version
Equivalent of https://github.com/ostreedev/ostree/pull/691
Basically, it's nicer to parse.

Closes: #665
Approved by: jlebon
2017-03-09 14:23:34 +00:00
Colin Walters
7cf366497f daemon: Log txn initiation
Down the line we should have a saner stringification, and also
log things like the invoking `loginuid` etc.

But this gets us something.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
fd6b30746e daemon: Move mainctx iteration out of main() into object
Preparation for idle exit - the daemon knows when it should idle exit. It's
tempting to move all of the mainloop logic there, but it'd be more code churn
than I want to do right now.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
927d9d061a daemon: Track transaction status, render it in systemd status
This is part of the idle exit work - the higher level daemon logic needs to
track running transaction state, so we can combine that with whether we have `> 0` clients.

(Yes, we need a better stringification of txns)

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
33e9f2a4cd daemon: use sd_notify() when we receieve a signal
Just to keep the status display up to date.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
d3772ffc4f daemon: Use sd_notify(STATUS=) to print how many clients are active
Now `systemctl status rpm-ostreed` will show it. In future patches I plan to add
whether we have an active txn and our pending idle exit timeout.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
98c1f43326 daemon: Add RegisterClient API
This is like what bluez does. With this, we have a stronger mechanism to avoid
races with future work to auto-exit on idle.  Registered clients
hold a reference to the daemon effectively.

Note that calling `UnregisterClient` is optional if the calling process is going
to exit soon - as is the case for using the command line binary via e.g.
`rpm-ostree status`.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
d4789e0886 daemon: use systemd notify(STOPPING=1) + ReleaseName combo
More info here:

 - https://lists.freedesktop.org/archives/dbus/2015-May/016671.html
 - https://github.com/cgwalters/test-exit-on-idle

This is preparatory work for doing exit-on-idle, which we want to do because we
currently leak memory badly. Even if we didn't leak, it's still better to not
have services sit around resident.

While I had the patient open, I did some code cleanup such as switching to
direct `g_main_context_iteration()` and using `sd_journal_print()` instead of
`g_warning()` etc.

Closes: #660
Approved by: jlebon
2017-03-07 22:33:58 +00:00
Colin Walters
7b8e05ffc2 Allow and start using C99 declaration-after-statement
The equivalent of https://github.com/ostreedev/ostree/pull/718
but for this codebase.

I just picked one example at random, there's plenty of others, but I don't want
to do any kind of tree-wide conversion since we have lots of outstanding
patches.

Closes: #664
Approved by: jlebon
2017-03-06 23:16:30 +00:00
Colin Walters
79ee224709 core: Log commit stats into journal
It's useful to see how many new objects were added and how much storage they
use. Just part of a general trend where I'd like to make a lot more extensive
use of the journal, particularly with structured logging.

Closes: #661
Approved by: jlebon
2017-03-06 22:19:10 +00:00
Colin Walters
e1e2bf9fbe compose-tree: Print nsolvables, consolidate disabled repos
This should help us if we ever find corrupt repodata; I thought
that might be happening in https://pagure.io/releng/issue/6602
Also, consolidate the disabled repos printing, since I think
we don't need to be verbose about those.

Closes: #613
Approved by: jlebon
2017-03-06 21:21:10 +00:00
Colin Walters
f9944e6d96 daemon: Remove several unused instance variables
These were leftovers from the very original code which did support exit-on-idle;
we're going to do it differently now.

Closes: #659
Approved by: jlebon
2017-03-06 16:14:38 +00:00
Colin Walters
a00b9f94fb daemon: Tell systemd when we're shutting down
This won't actually help with much right now; just splitting out this change to
highlight it as part of future work for exit-on-idle.

Closes: #659
Approved by: jlebon
2017-03-06 16:14:38 +00:00
Colin Walters
66425c3161 core: Do GPG verification before importing
While reading a recent conversation about GPG checking at treecompose
time, I had a sudden thought - were we actually doing verification
client side?  Turned out, we aren't.  That happens as part of
`dnf_transaction_commit()` which we don't use.

That function verifies every package at one go, but for us I think it's better
to do it before "importing". We shouldn't have untrusted bits that we've
unpacked (they might have suid binaries, for one thing).

This is an embarassing problem, but it's worth emphasizing that everyone should
be retrieving repodata at a minimum over TLS, which sets a baseline. On RHEL, we
already do pinned TLS, and there are discussions about extending that elsewhere.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1422157

Closes: #656
Approved by: jlebon
2017-03-06 15:19:43 +00:00
Colin Walters
127275fe0e build: Depend on ostree 2017.2, drop check for gpg symbol
Since somehow it's breaking in [Fedora's Koji](https://kojipkgs.fedoraproject.org//packages/rpm-ostree/2017.2/2.fc25/data/logs/x86_64/build.log)
and I don't feel like debugging it.  Let's just take a hard dependency
now and kill the conditional.

Closes: https://github.com/projectatomic/rpm-ostree/issues/653

Closes: #655
Approved by: jlebon
2017-03-03 14:34:57 +00:00
Colin Walters
3568ed9a83 core: Plug leak of pkgcache repo
This showed up as a number of duplicate fds to the pkgcache root in
`/proc/<pid>/fd` for the daemon.

Closes: #654
Approved by: jlebon
2017-03-03 13:38:40 +00:00
Jonathan Lebon
89693c0385 core: no longer embed treespec
In the case of client layering, we hackily use the treespec because
that's what the core understands (for now), but it really shouldn't be
part of the final commit, nor should we rely on it.

This patch starts the path towards moving us away from the treespec by
not embedding it in client layers, and instead directly inserting
layered packages under the "rpmostree.packages" key.

The SERVER_BASE case still embeds the treespec, since only the container
path uses it for now and it needs it.

Down the line, we'll want to make the treespec just one of the methods
by which we initialize RpmOstreeContext. But nothing stops us from
hiding that detail already.

Closes: #650
Approved by: cgwalters
2017-03-01 17:42:42 +00:00
Colin Walters
22048b25a7 scripts: Use tmpfs for /var/tmp, not the host's /tmp
We don't want to expose the host's `/tmp` since that means scripts could
potentially find things like the X11 socket or whatever.

To debug things better, add a quick bash script to run bwrap like the C code
does. Perhaps down the line we can add `rpm-ostree internals run-bwrap` or so.

Closes: #647
Approved by: jlebon
2017-02-28 23:37:15 +00:00
Jonathan Lebon
b364357f60 origin: fix indentation
Closes: #651
Approved by: cgwalters
2017-02-28 22:45:07 +00:00
Jonathan Lebon
051492b3e6 upgrader: switch to stateless model
This is the culmination of the work paved by the previous commits. We no
longer handle package set mutations in the upgrader, but rather let
clients modify the origin as needed for that. As such, the upgrader is
now completely memoryless. We no longer try to error out if a package is
already in the base and we no longer drop out packages previously
layered packages that are now in the base. *Which* packages actually get
layered is determined during deployment and not coded in the origin.

At an API level, the deployment variant returned (and thus what gets
printed by `status --json`) now has both "packages" and
"requested-packages". The former retains its original semantic: it's the
set of packages that are *actually* layered. The latter contains
everything from the origin, which includes the former set + dormant
packages.

Accordingly, the output of `rpm-ostree status` also now distinguishes
between "RequestedPackages" and "LayeredPackages". Extra: we now quote
provides that contain spaces.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Jonathan Lebon
6a5d4a2699 RpmOstreeOrigin: also cache initramfs args
Ideally, we want to parse out everything in the origin on init and then
cache the values to make it even easier to inspect it. We make this more
explicit by renaming the struct members to "cache_*" and also caching
the initramfs args. This doesn't actually save any allocations right
now (though it might in the future), but we do the same for everything
else, and it's nice to be consistent.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Jonathan Lebon
d0c8dddf99 RpmOstreeOrigin: remove is_locally_assembled()
There are multiple reasons for this. The first is that with the
introduction of "dormant" packages, it becomes impossible from simply
looking at a deployment's origin whether it was layered or not. We
*could* make the upgrader turn around and insert back the "real" final
packages that were layered. But we're really trying to reinforce the
concept that the origin is mutable and represents an input for the
upgrader. Derived data shouldn't live there.

Whether a deployment is actually layered or not instead is determined by
the deployment commit itself. The embedded treespec records the exact
packages that were layered, which we will later use to enhance status
output.

Closes: #646
Approved by: cgwalters
2017-02-27 19:53:19 +00:00
Colin Walters
b034381506 daemon: Implement "reload"
There are two main issues right now; first, we don't pick up manual changes to
`.origin` files, which occurs when one needs to sed it to remove `unconfigured`
for example. Second, we need to reload changes to the remotes.

Closes: #598
Approved by: jlebon
2017-02-24 21:36:45 +00:00
Colin Walters
229fdfa7f9 scripts: Bind rather than symlink /usr/etc → /etc
This fixes resolution of relative symlinks, which fixes installation
of `firewalld` in Fedora 25.

Closes: https://github.com/projectatomic/rpm-ostree/issues/638

Closes: #640
Approved by: jlebon
2017-02-24 20:43:05 +00:00
Colin Walters
0f795cae3f daemon: Squash a memleak of parent commit
Closes: #644
Approved by: jlebon
2017-02-24 18:55:36 +00:00
Colin Walters
cc20f32fb7 tree-wide: Use autoptr cleanup for HyQuery
This was a noticeable source of leaks.

Closes: #644
Approved by: jlebon
2017-02-24 18:55:36 +00:00
Jonathan Lebon
c0b4fbc0f5 util: drop custom set_prefix_error_from_errno
We can use libglnx functions for this now.

Closes: #643
Approved by: cgwalters
2017-02-23 20:57:51 +00:00
Jonathan Lebon
b7613dcc81 RpmOstreeOrigin: don't error on unconfigured state
This is a follow-up to 775c781 (#626). Really, RpmOstreeOrigin shouldn't
concern itself with whether the origin is unconfigured. Its main goal
should be to parse it out and make it easy for users to modify it. That
sort of business logic lives in the upgrader, which *should* be
concerned if the origin in unconfigured.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Jonathan Lebon
62c7242a98 upgrader: move origin mutation outside of the upgrader
This is the beginning of a move towards making the upgrader more
stateless and less hysteretic. Ideally, the only input required should
be an origin file. Users of the upgrader are in charge of modifying the
origin as needed. The main goal of this is to simplify the upgrader
model and a better separation of concerns.

This was already done for the initramfs-related entries. This patch
formalises this for most of the other inputs, except packages, which
will be done soon.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Jonathan Lebon
c437f90a83 upgrader: drop support for ignore scripts
This functionality is disabled right now, and if we want to re-enable
it, we'll want to do it in a way that ensures the ignore scripts are
permanent.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Jonathan Lebon
3858913c98 package_diff: make use of apply_revision_override
Dedup some more code. Might be worth it to just completely unify
package_diff with deploy at some point.

Closes: #634
Approved by: cgwalters
2017-02-21 21:12:09 +00:00
Chen Fan
7ee41f75b7 remove duplicate words in comment
Signed-off-by: Chen Fan <fan.chen@easystack.cn>

Closes: #635
Approved by: cgwalters
2017-02-21 14:47:59 +00:00
Colin Walters
279287d69d upgrader: Avoid "Freed 0 pkgcache branch" pluralization failure
Let's not hardcode a broken English-only version of `ngettext` here; rework the
message to avoid requiring pluralization.

Closes: #632
Approved by: jlebon
2017-02-17 20:57:59 +00:00
Colin Walters
f6842838d7 build: Deal with gperf 3.1 changing to size_t
https://bugzilla.redhat.com/show_bug.cgi?id=1424268

Closes: #631
Approved by: jlebon
2017-02-17 18:51:53 +00:00
Eduardo Mayorga
1f2170e10c libpriv: Reduce scope of variables
nchange_date, nchange_name, and nchange_text are declared in a larger
scope in rpmostree-rpm-util.c. Reduce the scope to satisfy cppcheck.

Closes: #628
Approved by: jlebon
2017-02-16 15:40:20 +00:00
Colin Walters
ddbaf19f1e importer: Error importing RPMs which install to /opt (outside of /usr)
See https://github.com/projectatomic/rpm-ostree/issues/233 - for RPMs which
place files in e.g. `/opt`, we have different behavior in the treecompose case
(silently drop it) versus package layering (does the wrong thing).

Since the unpacker right now is only used in the layering case, this just
ensures we'll get a consistent error there.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Colin Walters
325de9d898 unpacker: Check for filter errors while committing tmpfiles too
This masked an issue in a change I was working on in the filter.

Closes: #624
Approved by: jlebon
2017-02-14 22:15:56 +00:00
Colin Walters
775c7819b7 daemon: Avoid erroring out on startup/status with origin unconfigured-state
As part of an earlier cleanup of origin parsing, we started checking
the origin `unconfigured-state` even just starting the daemon, which
is kind of bad.

It's tempting to flip the default for the parser so that we *only* check
unconfigured state if we go to upgrade, but let's not do that in this patch.

Closes: #626
Approved by: jlebon
2017-02-14 14:00:09 +00:00
Colin Walters
ae90a9d2b8 upgrader: Avoid GLib warning on shutdown after failed update
The hash tables are only initialized if the initable init completes, but it
won't if something goes wrong there. In this case, an origin
`unconfigured-state`.

Closes: #626
Approved by: jlebon
2017-02-14 14:00:09 +00:00
Jonathan Lebon
8cf071127e unpacker: support root-owned /var & /run files
I'm not sure why we weren't doing this before, but we need to also
support files in /var and /run that are owned by root.

Related: RHBZ#1421781

Closes: #622
Approved by: cgwalters
2017-02-13 16:49:59 +00:00
Colin Walters
a66d27230d Add a cleanup command and DBus API
We sometimes talk about using `ostree admin undeploy`, but that
doesn't know about the pkgcache, and hence space there leaks
until the next rpm-ostree operation.

Just for this, we need to expose a cleanup command (and API).  But
we also need to support cleaning:

 - repomd
 - downloads (repo/tmp)

So let's start implementing that.

Closes: #614
Approved by: jlebon
2017-02-13 15:28:05 +00:00