Commit Graph

2603 Commits

Author SHA1 Message Date
Jonathan Lebon
a1014aee2f daemon: Perform kargs in cache-only mode
Users don't expect the tree to get updated here.

Closes: #1705

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
b104a289f7 daemon/kargs: Fix handling of staged deployments
When handling `GetDeploymentBootConfig()`, we would trip an assertion
when trying to read the full bootconfig from a staged deployment, which
of course doesn't have a full bootconfig yet. Rework this to add a new
`staged` key to the returned dict, in which case only `options` is
included. (Which is all `rpm-ostree kargs` needs anyway).

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
770e6e72e1 compose: Don't require full previous commit
Right now, if we want to commit onto the same ref, we require the full
previous commit. The previous commit full root is currently used as
follow:

- for handling `check-passwd` in the `previous` case
- as an optimization for SELinux policy matching (#1659)

Let's change the logic so that we gracefully handle partial commits.
This patch lowers full commit root loading in the one function that
really needs it (`rpmostree_passwd_compose_prep()`). We also turn off
the SELinux optimization if the commit is partial.

My goal here is to use this in coreos-assembler to seed the repo just
enough to keep the commit history intact. (Though we do have cache so
that we can still profit from the SELinux optimization if it's
populated).

Closes: #1704
Approved by: cgwalters
2018-12-07 21:51:14 +00:00
Jonathan Lebon
6e4d653292 compose: Factor out SELinux policy checkout
Simplifies return paths. Prep for next commit.

Closes: #1704
Approved by: cgwalters
2018-12-07 21:51:14 +00:00
Jonathan Lebon
9aee7d43ce libpriv/passwd-util: Rework error-handling
In the passwd/group migration code, rather than do a bunch of work and
then check for the error cases where we only migrate one of passwd and
group, just bring those checks and and queue the work at the end. This
simpifies the logic a bit since we don't have to maintain a
`perform_migrate` variable as well and instead can just return early in
the trival cases.

Closes: #1704
Approved by: cgwalters
2018-12-07 21:51:14 +00:00
Colin Walters
f811828543 rust/treefile: Add container key
This was omitted since in practice we aren't actually testing it,
the container path is mostly via `ex container` which uses keyfiles.

Closes: #1701

Closes: #1702
Approved by: jlebon
2018-12-06 19:03:06 +00:00
Colin Walters
ff80765a86 daemon: Drop unused mutex
Was reading the code while looking at
https://github.com/projectatomic/rpm-ostree/issues/1692
and blinked at the comment, then noticed it hasn't been used
since:  3e289ffab0

Closes: #1699
Approved by: jlebon
2018-12-06 16:20:48 +00:00
Jonathan Lebon
544d79fd3f compose: Fix EBADF in unified core mode without cachedir
If no cache dir is given in the workdir, we would alias the cache dir fd
to the workdir fd. But of course, this meant that we'd try to close the
same fd twice when freeing the compose context. Instead, let's just copy
the fd as is also done in the non-unified path.

Closes: #1697

Closes: #1698
Approved by: lucab
2018-12-06 13:44:19 +00:00
Jonathan Lebon
bdf3cda8db Print CVEs fixed in available updates
One question I often have when looking at the output of `status -a`:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
           Diff: 67 upgraded, 1 removed, 16 added
```

is "How serious and relevant are these advisories to me? How soon should
I reboot?". For the packages that I'm most familiar with, e.g. `kernel`
and `git-core`, I usually look up the advisory and check why it was
marked as a security update, mentioned CVEs, and how those affect me.

The updateinfo metadata includes a wealth of information that could be
useful here. In Fedora, CVEs treated by the security response team
result in RHBZs, which end up attached to the advisories and thus make
it into that metadata.

This patch tries to reduce friction in answering some of those questions
above by checking for those CVEs and printing a short description in the
output of `status -a`. Example:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                   CVE-2018-18065 CVE-2018-18066 net-snmp: various flaws [fedora-all]
                   https://bugzilla.redhat.com/show_bug.cgi?id=1637573
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                   CVE-2018-16862 kernel: cleancache: Infoleak of deleted files after reuse of old inodes
                   https://bugzilla.redhat.com/show_bug.cgi?id=1649017
                   CVE-2018-19407 kernel: kvm: NULL pointer dereference in vcpu_scan_ioapic in arch/x86/kvm/x86.c
                   https://bugzilla.redhat.com/show_bug.cgi?id=1652656
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
                   CVE-2018-19486 git: Improper handling of PATH allows for commands to executed from current directory
                   https://bugzilla.redhat.com/show_bug.cgi?id=1653143
           Diff: 67 upgraded, 1 removed, 16 added
```

Including the CVE name and RHBZ link also makes it easier to look for
more details if desired.

Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Jonathan Lebon
51d48db235 tests/utils/updateinfo: Rename function
Use a better function name for the shallow copy we do in
`copy_update_no_cols()`. Also add a better comment.

Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Jonathan Lebon
e7d256b4c8 tests/utils/updateinfo: Assuage flake8
Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Colin Walters
5554ca10d1 Add rpm-ostree-bootstatus.service
Capturing the system state at boot aids debugging.  This is a
trivial implementation; we could in the future do structured
logging too.

The high level goal here is to help us track system state in
Red Hat CoreOS.

Closes: #1693
Approved by: jlebon
2018-12-03 15:38:50 +00:00
Colin Walters
09e7caef4e compose: Add a CUtf8Buf copy of rojig_name
Avoids passing an allocated buffer from Rust to C; there's
controversy in the PR I sent to rust-lang around defining this as
supported.

Closes: #1691
Approved by: jlebon
2018-11-28 16:53:00 +00:00
Colin Walters
62744449dc rust: Use Fallible<> more
Since use of the `failure` crate has been a success, let's use
it a bit more.  The big thing to convert left is `treefile.rs` which
does need a custom error so we can stop abusing `io::ErrorKind::InvalidInput`.

Closes: #1690
Approved by: jlebon
2018-11-28 16:00:56 +00:00
Jonathan Lebon
01c349a558 libpriv/rpm-util: Tweak changelog entry indentation
Indent the changelog entries so it's easier to tell which entries belong
to which packages.

Before:

```
$ ros db diff -c
ostree diff commit old: rollback deployment (ef3c0e45ee1b874bc4952904a778084f0c32a0e06346e271786abf544dad54ff)
ostree diff commit new: booted deployment (c1d4c3ef571a53e5ab3dbd6ea1ec91a374ebd5f6ba9c0a99938d0649853588a5)
Upgraded:
  firefox 63.0.3-1.fc29.x86_64 -> 63.0.3-2.fc29.x86_64
* Wed Nov 21 2018 Martin Stransky <stransky@redhat.com> - 63.0.3-2
- Fixed mozbz#1507475 - crash when display changes (rhbz#1646151).

  httpd 2.4.37-3.fc29.x86_64 -> 2.4.37-5.fc29.x86_64
* Fri Nov 23 2018 Lubos Uhliarik <luhliari@redhat.com> - 2.4.37-5
- Resolves: #1652678 - TLS connection allowed while all protocols are forbidden

* Thu Nov 08 2018 Joe Orton <jorton@redhat.com> - 2.4.37-4
- add httpd.conf(5) (#1611361)
```

After:

```
$ ros db diff -c
ostree diff commit old: rollback deployment (ef3c0e45ee1b874bc4952904a778084f0c32a0e06346e271786abf544dad54ff)
ostree diff commit new: booted deployment (c1d4c3ef571a53e5ab3dbd6ea1ec91a374ebd5f6ba9c0a99938d0649853588a5)
Upgraded:
  firefox 63.0.3-1.fc29.x86_64 -> 63.0.3-2.fc29.x86_64
    * Wed Nov 21 2018 Martin Stransky <stransky@redhat.com> - 63.0.3-2
    - Fixed mozbz#1507475 - crash when display changes (rhbz#1646151).

  httpd 2.4.37-3.fc29.x86_64 -> 2.4.37-5.fc29.x86_64
    * Fri Nov 23 2018 Lubos Uhliarik <luhliari@redhat.com> - 2.4.37-5
    - Resolves: #1652678 - TLS connection allowed while all protocols are forbidden

    * Thu Nov 08 2018 Joe Orton <jorton@redhat.com> - 2.4.37-4
    - add httpd.conf(5) (#1611361)
```

Closes: #1687
Approved by: rfairley
2018-11-28 13:40:41 +00:00
Jonathan Lebon
0e8eab7bfc libpriv/rpm-util: Use g_print instead of printf
For consistency with the rest of the codebase. There were also a few
spots which were missing the space between the function name and the
opening parenthesis.

Closes: #1687
Approved by: rfairley
2018-11-28 13:40:41 +00:00
Colin Walters
a177f551b0 rust/ffi: Convert String creation API, extend docs
Create a new `_new_` naming convention, and extend the FFI
documentation to describe the new state as well as background assumptions.

Closes: #1688
Approved by: jlebon
2018-11-28 09:14:41 +00:00
Colin Walters
a2f5b7c1c0 rust/ffi: More "view" changes, for bytestrings
Change our existing "view as [u8] API", and also add one
that does a view as `OsStr`.  The motivation for the latter
is I noticed ithat `OsStr::from_bytes()` *doesn't* copy,
or rather it just copies the pointer value.  Rust's lifetime
inference ensures that the returned lifetime matches the input array.

I think the previous code in `treefile.rs` was confused about this.

Closes: #1688
Approved by: jlebon
2018-11-28 09:14:41 +00:00
Colin Walters
bfe637a877 rust/ffi: Convert str func to new "view" naming convention
More obvious one shouldn't leak the pointer.

Closes: #1688
Approved by: jlebon
2018-11-28 09:14:41 +00:00
Colin Walters
db0319beac rust/ffi: Change dfd handling to unwrap, create naming convention
I was going to add another usage of this function, and I think the
gerror stuff is unnecessary - if we are handed a bad file descriptor
(or a fd pointing to a regular file) that's something where we should
just abort.

While we're here, I'd like to codify expected usage in the function
names here.  If you like this I'll e.g. also change `str_from_nullable`
to `ffi_view_nullable_str`.

Closes: #1685
Approved by: jlebon
2018-11-27 15:42:08 +00:00
Jonathan Lebon
140704c105 daemon/transaction-types: Fix dnf cache override
Minor regression from #1587. There were places that were still doing
`dnf_context_set_cache_age()` manually, but those calls didn't exactly
have the intended effect since the core now handled caching itself.

The actual result was that the metadata was still being updated, but not
during the `dnf_repo_check` pass that the core does, but rather the
`Importing rpm-md` pass it does right after. So then, we were
incorrectly printing `(cached)` even though we'd update it afterwards.

Switch to the new way of doing things.

Closes: #1686
Approved by: cgwalters
2018-11-26 23:39:44 +00:00
Jonathan Lebon
1164b7b047 daemon/transaction-types: Drop unneeded arg
Drop the `force_refresh` boolean parameter since we only ever call it
with `TRUE`. I think this dates from an earlier implementation where we
did call it with differing values.

Small prep for next patch.

Closes: #1686
Approved by: cgwalters
2018-11-26 23:39:44 +00:00
Jonathan Lebon
2cc06ce89a configure.ac: Add "GitHub release" step to release workflow
Let's make use of the GitHub release feature to make it more prominent
on the "Releases" tab, but more importantly so that we can attach
vendored tarballs for downstreams. E.g. this will allow us to have a
correct `Source0` field in the Fedora spec file.

Related: #1683

Closes: #1684
Approved by: rfairley
2018-11-26 22:59:50 +00:00
Jonathan Lebon
ed2fefad85 Move reset out of experimental
This is relatively uncontroversial functionality that has already proved
useful when helping folks debug their stuff. Let's promote it to the
stable interface.

Closes: #1682
Approved by: rfairley
2018-11-22 20:47:50 +00:00
Jonathan Lebon
03a92c0981 app/ex: Nuke ex kargs compatibilty alias
It said so right there; "nuke in next version". No need to carry this
stuff forever. It's the final phase of feature promotion, like when one
officially moves out of their parents'.

Closes: #1682
Approved by: rfairley
2018-11-22 20:47:50 +00:00
Colin Walters
e0a976386a rust: Use failure crate for errors
In a lot of places we're abusing `io::Error(io::ErrorKind::InvalidInput)`
which is both verbose and inaccurate really.  Maybe in some
places we should be defining custom errors, but eh.

I like the `failure` crate.  Use it in just `utils.rs` for now.
Tweak our error handling FFI wrappers to accept `Display` since
all we do is convert the error to a string.

Closes: #1675
Approved by: lucab
2018-11-21 22:03:53 +00:00
Colin Walters
387da3a420 ci: Verify rustfmt
Split out the command into a separate `Makefile` that doesn't
use Automake so we can invoke it early.

Closes: #1674
Approved by: dustymabe
2018-11-21 21:16:03 +00:00
Colin Walters
6b5e475607 rust: Rerun rustfmt
Going to add a CI step next.

Closes: #1674
Approved by: dustymabe
2018-11-21 21:16:03 +00:00
Jonathan Lebon
17359bc0cc core: Tweak "Importing" messages
To make it more obvious what the difference between "Importing metadata"
and "Importing" is, add "rpm-md" to the first and "packages" to the
second.

Closes: #1681
Approved by: cgwalters
2018-11-21 13:54:35 +00:00
Colin Walters
da8be3f7b8 scripts: Add a hack to handle RHEL7's glibc-common.post
While we had an override for both the Fedora `glibc-all-langpacks.posttrans`
version and the RHEL7-era `glibc-common.post`, there were two
problems.

First, the RHEL7 version's lua calls `rpm.expand()` internally
rather than flagging the script itself for expansion.

Second, we also need to disable rofiles-fuse for it.

Closes: #1678
Approved by: jlebon
2018-11-21 08:14:24 +00:00
Jonathan Lebon
c9cbad94a3 daemon/utils: Finish OstreeAsyncProgress after pull
Otherwise, the object might still own an idle source on the main
context, which will cause issues if another pull operation happens
again. This wasn't causing issues before because in places where we did
do multiple pull operations, we would reuse the same
`OstreeAsyncProgress` object, and the second pull operation *did* do
`ostree_async_progress_finish()`. But that's no longer the case now with
66761916.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Jonathan Lebon
c7ce2564dc daemon/transaction: Emit progress end signal
This is not strictly necessary since the progress is considered ended on
the client side when the transaction is finished, but let's be nice.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Jonathan Lebon
71f0a30d83 ci: Go back to centos/7/atomic/smoketested
We should prefer the smoketested ref in general since it's more stable.
There is an updated glib2 there now. I also tested that layering-relabel
works (at least locally).

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
ce339dff9b vmcheck/multitest: Print when we're waiting
To help debug.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
11ae25792c build-sys: Add bindgen bits to .gitignore
Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
dcc78cc2f1 build-sys: Fix uid check when we don't have /usr/bin/cbindgen
Make this match the other case.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
c08e9620d6 vmcheck/multitest: Time execution of each test
For some reason c7 is consistently timing out, let's gather
data on how long our tests take to execute.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
33178cb621 vmcheck: Sync over libsolv and zchunk-libs too
In general our current CI/test system is susceptible to drift
between the container and AH.  The direction we should be
going is to have coreos-assembler solve this problem with
a SDK, but for now, let's ensure that the container's libsolv
makes it to the host, same thing we do for libostree.

Closes: #1676
Approved by: cgwalters
2018-11-20 18:06:34 +00:00
Colin Walters
89486eace1 core: Typo in indicatif commit: metdatametadata
A typo.

Closes: #1673
Approved by: arithx
2018-11-13 20:08:59 +00:00
Colin Walters
1dee43319c Use indicatif for progress
This turned out to be messier than I thought, because of two primary
factors; the biggest mess here of course is the indirection
through the DBus API.

The other problem is that previously we passed the string to render
each time, and with current indicatif that'd trigger a rerender.
Since (usually) don't change the "prefix string", rework the API.

Change the "percent/n_items" bits to use autocleanups as well, and
to take the prefix string as an initial argument.

Since the state expands to multiple components, also change the
API to use the `0-initialized` pattern rather than trying to
return an aggregate.

We also gain a "sub message" which we use to display e.g.
package names as we're doing checkouts.  Note this ends up
at the end, since otherwise everything else jumps around.

Closes: #1661
Approved by: rfairley
2018-11-09 13:59:43 +00:00
Colin Walters
0f5fc1afca daemon/transaction: Clear out libostree progress consistently
Prep for indicatif, the new progress implementation, which is now
more strict about overwriting tasks.  The `OstreeAsyncProgress`
object lingered on and could own tasks on the mainloop.  Narrow
the scope and avoid having one that crosses multiple pull requests.

Closes: #1661
Approved by: rfairley
2018-11-09 13:59:43 +00:00
Marcel-Lambacher
2c4231b376 app/status: Output corresponding status message if GPG key is missing
The previous behaviour was to simply return "Invalid signature"
if the corresponding GPG public key wasn't found.
This status message wasn't clear enough that the key is missing.
If the GPG public key is now missing, a corresponding status message will be issued.

Closes: #1650
Approved by: rfairley
2018-11-07 14:56:16 +00:00
Jonathan Lebon
c59a136ba7 compose: Initialize out_changed var before --print-only
Otherwise we can get fun undefined behaviour like the caller thinking
there was a change when we didn't even install anything.

Closes: #1669
Approved by: cgwalters
2018-11-06 19:04:47 +00:00
Jonathan Lebon
dd2134e54f compose: Drop 'ex' prefix for --unified-core in msg
That was the last bit in which we referred to the old unified core
option. Also change it in the source itself for completeness.

Closes: #1668
Approved by: cgwalters
2018-11-06 14:21:35 +00:00
Jonathan Lebon
c884a72033 compose: Allow --print-only without bwrap support
We don't need to run the bwrap self test if we just want to print the
manifest. I played with putting the self test in `impl_install`, though
e.g. `postprocess` also needs this so it wasn't quite right.

Closes: #1666
Approved by: cgwalters
2018-11-05 22:12:06 +00:00
Jonathan Lebon
dfb7bd8487 compose: Support all target repos in unified mode
Previously, we were limiting the target repo in unified mode to be a
bare-user repo located on the same filesystem (see message of previous
commit). This patch lifts this restriction by making a distinction
between the *build repo* and the *final* target repo.

To do this, we create a bare-user repo located near the pkgcache to
take advantage of hardlinks and devino caching at commit time. And only
after committing do we essentially `pull-local` into the final target
repo. This of course allows us to avoid potentially pulling across the
two filesystems file objects that are already present in the target
repo.

This will be used by coreos-assembler:
https://github.com/coreos/coreos-assembler/pull/190

Closes: #1490

Closes: #1657
Approved by: cgwalters
2018-11-05 21:29:31 +00:00
Jonathan Lebon
21886fc936 compose: Always put workdir on same filesystem as pkgcache
This ensures that we always get hardlinks when checking out of the
pkgcache. This works right now because we indirectly require the target
bare-user repo and the pkgcache to be on the same filesystem by setting
`no_copy_fallback` in the core (I say "indirectly" because that setting
only enforces the workdir to be on the same filesystem as the pkgcache
repo, but since the workdir is currently placed inside the bare-user
repo...).

However, I'd like to change the requirement of a bare-user repo so that
one can commit into a repo on a different file system or a repo of a
different type (e.g. archive repo). This is prep for that.

Closes: #1657
Approved by: cgwalters
2018-11-05 21:29:31 +00:00
Colin Walters
5d9e25653c libvm: Don't sync rust target/
Since it gets enormous, mine is 1.5GB right now.

Closes: #1665
Approved by: jlebon
2018-11-05 19:59:58 +00:00
Colin Walters
bd4d0d8e0a build-sys: With external cbindgen, drop Rust lib dependency
Perhaps an unexpected side benefit of slow compilation processes
is that one has an opportunity to reflect and ponder.

I realized during exactly such a moment that since we moved
`cbindgen` out of our library build, there's no need to wait
for the library to be built before we can start building the C
code.

This is a notable local quality-of-life development improvement.

Closes: #1665
Approved by: jlebon
2018-11-05 19:59:58 +00:00
Jonathan Lebon
55696824df compose: Use previous sepolicy if available
Rather than defaulting to the host system's SELinux policy, we can be
much more efficient here if we instead use the policy of the last commit
if available. Likely, the pkgcache is currently labeled with that
policy, which means we skip the relabeling phase before checkout. But
also, if the policy didn't change at all in the new rootfs, we also skip
the second relabeling phase after assembly.

Closes: #1659
Approved by: cgwalters
2018-11-05 19:20:40 +00:00