Commit Graph

2474 Commits

Author SHA1 Message Date
Colin Walters
65afbec3b1 rust: Split openat helpers into module
Will help use them in other parts of our code.

Closes: #1737
Approved by: jlebon
2019-01-23 16:00:16 +00:00
Jonathan Lebon
25d0213d15 rust/treefile: Include filename in more error msgs
This uses the `Context` feature of the failure crate to make error
messages more useful when we fail to open a file. The difference with
`map_err` is that one can still obtain the underlying error from the
context if need be. Though surprisingly, the normal `Display` for a
`Context` doesn't include the original error, so we essentially have to
do a prefix here (see [1]).

Before:

```
error: Failed to load YAML treefile: No such file or directory (os error 2)
```

After:

```
error: Failed to load YAML treefile: Can't open file "treecompose-post.sh": No such file or directory (os error 2)
```

[1] https://github.com/rust-lang-nursery/failure/issues/182

Closes: #1735
Approved by: cgwalters
2019-01-22 15:49:34 +00:00
Jonathan Lebon
1594140a33 rust/treefile: Convert more functions to Fallible
Slightly prep for next patch (was in the area so just did the whole
module).

Closes: #1735
Approved by: cgwalters
2019-01-22 15:49:34 +00:00
Jonathan Lebon
d9577618a1 Release 2019.1
Not a lot of changes, but a couple of important bugfixes.

Closes: #1736
Approved by: dustymabe
2019-01-21 17:54:25 +00:00
Jonathan Lebon
34bdffcbd2 packaging: Don't include checksums for libtool.m4 and configure
These are more files that get mangled at `%configure` time. These two
new ones specifically, I tripped on while building on ppc64le for RHEL7.
See https://bugzilla.redhat.com/show_bug.cgi?id=1113618 for more info.

Closes: #1731
Approved by: cgwalters
2019-01-15 19:20:36 +00:00
rfairley
a20d10e3d5 libpriv/util: Add date field in auto-versioning
This adds an optional date field to the prefix
passed by automatic_version_prefix. An example of specifying
the field is as follows:

10.<date:%Y>

And the fields progress like:

10.2018.0
10.2018.1
10.2018.2
10.2019.0

The date format creates a new "current date" string using
valid date directives passed into g_date_time_format().

If there is a problem reading the given date format,
an error is given and the next version is returned as NULL.

If no <date:...> tag is detected in the auto version prefix,
the same behavior as before (appending .1 and incrementing) occurs.

This may be helpful to avoid writing glue code to auto-update
the version if a date string in the commit version is desired.
Otherwise, --add-metadata-string=version= is an alternative for
complete customization.

Fixes: #1712

Closes: #1721
Approved by: jlebon
2019-01-14 18:49:41 +00:00
Colin Walters
89cdeb5131 Add rayon dependency, do postprocessing in parallel
Obviously doing these trivial functions in parallel doesn't matter,
but I wanted to dip my toes into Rayon (and Rust parallelism in
general).

`try_for_each()` is just...staggeringly simple and beautiful.

Closes: #1727
Approved by: jlebon
2019-01-07 18:50:49 +00:00
Colin Walters
255dc7027e compose: Minor further oxidation of postprocessing
Take a baby step into further oxidation by migrating the bit
to write our presets.

Closes: #1727
Approved by: jlebon
2019-01-07 18:50:49 +00:00
Colin Walters
56e6ddb8e5 compose: Change /etc/default/useradd to use HOME=/var/home
For a long time I've resisted encoding "policy" into rpm-ostree
as much as possible.  Doing so makes it more distribution specific
for example.  That said, for `/var/home` there argument for doing
this in rpm-ostree is that we already make that symlink in our
hardcoded rootfs.  So we might as well do the other fixups for it.

https://github.com/coreos/fedora-coreos-config/pull/18
https://pagure.io/workstation-ostree-config/pull-request/121
https://discussion.fedoraproject.org/t/adapting-user-home-in-etc-passwd/487/6
https://github.com/justjanne/powerline-go/issues/94

Closes: #1726
Approved by: jlebon
2019-01-07 15:35:37 +00:00
Colin Walters
bb4a17d0ba compose: Add a stub "postprocess" entrypoint in Rust
Prep for more code.

Closes: #1726
Approved by: jlebon
2019-01-07 15:35:37 +00:00
Colin Walters
43bd9f4329 core: Fix rpm-md repo caching
I'd been seeing hangs sometimes in "Importing metadata" and had
thought it was libsolv being slow, but actually the problem is
that while we explicitly check the libdnf repos,
`dnf_context_setup_sack_with_flags()` checks them again using
its `cache_age`.

Set the libdnf context's expiry to "never" after we've done the
checking so it's not checked twice.

This PR brought to you by airplane 🛫 travel and not wanting
to pay for expensive WiFi 💸.

Closes: #1723
Approved by: lucab
2018-12-25 08:39:16 +00:00
Jonathan Lebon
fa64ebfa72 rust: Bump dependencies
This leaves only indicatif outdated (see link in diff).

Closes: #1720
Approved by: cgwalters
2018-12-19 21:25:13 +00:00
Jonathan Lebon
514f4d2473 core: Explain a bit the base/layered split mismatch issue
This is a very commonly hit issue in Fedora land. Add some explanation
so users aren't just confused, as well as the `cleanup -m` pointer. Also
link back to the main issue.

Closes: #1717
Approved by: cgwalters
2018-12-19 20:40:17 +00:00
Jonathan Lebon
9a3cd0782b rust: Add Cargo.lock
We've been seeing more breakage recently due to much of the Rust
ecosystem sticking to minimum rustc versions more recent than ours. A
lockfile now means we have to periodically update it, but at least we
have much better control over our CI system and when we want to spend
time resolving package bump conflicts vs. just trying to get a small
patch in.

Closes: #1719
Approved by: cgwalters
2018-12-19 16:00:00 +00:00
Jonathan Lebon
f6de8f3dfc rust: Drop backtrace lock
We can build against the latest now that we bumped our minimum Rust
version.

Closes: #1716
Approved by: cgwalters
2018-12-17 22:06:49 +00:00
Jonathan Lebon
1f72fbb055 ci: Bump minimum Rust version to 1.29.2
This matches the latest DTS release on el7.

Closes: #1716
Approved by: cgwalters
2018-12-17 22:06:49 +00:00
Colin Walters
c60043c2db compose: Use dracut tmpdir under target root
The problem here is that bare-user has `user.ostreemeta` xattrs,
`tmpfs` (which bwrap uses for `/var/tmp` by default) doesn't support that.

Pass through the tmpdir to the target rootfs, which is the same place
as the repo in unified-core mode.

Ref: https://github.com/coreos/coreos-assembler/issues/254

Closes: #1716
Approved by: cgwalters
2018-12-17 22:06:49 +00:00
Jonathan Lebon
5202ce06c2 packaging: Don't include checksums for ltmain.sh
And `config.guess` and `config.sub`. These files get mangled by the
`%configure` macro when it tries to insert hardening compile flags and
so the checksums no longer match. This is an ugly hack akin to #1554
that requires an incision in the cargo vendor JSON.

Fedora does package a lot of these crate sources now which we
could use to drop these hacks, but not all the crates are packaged (I
counted 4 unpackaged top-level crates), and I'm not sure what their
states are in RHEL7/8 either.

Closes: #1715
Approved by: cgwalters
2018-12-15 14:21:19 +00:00
Jonathan Lebon
b889a51056 Release 2018.10
It's been a while and we have lots of goodies. Likely the last one
before the new year. 🎉

Closes: #1713
Approved by: cgwalters
2018-12-13 16:34:56 +00:00
Jonathan Lebon
24d401072a Cargo.toml: Fix backtrace crate to compatibile version
The latest backtrace release isn't compatible with 1.26.2. A patch for
this is underway in:

https://github.com/alexcrichton/backtrace-rs/pull/137

Though for now, let's just restrict the version since we should be
bumping our minimum rustc requirement soon-ish anyway.

Closes: #1713
Approved by: cgwalters
2018-12-13 16:34:56 +00:00
Jonathan Lebon
88c2700cdd daemon/deploy: Fix free() of override replace pkgs
We were using `g_strfreev()` to free the string array, but the strings
themselves were owned by the `modifiers` GVariantDict. Fix this and make
the comments about it more explicit. On my computer (and at least
Dusty's), this was only actually tripping up libc when passing more than
just one package on the CLI.

Closes: #1707

Closes: #1709
Approved by: cgwalters
2018-12-10 20:49:52 +00:00
Jonathan Lebon
a1014aee2f daemon: Perform kargs in cache-only mode
Users don't expect the tree to get updated here.

Closes: #1705

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
b104a289f7 daemon/kargs: Fix handling of staged deployments
When handling `GetDeploymentBootConfig()`, we would trip an assertion
when trying to read the full bootconfig from a staged deployment, which
of course doesn't have a full bootconfig yet. Rework this to add a new
`staged` key to the returned dict, in which case only `options` is
included. (Which is all `rpm-ostree kargs` needs anyway).

Closes: #1708
Approved by: cgwalters
2018-12-10 18:54:46 +00:00
Jonathan Lebon
770e6e72e1 compose: Don't require full previous commit
Right now, if we want to commit onto the same ref, we require the full
previous commit. The previous commit full root is currently used as
follow:

- for handling `check-passwd` in the `previous` case
- as an optimization for SELinux policy matching (#1659)

Let's change the logic so that we gracefully handle partial commits.
This patch lowers full commit root loading in the one function that
really needs it (`rpmostree_passwd_compose_prep()`). We also turn off
the SELinux optimization if the commit is partial.

My goal here is to use this in coreos-assembler to seed the repo just
enough to keep the commit history intact. (Though we do have cache so
that we can still profit from the SELinux optimization if it's
populated).

Closes: #1704
Approved by: cgwalters
2018-12-07 21:51:14 +00:00
Jonathan Lebon
6e4d653292 compose: Factor out SELinux policy checkout
Simplifies return paths. Prep for next commit.

Closes: #1704
Approved by: cgwalters
2018-12-07 21:51:14 +00:00
Jonathan Lebon
9aee7d43ce libpriv/passwd-util: Rework error-handling
In the passwd/group migration code, rather than do a bunch of work and
then check for the error cases where we only migrate one of passwd and
group, just bring those checks and and queue the work at the end. This
simpifies the logic a bit since we don't have to maintain a
`perform_migrate` variable as well and instead can just return early in
the trival cases.

Closes: #1704
Approved by: cgwalters
2018-12-07 21:51:14 +00:00
Colin Walters
f811828543 rust/treefile: Add container key
This was omitted since in practice we aren't actually testing it,
the container path is mostly via `ex container` which uses keyfiles.

Closes: #1701

Closes: #1702
Approved by: jlebon
2018-12-06 19:03:06 +00:00
Colin Walters
ff80765a86 daemon: Drop unused mutex
Was reading the code while looking at
https://github.com/projectatomic/rpm-ostree/issues/1692
and blinked at the comment, then noticed it hasn't been used
since:  3e289ffab0

Closes: #1699
Approved by: jlebon
2018-12-06 16:20:48 +00:00
Jonathan Lebon
544d79fd3f compose: Fix EBADF in unified core mode without cachedir
If no cache dir is given in the workdir, we would alias the cache dir fd
to the workdir fd. But of course, this meant that we'd try to close the
same fd twice when freeing the compose context. Instead, let's just copy
the fd as is also done in the non-unified path.

Closes: #1697

Closes: #1698
Approved by: lucab
2018-12-06 13:44:19 +00:00
Jonathan Lebon
bdf3cda8db Print CVEs fixed in available updates
One question I often have when looking at the output of `status -a`:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
           Diff: 67 upgraded, 1 removed, 16 added
```

is "How serious and relevant are these advisories to me? How soon should
I reboot?". For the packages that I'm most familiar with, e.g. `kernel`
and `git-core`, I usually look up the advisory and check why it was
marked as a security update, mentioned CVEs, and how those affect me.

The updateinfo metadata includes a wealth of information that could be
useful here. In Fedora, CVEs treated by the security response team
result in RHBZs, which end up attached to the advisories and thus make
it into that metadata.

This patch tries to reduce friction in answering some of those questions
above by checking for those CVEs and printing a short description in the
output of `status -a`. Example:

```
AvailableUpdate:
        Version: 29.20181202.0 (2018-12-02T08:37:50Z)
         Commit: dece5737a087d5c6038efdb86cb4512f867082ccfc6eb0fa97b2734c1f6d99c3
   GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
  SecAdvisories: FEDORA-2018-042156f164  Unknown    net-snmp-libs-1:5.8-3.fc29.x86_64
                   CVE-2018-18065 CVE-2018-18066 net-snmp: various flaws [fedora-all]
                   https://bugzilla.redhat.com/show_bug.cgi?id=1637573
                 FEDORA-2018-87ba0312c2  Moderate   kernel-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-core-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-4.19.5-300.fc29.x86_64
                 FEDORA-2018-87ba0312c2  Moderate   kernel-modules-extra-4.19.5-300.fc29.x86_64
                   CVE-2018-16862 kernel: cleancache: Infoleak of deleted files after reuse of old inodes
                   https://bugzilla.redhat.com/show_bug.cgi?id=1649017
                   CVE-2018-19407 kernel: kvm: NULL pointer dereference in vcpu_scan_ioapic in arch/x86/kvm/x86.c
                   https://bugzilla.redhat.com/show_bug.cgi?id=1652656
                 FEDORA-2018-f467c36c2b  Moderate   git-core-2.19.2-1.fc29.x86_64
                   CVE-2018-19486 git: Improper handling of PATH allows for commands to executed from current directory
                   https://bugzilla.redhat.com/show_bug.cgi?id=1653143
           Diff: 67 upgraded, 1 removed, 16 added
```

Including the CVE name and RHBZ link also makes it easier to look for
more details if desired.

Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Jonathan Lebon
51d48db235 tests/utils/updateinfo: Rename function
Use a better function name for the shallow copy we do in
`copy_update_no_cols()`. Also add a better comment.

Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Jonathan Lebon
e7d256b4c8 tests/utils/updateinfo: Assuage flake8
Closes: #1695
Approved by: rfairley
2018-12-05 18:56:49 +00:00
Colin Walters
5554ca10d1 Add rpm-ostree-bootstatus.service
Capturing the system state at boot aids debugging.  This is a
trivial implementation; we could in the future do structured
logging too.

The high level goal here is to help us track system state in
Red Hat CoreOS.

Closes: #1693
Approved by: jlebon
2018-12-03 15:38:50 +00:00
Colin Walters
09e7caef4e compose: Add a CUtf8Buf copy of rojig_name
Avoids passing an allocated buffer from Rust to C; there's
controversy in the PR I sent to rust-lang around defining this as
supported.

Closes: #1691
Approved by: jlebon
2018-11-28 16:53:00 +00:00
Colin Walters
62744449dc rust: Use Fallible<> more
Since use of the `failure` crate has been a success, let's use
it a bit more.  The big thing to convert left is `treefile.rs` which
does need a custom error so we can stop abusing `io::ErrorKind::InvalidInput`.

Closes: #1690
Approved by: jlebon
2018-11-28 16:00:56 +00:00
Jonathan Lebon
01c349a558 libpriv/rpm-util: Tweak changelog entry indentation
Indent the changelog entries so it's easier to tell which entries belong
to which packages.

Before:

```
$ ros db diff -c
ostree diff commit old: rollback deployment (ef3c0e45ee1b874bc4952904a778084f0c32a0e06346e271786abf544dad54ff)
ostree diff commit new: booted deployment (c1d4c3ef571a53e5ab3dbd6ea1ec91a374ebd5f6ba9c0a99938d0649853588a5)
Upgraded:
  firefox 63.0.3-1.fc29.x86_64 -> 63.0.3-2.fc29.x86_64
* Wed Nov 21 2018 Martin Stransky <stransky@redhat.com> - 63.0.3-2
- Fixed mozbz#1507475 - crash when display changes (rhbz#1646151).

  httpd 2.4.37-3.fc29.x86_64 -> 2.4.37-5.fc29.x86_64
* Fri Nov 23 2018 Lubos Uhliarik <luhliari@redhat.com> - 2.4.37-5
- Resolves: #1652678 - TLS connection allowed while all protocols are forbidden

* Thu Nov 08 2018 Joe Orton <jorton@redhat.com> - 2.4.37-4
- add httpd.conf(5) (#1611361)
```

After:

```
$ ros db diff -c
ostree diff commit old: rollback deployment (ef3c0e45ee1b874bc4952904a778084f0c32a0e06346e271786abf544dad54ff)
ostree diff commit new: booted deployment (c1d4c3ef571a53e5ab3dbd6ea1ec91a374ebd5f6ba9c0a99938d0649853588a5)
Upgraded:
  firefox 63.0.3-1.fc29.x86_64 -> 63.0.3-2.fc29.x86_64
    * Wed Nov 21 2018 Martin Stransky <stransky@redhat.com> - 63.0.3-2
    - Fixed mozbz#1507475 - crash when display changes (rhbz#1646151).

  httpd 2.4.37-3.fc29.x86_64 -> 2.4.37-5.fc29.x86_64
    * Fri Nov 23 2018 Lubos Uhliarik <luhliari@redhat.com> - 2.4.37-5
    - Resolves: #1652678 - TLS connection allowed while all protocols are forbidden

    * Thu Nov 08 2018 Joe Orton <jorton@redhat.com> - 2.4.37-4
    - add httpd.conf(5) (#1611361)
```

Closes: #1687
Approved by: rfairley
2018-11-28 13:40:41 +00:00
Jonathan Lebon
0e8eab7bfc libpriv/rpm-util: Use g_print instead of printf
For consistency with the rest of the codebase. There were also a few
spots which were missing the space between the function name and the
opening parenthesis.

Closes: #1687
Approved by: rfairley
2018-11-28 13:40:41 +00:00
Colin Walters
a177f551b0 rust/ffi: Convert String creation API, extend docs
Create a new `_new_` naming convention, and extend the FFI
documentation to describe the new state as well as background assumptions.

Closes: #1688
Approved by: jlebon
2018-11-28 09:14:41 +00:00
Colin Walters
a2f5b7c1c0 rust/ffi: More "view" changes, for bytestrings
Change our existing "view as [u8] API", and also add one
that does a view as `OsStr`.  The motivation for the latter
is I noticed ithat `OsStr::from_bytes()` *doesn't* copy,
or rather it just copies the pointer value.  Rust's lifetime
inference ensures that the returned lifetime matches the input array.

I think the previous code in `treefile.rs` was confused about this.

Closes: #1688
Approved by: jlebon
2018-11-28 09:14:41 +00:00
Colin Walters
bfe637a877 rust/ffi: Convert str func to new "view" naming convention
More obvious one shouldn't leak the pointer.

Closes: #1688
Approved by: jlebon
2018-11-28 09:14:41 +00:00
Colin Walters
db0319beac rust/ffi: Change dfd handling to unwrap, create naming convention
I was going to add another usage of this function, and I think the
gerror stuff is unnecessary - if we are handed a bad file descriptor
(or a fd pointing to a regular file) that's something where we should
just abort.

While we're here, I'd like to codify expected usage in the function
names here.  If you like this I'll e.g. also change `str_from_nullable`
to `ffi_view_nullable_str`.

Closes: #1685
Approved by: jlebon
2018-11-27 15:42:08 +00:00
Jonathan Lebon
140704c105 daemon/transaction-types: Fix dnf cache override
Minor regression from #1587. There were places that were still doing
`dnf_context_set_cache_age()` manually, but those calls didn't exactly
have the intended effect since the core now handled caching itself.

The actual result was that the metadata was still being updated, but not
during the `dnf_repo_check` pass that the core does, but rather the
`Importing rpm-md` pass it does right after. So then, we were
incorrectly printing `(cached)` even though we'd update it afterwards.

Switch to the new way of doing things.

Closes: #1686
Approved by: cgwalters
2018-11-26 23:39:44 +00:00
Jonathan Lebon
1164b7b047 daemon/transaction-types: Drop unneeded arg
Drop the `force_refresh` boolean parameter since we only ever call it
with `TRUE`. I think this dates from an earlier implementation where we
did call it with differing values.

Small prep for next patch.

Closes: #1686
Approved by: cgwalters
2018-11-26 23:39:44 +00:00
Jonathan Lebon
2cc06ce89a configure.ac: Add "GitHub release" step to release workflow
Let's make use of the GitHub release feature to make it more prominent
on the "Releases" tab, but more importantly so that we can attach
vendored tarballs for downstreams. E.g. this will allow us to have a
correct `Source0` field in the Fedora spec file.

Related: #1683

Closes: #1684
Approved by: rfairley
2018-11-26 22:59:50 +00:00
Jonathan Lebon
ed2fefad85 Move reset out of experimental
This is relatively uncontroversial functionality that has already proved
useful when helping folks debug their stuff. Let's promote it to the
stable interface.

Closes: #1682
Approved by: rfairley
2018-11-22 20:47:50 +00:00
Jonathan Lebon
03a92c0981 app/ex: Nuke ex kargs compatibilty alias
It said so right there; "nuke in next version". No need to carry this
stuff forever. It's the final phase of feature promotion, like when one
officially moves out of their parents'.

Closes: #1682
Approved by: rfairley
2018-11-22 20:47:50 +00:00
Colin Walters
e0a976386a rust: Use failure crate for errors
In a lot of places we're abusing `io::Error(io::ErrorKind::InvalidInput)`
which is both verbose and inaccurate really.  Maybe in some
places we should be defining custom errors, but eh.

I like the `failure` crate.  Use it in just `utils.rs` for now.
Tweak our error handling FFI wrappers to accept `Display` since
all we do is convert the error to a string.

Closes: #1675
Approved by: lucab
2018-11-21 22:03:53 +00:00
Colin Walters
387da3a420 ci: Verify rustfmt
Split out the command into a separate `Makefile` that doesn't
use Automake so we can invoke it early.

Closes: #1674
Approved by: dustymabe
2018-11-21 21:16:03 +00:00
Colin Walters
6b5e475607 rust: Rerun rustfmt
Going to add a CI step next.

Closes: #1674
Approved by: dustymabe
2018-11-21 21:16:03 +00:00
Jonathan Lebon
17359bc0cc core: Tweak "Importing" messages
To make it more obvious what the difference between "Importing metadata"
and "Importing" is, add "rpm-md" to the first and "packages" to the
second.

Closes: #1681
Approved by: cgwalters
2018-11-21 13:54:35 +00:00