Commit Graph

2433 Commits

Author SHA1 Message Date
Colin Walters
2c8a144c1b daemon: Move deploy parsing down into _new_deploy()
Follow to previous change; this should make it easier to add
a new parameter.  It also just makes more sense to centralize
the parsing here.

Closes: #1403
Approved by: jlebon
2018-06-12 20:29:56 +00:00
Colin Walters
826177c2a0 daemon: Push deploy GVariant parsing down a level
Subsequent patches will move it down farther; the high level
goal is to avoid touching many functions to add a parameter.

Closes: #1403
Approved by: jlebon
2018-06-12 20:29:56 +00:00
Jonathan Lebon
d436d48a2f libpriv/origin: Allow removing local RPMs by name only
This fixes a painful UX issue where one must use the full NEVRA when
uninstalling a locally layered RPM. Now, one can specify either the
NEVRA or the package name only. Though we still try to interpret the
request first as a NEVRA or a capability.

Closes: #1386

Closes: #1400
Approved by: cgwalters
2018-06-12 14:55:40 +00:00
Jonathan Lebon
073978aace app/override: Don't auto-enable cache-only if also installing
Otherwise, we won't be able to fetch the requested packages from the
repo.

Closes: #1384

Closes: #1402
Approved by: cgwalters
2018-06-12 14:52:27 +00:00
Colin Walters
27bd7b97bb Handle "pinned commits" specifically
There are a few scenarios today where one might deliver content
to a machine via an external transport.  For example, take the
scenario of a single server updated via USB drive.  While we
can provide a refspec...what should the remote be?  (This gets
into ostree collections).  There's nothing really that can
happen when typing `rpm-ostree upgrade` unless the USB stick
is plugged in.  That type of scenario should be emphasized
by pinning the commit - the machine is updated via an external
script.

Another case: we're experimenting embedding OSTree commits inside OCI
containers.  Here again since rpm-ostree can't understand how to
pull content from containers, it's saner to drop the refspec
bits, and pin to a commit.

Further enhancements will follow to make the admin experience more
obvious.

Closes: #1396
Approved by: jlebon
2018-06-11 20:37:58 +00:00
Colin Walters
26f04595b2 upgrade: Note if automatic updates are enabled
When e.g. FAH does the automatic update transition, let's remind
people who are doing it manually still.

Closes: #1399
Approved by: jlebon
2018-06-11 18:46:58 +00:00
Jonathan Lebon
eeb5213ddd upgrader: Add missing values to flags GType
We were missing a bunch of flags as part of the property GType. I was
getting a warning that's become fatal because we set
`G_DEBUG=fatal-warnings` for `make vmsync`, which is how I was testing
this:

```
rpm-ostree[1813]: value "((RpmOstreeSysrootUpgraderFlags) 48)" of type 'RpmOstreeSysrootUpgraderFlags' is invalid or out of range for property 'flags' of type 'RpmOstreeSysrootUpgraderFlags'
kernel: traps: pool[1816] trap int3 ip:7f47f4b4b6d5 sp:7f47e989d1f0 error:0 in libglib-2.0.so.0.5600.1[7f47f4af9000+115000]
```

I'm not quite sure why this is happening now though, since I use
`make vmsync` a lot. My guess would be something to do with how we
changed flag passing in d568e54d from "all of them" to "only those we
need", but I didn't verify that.

Closes: #1398
Approved by: cgwalters
2018-06-11 17:50:34 +00:00
Colin Walters
5fec2b1af3 build-sys: Add some gitignore for Rust bits
Just keeping status tidy.

Closes: #1397
Approved by: jlebon
2018-06-11 13:08:41 +00:00
Jonathan Lebon
4249dc8c63 libpriv: Directly parse NEVRAs, don't use branches
When I initially added support for local RPM layering (#657), I was
operating under the assumption that there is no reliable way to parse a
NEVRA back to its constituent elements. So we worked around this by
doing lookups in the pkgcache and comparing against branches converted
back to their NEVRAs, which was pretty hacky.

Later on, I learned that you *can* reliably parse a NEVRA back to its
elements because RPM forbids all fields from having a colon `:` and all
fields other than the package name to have a dash `-`. In fact, there is
a helper from `libdnf` to do exactly this.

This patch swaps out the old way of decomposing NEVRAs and converting
NEVRAs to their cache branch notation with a simpler more direct
approach using the helper function from `libdnf`.

Closes: #1395
Approved by: cgwalters
2018-06-08 20:51:30 +00:00
Jonathan Lebon
df4e1d7b75 libpriv: Move cache-branch functions to rpm-util.c
Prep for later commit.

I hesitated between `rpmostree-util.c` and `rpmostree-rpm-util.c`. In
the end, I decided on the latter since some of the functions require
rpmlib headers, which aren't currently included in `rpmostree-util.c`.

Closes: #1395
Approved by: cgwalters
2018-06-08 20:51:30 +00:00
Colin Walters
a8a5f236f1 packaging: Fix Rust vendoring
I made a subtle change at the last minute with the previous PR
to use `*` for the glob instead of `.`, because the tmpdir had a `.tmp`
file I didn't want.

But - this caused us to miss the `.cargo` directory which has
the config file.  And while I'd been testing builds with no network,
of course cargo was really pulling content from `~/.cargo`.

When I went to do a scratch build in Koji, that failed obviously.
I tested this makes things [work with a SRPM scratch](https://koji.fedoraproject.org/koji/taskinfo?taskID=27490830)
and in my dev container under `bwrap --unshare-net` with `mv ~/.cargo{,.orig}`.

Closes: #1394
Approved by: jlebon
2018-06-08 15:42:03 +00:00
Colin Walters
9f6f529f81 compose: Print out the input hash
We're seeing a case where the change detection apparently isn't
working; let's log the hash we got so it's easier to debug.

Closes: #1393
Approved by: jlebon
2018-06-07 17:33:17 +00:00
Colin Walters
c997fd99c2 rust: Fix unused-import in release mode
Closes: #1391
Approved by: jlebon
2018-06-06 15:52:48 +00:00
Colin Walters
7b69294b8a packaging: Support vendoring the Rust sources
This ends up being different than what landed in librsvg (that
was imported into ostree) because in rpm-ostree we've basically
been using `git archive`-based tarball generation rather than `make dist`
for a long time.  And supporting `make dist` looks like it'd get into
handling the `libdnf` bits and walking into `cmake` land so...yeah
let's not do that.

The canonical sources are in git (recursively via submodule),
except for the Rust sources, which cargo can download dynamically,
and with this patch we support glomming all of that together
into a tarball.

(And turn off `make dist` so people understand how we do it)

Tested by `make -f Makefile.dist-packaging dist-snapshot`, then
copying the resulting tarball into a container with `--net=none`
and building there.

Closes: #1391
Approved by: jlebon
2018-06-06 15:52:48 +00:00
Jonathan Lebon
98ed545e1e rust: Use flat_map for splitting package whitespace
This is mostly for fun, though I do like the final result as well.

Closes: #1390
Approved by: cgwalters
2018-06-05 22:02:34 +00:00
Jonathan Lebon
60d49c3413 rust: Pass through clippy
Some tweaks as suggested by rust-clippy.

Closes: #1390
Approved by: cgwalters
2018-06-05 22:02:34 +00:00
Colin Walters
702daa2335 rust: Fix automake warnings, add unit tests to glibutils
Automake was warning about duplicate `clean-local` definitions, let's
do the nonrecursive automake dance.

And while I'm here, let's add some Rust unit tests that actually run
on `make check` too, since the whole unit testing bits of Rust are
awesome.

(I also tweaked the propagate bits to use the nicer `is_null()` method)

Closes: #1389
Approved by: jlebon
2018-06-05 19:53:41 +00:00
Jonathan Lebon
34c3d5a02a app/install: Add --allow-inactive switch
For now all this switch does is turn off the scary warning. We also
tweak the warning to make it clear that this will be required in a
future release.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
625c17f9b0 daemon/deploy: Re-use base layer rsack
Follow-up to previous commit. Rather than allocating and destroying
RefSacks multiple times, hoist it out of those blocks so it can be
re-used.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
3c12a58b25 daemon/deploy: Warn on immediately inactive requests
This has been a huge point of confusion for users with the new
declarative way of specifying package requests (see #1216, #1287).
Essentially, doing `rpm-ostree install glibc` will work, but it's not
very useful because we don't really expect glibc to ever go away. As a
heuristic, let's work towards forbidding package requests that would
immediately become inactive. For now, we just print a warning.

Note that this still leaves the door open for layered pkgs that e.g.
become inactive during a rebase to a tree which has it baked in, and
then active again when rebasing back; supporting this was the major
advantage of switching to a declarative approach.

Closes: #1378
Approved by: cgwalters
2018-06-05 19:01:41 +00:00
Jonathan Lebon
315406d8cd daemon: Don't use absolute path for gdbus-codegen
This breaks builds on platforms that don't have `gdbus-codegen`
installed at that path.  This reverts #1153, which was needed at the
time, but things appear to be fixed now.

Closes: #1383
Approved by: cgwalters
2018-06-05 18:09:59 +00:00
Micah Abbott
de42ef701f docs: improve install/uninstall help
This adds some additional help to the man page and the RTD page around
using local packages with the install/uninstall commands.

Closes: #1388
Approved by: cgwalters
2018-06-05 15:59:52 +00:00
Jonathan Lebon
479406e6a5 Add support for YAML treefiles
Let's modernize and start supporting YAML treefiles. I'll dare make the
sweeping generalization that most people would prefer reading and
writing YAML over JSON.

This takes bits from coreos-assembler[1] that know how to serialize a
YAML file and spit it back out as a JSON and makes it into a shared lib
that we can link against. We could use this eventually for JSON inputs
as well to force a validation check before composing.

If we go this route, we could then turn on `--enable-rust` in FAHC for
now and drop the duplicate code in coreos-assembler.

[1] https://github.com/cgwalters/coreos-assembler

Closes: #1377
Approved by: cgwalters
2018-06-05 13:08:33 +00:00
Jonathan Lebon
9085e5144d tests/vmcheck: Fix rojig expected NEVRA for f28
Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
8f71aa95e6 tests/vmcheck: Disable rojig test on CentOS
I was honestly surprised to see this test was running all this time for
CentOS. As cool as it is, this test is really more meant for Fedora.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
d568e54d84 app: Simplify options GVariant generation
In the middle of adding yet another parameter to
`rpmostree_get_options_variant`, it dawned upon me that we weren't
gaining much from making it its own function. Just generating the
GVariant inline seems much cleaner since (1) we only actually have to
insert keys for values we want to specify, rather than always specifying
default `FALSE` values, and (2) the fact that the key name is right
there makes it self-documenting and thus doesn't require a bunch of
comments on the side.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
09b2ca0482 tests/vmcheck: Fix override remove test
We were trying to test that doing `override remove` on a layered package
errored out. But the test was bogus since `foo` is a base package here.
Let's just use a brand new package that will actually be layered.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Jonathan Lebon
d3d205f7af tests/vmcheck: Split basic layering tests in two
Just part of the continuing breakdown of our testsuites into smaller
pieces for more efficient allocation.

Will add a test case to the `layering-basic-2.sh` in a follow-up commit.

Closes: #1382
Approved by: jlebon
2018-06-04 14:10:21 +00:00
Colin Walters
89d130c596 .gitmodules: Update URL for libglnx
Seeing a 404 in the CentOS build for some reason...something like
older versions of git not following redirects maybe?

Update submodule: libglnx

Closes: #1374
Approved by: jlebon
2018-05-29 14:22:46 +00:00
Jonathan Lebon
bff3a54e1e daemon/upgrader: Consistently remove transient state
With the new support for pinning deployments, we need to also update
rpm-ostree to clean up the transient state as is now done in the ostree
sysroot upgrader.

This addresses that issue as well as tries to be a little cleaner in how
we clean up other transient state. Notably, we add a new helper function
to `RpmOstreeOrigin` to do this for us and use it in the upgrader. In
other cases, we do want this transient information since it allows us to
describe the deployment.

Closes: https://github.com/ostreedev/ostree/issues/1595

Closes: #1372
Approved by: cgwalters
2018-05-23 16:27:10 +00:00
Jonathan Lebon
592d6052b9 ci: Bump to F28
Closes: #1358
Approved by: cgwalters
2018-05-23 14:18:41 +00:00
Colin Walters
23129a15b7 compose: Add a missing return in error path
Was hitting a `SEGV` in a build that *might* be this.

Closes: #1371
Approved by: jlebon
2018-05-23 13:24:44 +00:00
Colin Walters
99901ac0dc daemon: Add an 'id' param to RegisterClient, log it
This makes the logs a bit more useful, but the ultimate goal
here is to write the originating client `id` to the cached update
data, so users know that e.g. `gnome-software` triggered it.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Colin Walters
7a7d7ad886 daemon: Load sd unit for callers, log it
The high level goal is to render in a better way what caused an
update: https://github.com/projectatomic/rpm-ostree/issues/247#issuecomment-386615707

This gets us for Cockpit:
`Initiated txn DownloadUpdateRpmDiff for client(dbus:1.28 unit:session-6.scope uid:0): /org/projectatomic/rpmostree1/fedora_atomic`
which isn't as good as I'd hoped; I was thinking we'd get `cockpit.service`
but actually Cockpit does invocations as a real login for good reason.

We get a similar result from the CLI.

Closes: #1368
Approved by: jlebon
2018-05-16 18:11:40 +00:00
Jonathan Lebon
377b1674da ci: Drop f27-sanity testsuite
This started out as an experiment to see how useful it'd be in
rpm-ostree. Again, the goal wasn't to leverage the rpm-ostree-specific
tests of a-h-t here, but rather running the *other* tests to ensure that
we weren't inadvertedly impacting userspace. In the end, as far as I
remember, it hasn't brought up anything since adding it here, so let's
bring the experiment to an end.

It might still be worth doing this same experiment in the ostree CI
instead, though I don't want to add more load there until we fully
switch over to the new CI infra.

Closes: #1369
Approved by: cgwalters
2018-05-16 15:19:13 +00:00
Jonathan Lebon
bb7b489d17 Release 2018.5
Tree seems sane, tests are passing, and no major outstanding issues.
Let's get all those new goodies out!

Closes: #1367
Approved by: cgwalters
2018-05-15 13:07:37 +00:00
Colin Walters
e57504a88a ci: Split codestyle checks into separate context
Prep for reworking the primary test to do vm-in-container, which
will temporarily be vm-in-container-in-vm.

See https://github.com/projectatomic/rpm-ostree/pull/1362

Closes: #1364
Approved by: jlebon
2018-05-14 19:50:36 +00:00
Colin Walters
8387f1c7c3 Add "ex-stage" update policy, support for ostree staged deployments
Following up to https://github.com/projectatomic/rpm-ostree/pull/1352
AKA 506910d930
which added an experimental flag to globally enable deployment
staging, let's add an `ex-stage` automatic update policy.

I chose to create a new `test-autoupdate-stage.sh` and rename
the previous one to `test-autoupdate-check.sh` in going with
the previous theme of smaller test files; it's
way faster to iterate on new tests when it's a new file. And adding
staging at the top would have been weird.

This was all quite straightforward, just plumbing through lots
of layers.

Closes: #1321
Approved by: jlebon
2018-05-14 19:03:56 +00:00
Jonathan Lebon
38b11d34e5 ci: Use continuous branch for now
We are hitting issues with a-h-t on CAHC right now failing due to an
`atomic scan` issue and preventing promotion of continuous content to
the smoketested branch. This is then blocking our CI which needs a newer
tree with the latest glib2 baked in. Let's unblock our CI for now by
rebasing on the continuous branch. We'll drop that hack once the a-h-t
issues are ironed out.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Jonathan Lebon
169fab6f4f ci: Drop BDB1539 hack
This shouldn't be needed anymore.

Closes: #1365
Approved by: cgwalters
2018-05-14 18:14:54 +00:00
Colin Walters
b03b6a2057 core: Tweak package checkout message, update libglnx
The common case is having layered packages and no overrides; seeing
`Applying 8 overlays` then `Building filesystem [0/8]` is redundant.
Tweak the progress to avoid the double message.  Also change the terminology
to clarify that each item is a package.

Also update libglnx to pull in progress bar newline fix.

Update submodule: libglnx

Closes: #1363
Approved by: jlebon
2018-05-10 01:13:38 +00:00
Jonathan Lebon
6fbf405be0 libglnx: Bump for copy_file_range fix
Fixes compilation on f28. Also prep for #1358.

Update submodule: libglnx

Closes: #1361
Approved by: cgwalters
2018-05-04 20:18:12 +00:00
Jonathan Lebon
beee865ed2 app/status: Add --advisories flag
The `status --verbose` output is getting quite long now that we print
repo information and diffs. Let's add a switch for only making
advisories more verbose so that it's easy to get more information
without having to use `-v` and scroll up if the diff is massive.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Jonathan Lebon
c1c78154b1 app/status: Print diff/advisories with pending deployment
Follow-up from #1344.

In the case where a cached update is created from an `upgrade` operation
(and soon, "stage" auto-update policy runs), we can just print the diff
and advisory info together with the pending deployment. This makes the
output look much more natural.

Closes: #1350
Approved by: cgwalters
2018-05-04 19:16:52 +00:00
Jonathan Lebon
33ea6fbc96 daemon/upgrader: Reuse sack from core for advisories
In #1344, we changed `upgrade` to always write the cached update. One
thing I hadn't noticed was that with pkglayering, loading the sack twice
meant we printed all that gory repo info twice.

Really, the much nicer thing to do is just to make an effort to try to
*keep* the original sack that the core used so that we skip all that
overhead completely.

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Jonathan Lebon
ab5452bc3a libpriv/core: Always load updateinfo when preparing
It just makes the sack more useful for other purposes this way. Callers
can just use `download_metadata` directly if they'd like to override
this. Prep for a follow-up patch.

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Jonathan Lebon
974a9f6f71 app/dbus-helpers: Factor out diff/advisory printing
Prep for printing this info as part of the deployments. I also snuck in
a small fix so that we don't print the `Diff:` line if there's no diff
(e.g. it's a no-op update).

Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Jonathan Lebon
a7adf82c95 app/status: Drop unused argument
Closes: #1357
Approved by: cgwalters
2018-05-04 14:22:56 +00:00
Colin Walters
f39eeee671 db: Add some error prefixing around rpmdb loading
In https://github.com/cockpit-project/cockpit/pull/9090#issuecomment-386564044
we're seeing:
`Txn DownloadUpdateRpmDiff ... failed: open(O_TMPFILE): No such file or directory`

Looking at the error paths in the rpmdb diff code, there's not really
any error prefixing.  Let's add some so this is easier to debug.

Closes: #1356
Approved by: jlebon
2018-05-04 13:02:19 +00:00
Colin Walters
5037c85330 tests: Extend staging to cover vmcheck/upgrades
This now works after https://github.com/ostreedev/ostree/pull/1570

However, I was hitting an issue with the temp httpd running the
tests serially, let's split them up.

Really the only sane thing is to run each of these tests in a fresh
VM; we'll do the VM-in-container pattern here hopefully soon.

Closes: #1355
Approved by: jlebon
2018-05-02 19:20:26 +00:00