Commit Graph

2195 Commits

Author SHA1 Message Date
Jonathan Lebon
7911b14f49 daemon: Fix cached-update including no-op diffs
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.

We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.

Closes: #1446

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Jonathan Lebon
a17b4b9be0 tests/utils: Drop empty inject-pkglist.py
This one snuck in as part of #1205. I had initially started writing
`inject-pkglist.c` in Python and somehow inadvertedly made it part of
a commit.

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Colin Walters
831507b0f0 scripts: Add RPMOSTREE_SCRIPT_TRACE debugging envvar
I used `env RPMOSTREE_SCRIPT_TRACE='strace -f -s2048' rpm-ostree compose ...`
to debug https://bugzilla.redhat.com/show_bug.cgi?id=1548050

Closes: #1472
Approved by: jlebon
2018-07-27 13:09:49 +00:00
Colin Walters
e858a30eca compose: Stabilize --unified-core
In line with the recent trend of marking things stable, and in
preparation for stabilizing `rojig://` - Let's stabilize the `--unified-core`
option for `compose tree`.

I'm not sure we could make it the default anytime soon; today it trips
over bugs in the PAM package in RHEL7 for example.  But it
works fine for Fedora, and I think the code/design are good enough to be stable.

Closes: #1465
Approved by: jlebon
2018-07-24 23:06:50 +00:00
Colin Walters
fa29f7acfa compose: Support arch-specific packages in YAML (and in JSON again)
Follow up to: https://github.com/projectatomic/rpm-ostree/pull/1459

We now honor arch-specific packages in YAML, and reject unknown
architectures.  I looked a little bit at how to avoid having hardcoded
arch lists, but it doesn't seem worth it right now.

Closes: #1468
Approved by: jlebon
2018-07-24 22:05:06 +00:00
Jonathan Lebon
1a82f210ae ci: Bump f28-compose by 10 minutes
We're consistently seeing timeouts on the `f28-compose` test. It's
likely due to a performance regression in the backend rather than
anything else. I filed a ticket to get an investigation going, though
for now at least, let's bump the timeout some more to keep patches
flowing.

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Jonathan Lebon
c016509548 libdnf: Bump and disable html and man pages
Skip building man pages and HTML docs for our embedded libdnf to speed
up builds.

This bump also pulls in a fix to ensure we never try to install src
packages from `dnf_context_install()`.

See: https://github.com/projectatomic/libdnf/pull/3

Update submodule: libdnf

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Jonathan Lebon
b66337e0cb ci: Pepper in date markers to help debug timing
This should help out with determining what steps take the most time.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
6ac6f3d086 treefile.rs: Deny unknown fields by default
Let's not make the same mistake we did with JSON where typoing a
field means it's silently ignored.  This actually caught a bug
in a YAML usage we had:

```
error: Failed to load YAML treefile: unknown field `install_langs`, expected one of ... `install-langs` ...
```

Yes, this is a compatibility break with the feature we just announced
but...I seriously doubt anyone (that isn't known to me) has converted
yet, and if they are excited enough to start using a two-week-old feature
they can adjust.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
1b4a73f071 tests: Make two more tests compatible with deployment staging
Unlike the kernel args one, these two were simple.

Closes: #1458
Approved by: jlebon
2018-07-21 13:58:21 +00:00
Colin Walters
0cc002fd8d build-sys: Use python3 for libdnf by default if available
Probably at some point libdnf will drop py2 support, but the
main reason I'm doing this is avoids a python2 dependency
for rpm-ostree for distributions that don't want that.

Note of course rpm-ostree itself doesn't use python, libdnf does.
And only for the python bindings, which we don't use either.  So
this is mostly just to DTRT automatically for the libdnf bits; down
the line we could probably add a patch to make the python fully
conditional.

Closes: #1460
Approved by: jlebon
2018-07-20 22:27:34 +00:00
Jonathan Lebon
ec19d67d74 ci: Stop rebasing Fedora onto updates ref
We can drop this hack now since there have been updates since GA. Also
add a note that trying to drop the rebase to continuous causes
`layering-relabel` to fail.

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
096004426c rebase: Add support for "custom origin" descriptions
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.

Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM).  This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.

There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.

This builds on prior work in
27bd7b97bb from #1396 .

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
95de58c29d Update libglnx
For `renameat2()` fix to build with latest glibc (e.g. Fedora rawhide).

Update submodule: libglnx

Closes: #1461
Approved by: jlebon
2018-07-17 18:35:38 +00:00
Colin Walters
d1839378d6 compose: Add some assertions around treefile context directory
Seeing a crash on in one of our builds, not sure yet what's
going wrong as I haven't gotten a core out, but this makes things
clearer.

($kingdom for `T`/`Option<T>` instead of nullable pointers...)

Closes: #1454
Approved by: jlebon
2018-07-12 18:30:55 +00:00
Colin Walters
035a2a72ba tests/kernel-args: Prep for staged
This test was hard to adapt for staging.  I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs.  Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults.  So I just deleted those bits.

Closes: #1453
Approved by: jlebon
2018-07-12 17:44:15 +00:00
Ruixin Bao
9373f13b0a doc: add repo_structure section
This commit plans to add repo_structure document to better explain
what is the meaning for each folder. A similar approach can be found
in https://github.com/openshift/openshift-ansible/blob/master/docs/repo_structure.md

Closes: #1452
Approved by: jlebon
2018-07-11 20:40:59 +00:00
Jonathan Lebon
caf66d6e45 vmcheck/autoupdate-check: Fix minor typos
Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
cfb5e7c04b libvm: add vm_get_journal_after_cursor
The cursor argument was getting munged up by SSH. Use a helper to make
sure we always get quoting right.

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
2e237513b0 rust/treefile: Avoid unnecessary dup() of fd
We weren't checking for an error from `dup()`, and further it shouldn't
be necessary.  This is the best I could come up with, though it's
obviously not pretty.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
d7b4046599 rust: Add a make rustfmt, run it
It is actually really nice that there's One Canonical Style, even
if I sometimes don't like some details of what rustfmt does.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
e731f59007 rust: Split out treefile into its own module
Further prep for adding more code here.  The `lib.rs` then is the
collection of glue functions; perhaps in some ideal future it could
be generated even.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
30ccaa4db6 build-sys: Use --frozen if Rust library was already built
The main rationale here is avoiding any Cargo internet access during
`sudo make install` type flows.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
2894d61fbf build-sys: Make Rust library naming/entrypoint more generic
Prep for adding more functionality beyond treefile bits.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
bac6d634bb build-sys: Move Rust include header under rust/ directory
It makes more sense to have the include live next to the associated
code, just like we do with C, even though the `cargo build` doesn't
touch it.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Yoginski
bcbd68d9c8 Explain how to remove packages from the base layer
Haven't seen it being mentioned in any manuals.
Closes: #1441
Approved by: cgwalters
2018-07-03 21:14:28 +00:00
Colin Walters
b1d7ef15d8 sysroot-core: Use new libostree sysroot pruning API
Use the new API introduced in
9131d8a4cc
which helps avoid relying on the magical deployment refs.  I thought
this would help with a livefs+staging issue, but it doesn't.  But we might
as well do it, it's adding another level of safety.

Closes: #1439
Approved by: jlebon
2018-07-03 20:18:00 +00:00
Colin Walters
222d730cce vmcheck/misc-2: Make compatible with staged default
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system.  It's tempting to do a `reset` before each one
and reboot but this makes things work.

There's some additional assertions here as I went through and
was debugging.

Prep for making staging the default.

Closes: #1438
Approved by: jlebon
2018-07-03 19:14:54 +00:00
Jonathan Lebon
6b79e9c681 Release 2018.6
The tree is in good condition and it's been a while. Let's do this!

Closes: #1436
Approved by: cgwalters
2018-06-29 17:32:57 +00:00
Jonathan Lebon
b94fef9f24 daemon/rollback: Really fix staged deployment case
Follow up to #1434. I was testing this inside my test VM, which didn't
have a rollback deployment (i.e. it was just the staged deployment and
the booted deployment). Testing the upcoming release on my real system
showed that this wasn't correctly fixed.

Closes: #1437
Approved by: cgwalters
2018-06-29 16:30:07 +00:00
Jonathan Lebon
beb8775d8e daemon/rollback: Error out if pending deployment is staged
It doesn't make sense to rollback a staged deployment. Give a hint
instead to use `cleanup -p` if needed. Otherwise, we dump core later on
in libostree.

This is technically a breaking change once staged deployments become the
default. However, I think it makes sense overall. We'll have to make
some noise about it.

Closes: #1434
Approved by: cgwalters
2018-06-28 22:01:14 +00:00
Jonathan Lebon
1b5ab9834e app/dbus-helpers: Return early if no diff to print
A new update might not have any package changes at all. In which case,
we shouldn't even try to print the diff, otherwise we'll fail on trying
to lookup the `upgraded` key. We did this check already from
`rpmostree_print_cached_update`, but not from `print_one_deployment`
(when printing the diff as part of the pending deployment), so we'd
error out on no-diff upgrades.

Let's just inline a check in the function directly instead of wrapping
every call site.

Closes: #1431
Approved by: cgwalters
2018-06-28 20:49:23 +00:00
Jonathan Lebon
201b020f60 ci: Bump version_id match to pick up ostree FAHC
Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Colin Walters
bc52e8eaf3 tests: Update for ostree BLS file naming change
We should probably add a libtest.sh API for so for this, but this is
the quick hack.  Fallout from https://github.com/ostreedev/ostree/pull/1654

Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Colin Walters
77e2e19c1f postprocess: Set default.target in /usr
Follow-up improvement after
https://github.com/openshift/os/pull/135

This should ensure it survives systemd's preset run on firstboot.

Although honestly...what we should *really* do is check whether
the `default.target` symlink target exists, and if not reset it
to `multi-user.target` so no one would have to care, but that'd
be conceptually separate from this, so I may do it later.

Closes: #1427
Approved by: jlebon
2018-06-25 20:45:32 +00:00
Colin Walters
d7342731ab Add a treefile option to stop creating /etc/machine-id
We actually want systemd's `ConditionFirstBoot` to fire.  The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.

However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`.  The correct thing
to do from the start is to use presets.  We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.

See: https://github.com/dustymabe/bootengine/pull/11

Closes: #1425
Approved by: jlebon
2018-06-25 16:19:00 +00:00
Jonathan Lebon
94ee42b03b daemon: Add a few missing transaction titles
Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Michael Scherer
14b104d9f1 Fix the link to Fedora Atomic json file
Closes: #1423
Approved by: jlebon
2018-06-22 14:13:06 +00:00
Colin Walters
a4a8c7cc96 rust/treefile: Fix mutate-os-release typo
It's tempting to enable this by default in YAML...one step at a
time though.
2018-06-21 11:53:06 -04:00
Micah Abbott
4c61a19f42 Clarify the use of --uninstall switch
While looking at the inline help for `install`, I thought we could do
better by specifically stating that the `--uninstall` switch only
operates on layered packages.  This might prevent any users from
trying to use the `--uninstall` switch on a package that is part of
the base deployment.

Closes: #1421
Approved by: jlebon
2018-06-20 14:26:33 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
8b66c830b7 app/builtins: Consistent description for --reboot
A bunch of commands had descriptions for `--reboot` that were copied
from the `upgrade` description. Let's just make them all the same to
keep it consistent.

Closes: #1418
Approved by: cgwalters
2018-06-18 13:49:41 +00:00
Jonathan Lebon
096b0554f0 man/rpm-ostree: Tweak install/uninstall docs
Minor tweaks to try to clarify that the `install` and `uninstall`
commands are for managing an overlay. This should hopefully make it less
likely that someone tries `uninstall` when really they want
`override remove`.

Also drop the paragraph about having to specify NEVRAs for removing
local RPMs since that's no longer necessary.

Closes: #1418
Approved by: cgwalters
2018-06-18 13:49:41 +00:00
Colin Walters
57475e5cc7 daemon: Further consolidate option parsing for deploy
Fewer variables and avoid extra strdupv calls.  I also fixed
handling of refspec canonicalization that I broke with a previous
change (we weren't actually setting `self->refspec` to the canonicalized
version).

Closes: #1414
Approved by: jlebon
2018-06-15 20:19:41 +00:00