Commit Graph

2243 Commits

Author SHA1 Message Date
Colin Walters
831507b0f0 scripts: Add RPMOSTREE_SCRIPT_TRACE debugging envvar
I used `env RPMOSTREE_SCRIPT_TRACE='strace -f -s2048' rpm-ostree compose ...`
to debug https://bugzilla.redhat.com/show_bug.cgi?id=1548050

Closes: #1472
Approved by: jlebon
2018-07-27 13:09:49 +00:00
Colin Walters
e858a30eca compose: Stabilize --unified-core
In line with the recent trend of marking things stable, and in
preparation for stabilizing `rojig://` - Let's stabilize the `--unified-core`
option for `compose tree`.

I'm not sure we could make it the default anytime soon; today it trips
over bugs in the PAM package in RHEL7 for example.  But it
works fine for Fedora, and I think the code/design are good enough to be stable.

Closes: #1465
Approved by: jlebon
2018-07-24 23:06:50 +00:00
Colin Walters
fa29f7acfa compose: Support arch-specific packages in YAML (and in JSON again)
Follow up to: https://github.com/projectatomic/rpm-ostree/pull/1459

We now honor arch-specific packages in YAML, and reject unknown
architectures.  I looked a little bit at how to avoid having hardcoded
arch lists, but it doesn't seem worth it right now.

Closes: #1468
Approved by: jlebon
2018-07-24 22:05:06 +00:00
Jonathan Lebon
1a82f210ae ci: Bump f28-compose by 10 minutes
We're consistently seeing timeouts on the `f28-compose` test. It's
likely due to a performance regression in the backend rather than
anything else. I filed a ticket to get an investigation going, though
for now at least, let's bump the timeout some more to keep patches
flowing.

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Jonathan Lebon
c016509548 libdnf: Bump and disable html and man pages
Skip building man pages and HTML docs for our embedded libdnf to speed
up builds.

This bump also pulls in a fix to ensure we never try to install src
packages from `dnf_context_install()`.

See: https://github.com/projectatomic/libdnf/pull/3

Update submodule: libdnf

Closes: #1463
Approved by: jlebon
2018-07-24 20:15:10 +00:00
Jonathan Lebon
b66337e0cb ci: Pepper in date markers to help debug timing
This should help out with determining what steps take the most time.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
6ac6f3d086 treefile.rs: Deny unknown fields by default
Let's not make the same mistake we did with JSON where typoing a
field means it's silently ignored.  This actually caught a bug
in a YAML usage we had:

```
error: Failed to load YAML treefile: unknown field `install_langs`, expected one of ... `install-langs` ...
```

Yes, this is a compatibility break with the feature we just announced
but...I seriously doubt anyone (that isn't known to me) has converted
yet, and if they are excited enough to start using a two-week-old feature
they can adjust.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
1b4a73f071 tests: Make two more tests compatible with deployment staging
Unlike the kernel args one, these two were simple.

Closes: #1458
Approved by: jlebon
2018-07-21 13:58:21 +00:00
Colin Walters
0cc002fd8d build-sys: Use python3 for libdnf by default if available
Probably at some point libdnf will drop py2 support, but the
main reason I'm doing this is avoids a python2 dependency
for rpm-ostree for distributions that don't want that.

Note of course rpm-ostree itself doesn't use python, libdnf does.
And only for the python bindings, which we don't use either.  So
this is mostly just to DTRT automatically for the libdnf bits; down
the line we could probably add a patch to make the python fully
conditional.

Closes: #1460
Approved by: jlebon
2018-07-20 22:27:34 +00:00
Jonathan Lebon
ec19d67d74 ci: Stop rebasing Fedora onto updates ref
We can drop this hack now since there have been updates since GA. Also
add a note that trying to drop the rebase to continuous causes
`layering-relabel` to fail.

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
096004426c rebase: Add support for "custom origin" descriptions
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.

Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM).  This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.

There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.

This builds on prior work in
27bd7b97bb from #1396 .

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
95de58c29d Update libglnx
For `renameat2()` fix to build with latest glibc (e.g. Fedora rawhide).

Update submodule: libglnx

Closes: #1461
Approved by: jlebon
2018-07-17 18:35:38 +00:00
Colin Walters
d1839378d6 compose: Add some assertions around treefile context directory
Seeing a crash on in one of our builds, not sure yet what's
going wrong as I haven't gotten a core out, but this makes things
clearer.

($kingdom for `T`/`Option<T>` instead of nullable pointers...)

Closes: #1454
Approved by: jlebon
2018-07-12 18:30:55 +00:00
Colin Walters
035a2a72ba tests/kernel-args: Prep for staged
This test was hard to adapt for staging.  I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs.  Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults.  So I just deleted those bits.

Closes: #1453
Approved by: jlebon
2018-07-12 17:44:15 +00:00
Ruixin Bao
9373f13b0a doc: add repo_structure section
This commit plans to add repo_structure document to better explain
what is the meaning for each folder. A similar approach can be found
in https://github.com/openshift/openshift-ansible/blob/master/docs/repo_structure.md

Closes: #1452
Approved by: jlebon
2018-07-11 20:40:59 +00:00
Jonathan Lebon
caf66d6e45 vmcheck/autoupdate-check: Fix minor typos
Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
cfb5e7c04b libvm: add vm_get_journal_after_cursor
The cursor argument was getting munged up by SSH. Use a helper to make
sure we always get quoting right.

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
2e237513b0 rust/treefile: Avoid unnecessary dup() of fd
We weren't checking for an error from `dup()`, and further it shouldn't
be necessary.  This is the best I could come up with, though it's
obviously not pretty.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
d7b4046599 rust: Add a make rustfmt, run it
It is actually really nice that there's One Canonical Style, even
if I sometimes don't like some details of what rustfmt does.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
e731f59007 rust: Split out treefile into its own module
Further prep for adding more code here.  The `lib.rs` then is the
collection of glue functions; perhaps in some ideal future it could
be generated even.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
30ccaa4db6 build-sys: Use --frozen if Rust library was already built
The main rationale here is avoiding any Cargo internet access during
`sudo make install` type flows.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
2894d61fbf build-sys: Make Rust library naming/entrypoint more generic
Prep for adding more functionality beyond treefile bits.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Colin Walters
bac6d634bb build-sys: Move Rust include header under rust/ directory
It makes more sense to have the include live next to the associated
code, just like we do with C, even though the `cargo build` doesn't
touch it.

Closes: #1444
Approved by: jlebon
2018-07-05 13:58:22 +00:00
Yoginski
bcbd68d9c8 Explain how to remove packages from the base layer
Haven't seen it being mentioned in any manuals.
Closes: #1441
Approved by: cgwalters
2018-07-03 21:14:28 +00:00
Colin Walters
b1d7ef15d8 sysroot-core: Use new libostree sysroot pruning API
Use the new API introduced in
9131d8a4cc
which helps avoid relying on the magical deployment refs.  I thought
this would help with a livefs+staging issue, but it doesn't.  But we might
as well do it, it's adding another level of safety.

Closes: #1439
Approved by: jlebon
2018-07-03 20:18:00 +00:00
Colin Walters
222d730cce vmcheck/misc-2: Make compatible with staged default
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system.  It's tempting to do a `reset` before each one
and reboot but this makes things work.

There's some additional assertions here as I went through and
was debugging.

Prep for making staging the default.

Closes: #1438
Approved by: jlebon
2018-07-03 19:14:54 +00:00
Jonathan Lebon
6b79e9c681 Release 2018.6
The tree is in good condition and it's been a while. Let's do this!

Closes: #1436
Approved by: cgwalters
2018-06-29 17:32:57 +00:00
Jonathan Lebon
b94fef9f24 daemon/rollback: Really fix staged deployment case
Follow up to #1434. I was testing this inside my test VM, which didn't
have a rollback deployment (i.e. it was just the staged deployment and
the booted deployment). Testing the upcoming release on my real system
showed that this wasn't correctly fixed.

Closes: #1437
Approved by: cgwalters
2018-06-29 16:30:07 +00:00
Jonathan Lebon
beb8775d8e daemon/rollback: Error out if pending deployment is staged
It doesn't make sense to rollback a staged deployment. Give a hint
instead to use `cleanup -p` if needed. Otherwise, we dump core later on
in libostree.

This is technically a breaking change once staged deployments become the
default. However, I think it makes sense overall. We'll have to make
some noise about it.

Closes: #1434
Approved by: cgwalters
2018-06-28 22:01:14 +00:00
Jonathan Lebon
1b5ab9834e app/dbus-helpers: Return early if no diff to print
A new update might not have any package changes at all. In which case,
we shouldn't even try to print the diff, otherwise we'll fail on trying
to lookup the `upgraded` key. We did this check already from
`rpmostree_print_cached_update`, but not from `print_one_deployment`
(when printing the diff as part of the pending deployment), so we'd
error out on no-diff upgrades.

Let's just inline a check in the function directly instead of wrapping
every call site.

Closes: #1431
Approved by: cgwalters
2018-06-28 20:49:23 +00:00
Jonathan Lebon
201b020f60 ci: Bump version_id match to pick up ostree FAHC
Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Colin Walters
bc52e8eaf3 tests: Update for ostree BLS file naming change
We should probably add a libtest.sh API for so for this, but this is
the quick hack.  Fallout from https://github.com/ostreedev/ostree/pull/1654

Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Colin Walters
77e2e19c1f postprocess: Set default.target in /usr
Follow-up improvement after
https://github.com/openshift/os/pull/135

This should ensure it survives systemd's preset run on firstboot.

Although honestly...what we should *really* do is check whether
the `default.target` symlink target exists, and if not reset it
to `multi-user.target` so no one would have to care, but that'd
be conceptually separate from this, so I may do it later.

Closes: #1427
Approved by: jlebon
2018-06-25 20:45:32 +00:00
Colin Walters
d7342731ab Add a treefile option to stop creating /etc/machine-id
We actually want systemd's `ConditionFirstBoot` to fire.  The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.

However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`.  The correct thing
to do from the start is to use presets.  We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.

See: https://github.com/dustymabe/bootengine/pull/11

Closes: #1425
Approved by: jlebon
2018-06-25 16:19:00 +00:00
Jonathan Lebon
94ee42b03b daemon: Add a few missing transaction titles
Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Michael Scherer
14b104d9f1 Fix the link to Fedora Atomic json file
Closes: #1423
Approved by: jlebon
2018-06-22 14:13:06 +00:00
Colin Walters
a4a8c7cc96 rust/treefile: Fix mutate-os-release typo
It's tempting to enable this by default in YAML...one step at a
time though.
2018-06-21 11:53:06 -04:00
Micah Abbott
4c61a19f42 Clarify the use of --uninstall switch
While looking at the inline help for `install`, I thought we could do
better by specifically stating that the `--uninstall` switch only
operates on layered packages.  This might prevent any users from
trying to use the `--uninstall` switch on a package that is part of
the base deployment.

Closes: #1421
Approved by: jlebon
2018-06-20 14:26:33 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
8b66c830b7 app/builtins: Consistent description for --reboot
A bunch of commands had descriptions for `--reboot` that were copied
from the `upgrade` description. Let's just make them all the same to
keep it consistent.

Closes: #1418
Approved by: cgwalters
2018-06-18 13:49:41 +00:00
Jonathan Lebon
096b0554f0 man/rpm-ostree: Tweak install/uninstall docs
Minor tweaks to try to clarify that the `install` and `uninstall`
commands are for managing an overlay. This should hopefully make it less
likely that someone tries `uninstall` when really they want
`override remove`.

Also drop the paragraph about having to specify NEVRAs for removing
local RPMs since that's no longer necessary.

Closes: #1418
Approved by: cgwalters
2018-06-18 13:49:41 +00:00
Colin Walters
57475e5cc7 daemon: Further consolidate option parsing for deploy
Fewer variables and avoid extra strdupv calls.  I also fixed
handling of refspec canonicalization that I broke with a previous
change (we weren't actually setting `self->refspec` to the canonicalized
version).

Closes: #1414
Approved by: jlebon
2018-06-15 20:19:41 +00:00
Jonathan Lebon
293baa222d treefile.rs: Add missing #[serde(rename)] attributes
I was surprised when I fired up GDB and found out that my
`etc-group-members` modifications weren't being obeyed. It turns out
serde was looking for the wrong field name. I did a quick sanity check
and found a few others.

Closes: #1417
Approved by: cgwalters
2018-06-15 18:48:34 +00:00
Colin Walters
5d75697dc4 status: Assert that we have deployments
I think this is what's showing up in a PR test:
21b68a68ff.2.1529072619059542132/artifacts/vmcheck/livefs.log
```
(rpm-ostree status:17017): Json-CRITICAL **: json_gvariant_serialize: assertion 'variant != NULL' failed
```

It shouldn't be happening; let's add an assertion which will give us a core
dump reliably instead of the "stumble on" `g_warning()` default.

Closes: #1416
Approved by: jlebon
2018-06-15 18:24:19 +00:00