Commit Graph

457 Commits

Author SHA1 Message Date
Colin Walters
5126b2774e Stabilize stage automatic update policy
Now that staging is the default, there's no reason for this
policy not to be stable too.

Closes: #1510
Approved by: jlebon
2018-08-21 01:01:45 +00:00
Colin Walters
b6d07487d5 Turn staged deployments on by default
We've put a lot of work into staged deployments, it's time
to pull the trigger and turn them on by default.  This is
a key step for enabling `stage` mode automatic updates by
default in e.g. Fedora CoreOS/Silverblue.

We add a new `--disable-staged` build-time option to flip
things back.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
44b39a7340 livefs: Require deployment staging
Staging fixes the `/etc` bug for livefs.  There's actually more
we could do here around taking advantage of staging for livefs;
for example, I think once the livefs is complete, we could just delete
the staged deployment.  And then we don't need to render on the next
boot the live status, etc.

Anyways, all that can come in the future.  This is prep for
enabling staging by default.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
e1b62d0f7d tests/vmcheck: Display human-readable status on jq failure
So it's easier to debug.

This inlines the helper into the only function that uses it.

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Colin Walters
2278b60a15 tests: Remove dead assert_status_jq
Nothing calls it today; looks like it was last used in
283b915ecf

Closes: #1430
Approved by: jlebon
2018-08-20 20:32:00 +00:00
Jonathan Lebon
4ccad2f64b app/status: Tweak output with --booted
Fix `--booted` was printing an extra line after `Deployments:`. While
we're there, also update the section name to `BootedDeployment:` in that
case to be more correct. (We're not printing all the deployments, only
the booted deployment.)

Closes: #1503
Approved by: cgwalters
2018-08-14 12:32:50 +00:00
Jonathan Lebon
45e162fb69 ci: Split compose test into two
The `f28-compose` test keeps timing out. Some time recently, I/O
performance of the internal OpenStack instance used for testing has
degraded. I have a ticket open to investigate the regression though
haven't had any luck so far.

Let's just take the easy way out and split the test into two testsuites.
This is obviously hacky, and sad, and unfortunate. But the PRs must keep
flowing until we finally wean off of OpenStack.

Closes: #1498
Approved by: cgwalters
2018-08-13 21:06:18 +00:00
Colin Walters
15aecff36a livefs: Rename --replace to --dangerous-do-not-use-replace
And don't mention "replace" in the error texts.
This is (now) known broken with kernel updates; making that not
dangerous isn't going to be easy.  I debated entirely removing it
but for now let's just make it harder to use.

Ref: https://github.com/projectatomic/rpm-ostree/issues/1495

Closes: #1497
Approved by: jlebon
2018-08-09 15:04:11 +00:00
Jonathan Lebon
7c87a553e4 tests/compose: Fix wrong assert in test-rojig-e2e
We should be expecting testpkg-1.1-1 here, not 1.0-1. This was passing
before because of the nondeterministic `find` output (fixed in the
previous commit) which could spit out the older rojig RPM.

Closes: #1491
Approved by: cgwalters
2018-08-07 16:00:13 +00:00
Jonathan Lebon
3891aa2563 tests/compose: Simplify test-rojig-e2e file search
This test was relying on the order in which `find` reports matching path
names to find the right RPM. This was failing for me locally sometimes
because it matched the wrong RPM file. Fix this by just directly
referencing the full path name since we can.

Closes: #1491
Approved by: cgwalters
2018-08-07 16:00:13 +00:00
Jonathan Lebon
dce98e4a53 tests/compose: Split out machineid-compat test
My fix to the testsuite in #1488 in which I made the `machineid-compat`
test part of `test-basic.sh` wasn't correct since the basic tests in
`libbasic-test.sh` also check that the default behaviour without the
`machineid-compat` option is to include it.

Let's just do this right and split out the `machineid-compat` test into
its own run.

Closes: #1491
Approved by: cgwalters
2018-08-07 16:00:13 +00:00
Jonathan Lebon
391a684b5f tests/compose: Fix run-test.sh regression
Regression from hasty hack in #1488. We want to return nonzero if the
test failed so that `parallel` fails too.

Closes: #1491
Approved by: cgwalters
2018-08-07 16:00:13 +00:00
Jonathan Lebon
e94f8c9b5f compose: Fix mutate-os-release handling
I noticed that the latest Fedora Atomic Host 28 and Silverblue did not
have an `OSTREE_VERSION` line in `/etc/os-release` even though both
specified `mutate-os-release` in their manifests. This turned out to be
due to the fact that `/usr/lib/os-release` is now a symlink to a
variant-specific file (e.g. `os-release-atomichost`), so we would
fallback to mutating `/usr/lib/os.release.d/os-release-fedora` instead.

Fix this by just taking the nuclear option of running `realpath` in the
rootfs directly. This is more maintainable than trying to keep up with
changes in variants/naming/etc. There's related discussions to this in
the original [PR](https://github.com/projectatomic/rpm-ostree/pull/410)
which introduced the feature re. resolving symlinks within the rootfs.

Closes: #1481
Approved by: jlebon
2018-08-06 01:32:49 +00:00
Jonathan Lebon
1c899c5e9e compose-tests: Rename dir based on failure
Should make it easier to tell which compose test failed rather than
going through the verbose output of parallel.

Closes: #1488
Approved by: jlebon
2018-08-06 00:33:56 +00:00
Colin Walters
588a0327db postprocess: Error if units with machineid-compat: false
Rather than silently ignoring it.

In theory...we could write to /usr/lib/systemd instead of `/etc`
but eh...I feel like what we really want to do is make it convenient
to write a preset file from the YAML.

(We could have an `add-files` content that takes values literally
 which would be nice in YAML and suck in JSON)

A general thread running through this is that for people making
*derivatives* of a CoreOS-like system, having to create their
own `exampleos-release` package is an annoying hurdle.

Anyways for now we're fixing the bug that we were silently ignoring
it.

Closes: #1488
Approved by: jlebon
2018-08-06 00:33:56 +00:00
Colin Walters
344aee1d76 rust: Add support for inline rojig spec files
The rojig spec is almost entirely rpm-ostree implementation details;
let's not have lots of people fork/duplicate it.  Rather add the bits
of rojig to the treefile that people need to define (most notably
the name).

Prep for stabilizing rojig.

I had a few false starts with this PR; managing ownership/lifetimes
across C/Rust is just complicated.  I got bit hard by the fact that
the workdir in `--unified-core` is really dfd-relative, and had to
do a dance to propagate the dfd into rust, as well as down into
the rojig builder.

Closes: #1484
Approved by: jlebon
2018-08-03 16:54:47 +00:00
Colin Walters
23badcd288 compose: Write a preset file to enable ostree-remount.service
I've lost count now of how many times people have hit variants
of https://github.com/projectatomic/centos-release-atomic-host-devel/pull/6
Let's just bake it in.

Closes: #1482
Approved by: jlebon
2018-08-01 20:11:38 +00:00
Colin Walters
628a3aa22f tests/compose: Write logs directly
Make logging work the same as it does for the vmcheck-STI work
(at some point I'll try to unify the 3 parallel+script implementions
 we have).  This fixes the problem that when the test times out,
the filename won't have `.txt` and S3 won't have the right MIME type.

Closes: #1479
Approved by: jlebon
2018-08-01 18:05:09 +00:00
Colin Walters
800402b00a tests/compose: Add some time logging
To help us profile.

Closes: #1479
Approved by: jlebon
2018-08-01 18:05:09 +00:00
Jonathan Lebon
67860c5ece app/pkg-builtins: Add --unchanged-exit-77
This is analogous to `upgrade --upgrade-unchanged-exit-77`, but for
`install`/`uninstall`. This way, one can determine whether the command
truly had an effect on the default deployment or not.

Since this works by comparing against the previous default deployment,
this works correctly even if we already had a pending deployment at the
beginning of the transaction.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
d35fbb665e app/pkg-builtins: Add --idempotent
Add a new `install/uninstall --idempotent` option to make it easier to
interact with the CLI through scripts. E.g. one doesn't have to check
first if a request has already been installed/uninstalled.

Closes: #1467

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
fcb061b19a app/status: Add --pending-exit-77 switch
This makes it easier for scripts to determine whether there is a pending
deployment instead of using `--json/--jsonpath`.

Closes: #1478
Approved by: cgwalters
2018-07-31 08:53:15 +00:00
Jonathan Lebon
7911b14f49 daemon: Fix cached-update including no-op diffs
The `cached-update` variant would mark a bunch of RPMs as upgraded even
if they didn't actually change. The issue turned out to be we were doing
the diff all wrong in the staged deployment case. I'm not sure what I
was thinking in #1344, but essentially, we were marking all layered RPMs
in the staged deployment as updates instead of only marking those
layered RPMs which were actually changed EVR.

We just simplify the approach here by directly doing a pkglist diff
between the booted and staged deployments and consuming that. That's
really all there is to it! Reduces the code quite a bit too.

Closes: #1446

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Jonathan Lebon
a17b4b9be0 tests/utils: Drop empty inject-pkglist.py
This one snuck in as part of #1205. I had initially started writing
`inject-pkglist.c` in Python and somehow inadvertedly made it part of
a commit.

Closes: #1455
Approved by: cgwalters
2018-07-28 06:53:40 +00:00
Colin Walters
e858a30eca compose: Stabilize --unified-core
In line with the recent trend of marking things stable, and in
preparation for stabilizing `rojig://` - Let's stabilize the `--unified-core`
option for `compose tree`.

I'm not sure we could make it the default anytime soon; today it trips
over bugs in the PAM package in RHEL7 for example.  But it
works fine for Fedora, and I think the code/design are good enough to be stable.

Closes: #1465
Approved by: jlebon
2018-07-24 23:06:50 +00:00
Colin Walters
fa29f7acfa compose: Support arch-specific packages in YAML (and in JSON again)
Follow up to: https://github.com/projectatomic/rpm-ostree/pull/1459

We now honor arch-specific packages in YAML, and reject unknown
architectures.  I looked a little bit at how to avoid having hardcoded
arch lists, but it doesn't seem worth it right now.

Closes: #1468
Approved by: jlebon
2018-07-24 22:05:06 +00:00
Jonathan Lebon
b66337e0cb ci: Pepper in date markers to help debug timing
This should help out with determining what steps take the most time.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
6ac6f3d086 treefile.rs: Deny unknown fields by default
Let's not make the same mistake we did with JSON where typoing a
field means it's silently ignored.  This actually caught a bug
in a YAML usage we had:

```
error: Failed to load YAML treefile: unknown field `install_langs`, expected one of ... `install-langs` ...
```

Yes, this is a compatibility break with the feature we just announced
but...I seriously doubt anyone (that isn't known to me) has converted
yet, and if they are excited enough to start using a two-week-old feature
they can adjust.

Closes: #1459
Approved by: cgwalters
2018-07-21 14:43:48 +00:00
Colin Walters
1b4a73f071 tests: Make two more tests compatible with deployment staging
Unlike the kernel args one, these two were simple.

Closes: #1458
Approved by: jlebon
2018-07-21 13:58:21 +00:00
Colin Walters
096004426c rebase: Add support for "custom origin" descriptions
We're looking to embed an ostree commit inside a container image,
to make it easier to transport around with other images.

Conceptually here the host system is tracking a container (just
like for rojig we're tracking an RPM).  This is the first step
towards making that support nicer; tooling can do
`rebase --custom-origin-url oscontainer://quay.io/exampleos@sha256:...`
and have that show up in `rpm-ostree status`.

There are two values, one intended to be machine readable (like
the `ostree://` and `rojig://` and one for humans which we
display when an admin types `rpm-ostree upgrade`.

This builds on prior work in
27bd7b97bb from #1396 .

Closes: #1406
Approved by: jlebon
2018-07-20 18:47:51 +00:00
Colin Walters
035a2a72ba tests/kernel-args: Prep for staged
This test was hard to adapt for staging.  I did the obvious change
of just rebooting after most of the `kargs` invocations, but much
later in the suite there was a test that compared vs the "pristine"
kargs.  Supporting that would be a bit hard...I briefly thought
about pinning etc. but it gets so messy to do in both staged and not-staged
defaults.  So I just deleted those bits.

Closes: #1453
Approved by: jlebon
2018-07-12 17:44:15 +00:00
Jonathan Lebon
caf66d6e45 vmcheck/autoupdate-check: Fix minor typos
Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
cfb5e7c04b libvm: add vm_get_journal_after_cursor
The cursor argument was getting munged up by SSH. Use a helper to make
sure we always get quoting right.

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Jonathan Lebon
fbee05bcc5 vmcheck/autoupdate-check.sh: Fix --check/--preview tests
Noticed this while looking at the logs for #1432.

Because --check and --preview exit with rc=77 when there are no updates,
we would actually stop early on in the test and marking it as SKIPPED.
Fix this by making sure we explicitly check for the $rc we expected when
using those switches.

I also added a final grep pass to make it easy to inspect whether we
skipped any tests. I was about to do this nicely in `multitest.py`
instead, though it may not be of this world much longer, so meh...

Closes: #1450
Approved by: cgwalters
2018-07-11 13:56:37 +00:00
Colin Walters
bfd39aecf3 status: Rework auto-update status display
First, split it into its own section; it's important enough to merit it.
Second, explicitly reference the systemd timer/service units.  For
example, a question I often have is "when is the next run" and of course
you can get that rpm `systemctl status rpm-ostreed-automatic.timer` but
you have to know that, and the reminder helps.

(I briefly looked at implementing the `Trigger` line from `systemctl status`
 but it's not entirely trivial...tempting to just fork off a `systemctl status | grep `)

Prep for unifying this text with the message we print when one does
`rpm-ostree upgrade` when auto-updates are enabled.

Closes: #1432
Approved by: jlebon
2018-07-10 13:26:37 +00:00
Colin Walters
7fa15cf078 app: Add an updateupgrade alias
Same as `dnf` for example, and `flatpak` today uses `update`, so
let's do both.

Closes: https://github.com/projectatomic/rpm-ostree/issues/1448

Closes: #1449
Approved by: jlebon
2018-07-09 14:21:32 +00:00
Colin Walters
222d730cce vmcheck/misc-2: Make compatible with staged default
First the pinning tests would try to pin a staged deployment,
and some of the later tests here depend on a subtle way on the
state of the system.  It's tempting to do a `reset` before each one
and reboot but this makes things work.

There's some additional assertions here as I went through and
was debugging.

Prep for making staging the default.

Closes: #1438
Approved by: jlebon
2018-07-03 19:14:54 +00:00
Colin Walters
bc52e8eaf3 tests: Update for ostree BLS file naming change
We should probably add a libtest.sh API for so for this, but this is
the quick hack.  Fallout from https://github.com/ostreedev/ostree/pull/1654

Closes: #1433
Approved by: jlebon
2018-06-28 19:33:11 +00:00
Jonathan Lebon
07ecfca3d9 Add new reset command
Add a new `reset` command that makes it easy to blow away all
customizations: overlays, overrides, and initramfs. One can use flags to
only reset some of the customizations.

I placed this under `ex` out of conservatism. It's a pretty simple
command with simple behaviour, though the features it relies on
(no-layering, no-initramfs) are brand new. We can move it out of there
in a release or two?

Closes: #1387

Closes: #1419
Approved by: cgwalters
2018-06-26 20:32:28 +00:00
Jonathan Lebon
c84b8b81db app: Lift kargs out of experimental
It's been in experimental for a while now, and we've had good feedback
that it's working. With #1392 fixed, it seems in a good position to
declare stable and commit to that API. This also helps empty out `ex` a
bit.

Closes: #1428
Approved by: cgwalters
2018-06-26 19:14:56 +00:00
Colin Walters
77e2e19c1f postprocess: Set default.target in /usr
Follow-up improvement after
https://github.com/openshift/os/pull/135

This should ensure it survives systemd's preset run on firstboot.

Although honestly...what we should *really* do is check whether
the `default.target` symlink target exists, and if not reset it
to `multi-user.target` so no one would have to care, but that'd
be conceptually separate from this, so I may do it later.

Closes: #1427
Approved by: jlebon
2018-06-25 20:45:32 +00:00
Colin Walters
d7342731ab Add a treefile option to stop creating /etc/machine-id
We actually want systemd's `ConditionFirstBoot` to fire.  The
primary rationale here is that we're adopting Ignition for Fedora CoreOS,
and having `ConditionFirstBoot=` function will help a lot, as the idea
is it only runs once.

However, I discovered that this breaks the `units` directive for example,
as systemd blows away all the unit state in `/etc`.  The correct thing
to do from the start is to use presets.  We could add an implementation of
`units` which works with this on and instead writes a preset file but...eh.
My plan is to at some point introduce an "epoch" and flip various defaults,
this one, `tmp-is-dir`, the passwd file handling, etc.

See: https://github.com/dustymabe/bootengine/pull/11

Closes: #1425
Approved by: jlebon
2018-06-25 16:19:00 +00:00
Jonathan Lebon
b302aa45bc daemon/deploy: Allow layering with no-layering option
We added the `no-layering` option, but made it conflicting with
`--install`. This loosens that requirement so that one can do e.g.

    rpm-ostree uninstall --all -install foobar

to essentially remove all layered packages and then add back `foobar`.
Prep for `reset` command.

Closes: #1424
Approved by: cgwalters
2018-06-22 15:07:42 +00:00
Ruixin Bao
92f08ea80b upgrader/vmcheck: fix kargs get lost when doing multiple operations
Kernel arguments added from `rpm-ostree ex kargs` can be lost
after doing multiple deploy related operations (e.g upgrade, install etc).

The root cause of that is because when doing the deployment
(ostree_sysroot_deploy_tree), the kernel arguments from the
pending deployment were never passed in.

This commit adds logic to pass in kernel arguments from pending deployment
to fix the above mentioned problem

Closes: https://github.com/projectatomic/rpm-ostree/issues/1392

Closes: #1420
Approved by: cgwalters
2018-06-18 16:13:57 +00:00
Jonathan Lebon
fb006b5b43 app/uninstall: Add --all switch to drop all layering
This is one part of #1387 and is prep for a single higher level command
that makes it easier to remove all customizations (layering *and*
overrides).

Closes: #1408
Approved by: cgwalters
2018-06-14 12:36:15 +00:00
Jonathan Lebon
72e427cc3d app/override: Don't crash on no change
Doing `override reset --all` may be a no-op if there are no overrides
currently applied. But we were trying to always print the tree diff
under the assumption that all override invocations have to result in a
new deployment.

Fix this by using the same default deployment check that `upgrade` does
before printing.

Closes: #1407
Approved by: cgwalters
2018-06-13 20:32:37 +00:00
Jonathan Lebon
d436d48a2f libpriv/origin: Allow removing local RPMs by name only
This fixes a painful UX issue where one must use the full NEVRA when
uninstalling a locally layered RPM. Now, one can specify either the
NEVRA or the package name only. Though we still try to interpret the
request first as a NEVRA or a capability.

Closes: #1386

Closes: #1400
Approved by: cgwalters
2018-06-12 14:55:40 +00:00
Jonathan Lebon
073978aace app/override: Don't auto-enable cache-only if also installing
Otherwise, we won't be able to fetch the requested packages from the
repo.

Closes: #1384

Closes: #1402
Approved by: cgwalters
2018-06-12 14:52:27 +00:00
Colin Walters
27bd7b97bb Handle "pinned commits" specifically
There are a few scenarios today where one might deliver content
to a machine via an external transport.  For example, take the
scenario of a single server updated via USB drive.  While we
can provide a refspec...what should the remote be?  (This gets
into ostree collections).  There's nothing really that can
happen when typing `rpm-ostree upgrade` unless the USB stick
is plugged in.  That type of scenario should be emphasized
by pinning the commit - the machine is updated via an external
script.

Another case: we're experimenting embedding OSTree commits inside OCI
containers.  Here again since rpm-ostree can't understand how to
pull content from containers, it's saner to drop the refspec
bits, and pin to a commit.

Further enhancements will follow to make the admin experience more
obvious.

Closes: #1396
Approved by: jlebon
2018-06-11 20:37:58 +00:00
Colin Walters
26f04595b2 upgrade: Note if automatic updates are enabled
When e.g. FAH does the automatic update transition, let's remind
people who are doing it manually still.

Closes: #1399
Approved by: jlebon
2018-06-11 18:46:58 +00:00