Commit Graph

44 Commits

Author SHA1 Message Date
OpenShift Merge Robot
be62a01bff
Merge pull request #2106 from jlebon/pr/fix-admin-test
tests/admin-test: Ensure that commits are 1s apart
2020-05-20 19:05:35 +02:00
Colin Walters
191ce95ca1 admin-test: Show err.txt on unexpected failure
In a CI run I think one of these `ostree show` commands is failing.
While that output would be useful, the actual `err.txt` usually
has what we want too.
2020-05-20 15:57:10 +00:00
Jonathan Lebon
70ebb91aa3 tests/admin-test: Ensure that commits are 1s apart
Otherwise the new check we added there to verify that upgrading without
`--allow-downgrade` fails itself fails.

See: https://github.com/ostreedev/ostree/pull/2099#issuecomment-629805840
2020-05-20 10:48:44 -04:00
Colin Walters
f14aa894d3 ci: Test for clock skew
I saw `tests/test-admin-deploy.none.sh` fail in one CI run, and
I want to check if it was because of clock skew, so fail
fast if we detect that.

xref https://github.com/ostreedev/ostree/pull/2099#issuecomment-629805375
2020-05-18 18:48:28 +00:00
Jonathan Lebon
79079c2657 lib/upgrader: Pull with timestamp-check-from-rev
For the same reason as https://github.com/coreos/rpm-ostree/pull/2094.
What we care most about is that the new commit we pull is newer than the
one we're currently sitting on, not necessarily that it's newer than the
branch itself, which it might not be if e.g. we're trying to deploy a
commit older than the tip but still newer than the deployment (via
`--override-commit`).
2020-05-14 15:27:29 -04:00
Javier Martinez Canillas
d42f970ed3 lib/bootconfig-parser: Always include deployment index in BLS title
If there are different deployments for the same commit version, the BLS
snippets will have the same title fields (but different version fields):

$ grep title *
ostree-1-testos.conf:title TestOS 42 20190902.0 (ostree)
ostree-2-testos.conf:title TestOS 42 20190902.0 (ostree)
ostree-3-testos.conf:title TestOS 42 20190902.0 (ostree)

But bootloaders could expect the title field to be unique for BLS files.
For example, the zipl bootloader used in the s390x architecture uses the
field to name the boot sections that are created from the BLS snippets.

So two BLS snippets having the same title would lead to zipl failing to
create the IPL boot sections because they would have duplicated names:

$ zipl
Using config file '/etc/zipl.conf'
Using BLS config file '/boot/loader/entries/ostree-3-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-2-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-1-testos.conf'
Error: Config file '/etc/zipl.conf': Line 0: section name 'TestOS 42 20190902.0 (ostree)' already specified

Avoid this by always including the deployment index along with the commit
version in the title field, so this will be unique even if there are BLS
files for deployments that use the same commit version:

$ grep title *
ostree-1-testos.conf:title TestOS 42 20190902.0 (ostree:2)
ostree-2-testos.conf:title TestOS 42 20190902.0 (ostree:1)
ostree-3-testos.conf:title TestOS 42 20190902.0 (ostree:0)

$ zipl
Using config file '/etc/zipl.conf'
Using BLS config file '/boot/loader/entries/ostree-3-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-2-testos.conf'
Using BLS config file '/boot/loader/entries/ostree-1-testos.conf'
Building bootmap in '/boot'
Building menu 'zipl-automatic-menu'
Adding #1: IPL section 'TestOS 42 20190902.0 (ostree:0)' (default)
Adding #2: IPL section 'TestOS 42 20190902.0 (ostree:1)'
Adding #3: IPL section 'TestOS 42 20190902.0 (ostree:2)'
Preparing boot device: dasda (0120).
Done.

Closes: #1911
Approved by: cgwalters
2019-09-02 21:08:58 +00:00
Colin Walters
ab73d9f525 admin/init-fs: Add a --modern switch
This skips creating the default stuff in the physical sysroot.
I don't recall why I did that to be honest; it originated with
the first commit of this file.  It might not have ever been
necessary.

In any case, it's not necessary now with Fedora CoreOS, so
prune it and let's have a clean `/`.

Keep the old behavior by default though to avoid breaking anyone.

Closes: #1894
Approved by: ajeddeloh
2019-08-08 20:02:11 +00:00
Matthew Leeds
dd6844a61e tests: Escape periods when appropriate
Don't match any character in a regular expression when we only want to
match a period.

Closes: #1834
Approved by: rfairley
2019-04-11 14:28:32 +00:00
Jonathan Lebon
978cffed70 lib/deploy: Don't include version twice in bootmenu title
In Silverblue right now, the boot menu title looks like this:

    Fedora 29.20190301.0 (Workstation Edition) 29.20190301.0 (ostree)

This is because RPM-OSTree's `mutate-os-release` feature is enabled,
which injects the OSTree version string directly into `VERSION` and
`PRETTY_NAME`. So appending the version string again is a bit redundant.
Let's just do a simple substring check here before adding the version to
the title.

Closes: #1829
Approved by: cgwalters
2019-03-09 19:44:15 +00:00
Javier Martinez Canillas
9f48e212a3 deploy: Change BootLoaderSpec filenames so they can be used for sorting
Currently the BLS snippets are named ostree-$ID-$VARIANT_ID-$index.conf,
but the BLS config files are actually sorted by using the version field
which is the inverse of the index.

In most places, _ostree_sysroot_read_boot_loader_configs() is used to
get the BLS files and this function already returns them sorted by the
version field. The only place where the index trailing number is used is
in the ostree-grub-generator script that lists the BLS files to populate
the grub config file.

But for some bootloaders the BLS filename is the criteria for sorting by
taking the filename as a string version. So on these bootloaders the BLS
entries will be listed in the reverse order.

To avoid that, change the BLS snippets filename to have the version field
instead of the index and also to have the version before deployment name.

Make the filenames to be of the form ostree-$version-$ID-$VARIANT_ID.conf
so the version is before the deployment name.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>

Closes: #1654
Approved by: cgwalters
2018-06-27 18:08:28 +00:00
Colin Walters
6873650cae lib/sysroot: Add OSTREE_EX_STAGE_DEPLOYMENTS environment variable
I feel like I'm drowning in a pile of experimental-but-almost-stable
features...

Anyways, since we made the feature opt-in in rpm-ostree in
https://github.com/projectatomic/rpm-ostree/pull/1352
let's mirror that a bit here with an environment variable so people
can play with it more easily.

The tests needed some tweaks; specifically we need to reload the
status fact after making changes.  I'm still a bit uncertain
about the Ansible-as-tests.

But we add an upgrade test that uses the new environment variable.

Closes: #1583
Approved by: jlebon
2018-05-11 12:39:17 +00:00
Colin Walters
25ba8db987 lib/deploy: Throw an error if trying to stage when not ostree-booted
There's no reason to do this.  I didn't actually hit this problem,
but it's a corner case that just occurred to me while working on
the code.

I think callers should be adapted to skip trying to use staging
if there's no booted deployment.

Closes: #1568
Approved by: jlebon
2018-05-02 13:27:01 +00:00
Colin Walters
7357d346c4 tests: Better error message if target is not a symlink
I broke the code for this and ended up adding this to debug it.

Closes: #1535
Approved by: jlebon
2018-04-11 19:11:07 +00:00
Marcus Folkesson
6bf4b3e1d8 Add SPDX-License-Identifier to source files
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.

Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
  text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
  environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
  license for their code is respected

See http://spdx.org for further reading.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1439
Approved by: cgwalters
2018-01-30 20:03:42 +00:00
Colin Walters
795a953a36 core: Add standard SOURCE_TITLE metadata key
This is a freeform string useful to track/display when a commit is "derived"
from some other format.  For example, in the rpm-ostree test we make a
`vmcheck` ref that conceptually overlays the default ref like
`fedora-atomic:fedora/26/x86_64/atomic-host`.

My current patch sets the source title to e.g.
"Dev overlay on fedora-atomic:fedora/26/x86_64/atomic-host".

Another case I'm working on now is importing OCI images to use
as host images.  For that case, the source title is
With this patch we could then set the original OCI image name + tag
as the source name, like:
"oci:cgwalters/demo-custom-fedora-atomic-host:26".

Closes: #1296
Approved by: jlebon
2017-10-23 14:19:41 +00:00
Dan Nicholson
adac42b6ef repo: Add add-remotes-config-dir option
This option allows a repo to explicitly opt out of adding new remotes in
a remotes configuration directory. This currently defaults to true for
system repos and false for non-system repos to maintain legacy behavior
that non-system repos don't add remotes in a configuration directory.
That would be problematic for flatpak, which specifies a remotes config
dir but adds remotes in ways that are incompatible with it.

So, what this really does is allow system repos to control whether they
want to add remotes in the config dir or not. That's important if your
flatpak repo is the system repo like at Endless.

Closes: #1134

Closes: #1155
Approved by: cgwalters
2017-09-11 10:53:20 +00:00
Colin Walters
3594bb2d0f lib: Add a private helper to abort txns, use in sysroot cleanup
Steal some code from flatpak for this, which allows porting a few more things to
new style. I started on a public API version of this but was trying to roll some
other things into it and it snowballed. Let's do this version since it's easy
for now.

While here I changed things so that `generate_deployment_refs()` now just uses
`_set_ref_immediate()` rather than requring a txn.

Also, AFAICS there was no test coverage of `generate_deployment_refs()`; I tried
commenting it out and at least `admin-test.sh` passed. Add some coverage of this
- I verified that with this commenting out bits of that function cause the test
to fail.

Closes: #1132
Approved by: jlebon
2017-09-08 16:25:06 +00:00
Jonathan Lebon
9342be6e34 ostree-sysroot: make simple_write_deployment smarter
This is a follow-up to https://github.com/ostreedev/ostree/pull/1097.
We make simple_write_deployment smart enough so that it can be used for
rpm-ostree's purposes. This is mostly an upstreaming of logic that
already existed there.

Notably we correctly append NOT_DEFAULT deployments *after* the booted
deployment and we now support RETAIN_PENDING and RETAIN_ROLLBACK flags
to have more granularity on deployment pruning.

Expose these new flags on the CLI using new options (as well as expose
the previously existing NOT_DEFAULT flag as --not-as-default).

I couldn't add tests for --retain-pending because the merge deployment
is always the topmost one. Though I did check that it worked in a VM.

Closes: #1110
Approved by: cgwalters
2017-08-25 01:02:15 +00:00
Colin Walters
8642ef5ab3 lib/deploy: Use a FIFREEZE/FITHAW cycle for /boot
See: http://marc.info/?l=linux-fsdevel&m=149520244919284&w=2

XFS doesn't flush the journal on `syncfs()`. GRUB doesn't know how to follow the
XFS journal, so if the filesystem is in a dirty state (possible with xfs
`/boot`, extremely likely with `/`, if the journaled data includes content for
`/boot`, the system may be unbootable if a system crash occurs.

Fix this by doing a `FIFREEZE`+`FITHAW` cycle.  Now, most people
probably would have replaced the `syncfs()` invocation with those two
ioctls.  But this would have become (I believe) the *only* place in
libostree where we weren't safe against interruption.  The failure
mode would be ugly; nothing else would be able to write to the filesystem
until manual intervention.

The real fix here I think is to land an atomic `FIFREEZETHAW` ioctl
in the kernel.  I might try a patch.

In the meantime though, let's jump through some hoops and set up
a "watchdog" child process that acts as a fallback unfreezer.

Closes: https://github.com/ostreedev/ostree/issues/876

Closes: #1049
Approved by: jlebon
2017-08-08 16:09:04 +00:00
Colin Walters
e0346c1494 Add a notion of "physical" sysroot, use for remote writing
(Note this PR was reverted in <https://github.com/ostreedev/ostree/pull/902>;
 this version should be better)

Using `${sysroot}` to mean the physical storage root: We don't want to write to
`${sysroot}/etc/ostree/remotes.d`, since nothing will read it, and really
`${sysroot}` should just have `/ostree` (ideally). Today the Anaconda rpmostree
code ends up writing there. Fix this by adding a notion of "physical" sysroot.
We determine whether the path is physical by checking for `/sysroot`, which
exists in deployment roots (and there shouldn't be a `${sysroot}/sysroot`).

In order to unit test this, I added a `--sysroot` argument to `remote add`.
However, doing this better would require reworking the command line parsing for
the `remote` argument to support specifying `--repo` or `--sysroot`, and I
didn't quite want to do that yet in this patch.

This second iteration of this patch fixes the bug we hit the first time;
embarassingly enough I broke `ostree remote list` finding system remotes.
The fix is to have `ostree_repo_open()` figure out whether it's the same
as `/ostree/repo` for now.

Down the line...we might consider having the `ostree remote` command line itself
instatiate an `OstreeSysroot` by default, but this maximizes compatibility; we
just have to pay a small cost that `ostree` usage outside of that case like
`ostree static-delta` in a releng Jenkins job or whatever will do this `stat()`
too.

Closes: https://github.com/ostreedev/ostree/issues/892

Closes: #1008
Approved by: mbarnes
2017-07-18 18:58:06 +00:00
Anton Gerasimov
64ab8334b7 lib/sysroot: Add API to get pending/rollback for given stateroot
This imports a function that is used in rpm-ostree, and it's also intended for
use by https://github.com/advancedtelematic/aktualizr to display
what deployment we're going to boot next after the reboot.

Updated-by: Colin Walters <walters@verbum.org>

Closes: #897
Approved by: OYTIS
2017-06-14 09:56:01 +00:00
Colin Walters
cad42d9601 Revert "Add a notion of "physical" sysroot, use for remote writing"
This reverts commit 1eff3e8343. There
are a few issues with it.  It's not a critical thing for now, so
let's ugly up the git history and revisit when we have time to
debug it and add more tests.

Besides the below issue, I noticed that the simple `ostree remote add`
now writes to `/ostree/repo/config` because we *aren't* using the
`--sysroot` argument.

Closes: https://github.com/ostreedev/ostree/issues/901

Closes: #902
Approved by: mike-nguyen
2017-06-02 14:11:58 +00:00
Colin Walters
1eff3e8343 Add a notion of "physical" sysroot, use for remote writing
Using `${sysroot}` to mean the physical storage root: We don't want to write to
`${sysroot}/etc/ostree/remotes.d`, since nothing will read it, and really
`${sysroot}` should just have `/ostree` (ideally). Today the Anaconda rpmostree
code ends up writing there. Fix this by adding a notion of "physical" sysroot.
We determine whether the path is physical by checking for `/sysroot`, which
exists in deployment roots (and there shouldn't be a `${sysroot}/sysroot`).

In order to unit test this, I added a `--sysroot` argument to `remote add`.
However, doing this better would require reworking the command line parsing for
the `remote` argument to support specifying `--repo` or `--sysroot`, and I
didn't quite want to do that yet in this patch.

Closes: https://github.com/ostreedev/ostree/issues/892

Closes: #896
Approved by: jlebon
2017-06-01 18:15:56 +00:00
Simon McVittie
9a3f82caae Sourced test snippets: remove shebang and make non-executable
They are installed non-executable, which makes Debian's Lintian
packaging consistency check complain that #! is only useful
in executable scripts. But in fact they are not useful to execute
directly (they rely on setup being done in the script that sources
them), so just chmod them -x.

Closes: #652
Approved by: cgwalters
2017-01-19 13:54:59 +00:00
Gatis Paeglis
b6ec7526b5 u-boot: Merge ostree's and systems uEnv.txt
This is a proper fix for:
https://bugzilla.gnome.org/show_bug.cgi?id=755787

With this patch, an admin (system builder) can now:

1) Edit /usr/lib/ostree-boot/uEnv.txt
2) Deploy the new tree. OSTree will append system's uEnv.txt
   to the OSTree's managed uEnv.txt (loader/uEnv.txt).

It is common for u-boot systems to read in an extra env
from external /uEnv.txt. The same file OSTree uses to pass
in its env. With this patch /uEnv.txt now contains OSTree's
env + custom env added by system builders.

Closes: #466
Approved by: cgwalters
2016-08-27 13:11:22 +00:00
Gatis Paeglis
4e81548447 Introducing ostree-grub-generator
ostree-grub-generator can be used to customize
the generated grub.cfg file. Compile time
decision ostree-grub-generator vs grub2-mkconfig
can be overwritten with the OSTREE_GRUB2_EXEC
envvar - useful for auto tests and OS installers.

Why this alternative approach:

1) The current approach is less flexible than using a
   custom 'ostree-grub-generator' script. Each system can
   adjust this script for its needs, instead of using the
   hardcoded values from ostree-bootloader-grub2.c.

2) Too much overhead on embedded to generate grub.cfg
   via /etc/grub.d/ configuration files. It is still
   possible to do so, even with this patch applied.
   No need to install grub2 package on a target device.

3) The grub2-mkconfig code path has other issues:
   https://bugzilla.gnome.org/show_bug.cgi?id=761180

Task: https://bugzilla.gnome.org/show_bug.cgi?id=762220

Closes: #228
Approved by: cgwalters
2016-04-04 14:19:35 +00:00
Simon McVittie
1dd85513e5 tests/admin-test.sh: this is a bash script, not a POSIX sh script
The "function foo()" syntax is bash-specific, and Colin indicated in
PR #226 that he prefers to require bash rather than trying to support
every POSIX shell.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #232
Approved by: cgwalters
2016-03-31 14:22:06 +00:00
Simon McVittie
35f412c86a tests/admin-test.sh: add #!/bin/sh
This is executable and appears to be intended to be executed directly,
so it should have a #! line.
2016-03-21 13:02:25 -04:00
Colin Walters
d49718a619 deploy: Bump the mtime on ostree/deploy after deployments finish
This allows other processes (e.g. rpm-ostreed) to monitor for external
changes (e.g. if someone does `ostree admin undeploy`) in a relatively
sane fashion.

Specifically, I'm trying to fix:
https://github.com/projectatomic/rpm-ostree/issues/220
2016-03-03 08:52:42 -05:00
Colin Walters
d25212f04a tests: Port to glib-tap.mk, make make check run all of the tests
OSTree's code for testing predates the `glib-tap.mk` making its
way into GLib.  Let's switch to it, as it provides a number
of advantages.

By far the biggest advantage is that `make check` can start to run
most of the tests *in addition* to having them work installed.

This commit keeps the installed tests working, but `make check` turns
out to be really broken because...our TAP usage has bitrotted to say
the least.  Fix that all up.

Do some hacks so that the tests work uninstalled as well - in
particular, `glib-tap.mk` and the bits encoded into
`g_test_build_filename()` assume *recursive* Automake (blah).  Work
around that by creating a symlink when installed to loop back.
2016-03-03 08:50:19 -05:00
Colin Walters
ca57ec4aa5 repo: Port -refs.c to openat()
I'd like to incrementally convert all of `ostree-repo*.c` to
fd-relative usage, so that we can sanely introduce
`ostree_repo_new_at()` which doesn't involve GFile.

This one is medium risk, but passes the test suite.
2016-01-28 14:57:13 -05:00
Colin Walters
5ebe43859d tests: Use "bash strict mode"
I noticed in the static deltas tests, there were some tests that
should have been under `-o pipefail` to ensure we properly propagate
errors.

There were a few places where we were referencing undefined variables.

Overall, this is clearly a good idea IMO.
2016-01-27 11:44:10 -05:00
Matthew Barnes
2210a79bfa upgrade: Add --override-commit=CHECKSUM option
Adds an entry to the origin file to force the OstreeSysrootUpgrader to
pull and deploy the given checksum.

  [origin]
  override-commit=CHECKSUM

If the option is not given, any such entry is explicitly removed from
the origin file to ensure we upgrade to the latest available commit.
2015-10-13 12:34:57 -04:00
Matthew Barnes
df0cae4375 tests: Export OSTREE_SYSROOT in setup_os_repository
Eliminates the need for constantly passing --sysroot=sysroot, but
also makes ostree place remote configs for sysroot/ostree/repo in
sysroot/etc/ostree/remotes.d where they should have been all along.
2015-07-16 12:49:34 -04:00
Colin Walters
20bf7692a9 tests: Add a crosscheck for syslinux bootloader config generation
And actually wire this one up in admin-test.sh.
2015-05-28 14:21:30 -04:00
Giuseppe Scrivano
aea173b3f8 tests: enforce ${CMD_PREFIX} on all ostree processes
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 18:32:24 -05:00
Colin Walters
6bbfa5f85a admin: Add set-origin command
See projectatomic/rpm-ostree#42 for rationale. There are two high
level use cases:

 - If the OS comes unconfigured, this is a way to point it at a repo of your choice.
 - To switch between repositories while keeping the same branch easily.
2015-01-19 13:55:20 -05:00
Colin Walters
038ba3dade tests: Fix u-boot test failure
This bit needs to be specific to syslinux.
2014-11-25 15:21:12 -05:00
Colin Walters
97519b6c22 admin-test: Fix two typos in bootloader UI change
Based on a patch from Giuseppe Scrivano <gscrivan@redhat.com>
2014-11-06 18:22:20 -05:00
Matthew Barnes
e5b147c643 libostree: Reformat bootloader titles
Use the pattern:

  $PRETTY_NAME [$COMMIT_VERSION] (ostree[:$OSNAME][:$DEPLOYMENT_INDEX])

$OSNAME is only shown if there are multiple values.

$COMMIT_VERSION refers to the version tag in the commit's metadata.

$DEPLOYMENT_INDEX is only shown if no $COMMIT_VERSION is available.

https://bugzilla.gnome.org/show_bug.cgi?id=739416
2014-11-03 20:43:36 -05:00
Colin Walters
b43ce2329e tests: Add some versioning metadata and test it appears in status output
Followup to previous commits.
2014-10-23 11:24:44 -04:00
Owen W. Taylor
cc180f5d38 Test 'ostree admin --print-current-dir'
Add a test for the --print-current-dir option

https://bugzilla.gnome.org/show_bug.cgi?id=731051
2014-09-26 11:55:11 -04:00
Colin Walters
82fb07e4ae deploy: Also honor multiply-specified kernel args with existing config
If we deployed a new tree inside an existing OS, inheriting kernel
args, we need to use append() instead of replace() to avoid collapsing
multiply specified kernel arguments like console=/dev/foo
console=/dev/bar.

Reported-by: Dusty Mabe <dusty@dustymabe.com>
2014-08-03 14:34:28 -04:00
Colin Walters
5bf6099a8b Unify uboot and syslinux test cases
The test-admin-deploy-1 was copied into -uboot at some point.  But
really they should be testing exactly the same thing, except for
the bootloader backend.

Unify these tests by extracting a common test core.
2014-01-15 09:48:04 -05:00