Commit Graph

4699 Commits

Author SHA1 Message Date
Colin Walters
40d6f6b5ee tests: Add a pre-signed-pull.sh test
I'm thinking about adding an implementation of ed25519 signatures
with OpenSSL (so we can ship the feature with Fedora CoreOS
without requiring an additional library) and in preparation for
that it's essential that we validate that libsodium-generated
signatures and OpenSSL-generated signatures are compatible.

I don't know if they are yet actually, but the goal of this
new test is to add a pre-generated repository with a signed
commit generated by libsodium.

This will catch if e.g. there's ever a change in libsodium,
or if existing libsodium implementation versions (e.g. the
one in Debian) might differ from what we ship here.
2020-06-11 18:56:35 +00:00
OpenShift Merge Robot
a128eb551a
Merge pull request #2094 from zpiotr/patch-1
Changing link to file about contributing, in readme.
2020-06-10 14:57:14 -04:00
NEPO
b82c296198 README.md: Fix link to CONTRIBUTING.md
We should link to the target and not the symlink.
2020-06-10 11:31:16 -07:00
OpenShift Merge Robot
bd9b4ea731
Merge pull request #2121 from cgwalters/arch-key
core: Add OSTREE_COMMIT_META_KEY_ARCH
2020-06-09 02:58:59 +02:00
OpenShift Merge Robot
be8dcd3df2
Merge pull request #2126 from agners/improve-ostree-checkout-man
Improve checkout man page
2020-06-08 12:50:43 -04:00
Stefan Agner
38aa912a0e man/checkout: document missing options
Document missing options in the ostree checkout man page.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-06-08 18:23:10 +02:00
Stefan Agner
832ca09891 checkout: use FILE as option argument string for --skip-list
Align with --from-file and use 'FILE' instead of 'PATH' as option
argument string. No functional change, this is only cosmetics.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-06-08 15:43:32 +02:00
Stefan Agner
843482e589 man/checkout: fix short name option of --user-mode
The short name option of --user-mode is -U.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-06-08 15:43:26 +02:00
Colin Walters
29dddf38d3 core: Add OSTREE_COMMIT_META_KEY_ARCHITECTURE
Add a standard key for this.  We actually had a case in OpenShift
builds recently where a `ppc64le` image was pushed over an `x86_64`
one and this started failing at runtime with a not immediately
obvious error.

I'll probably end up changing rpm-ostree at least to use
the RPM architecture for this key and fail if it doesn't match
the booted value.

Possibly that should live in ostree but it would involve adding
architecture schema here, which gets into a big mess.  Let's
just standardize the key.

xref e02ef2683d
2020-06-05 17:52:42 +00:00
OpenShift Merge Robot
e36d52fe55
Merge pull request #2123 from cgwalters/all-your-base-have-arguments
commit: Note in help that --base takes an argument
2020-06-04 17:34:52 -04:00
Colin Walters
b137895022 commit: Note in help that --base takes an argument
I was trying to use this in some testing work and was confused for a minute.
2020-06-04 21:06:52 +00:00
OpenShift Merge Robot
d9fc1dd55d
Merge pull request #2122 from cgwalters/testrs-webserver
tests/rust: Extract a with_webserver_in helper wrapper
2020-06-04 16:18:16 +02:00
Colin Walters
1cd902cd1a tests/rust: Extract a with_webserver_in helper wrapper
It's much cleaner if the Tokio stuff stays in `test.rs`, and
easier to write tests if the function is synchronous.

Prep for further tests.
2020-06-04 13:04:35 +00:00
OpenShift Merge Robot
25986126c7
Merge pull request #2048 from cgwalters/rust-cmdspec-tests
Add new Rust-based tests
2020-06-02 20:31:30 +02:00
OpenShift Merge Robot
12937211e1
Merge pull request #2119 from cgwalters/bumpsplit-rustfmt
bupsplit: rustfmt(*)
2020-06-02 18:22:48 +02:00
Colin Walters
97cda7ff4d bupsplit: rustfmt(*)
Let's use the standard rustfmt style.
Also remove unused parenthesis which rust-analyzer was complaining
about.

Also add a `.gitignore`.
2020-06-01 13:53:55 +00:00
OpenShift Merge Robot
d6ce1b09cf
Merge pull request #2118 from cgwalters/error-prefix-parsing
lib: Add error prefixing with specific object when loading
2020-05-29 18:52:23 -04:00
Colin Walters
167edbe63a pull: Add error prefixing with specific object when parsing
One OpenShift user saw this from rpm-ostree:
```
client(id:cli dbus:1.583 unit:machine-config-daemon-host.service uid:0) added; new total=1
Initiated txn UpdateDeployment for client(id:cli dbus:1.583 unit:machine-config-daemon-host.service uid:0): /org/projectatomic/rpmostree1/rhcos
Txn UpdateDeployment on /org/projectatomic/rpmostree1/rhcos failed: File header size 4294967295 exceeds size 0
```

which isn't very helpful.  Let's add some error
prefixing here which would at least tell us which
object was corrupted.
2020-05-29 13:12:02 +00:00
OpenShift Merge Robot
c6eade5ce5
Merge pull request #2117 from cgwalters/pull-signapi-default-explicit
remote-add: Default to explicit sign-verify backends
2020-05-28 13:52:48 -04:00
Colin Walters
68ebf743cd remote-add: Default to explicit sign-verify backends
In 588f42e8c6
we added a way to add keys for sign types when doing
a `remote add`, and in https://github.com/ostreedev/ostree/pull/2105
we extended `sign-verify` to support *limiting* to an explicit
set.

This PR changes the *default* for `remote add` to combine
the two - when providing an explicit `--sign-verify=type`,
we now limit the accepted types to only those.
2020-05-28 00:41:34 +00:00
Colin Walters
1f637bf341 Add new Rust-based tests
There's a lot going on here.  First, this is intended to run
nicely as part of the new [cosa/kola ext-tests](https://github.com/coreos/coreos-assembler/pull/1252).

With Rust we can get one big static binary that we can upload,
and include a webserver as part of the binary.  This way we don't
need to do the hack of running a container with Python or whatever.

Now, what's even better about Rust for this is that it has macros,
and specifically we are using [commandspec](https://github.com/tcr/commandspec/)
which allows us to "inline" shell script.  I think the macros
could be even better, but this shows how we can intermix
pure Rust code along with using shell safely enough.

We're using my fork of commandspec because the upstream hasn't
merged [a few PRs](https://github.com/tcr/commandspec/pulls?q=is%3Apr+author%3Acgwalters+).

This model is intended to replace *both* some of our
`make check` tests as well.

Oh, and this takes the obvious step of using the Rust OSTree bindings
as part of our tests.  Currently the "commandspec tests" and "API tests"
are separate, but nothing stops us from intermixing them if we wanted.

I haven't yet tried to write destructive tests with this but
I think it will go well.
2020-05-27 21:59:23 +00:00
OpenShift Merge Robot
854099802f
Merge pull request #2116 from cgwalters/kolainst
tests/kola: Move to tests/kolainst
2020-05-27 11:43:22 -04:00
Colin Walters
718cca8055 tests/kola: Move to tests/kolainst
Follow the precedent set in https://github.com/coreos/rpm-ostree/pull/2106
and rename the directory, to more clearly move away from the
"uninstalled" test model.  Prep for Rust-based tests.
2020-05-27 15:16:50 +00:00
OpenShift Merge Robot
936301608a
Merge pull request #2113 from cgwalters/prepare-root-sysroot-ro
Move ro /sysroot bind mount of /etc into initramfs
2020-05-26 11:38:42 -04:00
OpenShift Merge Robot
8e025978e1
Merge pull request #2105 from cgwalters/pull-signapi-explicit
pull: Add support for sign-verify=<list>
2020-05-24 14:53:46 -04:00
Colin Walters
3564225917 Move ro /sysroot bind mount of /etc into initramfs
We recently disabled the read-only /sysroot handling:
e35b82fb89

The core problem was that a lot of services run early in the
real root and want write access to things like `/var` and `/etc`.

In trying to do remounts while the system is running we introduce
too many race conditions.

Instead, just make the `/etc` bind mount in the initramfs right
after we set up the main root.  This is much more natural really,
and avoids all race conditions since nothing is running in the
sysroot yet.

The main awkward part is that since we're not linking
`ostree-prepare-root` to GLib (yet) we have a hacky parser
for the config file.  But, this is going to be fine I think.

In order to avoid parsing the config twice, pass state from
`ostree-prepare-root` to `ostree-remount` via a file in `/run`.
2020-05-24 18:46:28 +00:00
Colin Walters
5cb9d0df38 pull: Add support for sign-verify=<list>
The goal here is to move the code towards a model
where the *client* can explicitly specify which signature types
are acceptable.

We retain support for `sign-verify=true` for backwards compatibility.
But in that configuration, a missing public key is just "no signatures found".

With `sign-verify=ed25519` and no key configured, we can
explicitly say `No keys found for required signapi type ed25519`
which is much, much clearer.

Implementation side, rather than maintaining `gboolean sign_verify` *and*
`GPtrArray sign_verifiers`, just have the array.  If it's `NULL` that means
not to verify.

Note that currently, an explicit list is an OR of signatures, not AND.
In practice...I think most people are going to be using a single entry
anyways.
2020-05-22 19:10:32 +00:00
OpenShift Merge Robot
8801e38bba
Merge pull request #2110 from jlebon/pr/fix-admin-tests2
tests/admin-test: Fix --allow-downgrade check
2020-05-22 20:27:43 +02:00
Jonathan Lebon
6730acc350 tests/admin-test: Fix --allow-downgrade check
We were doing a check to verify that `ostree admin upgrade` wouldn't
accept a downgrade without `--allow-downgrade`. However, there's no
guarantee that the commit it's upgrading from is older than HEAD^ (what
we're upgrading to). Specifically, if the test runs fast enough, the
timestamps could be equal, since the lowest resolution is seconds.

Rework the test so that we first upgrade to HEAD, which we're sure is at
least 1 second apart from HEAD^, and *then* check that downgrade
protection is enforced.

We also can't use `rev-parse testos/buildmaster/x86_64-runtime` as a way
to know what commit the host is sitting on since the ref might've gone
ahead. Instead, just use `ostree admin status | head -n1`. (I played
with using the `ostree/I/J/K` refs, but those depend on what the
boot/subbootversion is and can easily change if we change previous
tests).
2020-05-22 13:59:36 -04:00
OpenShift Merge Robot
6be014daa8
Merge pull request #2111 from strugee/patch-1
Fix typo
2020-05-22 16:36:21 +02:00
AJ Jordan
0487b498ad
Fix typo 2020-05-21 23:22:44 -07:00
OpenShift Merge Robot
0c8701b896
Merge pull request #2108 from jlebon/pr/back-out-ro-sysroot
switchroot/remount: Neuter sysroot.readonly for now
2020-05-21 17:58:04 +02:00
Jonathan Lebon
e35b82fb89 switchroot/remount: Neuter sysroot.readonly for now
We're hitting issues with the read-only remounts racing with various
services coming up. Let's neuter it for now until we rework how it
works.

See: https://github.com/coreos/fedora-coreos-tracker/issues/488
2020-05-20 16:23:59 -04:00
OpenShift Merge Robot
be62a01bff
Merge pull request #2106 from jlebon/pr/fix-admin-test
tests/admin-test: Ensure that commits are 1s apart
2020-05-20 19:05:35 +02:00
OpenShift Merge Robot
6586d240c2
Merge pull request #2107 from cgwalters/more-commit-clocking
admin-test: Show err.txt on unexpected failure
2020-05-20 18:33:51 +02:00
Colin Walters
191ce95ca1 admin-test: Show err.txt on unexpected failure
In a CI run I think one of these `ostree show` commands is failing.
While that output would be useful, the actual `err.txt` usually
has what we want too.
2020-05-20 15:57:10 +00:00
Jonathan Lebon
70ebb91aa3 tests/admin-test: Ensure that commits are 1s apart
Otherwise the new check we added there to verify that upgrading without
`--allow-downgrade` fails itself fails.

See: https://github.com/ostreedev/ostree/pull/2099#issuecomment-629805840
2020-05-20 10:48:44 -04:00
OpenShift Merge Robot
5a82d75adb
Merge pull request #2102 from cgwalters/test-clockskew-check
ci: Test for clock skew
2020-05-18 21:39:30 +02:00
Colin Walters
f14aa894d3 ci: Test for clock skew
I saw `tests/test-admin-deploy.none.sh` fail in one CI run, and
I want to check if it was because of clock skew, so fail
fast if we detect that.

xref https://github.com/ostreedev/ostree/pull/2099#issuecomment-629805375
2020-05-18 18:48:28 +00:00
OpenShift Merge Robot
0d7132ec79
Merge pull request #2101 from cgwalters/signapi-no-explicit-on
pull: Only have API to disable signapi for local pulls
2020-05-18 00:06:04 -07:00
Colin Walters
5a47c926c1 pull: Only have API to disable signapi for local pulls
There's a lot of historical baggage associated with GPG verification
and `ostree pull` versus `ostree pull-local`.  In particular nowadays,
if you use a `file://` remote things are transparently optimized
to e.g. use reflinks if available.

So for anyone who doesn't trust the "remote" repository, you should
really go through through the regular
`ostree remote add --sign-verify=X file://`
path for example.

Having a mechanism to say "turn on signapi verification" *without*
providing keys goes back into the "global state" debate I brought
up in https://github.com/ostreedev/ostree/issues/2080

It's just much cleaner architecturally if there is exactly one
path to find keys: from a remote config.

So here in contrast to the GPG code, for `pull-local` we explictily
disable signapi validation, and the `ostree_repo_pull()` API just
surfaces flags to disable it, not enable it.
2020-05-17 13:52:24 +00:00
OpenShift Merge Robot
b5e51d8c08
Merge pull request #2100 from cgwalters/make-install-kola-tests
ci: Install kola tests
2020-05-15 14:38:00 -07:00
Colin Walters
48f5a1885e ci: Install kola tests
This builds on
https://github.com/coreos/coreos-assembler/pull/1441
to install our tests rather than running them from the source
directory.  This model will more cleanly allow us to ship
our tests along with a test container or elsewhere, separate
from the source directory.

Also prep for https://github.com/ostreedev/ostree/pull/2048
2020-05-15 19:40:54 +00:00
OpenShift Merge Robot
11b09ec3cf
Merge pull request #2099 from jlebon/pr/timestamp-check-from-rev
lib/pull: Add `timestamp-check-from-rev`
2020-05-15 08:16:38 -07:00
Jonathan Lebon
79079c2657 lib/upgrader: Pull with timestamp-check-from-rev
For the same reason as https://github.com/coreos/rpm-ostree/pull/2094.
What we care most about is that the new commit we pull is newer than the
one we're currently sitting on, not necessarily that it's newer than the
branch itself, which it might not be if e.g. we're trying to deploy a
commit older than the tip but still newer than the deployment (via
`--override-commit`).
2020-05-14 15:27:29 -04:00
Jonathan Lebon
c8efce0656 lib/pull: Add timestamp-check-from-rev
The way `timestamp-check` works might be too restrictive in some
situations. Essentially, we need to support the case where users want to
pull an older commit than the current tip, but while still guaranteeing
that it is newer than some even older commit.

This will be used in Fedora CoreOS. For more information see:
https://github.com/coreos/rpm-ostree/pull/2094
https://github.com/coreos/fedora-coreos-tracker/issues/481
2020-05-14 14:00:42 -04:00
OpenShift Merge Robot
f145d18369
Merge pull request #2098 from cgwalters/finalize-requires-mounts
finalize: Add RequiresMountsFor=/boot too
2020-05-13 12:21:14 -07:00
Colin Walters
82679ce834 finalize: Add RequiresMountsFor=/boot too
In https://bugzilla.redhat.com/show_bug.cgi?id=1827712
some OpenShift CI is seeing `/boot` being unmounted before
`ostree-finalize-staged.service` runs or completes.

We finally tracked this down to a bug elsewhere, but
I think we should add this because it clearly shows
our requirements.
2020-05-13 18:52:45 +00:00
OpenShift Merge Robot
4b8354d478
Merge pull request #2097 from cgwalters/sign-verifier
pull: Further cleanup signapi verification
2020-05-12 11:38:58 -07:00
OpenShift Merge Robot
371955549e
Merge pull request #2096 from cgwalters/test-staged-delay
tests/staged-delay.sh: New test
2020-05-12 10:41:29 -07:00