Commit Graph

4765 Commits

Author SHA1 Message Date
OpenShift Merge Robot
2aa5ec7c51
Merge pull request #2135 from mwleeds/test-symbols-check-for-example
tests: Check that example symbol isn't released
2020-06-18 09:49:43 -04:00
Matthew Leeds
d21181653e tests: Check that example symbol isn't released
For the motivation for this see #2132.
2020-06-17 15:28:15 -07:00
OpenShift Merge Robot
33c95812bb
Merge pull request #2132 from cgwalters/remove-unused-symbol
libostree-devel.sym: Remove nonexistent stub symbol
2020-06-17 16:18:26 -04:00
Jonathan Lebon
516c1340b3 ci: Remove libpaprci/ directory
And move everything that was in it directly in `ci/`. There's a bunch
more cleanups here that we need to do (and more changes to upstream from
the rpm-ostree copies of this).
2020-06-17 15:48:31 -04:00
OpenShift Merge Robot
1f7f5a1db0
Merge pull request #2134 from d4s/wip/d4s/fix_abort_on_verify
Fix abort on verify
2020-06-17 15:44:27 -04:00
Colin Walters
9336837194 libostree-devel.sym: Remove nonexistent stub symbol
This should have been removed when we added symbols to this list.
2020-06-17 19:38:26 +00:00
Denis Pynkin
ede93dc2ef sign/ed25519: fix return value if no correct keys in file
Fix the return value if file doesn't contains correct public key(s).

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-06-17 19:45:05 +03:00
Denis Pynkin
ce73876389 sign/ed25519: fix the abort in case of incorrect public key
We need to check the size of public key before trying to use it.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-06-17 19:44:56 +03:00
Jonathan Lebon
eb3fe35b06 ci: Import latest ci-commitmessage-submodules from rpm-ostree
Especially for https://github.com/coreos/rpm-ostree/pull/2079.
2020-06-17 10:39:51 -04:00
Colin Walters
36258036ae signapi: Change API to also return a success message
This is the dual of 1f3c8c5b3d
where we output more detail when signapi fails to validate.

Extend the API to return a string for success, which we output
to stdout.

This will help the test suite *and* end users validate that the expected
thing is happening.

In order to make this cleaner, split the "verified commit" set
in the pull code into GPG and signapi verified sets, and have
the signapi verified set contain the verification string.

We're not doing anything with the verification string in the
pull code *yet* but I plan to add something like
`ostree pull --verbose` which would finally print this.
2020-06-17 00:33:47 +00:00
OpenShift Merge Robot
b2dde24f00
Merge pull request #2130 from jlebon/pr/bump-libglnx
libglnx: Bump to latest
2020-06-16 22:59:55 +02:00
Jonathan Lebon
b3694b55ac libglnx: Bump to latest
For `copy_file_range` fix:

https://gitlab.gnome.org/GNOME/libglnx/-/merge_requests/18

Update submodule: libglnx
2020-06-16 16:31:22 -04:00
OpenShift Merge Robot
4b32cc5195
Merge pull request #2129 from cgwalters/ed25519-errors
sign/ed25519: Output failed signatures in error message
2020-06-16 18:15:21 +02:00
Colin Walters
1f3c8c5b3d sign/ed25519: Output failed signatures in error message
To aid debuggability, when we find a commit that isn't signed
by our expected key, output a specific error message with the
key.

(And then add code to switch to just printing the count beyond 3
 because the test suite injects 100 keys and hopefully no one
 ever actually does that)
2020-06-16 18:20:54 +03:00
OpenShift Merge Robot
fa70ab417b
Merge pull request #2128 from cgwalters/verify-pre-signed
tests: Add a pre-signed-pull.sh test
2020-06-15 15:29:23 -04:00
Colin Walters
40d6f6b5ee tests: Add a pre-signed-pull.sh test
I'm thinking about adding an implementation of ed25519 signatures
with OpenSSL (so we can ship the feature with Fedora CoreOS
without requiring an additional library) and in preparation for
that it's essential that we validate that libsodium-generated
signatures and OpenSSL-generated signatures are compatible.

I don't know if they are yet actually, but the goal of this
new test is to add a pre-generated repository with a signed
commit generated by libsodium.

This will catch if e.g. there's ever a change in libsodium,
or if existing libsodium implementation versions (e.g. the
one in Debian) might differ from what we ship here.
2020-06-11 18:56:35 +00:00
OpenShift Merge Robot
a128eb551a
Merge pull request #2094 from zpiotr/patch-1
Changing link to file about contributing, in readme.
2020-06-10 14:57:14 -04:00
NEPO
b82c296198 README.md: Fix link to CONTRIBUTING.md
We should link to the target and not the symlink.
2020-06-10 11:31:16 -07:00
OpenShift Merge Robot
bd9b4ea731
Merge pull request #2121 from cgwalters/arch-key
core: Add OSTREE_COMMIT_META_KEY_ARCH
2020-06-09 02:58:59 +02:00
OpenShift Merge Robot
be8dcd3df2
Merge pull request #2126 from agners/improve-ostree-checkout-man
Improve checkout man page
2020-06-08 12:50:43 -04:00
Stefan Agner
38aa912a0e man/checkout: document missing options
Document missing options in the ostree checkout man page.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-06-08 18:23:10 +02:00
Stefan Agner
832ca09891 checkout: use FILE as option argument string for --skip-list
Align with --from-file and use 'FILE' instead of 'PATH' as option
argument string. No functional change, this is only cosmetics.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-06-08 15:43:32 +02:00
Stefan Agner
843482e589 man/checkout: fix short name option of --user-mode
The short name option of --user-mode is -U.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-06-08 15:43:26 +02:00
Colin Walters
29dddf38d3 core: Add OSTREE_COMMIT_META_KEY_ARCHITECTURE
Add a standard key for this.  We actually had a case in OpenShift
builds recently where a `ppc64le` image was pushed over an `x86_64`
one and this started failing at runtime with a not immediately
obvious error.

I'll probably end up changing rpm-ostree at least to use
the RPM architecture for this key and fail if it doesn't match
the booted value.

Possibly that should live in ostree but it would involve adding
architecture schema here, which gets into a big mess.  Let's
just standardize the key.

xref e02ef2683d
2020-06-05 17:52:42 +00:00
OpenShift Merge Robot
e36d52fe55
Merge pull request #2123 from cgwalters/all-your-base-have-arguments
commit: Note in help that --base takes an argument
2020-06-04 17:34:52 -04:00
Colin Walters
b137895022 commit: Note in help that --base takes an argument
I was trying to use this in some testing work and was confused for a minute.
2020-06-04 21:06:52 +00:00
OpenShift Merge Robot
d9fc1dd55d
Merge pull request #2122 from cgwalters/testrs-webserver
tests/rust: Extract a with_webserver_in helper wrapper
2020-06-04 16:18:16 +02:00
Colin Walters
1cd902cd1a tests/rust: Extract a with_webserver_in helper wrapper
It's much cleaner if the Tokio stuff stays in `test.rs`, and
easier to write tests if the function is synchronous.

Prep for further tests.
2020-06-04 13:04:35 +00:00
OpenShift Merge Robot
25986126c7
Merge pull request #2048 from cgwalters/rust-cmdspec-tests
Add new Rust-based tests
2020-06-02 20:31:30 +02:00
OpenShift Merge Robot
12937211e1
Merge pull request #2119 from cgwalters/bumpsplit-rustfmt
bupsplit: rustfmt(*)
2020-06-02 18:22:48 +02:00
Colin Walters
97cda7ff4d bupsplit: rustfmt(*)
Let's use the standard rustfmt style.
Also remove unused parenthesis which rust-analyzer was complaining
about.

Also add a `.gitignore`.
2020-06-01 13:53:55 +00:00
OpenShift Merge Robot
d6ce1b09cf
Merge pull request #2118 from cgwalters/error-prefix-parsing
lib: Add error prefixing with specific object when loading
2020-05-29 18:52:23 -04:00
Stefan Agner
5f08649f51 deploy: support devicetree directory
Add support for a devicetree directory at /usr/lib/modules/$kver/dtb/.
In ARM world a general purpose distribution often suppports multiple
boards with a single operating system. However, OSTree currently only
supports a single device tree, which does not allow to use the same
OSTree on different ARM machines. In this scenario typically the boot
loader selects the effective device tree.

This adds device tree directory support for the new boot artefact
location under /usr/lib/modules. If the file `devicetree` does not
exist, then the folder dtb will be checked. All devicetrees are hashed
into the deployment hash. This makes sure that even a single devicetree
change leads to a new deployment and hence can be rolled back.

The loader configuration has a new key "devicetreepath" which contains
the path where devicetrees are stored. This is also written to the
U-Boot variable "fdtdir". The boot loader is expected to use this path
to load a particular machines device tree from.

Closes: #1900
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-05-29 19:56:11 +02:00
Colin Walters
167edbe63a pull: Add error prefixing with specific object when parsing
One OpenShift user saw this from rpm-ostree:
```
client(id:cli dbus:1.583 unit:machine-config-daemon-host.service uid:0) added; new total=1
Initiated txn UpdateDeployment for client(id:cli dbus:1.583 unit:machine-config-daemon-host.service uid:0): /org/projectatomic/rpmostree1/rhcos
Txn UpdateDeployment on /org/projectatomic/rpmostree1/rhcos failed: File header size 4294967295 exceeds size 0
```

which isn't very helpful.  Let's add some error
prefixing here which would at least tell us which
object was corrupted.
2020-05-29 13:12:02 +00:00
OpenShift Merge Robot
c6eade5ce5
Merge pull request #2117 from cgwalters/pull-signapi-default-explicit
remote-add: Default to explicit sign-verify backends
2020-05-28 13:52:48 -04:00
Colin Walters
68ebf743cd remote-add: Default to explicit sign-verify backends
In 588f42e8c6
we added a way to add keys for sign types when doing
a `remote add`, and in https://github.com/ostreedev/ostree/pull/2105
we extended `sign-verify` to support *limiting* to an explicit
set.

This PR changes the *default* for `remote add` to combine
the two - when providing an explicit `--sign-verify=type`,
we now limit the accepted types to only those.
2020-05-28 00:41:34 +00:00
Colin Walters
1f637bf341 Add new Rust-based tests
There's a lot going on here.  First, this is intended to run
nicely as part of the new [cosa/kola ext-tests](https://github.com/coreos/coreos-assembler/pull/1252).

With Rust we can get one big static binary that we can upload,
and include a webserver as part of the binary.  This way we don't
need to do the hack of running a container with Python or whatever.

Now, what's even better about Rust for this is that it has macros,
and specifically we are using [commandspec](https://github.com/tcr/commandspec/)
which allows us to "inline" shell script.  I think the macros
could be even better, but this shows how we can intermix
pure Rust code along with using shell safely enough.

We're using my fork of commandspec because the upstream hasn't
merged [a few PRs](https://github.com/tcr/commandspec/pulls?q=is%3Apr+author%3Acgwalters+).

This model is intended to replace *both* some of our
`make check` tests as well.

Oh, and this takes the obvious step of using the Rust OSTree bindings
as part of our tests.  Currently the "commandspec tests" and "API tests"
are separate, but nothing stops us from intermixing them if we wanted.

I haven't yet tried to write destructive tests with this but
I think it will go well.
2020-05-27 21:59:23 +00:00
OpenShift Merge Robot
854099802f
Merge pull request #2116 from cgwalters/kolainst
tests/kola: Move to tests/kolainst
2020-05-27 11:43:22 -04:00
Colin Walters
718cca8055 tests/kola: Move to tests/kolainst
Follow the precedent set in https://github.com/coreos/rpm-ostree/pull/2106
and rename the directory, to more clearly move away from the
"uninstalled" test model.  Prep for Rust-based tests.
2020-05-27 15:16:50 +00:00
OpenShift Merge Robot
936301608a
Merge pull request #2113 from cgwalters/prepare-root-sysroot-ro
Move ro /sysroot bind mount of /etc into initramfs
2020-05-26 11:38:42 -04:00
OpenShift Merge Robot
8e025978e1
Merge pull request #2105 from cgwalters/pull-signapi-explicit
pull: Add support for sign-verify=<list>
2020-05-24 14:53:46 -04:00
Colin Walters
3564225917 Move ro /sysroot bind mount of /etc into initramfs
We recently disabled the read-only /sysroot handling:
e35b82fb89

The core problem was that a lot of services run early in the
real root and want write access to things like `/var` and `/etc`.

In trying to do remounts while the system is running we introduce
too many race conditions.

Instead, just make the `/etc` bind mount in the initramfs right
after we set up the main root.  This is much more natural really,
and avoids all race conditions since nothing is running in the
sysroot yet.

The main awkward part is that since we're not linking
`ostree-prepare-root` to GLib (yet) we have a hacky parser
for the config file.  But, this is going to be fine I think.

In order to avoid parsing the config twice, pass state from
`ostree-prepare-root` to `ostree-remount` via a file in `/run`.
2020-05-24 18:46:28 +00:00
Colin Walters
5cb9d0df38 pull: Add support for sign-verify=<list>
The goal here is to move the code towards a model
where the *client* can explicitly specify which signature types
are acceptable.

We retain support for `sign-verify=true` for backwards compatibility.
But in that configuration, a missing public key is just "no signatures found".

With `sign-verify=ed25519` and no key configured, we can
explicitly say `No keys found for required signapi type ed25519`
which is much, much clearer.

Implementation side, rather than maintaining `gboolean sign_verify` *and*
`GPtrArray sign_verifiers`, just have the array.  If it's `NULL` that means
not to verify.

Note that currently, an explicit list is an OR of signatures, not AND.
In practice...I think most people are going to be using a single entry
anyways.
2020-05-22 19:10:32 +00:00
OpenShift Merge Robot
8801e38bba
Merge pull request #2110 from jlebon/pr/fix-admin-tests2
tests/admin-test: Fix --allow-downgrade check
2020-05-22 20:27:43 +02:00
Jonathan Lebon
6730acc350 tests/admin-test: Fix --allow-downgrade check
We were doing a check to verify that `ostree admin upgrade` wouldn't
accept a downgrade without `--allow-downgrade`. However, there's no
guarantee that the commit it's upgrading from is older than HEAD^ (what
we're upgrading to). Specifically, if the test runs fast enough, the
timestamps could be equal, since the lowest resolution is seconds.

Rework the test so that we first upgrade to HEAD, which we're sure is at
least 1 second apart from HEAD^, and *then* check that downgrade
protection is enforced.

We also can't use `rev-parse testos/buildmaster/x86_64-runtime` as a way
to know what commit the host is sitting on since the ref might've gone
ahead. Instead, just use `ostree admin status | head -n1`. (I played
with using the `ostree/I/J/K` refs, but those depend on what the
boot/subbootversion is and can easily change if we change previous
tests).
2020-05-22 13:59:36 -04:00
OpenShift Merge Robot
6be014daa8
Merge pull request #2111 from strugee/patch-1
Fix typo
2020-05-22 16:36:21 +02:00
AJ Jordan
0487b498ad
Fix typo 2020-05-21 23:22:44 -07:00
OpenShift Merge Robot
0c8701b896
Merge pull request #2108 from jlebon/pr/back-out-ro-sysroot
switchroot/remount: Neuter sysroot.readonly for now
2020-05-21 17:58:04 +02:00
Jonathan Lebon
e35b82fb89 switchroot/remount: Neuter sysroot.readonly for now
We're hitting issues with the read-only remounts racing with various
services coming up. Let's neuter it for now until we rework how it
works.

See: https://github.com/coreos/fedora-coreos-tracker/issues/488
2020-05-20 16:23:59 -04:00
OpenShift Merge Robot
be62a01bff
Merge pull request #2106 from jlebon/pr/fix-admin-test
tests/admin-test: Ensure that commits are 1s apart
2020-05-20 19:05:35 +02:00