Commit Graph

4663 Commits

Author SHA1 Message Date
Colin Walters
191ce95ca1 admin-test: Show err.txt on unexpected failure
In a CI run I think one of these `ostree show` commands is failing.
While that output would be useful, the actual `err.txt` usually
has what we want too.
2020-05-20 15:57:10 +00:00
OpenShift Merge Robot
5a82d75adb
Merge pull request #2102 from cgwalters/test-clockskew-check
ci: Test for clock skew
2020-05-18 21:39:30 +02:00
Colin Walters
f14aa894d3 ci: Test for clock skew
I saw `tests/test-admin-deploy.none.sh` fail in one CI run, and
I want to check if it was because of clock skew, so fail
fast if we detect that.

xref https://github.com/ostreedev/ostree/pull/2099#issuecomment-629805375
2020-05-18 18:48:28 +00:00
OpenShift Merge Robot
0d7132ec79
Merge pull request #2101 from cgwalters/signapi-no-explicit-on
pull: Only have API to disable signapi for local pulls
2020-05-18 00:06:04 -07:00
Colin Walters
5a47c926c1 pull: Only have API to disable signapi for local pulls
There's a lot of historical baggage associated with GPG verification
and `ostree pull` versus `ostree pull-local`.  In particular nowadays,
if you use a `file://` remote things are transparently optimized
to e.g. use reflinks if available.

So for anyone who doesn't trust the "remote" repository, you should
really go through through the regular
`ostree remote add --sign-verify=X file://`
path for example.

Having a mechanism to say "turn on signapi verification" *without*
providing keys goes back into the "global state" debate I brought
up in https://github.com/ostreedev/ostree/issues/2080

It's just much cleaner architecturally if there is exactly one
path to find keys: from a remote config.

So here in contrast to the GPG code, for `pull-local` we explictily
disable signapi validation, and the `ostree_repo_pull()` API just
surfaces flags to disable it, not enable it.
2020-05-17 13:52:24 +00:00
OpenShift Merge Robot
b5e51d8c08
Merge pull request #2100 from cgwalters/make-install-kola-tests
ci: Install kola tests
2020-05-15 14:38:00 -07:00
Colin Walters
48f5a1885e ci: Install kola tests
This builds on
https://github.com/coreos/coreos-assembler/pull/1441
to install our tests rather than running them from the source
directory.  This model will more cleanly allow us to ship
our tests along with a test container or elsewhere, separate
from the source directory.

Also prep for https://github.com/ostreedev/ostree/pull/2048
2020-05-15 19:40:54 +00:00
OpenShift Merge Robot
11b09ec3cf
Merge pull request #2099 from jlebon/pr/timestamp-check-from-rev
lib/pull: Add `timestamp-check-from-rev`
2020-05-15 08:16:38 -07:00
Jonathan Lebon
79079c2657 lib/upgrader: Pull with timestamp-check-from-rev
For the same reason as https://github.com/coreos/rpm-ostree/pull/2094.
What we care most about is that the new commit we pull is newer than the
one we're currently sitting on, not necessarily that it's newer than the
branch itself, which it might not be if e.g. we're trying to deploy a
commit older than the tip but still newer than the deployment (via
`--override-commit`).
2020-05-14 15:27:29 -04:00
Jonathan Lebon
c8efce0656 lib/pull: Add timestamp-check-from-rev
The way `timestamp-check` works might be too restrictive in some
situations. Essentially, we need to support the case where users want to
pull an older commit than the current tip, but while still guaranteeing
that it is newer than some even older commit.

This will be used in Fedora CoreOS. For more information see:
https://github.com/coreos/rpm-ostree/pull/2094
https://github.com/coreos/fedora-coreos-tracker/issues/481
2020-05-14 14:00:42 -04:00
OpenShift Merge Robot
f145d18369
Merge pull request #2098 from cgwalters/finalize-requires-mounts
finalize: Add RequiresMountsFor=/boot too
2020-05-13 12:21:14 -07:00
Colin Walters
82679ce834 finalize: Add RequiresMountsFor=/boot too
In https://bugzilla.redhat.com/show_bug.cgi?id=1827712
some OpenShift CI is seeing `/boot` being unmounted before
`ostree-finalize-staged.service` runs or completes.

We finally tracked this down to a bug elsewhere, but
I think we should add this because it clearly shows
our requirements.
2020-05-13 18:52:45 +00:00
OpenShift Merge Robot
4b8354d478
Merge pull request #2097 from cgwalters/sign-verifier
pull: Further cleanup signapi verification
2020-05-12 11:38:58 -07:00
OpenShift Merge Robot
371955549e
Merge pull request #2096 from cgwalters/test-staged-delay
tests/staged-delay.sh: New test
2020-05-12 10:41:29 -07:00
Colin Walters
9509a4bc94 pull: Further cleanup signapi verification
Previously in the pull code, every time we went to verify
a commit we would re-initialize an `OstreeSign` instance
of each time, re-parse the remote configuration
and re-load its public keys etc.

In most cases this doesn't matter really because we're
pulling one commit, but if e.g. pulling a commit with
history would get a bit silly.

This changes things so that the pull code initializes the
verifiers once, and reuses them thereafter.

This is continuing towards changing the code to support
explicitly configured verifiers, xref
https://github.com/ostreedev/ostree/issues/2080
2020-05-12 15:20:26 +00:00
Colin Walters
474e1e2a33 tests/staged-delay.sh: New test
Attempting to reproduce
https://bugzilla.redhat.com/show_bug.cgi?id=1827712
but no dice yet.
2020-05-11 21:31:13 +00:00
OpenShift Merge Robot
4293c36188
Merge pull request #2095 from cgwalters/sign-get-all
signing: Change API to create instances directly
2020-05-11 18:41:35 +02:00
Colin Walters
a9a81f3a29 signing: Change API to create instances directly
This cleans up the verification code; it was weird how
we'd get the list of known names and then try to create
an instance from it (and throw an error if that failed, which
couldn't happen).
2020-05-10 14:18:55 +00:00
OpenShift Merge Robot
f572206f15
Merge pull request #2092 from cgwalters/sign-verify-ed25519-explicit
remote-add: Add --sign-verify=KEYTYPE=[inline|file]:PUBKEYREF
2020-05-10 14:22:41 +02:00
Colin Walters
588f42e8c6 remote-add: Add --sign-verify=KEYTYPE=[inline|file]:PUBKEYREF
Per https://github.com/ostreedev/ostree/issues/2080#issuecomment-623614483

A huge benefit of ed25519 (and ECC in general) is that keys are very
short - short enough that it's completely reasonable to inline
them into a command line argument.

And I think that's a good model; it makes the keys very visible.

For example, someone could easily copy-paste a commandline
argument from a webpage (secured via TLS) that says to run
`ostree remote add --sign-verify=ed25519=inline:KEY`.
2020-05-09 20:07:44 +00:00
OpenShift Merge Robot
4c01c0385f
Merge pull request #2093 from cgwalters/sysroot-requires
ostree-prepare-root: Requires=sysroot.mount
2020-05-08 21:54:51 +02:00
Colin Walters
e1b6398787 ostree-prepare-root: Requires=sysroot.mount
With just `After=` we'll still try to run in the scenario
where `sysroot.mount` fails because the rootfs didn't appear.
And this will end up spewing an error which can confuse people
into thinking something is wrong at the ostree level.

This has come up numerous times w/{Fedora,RHEL} CoreOS, most
recently while looking at
https://bugzilla.redhat.com/show_bug.cgi?id=1803130
2020-05-08 12:50:55 +00:00
OpenShift Merge Robot
59173c9604
Merge pull request #2091 from cgwalters/commit-w-xor-x
commit: Add --mode-ro-executables
2020-05-06 22:11:45 +02:00
Colin Walters
7392259332 commit: Add --mode-ro-executables option
I think we should encourage removing the writable bits from
executables.  This has happened to me:
https://thomask.sdf.org/blog/2019/11/09/take-care-editing-bash-scripts.html

And not having the writable bit may help prevent hardlink
corruption with OSTree in some cases.

We can't do this by default, but add a convenient CLI flag
for it.
2020-05-06 19:41:27 +00:00
OpenShift Merge Robot
ba874c525f
Merge pull request #2090 from cgwalters/sign-default-type
signing: Add #define OSTREE_SIGN_NAME_ED25519
2020-05-04 14:55:23 +02:00
Colin Walters
811082063c signing: Add #define OSTREE_SIGN_NAME_ED25519
Using `#define` or constants instead of strings helps avoid
typos and encourages documentation.
2020-05-03 17:04:27 +00:00
OpenShift Merge Robot
654983ee31
Merge pull request #2089 from dbnicholson/flags-enums-as-bitfields
lib: Coerce flags enums to GIR bitfields
2020-05-01 22:29:37 +02:00
OpenShift Merge Robot
e6bc24804c
Merge pull request #2087 from cgwalters/test-staged-deploy-cleanup
tests/staged-deploy: Cleanup initial state
2020-05-01 22:10:53 +02:00
Dan Nicholson
dc69f56de6 lib: Coerce flags enums to GIR bitfields
The GI scanner decides if an `enum` is really a `bitfield` if it finds
any values that have left shifts. With an `enumeration`, the
introspecting language may error or convert to a different type if the
user tries to combine values. Change all Flags `enum`s to use
left-shifted values so that they're represented as `bitfield`s in the
GIR.

The primary bug here is that you can't combine `REFS_ONLY` and
`NO_PRUNE` when calling `OSTree.Repo.prune()` from an introspected
language.

This is an IABI break since the typelib will change from `enumeration`
to `bitfield`. `OstreeRepoImportFlags` is internal but the change is
included here to prepare for a subsequent name that would require bit
shifting to operate correctly as a flag.
2020-04-30 15:13:31 -06:00
Colin Walters
37e1921b51 tests/staged-deploy: Cleanup initial state
I'm using [cosa build-fast](https://github.com/coreos/coreos-assembler/pull/1371)
and this test doesn't like starting out with two deployments.  Clean
things up to one at the start just to simplify things.
2020-04-29 01:47:14 +00:00
OpenShift Merge Robot
240f6d3ac6
Merge pull request #2084 from d4s/wip/d4s/expose_signapi_metadata
signapi: expose metadata format and key
2020-04-29 03:19:11 +02:00
Denis Pynkin
ee5d0f350f signapi: expose metadata format and key
Explicitly expose functions for querying the metadata format
and key name used by OstreeSign object:
 - ostree_sign_metadata_format
 - ostree_sign_metadata_key

This allows to use the same metadata format and key name
by 3-rd party applications using signapi.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-04-24 16:34:50 +00:00
OpenShift Merge Robot
a05c99d933
Merge pull request #2083 from agners/doc-updates
Doc updates
2020-04-24 14:07:52 +02:00
Stefan Agner
ce5dfadbd7 docs: extend repository types
Clarify where metadata are stored exactly in the `bare-user` case.
Make the first sentence of `bare-user` and `bare-user-only` paragraph
symetric to make it easier to jump to the right paragraph for readers
in a hury. Stree out that `bare-user-only` may loose metadata.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-04-24 13:12:47 +02:00
Stefan Agner
b43c0be347 docs: extend object type documentation
Extend the object type documentation with file endings used for the
individual type. Also clarify in which situation content type objects
are used and why they do not match the SHA256 hash today.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-04-24 13:05:15 +02:00
Stefan Agner
b7662aaf33 docs: clarify archive repo type
Today `archive-z2` is still used as the default string to indicate
a `archive` type repository. Make clear that this is the way it is
intended. Otherwise users might think they use an no longer supported
OSTree repository since the mode string is still `archive-z2`.

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
2020-04-24 13:04:18 +02:00
OpenShift Merge Robot
66527efcb8
Merge pull request #2082 from cgwalters/finalize-sandbox
finalize-staged: Add ProtectHome=yes and ReadOnlyPaths=/etc
2020-04-22 18:51:15 +02:00
OpenShift Merge Robot
8996ead2ea
Merge pull request #2081 from cgwalters/deploy-etc
deploy: Add --no-merge
2020-04-22 16:34:57 +02:00
Colin Walters
47a3096ab8 finalize-staged: Add ProtectHome=yes and ReadOnlyPaths=/etc
Same motivation as
https://github.com/coreos/rpm-ostree/pull/2060

I tried `InaccessiblePaths=/var` first and was very sad to find
out we have one tiny exception that breaks it.  Otherwise it'd
be so elegant.  Maybe in the future we split out that one thing
to a separate `ostree-finalized-stage-var.service` that's just
`ExecStart=/bin/rm -vf /var/.updated` and is otherwise
`ProtectSystem=strict` etc.
2020-04-22 14:28:33 +00:00
Colin Walters
368574b657 deploy: Add --no-merge
All of the underlying libostree APIs have supported passing `NULL`
for a merge deployment for...a long time.  But we never plumbed
it up into the CLI.

Add a `--no-merge` option to aid people who want to do a "factory reset":
https://github.com/ostreedev/ostree/issues/1793
2020-04-22 14:09:54 +00:00
OpenShift Merge Robot
391ad0ec21
Merge pull request #2079 from cgwalters/pull-split-sign-verify
lib: Move gpg/signapi bits into ostree-repo-pull-verify.c
2020-04-18 13:57:08 -04:00
Colin Walters
1b8fed247c lib: Move gpg/signapi bits into ostree-repo-pull-verify.c
`ostree-repo-pull.c` is huge; separate some of the GPG/signing
verification functions into their own file so they're more easily seen.
2020-04-18 13:35:36 +00:00
Colin Walters
12984ddef2 lib: Move pull struct definition into repo-pull-private.h
Prep for further splitting up `ostree-repo-pull.c`.
2020-04-18 13:34:57 +00:00
OpenShift Merge Robot
7d51cee0a8
Merge pull request #2077 from jlebon/pr/transaction-cleanup
lib/commit: Check that dirent is a directory before cleaning
2020-04-17 15:15:12 -04:00
Jonathan Lebon
8ece36c28a lib/commit: Check that dirent is a directory before cleaning
I've only noticed this by inspection. But I think it's possible for
`cleanup_txn_dir` to get called with the `staging-...-lock` file since
it matches the prefix.

Make the checking here stronger by verifying that it's a directory. If
it's not a directory (lockfile), then follow the default pruning expiry
logic so that we still cleanup stray lockfiles eventually.
2020-04-17 14:29:13 -04:00
Jonathan Lebon
df065ad766 lib: Rename function for staging dir check
Rename the function to more accurately reflect what it does, which is to
check whether the filename has the `staging-` prefix.
2020-04-17 14:28:20 -04:00
Jonathan Lebon
bb8fd5a2c4 lib/commit: Add more error prefixing
We think we're hitting an error in that function in the Fedora infra.
Add some more error prefixing to help debugging.
2020-04-17 14:27:16 -04:00
OpenShift Merge Robot
95be6676c0
Merge pull request #2076 from d4s/wip/d4s/rename_with-libsodium_opt
sign: rename option for enabling ed25519
2020-04-17 13:30:36 -04:00
Denis Pynkin
6aeca233f6 sign: rename option for enabling ed25519
Use option `--with-ed25519-libsodium` instead of
`--with-libsodium` to enable ed25519 signature engine.

This allows to use later different implementations of ed25519
signing/verification. For instance, based on openssl.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-04-17 19:26:52 +03:00
OpenShift Merge Robot
351bc56f16
Merge pull request #2073 from cgwalters/pull-private-cleanup
lib: Move internal binding verification API to repo.c
2020-04-17 11:49:59 -04:00