3139 Commits

Author SHA1 Message Date
Denis Pynkin
200efd7d44 builtin/sign: add option 'keys-dir'
Option '--keys-dir' is used for redefinition of default directories with
public/revoked keys. If keys directory is set then default directories
are ignored and target directory is expected to contain following
structure for ed25519 signature mechanism:

dir/
  trusted.ed25519      <- file with trusted keys
  revoked.ed25519      <- file with revoked keys
  trusted.ed25519.d/   <- directory with files containing trusted keys
  revoked.ed25519.d/   <- directory with files containing revoked keys

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
ee12b7e774 lib/sign: add revoking mechanism for ed25519 keys
Skip public keys verification if key is marked as invalid key.
Allow to redefine system-wide directories for ed25519 verification.
Minor bugfixes.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
ceaf6d7f54 lib/sign: add ostree_seign_clear_keys function
Add the function for implicit cleanup of all loaded keys.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
eb8e501ece lib/sign: minor optimisation for ed25519
Exclude unneeded conversion while load keys from files.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
7fa7c3c4f9 sign: use common function for loading public keys during pulling
Add function `_load_public_keys()` to pre-load public keys according
remote's configuration. If no keys configured for remote, then use
system-wide configuration.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
f0181adff3 lib/sign: allow to add keys as base64 string for ed25519
Allow to add public and secret key for ed25519 module as based64 string.
This allows to use common API for pulling and builtins without knowledge
of used signature algorithm.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
bc4488692c lib/sign: public API optimisation
Removed from public `ostree_sign_detached_metadata_append` function.
Renamed `metadata_verify` into `data_verify` to fit to real
functionality.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
95ab57c17e lib/sign-ed25519: cleanup unneeded code
Removed unused code.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
ea291a0605 builtin/sign: allow to use multiple public keys for verification
`ostree sign` is able to use several public keys provided via arguments
and via file with keys.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
557f423609 sign: fix memory leaks and code cleanup
Return `const char *` instead of copy of the string -- this allow to
avoid unneeded copying and memory leaks in some constructions.
Minor code cleanup and optimisations.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
36e4667973 builtin/sign: allow to sign with keys from secret file
Read keys from secret file provided by `--keys-file=` option.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
94447617db lib/sign: read ed25519 public keys from well known places
If not provided key of file name with keys for remote, then try to use
system defaults:
- /etc/ostree/trusted.ed25519
- /etc/ostree/trusted.ed25519.d/*
- /usr/share/ostree/trusted.ed25519
- /usr/share/ostree/trusted.ed25519.d/*

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
073876d9b2 lib/sign: add support of file with valid keys for remote
Allow to use custom file with public keys for remote.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
91cc294d05 lib/sign: disable mandatory signature check
Do not check the signature check by default.
Need to enable it explicitly.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
3386893deb sign: check signatures for pulled commits
If `verification-key` is set for remote it is used as a public key for
checking the commit pulled from that remote.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
2831028c41 sign: fixes for ed25519 for loading public keys from files
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
fe3a839ae7 builtin/sign: remove libsodium dependency
Now do not need to compile/link builtin with external dependencies.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
a8521a7c3b sign: fix error return for dummy module
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
2d39126628 sign: fix unneeded objects creation
Do not create objects just for supported modules list.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
4b9232b1fe builtin/sign: remove libsodium-specific code
Use only common sign API without libsoduim parts.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
06cfcd9a8a builtin/sign: allow to provide the file with public keys
Added option `--keys-file` for `ostree sign`.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
2303202c86 sign: API changes for public keys and CLI keys format
API changes:
- added function `ostree_sign_add_pk()` for multiple public keys using.
- `ostree_sign_set_pk()` now substitutes all previously added keys.
- added function `ostree_sign_load_pk()` allowed to load keys from file.
- `ostree_sign_ed25519_load_pk()` able to load the raw keys list from file.
- use base64 encoded public and private ed25519 keys for CLI and keys file.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
c09df18454 lib/sign: enable verification for pulling
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
e133cb7b74 sign: allow to sign commits from CLI
Add signing ability to commit builtin.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
c3608aa56a sign: add new builtin for signing
This builtin allows to sign and verify commit with new signature
mechanism. At the moment it is possible to use 'dummy' and 'ed25519'
signing modules.

'dummy' module use any ASCII string from command line as a key for
commit's signing or verification.

Support of ed25519 signature is implemented with `libsoium` library.
Secret and public key should be provided in hex presentation via
command line.

Based on 'gpg-sign' source.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
edbbe1c4f2 lib/sign: initial implementation
Added the initial version of signing interface allowing to allowing to
sign and verify commits.
Implemented initial signing modules:
 - dummy -- simple module allowing to sign/verify with ASCII string
 - ed25519 -- module allowing to sign/verify commit with ed25519
   (EdDSA) signature scheme provided by libsodium library.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Colin Walters
b3bbbd1542 repo/commit: Add support for --selinux-policy-from-base
The [dev-overlay](332c6ab3b9/src/cmd-dev-overlay)
script shipped in coreos-assembler mostly exists to deal
with the nontrivial logic around SELinux policy.  Let's make
the use case of "commit some binaries overlaying a base tree, using
the base's selinux policy" just require a magical
`--selinux-policy-from-base` argument to `ostree commit`.

A new C API was added to implement this in the case of `--tree=ref`;
when the base directory is already checked out, we can just reuse
the existing logic that `--selinux-policy` was using.

Requires: https://github.com/ostreedev/ostree/pull/2039
2020-03-24 16:34:26 +00:00
Colin Walters
a717535490 main/commit: Rework control flow to use --tree=X path
Rework the simple cases of "commit ." and "commit argv[1]" to
generate the more general "--tree=X --tree=Y" path, so that we
only have one primary control flow here.

Prep for a future patch around loading SELinux policy from
the first argument.
2020-03-23 19:03:16 +00:00
Matthew Leeds
53d1efbaab lib/fetcher-util: retry download on G_IO_ERROR_PARTIAL_INPUT
Add G_IO_ERROR_PARTIAL_INPUT to the list of error codes caused by
transient networking errors which lead us to retry the request. When
attempting to install the spotify flatpak you often get the error
message "Connection terminated unexpectedly" and the download of the deb
file fails. In this case, libsoup is setting G_IO_ERROR_PARTIAL_INPUT
and sometimes a subsequent download attempt is successful, so we should
treat it as transient.

Ideally we would behave as wget does in this case and retry the download
picking up where we left off in the file rather than starting over, but
that would require changes to libsoup I think.

Sadly this patch does not fix the flatpak installation of spotify in the
face of such errors, because flatpak doesn't use libostree to download
extra data, but presumably it's possible we could encounter such an
error pulling from an ostree repo, so the patch is still correct.
2020-03-19 15:28:08 -07:00
Jonathan Lebon
fd822a8dfd lib/pull: Don't leave commits pulled by depth as partial
When pulling full parent commits via e.g. `--depth N`, we weren't
unmarking them as partial in the out path.

Closes: #2035
2020-03-19 10:10:50 -04:00
Jonathan Lebon
f7a8617c01 bin/diff: Clarify documentation around REV and DIR syntax
Related: #2032
2020-03-18 10:32:52 -04:00
Colin Walters
0b6ac72b7f main/pin: Fix usage of GError
This regressed in 2db79fb398
I noticed this while finally getting the installed tests to run
in FCOS via kola and `ostree admin pin 0` is now aborting because
we were returning TRUE, but no error set.

I don't see a reason to try to continue on if we hit an error;
the original reporter was requesting support for multiple arguments,
but not "ignore invalid requests".
2020-03-18 02:15:34 +00:00
Colin Walters
6a27dff941 main: Also automatically remount rw /sysroot for ostree pull etc.
See https://github.com/coreos/fedora-coreos-tracker/issues/343
When we added the read-only sysroot support it broke using "raw"
`ostree pull` and `ostree refs --create` and all of the core repo
CLIs that just operate on a repo and not a sysroot.

Fixing this is a bit ugly as it "layer crosses" things even more.
Extract a helper function that works in both cases.
2020-03-13 20:53:26 +00:00
Colin Walters
c6085ebd5e Release 2020.2
"Brown paper bag" release that actually sets the
`is_release_build=yes` flag and also fixes the
`Since:` on a few new functions.
2020-02-21 14:45:49 +00:00
Dan Nicholson
3557e4e165 lib: Fix Since versions for 2020.1
These had been added assuming 2019.7 would be the next version, but now
it's 2020 and there's been a release. In the case of
`OstreeCommitSizesEntry`, I'd forgotten to move it forward from 2019.5
to 2019.7 in the time between when I started working on the feature and
it landed.
2020-02-20 15:47:41 -07:00
OpenShift Merge Robot
b39c03ab97
Merge pull request #2013 from cgwalters/prepareroot-comment
prepare-root: Add a comment about the role of this service
2020-02-20 18:30:05 +01:00
Colin Walters
04c85fa101 Release 2020.1
New year, new release!
2020-02-20 16:35:47 +00:00
OpenShift Merge Robot
010e6c38bc
Merge pull request #1984 from jlebon/pr/supp-groups
lib/repo: Create repo directories as 0775
2020-02-20 03:07:41 +01:00
Colin Walters
d5bfbc6715 prepare-root: Add a comment about the role of this service
Came up on an IRC question, docs for this are scattered around.
2020-02-20 00:35:32 +00:00
Jonathan Lebon
7085a50297 lib/repo: Create repo directories as 0775
For repo structure directories like `objects`, `refs`, etc... we should
be more permissive and let the system's `umask` narrow down the
permission bits as wanted.

This came up in a context where we want to be able to have read/write
access on an OSTree repo on NFS from two separate OpenShift apps by
using supplemental groups[1] so we don't require SCCs for running as the
same UID (supplemental groups are part of the default restricted SCC).

[1] https://docs.openshift.com/container-platform/3.11/install_config/persistent_storage/persistent_storage_nfs.html#nfs-supplemental-groups
2020-02-18 11:26:46 -05:00
Colin Walters
0dd8dec2c9 deploy: Avoid trying to change immutable state unnecessarily
For some reason I haven't fully debugged (probably a recent
kernel change), in the case where the immutable bit isn't set,
trying to call `EXT2_IOC_SETFLAGS` without it set returns `EINVAL`.

Let's avoid calling the `ioctl()` if we don't have anything to do.

This fixes a slew of `make check` failures here in my toolbox
environment.

(kernel is `5.5.0-0.rc6.git0.1.fc32.x86_64` with `xfs`)
2020-02-14 16:37:38 +00:00
OpenShift Merge Robot
96fb1decf4
Merge pull request #1959 from cgwalters/ostree-verity
Initial fs-verity support
2020-01-28 13:01:40 -08:00
OpenShift Merge Robot
8a9a496501
Merge pull request #1957 from dbnicholson/commit-sizes
Upstream Endless sizes metadata changes
2020-01-27 08:25:59 -08:00
Colin Walters
58fa579b1b Initial fs-verity support
Using fs-verity is natural for OSTree because it's file-based,
as opposed to block based (like dm-verity).  This only covers
files - not symlinks or directories.  And we clearly need to
have integrity for the deployment directories at least.

Also, what we likely need is an API that supports signing files
as they're committed.

So making this truly secure would need a lot more work.  Nevertheless,
I think it's time to start experimenting with it.  Among other things,
it does *finally* add an API that makes files immutable, which will
help against some accidental damage.

This is basic enablement work that is being driven by
Fedora CoreOS; see also https://github.com/coreos/coreos-assembler/pull/876
2020-01-27 15:44:16 +00:00
OpenShift Merge Robot
08e292bc14
Merge pull request #1877 from dbnicholson/specific-gpg-errors
More specific GPG errors
2020-01-26 09:20:51 -08:00
Dan Nicholson
11ad68647a ostree/trivial-httpd: Add log message for autoexit
This is useful when checking if the daemon actually exited since we
don't store the child PID anywhere.
2020-01-24 13:11:54 -07:00
Dan Nicholson
fbf5a94e0a ostree/trivial-httpd: Fix --autoexit with --daemonize and --log-file
When --autoexit is used with --daemonize and --log-file, the program
never exits when the root directory is deleted. I believe what happens
is that g_file_new_for_path triggers the glib worker context to be
started to talk to GVfs. Once the program forks, the parent exits and
the thread iterating the worker context is gone. The file monitor then
never receives any events because the inotify helper also runs from the
worker context.

Move the fork earlier just after parsing and validating the command line
arguments. In order to handle setup errors in the child, a pipe is
opened and the parents waits until the child writes a status byte to it.
If the byte is 0, the parent considers the child setup successful and
exits and the child carries on as a daemon. Notably, the child doesn't
reopen stderr to /dev/null until after this so that it can send error
messages there.

Fixes: #1941
2020-01-24 13:11:54 -07:00
Dan Nicholson
0fbfc0b207 lib/gpg: Add more specific OstreeGpgError codes
Currently `ostree_gpg_verify_result_require_valid_signature` always
returns an error that the key used for the signature is missing from the
keyring. However, all that's been determined is that there are no valid
signatures. The error could also be from an expired signature, an
expired key, a revoked key or an invalid signature.

Provide values for these missing errors and return them from
`ostree_gpg_verify_result_require_valid_signature`. The description of
each result is appended to the error message, but since the result can
contain more than one signature but only a single error can be returned,
the status of the last signature is used for the error code. See the
comment for rationale.

Related: flatpak/flatpak#1450
2020-01-24 13:02:17 -07:00
Dan Nicholson
c54a3b5daa lib/gpg: Prefer declare-and-initialize style
As noted in
https://github.com/ostreedev/ostree/pull/1872#discussion_r295408768.
2020-01-24 13:02:09 -07:00
Dan Nicholson
97c831dd5f bin/show: Add --print-sizes option to show sizes metadata
Use the new `ostree_commit_get_object_sizes()` API to read the
`ostree.sizes` commit metadata and print a summary.
2020-01-20 20:46:31 -07:00