Commit Graph

4699 Commits

Author SHA1 Message Date
Denis Pynkin
c09df18454 lib/sign: enable verification for pulling
Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
e133cb7b74 sign: allow to sign commits from CLI
Add signing ability to commit builtin.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
c3608aa56a sign: add new builtin for signing
This builtin allows to sign and verify commit with new signature
mechanism. At the moment it is possible to use 'dummy' and 'ed25519'
signing modules.

'dummy' module use any ASCII string from command line as a key for
commit's signing or verification.

Support of ed25519 signature is implemented with `libsoium` library.
Secret and public key should be provided in hex presentation via
command line.

Based on 'gpg-sign' source.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
edbbe1c4f2 lib/sign: initial implementation
Added the initial version of signing interface allowing to allowing to
sign and verify commits.
Implemented initial signing modules:
 - dummy -- simple module allowing to sign/verify with ASCII string
 - ed25519 -- module allowing to sign/verify commit with ed25519
   (EdDSA) signature scheme provided by libsodium library.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Denis Pynkin
84c8164610 Add libsodium dependency
Allow to configure with libsodium flag.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:22:41 +03:00
OpenShift Merge Robot
7a95929867
Merge pull request #2041 from cgwalters/auto-sepolicy
repo/commit: Add support for --selinux-policy-from-base
2020-03-24 20:06:55 +01:00
Colin Walters
b3bbbd1542 repo/commit: Add support for --selinux-policy-from-base
The [dev-overlay](332c6ab3b9/src/cmd-dev-overlay)
script shipped in coreos-assembler mostly exists to deal
with the nontrivial logic around SELinux policy.  Let's make
the use case of "commit some binaries overlaying a base tree, using
the base's selinux policy" just require a magical
`--selinux-policy-from-base` argument to `ostree commit`.

A new C API was added to implement this in the case of `--tree=ref`;
when the base directory is already checked out, we can just reuse
the existing logic that `--selinux-policy` was using.

Requires: https://github.com/ostreedev/ostree/pull/2039
2020-03-24 16:34:26 +00:00
OpenShift Merge Robot
bdccbb16d4
Merge pull request #2042 from cgwalters/pull-repeated-no-flake
tests/pull-repeated: Bump up retries to match max fails
2020-03-24 16:49:14 +01:00
Colin Walters
ea16f7603d tests/pull-repeated: Bump up retries to match max fails
This test keeps occasionally failing in CI - as expected, because
we retry 8 times for an object but it's completely possible for
us to hit the <0.5% chance of 50% failure 8 times in a row.

Since the max errors from the server is 100, set retries to the
same thing.
2020-03-24 12:24:46 +00:00
OpenShift Merge Robot
41d3dfa7b8
Merge pull request #2039 from cgwalters/commit-cli-cleanup
main/commit: Rework control flow to use --tree=X path
2020-03-24 02:22:22 +01:00
Colin Walters
a717535490 main/commit: Rework control flow to use --tree=X path
Rework the simple cases of "commit ." and "commit argv[1]" to
generate the more general "--tree=X --tree=Y" path, so that we
only have one primary control flow here.

Prep for a future patch around loading SELinux policy from
the first argument.
2020-03-23 19:03:16 +00:00
OpenShift Merge Robot
d87301b111
Merge pull request #2040 from cgwalters/itest-require-writable
tests/kola: Two test fixes
2020-03-23 20:01:04 +01:00
Colin Walters
635fd0aa9c tests/kola: Two test fixes
Now that we're actually running this in CI.
2020-03-21 20:48:15 +00:00
OpenShift Merge Robot
89fdbf1b0e
Merge pull request #2037 from mwleeds/retry-on-partial-input
lib/fetcher-util: retry download on G_IO_ERROR_PARTIAL_INPUT
2020-03-20 14:35:14 +01:00
Matthew Leeds
53d1efbaab lib/fetcher-util: retry download on G_IO_ERROR_PARTIAL_INPUT
Add G_IO_ERROR_PARTIAL_INPUT to the list of error codes caused by
transient networking errors which lead us to retry the request. When
attempting to install the spotify flatpak you often get the error
message "Connection terminated unexpectedly" and the download of the deb
file fails. In this case, libsoup is setting G_IO_ERROR_PARTIAL_INPUT
and sometimes a subsequent download attempt is successful, so we should
treat it as transient.

Ideally we would behave as wget does in this case and retry the download
picking up where we left off in the file rather than starting over, but
that would require changes to libsoup I think.

Sadly this patch does not fix the flatpak installation of spotify in the
face of such errors, because flatpak doesn't use libostree to download
extra data, but presumably it's possible we could encounter such an
error pulling from an ostree repo, so the patch is still correct.
2020-03-19 15:28:08 -07:00
OpenShift Merge Robot
d0b3e76fa8
Merge pull request #2036 from jlebon/pr/partial-parent
lib/pull: Don't leave commits pulled by depth as partial
2020-03-19 18:05:00 +01:00
OpenShift Merge Robot
371a327dc2
Merge pull request #2025 from cgwalters/use-kola-run-ext
tests: Rework tests/installed → tests/kola
2020-03-19 17:32:23 +01:00
Colin Walters
b93180a4d3 tests: Rework tests/installed → tests/kola
Previously we made an effort to use the [Fedora Standard Test Interface](https://docs.fedoraproject.org/en-US/ci/standard-test-interface/).
This effort was not very successful; the primary thing that
it really died on is Ansible just didn't support rebooting
very well.  I think that's since gotten better, but even
then, Ansible wasn't the best thing for a test framework
for us anyways.

In the meantime Fedora CoreOS happened emphasizing Ignition
and not "post-hoc reconciliation" models like Ansible over
ssh.

And, [coreos-assembler](https://github.com/coreos/coreos-assembler) happened too.

Furthermore, we really need to test OSTree's interaction
with Ignition as we've invented several special things there.

Then most recently, I've been working on having
cosa/kola support running externally defined tests:
https://github.com/coreos/coreos-assembler/pull/1215

There's a lot of things to clean up after this but at least this
works for me:

```
$ cd /srv/fcos
$ cosa kola run -- --parallel 4 --output-dir tmp/kola -E ~/src/github/ostreedev/ostree/ 'ext.ostree.*'
```

NOTE: This *does not* drop ostree binaries into the target.  See:
https://github.com/coreos/coreos-assembler/pull/1252#issuecomment-600623315

This drops our dependency on Python in the installed tests, and
also fixes a few bugs that came up.

I disabled the `itest-bare-user-root.sh` one because it's
entangled with the shell script infrastructure for the unit tests.
2020-03-19 16:01:57 +00:00
Jonathan Lebon
fd822a8dfd lib/pull: Don't leave commits pulled by depth as partial
When pulling full parent commits via e.g. `--depth N`, we weren't
unmarking them as partial in the out path.

Closes: #2035
2020-03-19 10:10:50 -04:00
OpenShift Merge Robot
ab22e9dacb
Merge pull request #2034 from jlebon/pr/diff-docs
bin/diff: Clarify documentation around REV and DIR syntax
2020-03-19 01:06:15 +01:00
Jonathan Lebon
f7a8617c01 bin/diff: Clarify documentation around REV and DIR syntax
Related: #2032
2020-03-18 10:32:52 -04:00
OpenShift Merge Robot
f7582ce87e
Merge pull request #2033 from cgwalters/pin-error-handling
main/pin: Fix usage of GError
2020-03-18 14:42:42 +01:00
Colin Walters
0b6ac72b7f main/pin: Fix usage of GError
This regressed in 2db79fb398
I noticed this while finally getting the installed tests to run
in FCOS via kola and `ostree admin pin 0` is now aborting because
we were returning TRUE, but no error set.

I don't see a reason to try to continue on if we hit an error;
the original reporter was requesting support for multiple arguments,
but not "ignore invalid requests".
2020-03-18 02:15:34 +00:00
OpenShift Merge Robot
35c8fd3722
Merge pull request #2030 from jlebon/pr/2020.3
Release 2020.3
2020-03-14 16:17:21 +01:00
Jonathan Lebon
8f2c240326 Post-release version bump 2020-03-13 21:59:35 -04:00
Jonathan Lebon
6ed48234ba Release 2020.3
Let's do another release to get the `sysroot.readonly` fixes into FCOS
and unpin ostree and rpm-ostree there.
2020-03-13 21:59:05 -04:00
OpenShift Merge Robot
5164dcbe78
Merge pull request #2027 from cgwalters/pull-auto-mnt-namespace
main: Also automatically remount rw /sysroot for `ostree pull` etc.
2020-03-14 02:52:43 +01:00
Colin Walters
6a27dff941 main: Also automatically remount rw /sysroot for ostree pull etc.
See https://github.com/coreos/fedora-coreos-tracker/issues/343
When we added the read-only sysroot support it broke using "raw"
`ostree pull` and `ostree refs --create` and all of the core repo
CLIs that just operate on a repo and not a sysroot.

Fixing this is a bit ugly as it "layer crosses" things even more.
Extract a helper function that works in both cases.
2020-03-13 20:53:26 +00:00
OpenShift Merge Robot
2d901957b8
Merge pull request #2029 from jlebon/pr/upgrade-tests
ci: use `fcosKola` for running kola tests
2020-03-13 21:53:04 +01:00
Jonathan Lebon
cbb3f0a8c2 ci: use fcosKola for running kola tests
We were using `--no-test-exit-error` for upgrade tests but weren't
actually checking for test failures after.

Instead of running kola directly, just use the `fcosKola` custom step
which automatically takes care of e.g. running tests in parallel and
archiving results.
2020-03-13 14:35:53 -04:00
OpenShift Merge Robot
94ccbcca5e
Merge pull request #2028 from cgwalters/ci-test-upgrades
ci: Test kola --upgrades
2020-03-13 19:18:40 +01:00
Colin Walters
0302262e2f ci: Test kola --upgrades
This one is very relevant for ostree.
2020-03-13 17:29:58 +00:00
OpenShift Merge Robot
26a2be0578
Merge pull request #2018 from jlebon/pr/migrate-to-coreos-ci
ci: migrate to new coreos-ci project
2020-03-04 17:32:56 -05:00
Jonathan Lebon
24691c509c ci: migrate to new coreos-ci project
Use the new custom steps. I think we could simplify things further by
using `fcosBuild` but let's start with this for now.
2020-03-02 14:22:33 -05:00
OpenShift Merge Robot
acd3b24fd1
Merge pull request #2015 from cgwalters/release
Release 2020.2
2020-02-21 10:16:50 -05:00
Colin Walters
6bedfcfdc3 Post-release version bump 2020-02-21 14:48:00 +00:00
Colin Walters
c6085ebd5e Release 2020.2
"Brown paper bag" release that actually sets the
`is_release_build=yes` flag and also fixes the
`Since:` on a few new functions.
2020-02-21 14:45:49 +00:00
OpenShift Merge Robot
73742252e2
Merge pull request #2014 from ostreedev/fixup-since-versions
lib: Fix Since versions for 2020.1
2020-02-20 18:27:28 -05:00
Dan Nicholson
3557e4e165 lib: Fix Since versions for 2020.1
These had been added assuming 2019.7 would be the next version, but now
it's 2020 and there's been a release. In the case of
`OstreeCommitSizesEntry`, I'd forgotten to move it forward from 2019.5
to 2019.7 in the time between when I started working on the feature and
it landed.
2020-02-20 15:47:41 -07:00
OpenShift Merge Robot
b39c03ab97
Merge pull request #2013 from cgwalters/prepareroot-comment
prepare-root: Add a comment about the role of this service
2020-02-20 18:30:05 +01:00
Colin Walters
7db7cfc407 Post-release version bump 2020-02-20 16:37:07 +00:00
Colin Walters
04c85fa101 Release 2020.1
New year, new release!
2020-02-20 16:35:47 +00:00
OpenShift Merge Robot
010e6c38bc
Merge pull request #1984 from jlebon/pr/supp-groups
lib/repo: Create repo directories as 0775
2020-02-20 03:07:41 +01:00
Colin Walters
d5bfbc6715 prepare-root: Add a comment about the role of this service
Came up on an IRC question, docs for this are scattered around.
2020-02-20 00:35:32 +00:00
OpenShift Merge Robot
9eb4506c1b
Merge pull request #2012 from clime/patch-2
Update ostree-pull.xml with info about pulled refs location and access
2020-02-19 20:32:23 +01:00
clime
8e65366ffd Update ostree-pull.xml with info about pulled refs location and access 2020-02-19 20:12:23 +01:00
Jonathan Lebon
7085a50297 lib/repo: Create repo directories as 0775
For repo structure directories like `objects`, `refs`, etc... we should
be more permissive and let the system's `umask` narrow down the
permission bits as wanted.

This came up in a context where we want to be able to have read/write
access on an OSTree repo on NFS from two separate OpenShift apps by
using supplemental groups[1] so we don't require SCCs for running as the
same UID (supplemental groups are part of the default restricted SCC).

[1] https://docs.openshift.com/container-platform/3.11/install_config/persistent_storage/persistent_storage_nfs.html#nfs-supplemental-groups
2020-02-18 11:26:46 -05:00
OpenShift Merge Robot
13666dcd50
Merge pull request #2009 from cgwalters/testsuite
deploy: Avoid trying to change immutable state unnecessarily
2020-02-14 22:56:25 +01:00
Colin Walters
0dd8dec2c9 deploy: Avoid trying to change immutable state unnecessarily
For some reason I haven't fully debugged (probably a recent
kernel change), in the case where the immutable bit isn't set,
trying to call `EXT2_IOC_SETFLAGS` without it set returns `EINVAL`.

Let's avoid calling the `ioctl()` if we don't have anything to do.

This fixes a slew of `make check` failures here in my toolbox
environment.

(kernel is `5.5.0-0.rc6.git0.1.fc32.x86_64` with `xfs`)
2020-02-14 16:37:38 +00:00
OpenShift Merge Robot
7b28d01cd5
Merge pull request #2003 from crobinso/doc-link-fix
docs: Fix 'package layering' rpm-ostree link
2020-02-10 20:28:40 +01:00