3447 Commits

Author SHA1 Message Date
Colin Walters
87d115706e checkout: Save errno when re-throwing
I was seeing an `EPERM`  here which was confusing.
It turned out the real error was `EEXIST`.

Since we're referring to the original error, but we do a
lot of computation in the middle, we need to save errno.
2021-08-26 15:17:36 -04:00
Luca BRUNO
c64b4bceba
lib: improve transactions auto-cleanup logic
This fixes some aspects of OstreeRepoAutoTransaction and re-aligns
it with the logic in flatpak. Specifically:
 * link to the underlying repo through refcounting
 * bridge internal errors to warning messages
 * verify the input pointer type

This is a preparation step before exposing this logic as a public API.
2021-08-25 16:17:52 +00:00
Luca BRUNO
3e2360e3bb
lib/commit: automatically skip xattrs in bare-user-only mode 2021-08-24 07:56:46 +00:00
Luca BRUNO
f75552e15c
builtins/commit: set up relevant flags in bare-user-only mode
This detects bare-user-only mode and automatically enables a
commit modifier with relevant flags.
2021-08-24 07:56:19 +00:00
Luca BRUNO
0276f4dc0c
lib/diff: automatically skip xattrs in bare-user-only mode 2021-08-24 07:56:18 +00:00
Dan Nicholson
58a683f8f0 bin/remote: Rename list-gpg-keys to gpg-list-keys
As pointed out in the original review, `gpg-list-keys` fits better
alongside the existing `gpg-import`.

Changes were done with:

```
git grep -l list-gpg-keys | xargs sed -i 's/list-gpg-keys/gpg-list-keys/'
for src in $(git ls-files '*list-gpg-keys*'); do
  dst=${src/list-gpg-keys/gpg-list-keys}
  git mv "$src" "$dst"
done
```
2021-08-23 15:30:31 -06:00
Luca BRUNO
06ff77cfeb
lib/diff: ignore xattrs if disabled on either repos
This fixes the logic to detect whether xattrs should be automatically
ignored when diffing.
2021-08-23 13:31:13 +00:00
Colin Walters
2f675cf1b1
Merge pull request #2412 from lucab/ups/lib-commit-canonicalize
lib/commit: autofix permissions for bare-user-only
2021-08-20 15:12:21 -04:00
Colin Walters
98f3fe3d8e
Merge pull request #2401 from dbnicholson/gpg-key-info
Remote GPG key info
2021-08-20 15:11:11 -04:00
Luca BRUNO
8a5241dd6a
lib/commit: autofix permissions for bare-user-only
This tweaks commit logic to detect bare-user-only repositories and
canonicalize permissions automatically.
2021-08-20 16:11:04 +00:00
Luca BRUNO
d41fcd17a7
lib/repo/checkout: use canonical perms in bare-user-only mode
This automatically enables canonical permissions for checkouts in
bare-user-only mode.
2021-08-19 16:09:54 +00:00
Luca BRUNO
c6b72f527b
lib/core/checksum: add flag to use canonical permissions
This adds a new `OSTREE_CHECKSUM_FLAGS_CANONICAL_PERMISSIONS`
checksumming flag, which is needed in bare-user-only mode
to ignore local IDs.
2021-08-19 14:08:36 +00:00
Colin Walters
cac4f2133b
Merge pull request #2414 from lucab/ups/cli-commit-modifier-autoptr
builtins/commit: move commit modifier to auto-cleanup
2021-08-18 09:10:17 -04:00
Luca BRUNO
b079c11381
builtins/commit: move commit modifier to auto-cleanup
This reduces the usage of goto cleanup logic by porting the commit
modifier pointer to autoptr.
2021-08-18 09:06:26 +00:00
Luca BRUNO
5a3d5fb86f
builtins/commit: check for conflicting permissions options
This explicitly checks for commit command options asking for both
non-zero UID/GID and canonical permissions at the same time,
which are incompatible.
2021-08-18 08:16:26 +00:00
Jonathan Lebon
0f95e4e5ee ostree/dump: Fix free'ing a static string
Reported-by: Seth Arnold <seth.arnold@canonical.com>
2021-08-03 16:49:49 -04:00
Jonathan Lebon
738831c50b lib/sysroot: Fix error message about creating /var/lib
Reported-by: Seth Arnold <seth.arnold@canonical.com>
2021-08-03 16:49:49 -04:00
Jonathan Lebon
75b17937cf lib/sign-dummy: Handle incorrect signatures correctly
We need to check all signatures for one which passes, not just fail on
the first one.

Reported-by: Seth Arnold <seth.arnold@canonical.com>
2021-08-03 16:49:49 -04:00
刘建强
28174970c7 fix: Avoid wild pointers
Pointer command is dangerous if there is no assignment.

Log: Avoid wild pointers
2021-07-28 09:40:37 +08:00
Dan Nicholson
81df5c8aba fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys() 2021-07-15 17:03:45 -06:00
Dan Nicholson
814e481fff fixup! bin/remote: Add list-gpg-keys subcommand 2021-07-15 16:25:13 -06:00
Dan Nicholson
30c054b521 fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys() 2021-07-15 16:24:36 -06:00
Dan Nicholson
90a3bda1f8 bin/remote: Include update URLs in list-gpg-keys 2021-07-15 15:50:04 -06:00
Dan Nicholson
27dc5d7d38 lib/repo: Include WKD update URLs in GPG key listing
If the key UID contains a valid email address, include the GPG WKD
update URLs in GVariant returned by ostree_repo_remote_get_gpg_keys().
2021-07-15 15:50:04 -06:00
Dan Nicholson
4fa403aee5 libotutil: Add helper for GPG WKD update URLs
Calculate the advanced and direct update URLs for the key discovery
portion[1] of the OpenPGP Web Key Directory specification, and include
the URLs in the key listing in ostree_repo_remote_get_gpg_keys(). These
URLs can be used to locate updated GPG keys for the remote.

1. https://datatracker.ietf.org/doc/html/draft-koch-openpgp-webkey-service#section-3.1
2021-07-15 15:50:04 -06:00
Dan Nicholson
fbff05e28d libotutil: Import implementation of zbase32 encoding
This will be used to implement the PGP Web Key Directory (WKD) URL
generation. This is a slightly cleaned up implementation[1] taken from
the zbase32 author's original implementation[2]. It provides a single
zbase32_encode API to convert a set of bytes to the zbase32 encoding.

I believe this should be acceptable for inclusion in ostree. The license
in the source files is BSD style while the original repo LICENSE file
claims the Creative Commons CC0 1.0 Universal license, which is public
domain.

1. https://github.com/dbnicholson/libbase32/tree/for-ostree
2. https://github.com/zooko/libbase32
2021-07-15 15:50:04 -06:00
Dan Nicholson
74fb0c5f78 bin/remote: Add list-gpg-keys subcommand
This provides a wrapper for the `ostree_repo_remote_get_gpg_keys`
function to show the GPG keys associated with a remote. This is
particularly useful for validating that GPG key updates have been
applied. Tests are added, which checks the
`ostree_repo_remote_get_gpg_keys` API by extension.
2021-07-15 15:50:04 -06:00
Dan Nicholson
a50f6d0b9f lib/repo: Add ostree_repo_remote_get_gpg_keys()
This function enumerates the trusted GPG keys for a remote and returns
an array of `GVariant`s describing them. This is useful to see which
keys are collected by ostree for a particular remote. The same
information can be gathered with `gpg`. However, since ostree allows
multiple keyring locations, that's only really useful if you have
knowledge of how ostree collects GPG keyrings.

The format of the variants is documented in
`OSTREE_GPG_KEY_GVARIANT_FORMAT`. This format is primarily a copy of
selected fields within `gpgme_key_t` and its subtypes. The fields are
placed within vardicts rather than using a more efficient tuple of
concrete types. This will allow flexibility if more components of
`gpgme_key_t` are desired in the future.
2021-07-15 15:50:04 -06:00
Dan Nicholson
fc073654dc lib/repo: Allow preparing GPG verifier without global keyrings
Currently the verifier decides whether to include the global keyrings
based on whether the specified remote has its own keyring or not. Allow
callers to exclude the global keyrings even when that's not the case.
This will be used in a subsequent commit in order to get the GPG keys
only associated with a remote.
2021-07-15 15:50:04 -06:00
Dan Nicholson
c8715c123e lib/repo: Factor out GPG verifier preparation
In order to use the GPG verifier, it needs to be seeded with GPG keys
after instantation. Currently this is only used for verifying data, but
it will also be used for getting a list of trusted GPG keys in a
subsequent commit.
2021-07-15 15:50:04 -06:00
Dan Nicholson
dba2cdcbac lib/repo: Factor out GPG verifier key imports
Currently the verifier only imports all the GPG keys when verifying
data, but it would also be useful for inspecting the trusted keys.
2021-07-15 15:50:04 -06:00
Luca BRUNO
38c14b3745
Release 2021.3 2021-07-12 08:38:38 +00:00
Jonathan Lebon
301b52704e ostree-remount: Order before systemd-rfkill.*
The `systemd-rfkill.*` service falls in the category of early things
that need write access to `/var`, so we need to make sure we run before
or it might hit the read-only sysroot.

The long-term fix for this is
https://github.com/ostreedev/ostree/issues/2115.

Closes: https://github.com/coreos/fedora-coreos-tracker/issues/746
2021-06-22 11:22:47 -04:00
Luca BRUNO
70a8f56ce1
lib/commit: respect SOURCE_DATE_EPOCH for commit timestamp
This tweaks `ostree_repo_write_commit` so that it checks for the
envinroment variable `SOURCE_DATE_EPOCH` as a way to override
the current time, which is used as the commit timestamp.

Ref: https://reproducible-builds.org/docs/source-date-epoch/
Ref: https://reproducible-builds.org/specs/source-date-epoch/
2021-06-22 12:15:18 +00:00
Dan Nicholson
d9ef76a598 Don't fail build when systemd unit path not defined
In configure the systemd unit path is optional, but in the code it's
assumed to be defined. Add an `#ifdef` that throws an error when it's
not defined like the handling of `HAVE_LIBMOUNT` below it.
2021-06-18 12:35:41 -06:00
Jonathan Lebon
c9a318faf4
Merge pull request #2375 from cgwalters/generator-remount
Use generator to enable ostree-remount.service and ostree-finalize-staged.path
2021-06-18 10:25:35 -04:00
Colin Walters
73e3ccc401 Use generator to enable ostree-remount.service and ostree-finalize-staged.path
We struggled for a long time with enablement of our "internal units",
trying to follow the philosophy that units should only be enabled
by explicit preset.

See https://bugzilla.redhat.com/show_bug.cgi?id=1451458
and https://github.com/coreos/rpm-ostree/pull/1482
etc.

And I just saw chat (RH internal on a proprietary system sadly) where
someone hit `ostree-remount.service` not being enabled in CentOS8.

Thinking about this more, I realized we've shipped a systemd generator
for a long time and while its only role until now was to generate `var.mount`,
but by using it to force on our internal units, we don't require
people to deal with presets anymore.

Basically we're inverting things so that "if ostree= is on the kernel
cmdline, then enable our units" and not "enable our units, but have
them use ConditionKernelCmdline=ostree to skip".

Drop the weird gyrations we were doing around `ostree-finalize-staged.path`
too; forking `systemctl start` is just asking for bugs.

So after this, hopefully we won't ever again have to think about
distribution presets and our units.
2021-06-16 09:40:28 -04:00
Christian Kellner
f653545833 Fix small typo in ostree-sysroot.c 2021-06-15 23:02:10 +02:00
Colin Walters
edf7477ee9 deploy: Warn if we find content in the deployment's /var
This will be ignored, so let's make it very clear
people are doing something wrong.  Motivated by a bug
in a build pipeline that injected `/var/lib/rpm` into an ostree
commit which ended up crashing rpm-ostree because it was an empty db
which it wasn't expecting.

It *also* turns out rpm-ostree is incorrectly dumping content in the
deployment `/var` today, which is another bug.
2021-06-10 07:33:17 -04:00
Dan Nicholson
0cd9dfe815 repo: Use g_new for OstreeRepoAutoLock
GSlice is effectively deprecated and has little to no advantage over
using the system allocator on Linux.
2021-06-07 13:14:05 -06:00
Dan Nicholson
89f4ce2c1d repo: Make locking precondition failures fatal
Use `g_error` and `g_assert*` rather than `g_return*` when checking the
locking preconditions so that failures result in the program
terminating. Since this code is protecting filesystem data, we'd rather
crash than delete or corrupt data unexpectedly.

`g_error` is used when the error is due to the caller requesting an
invalid transition like attempting to pop a lock type that hasn't been
taken. It also provides a semi-useful message about what happened.
2021-06-05 09:15:34 -06:00
Dan Nicholson
ccef9784d7 repo: Make locking per-OstreeRepo
Previously each thread maintained its own lock file descriptor
regardless of whether the thread was using the same `OstreeRepo` as
another thread. This was very safe but it made certain multithreaded
procedures difficult. For example, if a main thread took an exclusive
lock and then spawned worker threads, it would deadlock if one of the
worker threads tried to acquire the lock.

This moves the file descriptor from thread local storage to the
`OstreeRepo` structure so that threads using the same `OstreeRepo` can
share the lock. A mutex guards against threads altering the lock state
concurrently.

Fixes: #2344
2021-06-05 09:15:32 -06:00
Dan Nicholson
c3ada6fa7a repo: Require lock type in ostree_repo_lock_pop
This simplifies the lock state management considerably since the
previously pushed type doesn't need to be tracked. Instead, 2 counters
are kept to track how many times each lock type has been pushed. When
the number of exclusive locks drops to 0, the lock transitions back to
shared.
2021-06-05 09:07:39 -06:00
Colin Walters
0f36d8c221 repo: Make locking APIs public
Doing anything even somewhat sophisticated requires this;
turns out our own `ostree prune` CLI wants this, e.g.
https://github.com/ostreedev/ostree/issues/2337

Closes: https://github.com/ostreedev/ostree/issues/2286
2021-06-05 09:00:21 -06:00
Timothée Ravier
02527f115e *: rename master to main in tests & examples 2021-05-07 16:55:03 +02:00
Colin Walters
0f3bccf640 pull: Cleanly error out on unknown schemes
Previous to this we'd trip an assertion `abort()` deep in the curl code if e.g.
a user did `ostree remote add foo htttp://...` etc.

Motivated by considering supporting "external remotes" where code outside
ostree does a pull, but we want to reuse the signing verification infrastructure.
2021-04-27 13:11:18 -04:00
Dan Nicholson
e660855796 tests: Test without a cache directory by default
Several tests generate summaries and then expect to use the generated
summary immediately. However, this can cause intermittent test failures
when they inadvertantly get a cached summary file. This typically
happens when the test is run on a filesystem that doesn't support user
extended attributes. In that case, the caching code can only use the
last modified time, which only has 1 second granularity. If tests don't
carefully manage the summary modification times or the repo cache then
they are likely subject to races in some test environments.

This introduces an environment variable `OSTREE_SKIP_CACHE` that
prevents the repo from using a cache directory. This is enabled by
default in tests and disabled for tests that are a explicitly trying to
test the caching behavior.

Fixes: #2313
Fixes: #2351
2021-04-19 11:09:44 -06:00
Colin Walters
6a72674ec6 Release 2021.2 2021-04-15 13:02:48 -04:00
Colin Walters
9b4bffc454 repo: Ensure we set the size for regfile inline
Need to clean up the internal APIs for this.
2021-04-09 22:29:55 +00:00
Colin Walters
6f84aff0ae repo: Add ostree_repo_write_regfile
This API is push rather than pull, which makes it much more
suitable to use cases like parsing a tar file from external
code.

Now, we have a large mess in this area internally because
the original file writing code was pull based, but static
deltas hit the same problem of wanting a push API, so I added
this special `OstreeRepoBareContent` just for writing regular
files from a push API.

Eventually...I'd like to deprecate the pull based API,
and rework things so that for regular files the push API
is the default, and then `write_content_object()` would
be split up into archive/bare cases.

In this world the `ostree_repo_write_content()` API would
then need to hackily bridge pull to push and it'd be
less efficient.

Anyways for now due to this bifurcation, this API only
works on non-archive repositories, but that's fine for
now because that's what I want for the `ostree-ext-container`
bits.
2021-04-09 21:54:44 +00:00