Commit Graph

5120 Commits

Author SHA1 Message Date
Dan Nicholson
89f4ce2c1d repo: Make locking precondition failures fatal
Use `g_error` and `g_assert*` rather than `g_return*` when checking the
locking preconditions so that failures result in the program
terminating. Since this code is protecting filesystem data, we'd rather
crash than delete or corrupt data unexpectedly.

`g_error` is used when the error is due to the caller requesting an
invalid transition like attempting to pop a lock type that hasn't been
taken. It also provides a semi-useful message about what happened.
2021-06-05 09:15:34 -06:00
Dan Nicholson
ccef9784d7 repo: Make locking per-OstreeRepo
Previously each thread maintained its own lock file descriptor
regardless of whether the thread was using the same `OstreeRepo` as
another thread. This was very safe but it made certain multithreaded
procedures difficult. For example, if a main thread took an exclusive
lock and then spawned worker threads, it would deadlock if one of the
worker threads tried to acquire the lock.

This moves the file descriptor from thread local storage to the
`OstreeRepo` structure so that threads using the same `OstreeRepo` can
share the lock. A mutex guards against threads altering the lock state
concurrently.

Fixes: #2344
2021-06-05 09:15:32 -06:00
Dan Nicholson
eb09207e1a build-sys: Bump required GLib to 2.44
This will allow usage of `GMutexLocker`. This should be available on
many older distros:

* RHEL 7 - 2.56.1
* RHEL 8 - 2.56.4
* Debian 9 stretch (oldstable) - 2.50.3
* Debian 10 buster (stable) - 2.58.3
* Ubuntu 16.04 xenial - 2.48.2
* Ubuntu 18.04 bionic - 2.56.4
2021-06-05 09:12:14 -06:00
Dan Nicholson
c3ada6fa7a repo: Require lock type in ostree_repo_lock_pop
This simplifies the lock state management considerably since the
previously pushed type doesn't need to be tracked. Instead, 2 counters
are kept to track how many times each lock type has been pushed. When
the number of exclusive locks drops to 0, the lock transitions back to
shared.
2021-06-05 09:07:39 -06:00
Colin Walters
0f36d8c221 repo: Make locking APIs public
Doing anything even somewhat sophisticated requires this;
turns out our own `ostree prune` CLI wants this, e.g.
https://github.com/ostreedev/ostree/issues/2337

Closes: https://github.com/ostreedev/ostree/issues/2286
2021-06-05 09:00:21 -06:00
Dan Nicholson
a7b590f7ae
Merge pull request #2367 from dbnicholson/gpg-2.3.0-expire-test-regression
tests/gpg: Don't assert subkey expiration when only primary expired
2021-06-03 16:56:07 -06:00
Dan Nicholson
9c20d162a4 tests/gpg: Don't assert subkey expiration when only primary expired
In gnupg 2.3.0[1], if a primary key is expired and a subkey does not
have an expiration or its expiration is older than the primary key, the
subkey's expiration will be reported as the primary's. Previously a
subkey without an expiration would not report one regardless of the
primary key's expiration.

This caused a regression in a test setting an expiration on a primary
key. The test was checking that the subkey was not expired by asserting
that there was no `Key expired` line in the signature verification
output. With gnupg 2.3.0+, it will show as expired, causing the test to
fail.

Remove the assertion since it's not consistent across gnupg versions. In
practice we don't care whether the subkey is considered expired or not
as long as the signature verification fails when the primary key is
expired.

1. https://dev.gnupg.org/T3343

Fixes: #2359
2021-05-28 15:35:58 -06:00
Colin Walters
5b8ce9e614
Merge pull request #2364 from cgwalters/fix-ci-f34
ci: Fix staged-delay to work with newer systemd
2021-05-26 08:58:27 -04:00
Colin Walters
08fc6d92e7
Merge pull request #2365 from dbnicholson/apidoc-permissions
workflow/docs: Give token write permission to push gh-pages
2021-05-25 16:37:07 -04:00
Colin Walters
6664ee4ed4 ci: Fix staged-delay to work with newer systemd
Yeah, we should stop parsing the text; I need to dig at that
at some point.
2021-05-25 16:28:32 -04:00
Dan Nicholson
bd42df4c9b workflow/docs: Give token write permission to push gh-pages
The ostree repo has read permissions set for workflows, which prevents
the documentation job from pushing the built docs to the gh-pages
branch. Raise the job's permissions to write for repo contents to allow
that.
2021-05-25 14:25:28 -06:00
Colin Walters
27a3af6b1b
Merge pull request #2363 from dbnicholson/publish-api-docs
Publish libostree API docs
2021-05-25 15:10:47 -04:00
Dan Nicholson
e19840a252 docs: Copy in API docs and add link
Make a copy of `apidoc/html` to `docs/reference` and then tell Jekyll to
include it verbatim. This will include the gtk-doc API docs on the
static site. A link is added to the main index.

A script is added to do the copy (a symlink won't do) and is setup to
run before Jekyll in the GitHub workflow. Ideally this would be a local
Jekyll plugin to make the process automatic, but the github-pages gem
doesn't allow that.
2021-05-21 10:46:49 -06:00
Dan Nicholson
19a306ecef docs: Add github workflow for building and publishing docs
This uses the Jekyll Actions GitHub action to push the rendered docs to
the gh-pages branch rather than GitHub's automated docs flow. That will
allow greater control over how the docs are generated. Pushing to the
gh-pages branch only happens on pushes to main. For pull requests, the
docs are only built.
2021-05-21 10:45:08 -06:00
Dan Nicholson
3c7449397a docs: Provide bundler setup for building site locally
This mimics the GitHub Pages environment so that you can build and serve
the site locally for testing. It's will also be required later for using
Jekyll Actions[1] instead of the automated GitHub Pages flow.

1. https://github.com/marketplace/actions/jekyll-actions
2021-05-21 10:13:15 -06:00
Dan Nicholson
e4105a0366 docs: Fix CONTRIBUTING link
This returns a 404 since the site is already generated from the docs
directory. Furthermore, the `CONTRIBUTING.md` markdown file isn't in the
generated site, just the HTML.

Instead, use jekyll's `link` tag to create the link. Unfortunately,
before jekyll 4.0 (github-pages uses 3.9), you have to prepend the base
URL.
2021-05-20 16:45:12 -06:00
Colin Walters
6c904c630f
Merge pull request #2358 from travier/rename
*: rename master branch to main
2021-05-11 11:45:38 -04:00
Timothée Ravier
a709d4f846 *: rename master branch to main (external repos) 2021-05-07 16:55:03 +02:00
Timothée Ravier
02527f115e *: rename master to main in tests & examples 2021-05-07 16:55:03 +02:00
Timothée Ravier
b8cca6cef1 *: rename master branch to main 2021-05-07 16:55:03 +02:00
Timothée Ravier
135675243a packit: update for F34, rawhide branch & master rename 2021-05-07 16:54:57 +02:00
Colin Walters
733093461c
Merge pull request #2356 from miabbott/usretc
docs: typo fix for /usr/etc
2021-05-03 13:10:09 -04:00
Micah Abbott
7893e1907b docs: typo fix for /usr/etc 2021-05-03 10:34:01 -04:00
Dan Nicholson
af723f9a2a
Merge pull request #2355 from jlebon/pr/etc-merge-moar-docs
docs: Add more details about 3-way merge
2021-04-30 10:14:52 -06:00
Jonathan Lebon
788e171b7c docs: Add more details about 3-way merge
This came up a few times so let's go into more details in the docs.
2021-04-30 10:50:15 -04:00
Jonathan Lebon
89481ca4be
Merge pull request #2354 from cgwalters/fix-gh-action
ci: Fix GH action for rustfmt
2021-04-27 15:23:02 -04:00
Jonathan Lebon
60c14240ab
Merge pull request #2353 from cgwalters/uri-scheme-hardening
pull: Cleanly error out on unknown schemes
2021-04-27 14:09:10 -04:00
Colin Walters
0f3bccf640 pull: Cleanly error out on unknown schemes
Previous to this we'd trip an assertion `abort()` deep in the curl code if e.g.
a user did `ostree remote add foo htttp://...` etc.

Motivated by considering supporting "external remotes" where code outside
ostree does a pull, but we want to reuse the signing verification infrastructure.
2021-04-27 13:11:18 -04:00
Colin Walters
73a896e62c ci: Fix GH action for rustfmt
Since we now have a toplevel workspace, just use that.
2021-04-27 13:10:07 -04:00
Colin Walters
0fcf4a3f30
Merge pull request #2352 from dbnicholson/skip-summary-cache-envvar
tests: Test without a cache directory by default
2021-04-20 08:57:28 -04:00
Dan Nicholson
e660855796 tests: Test without a cache directory by default
Several tests generate summaries and then expect to use the generated
summary immediately. However, this can cause intermittent test failures
when they inadvertantly get a cached summary file. This typically
happens when the test is run on a filesystem that doesn't support user
extended attributes. In that case, the caching code can only use the
last modified time, which only has 1 second granularity. If tests don't
carefully manage the summary modification times or the repo cache then
they are likely subject to races in some test environments.

This introduces an environment variable `OSTREE_SKIP_CACHE` that
prevents the repo from using a cache directory. This is enabled by
default in tests and disabled for tests that are a explicitly trying to
test the caching behavior.

Fixes: #2313
Fixes: #2351
2021-04-19 11:09:44 -06:00
Colin Walters
2458df446f
Merge pull request #2350 from smcv/trap-err
libtest: On failure, make it clearer what has happened
2021-04-18 08:54:49 -04:00
Simon McVittie
9d48d39a4b libtest: On failure, make it clearer what has happened
If we fail as a result of `set -x`, It's often not completely obvious
which command failed or how. Use a trap on ERR to show the command that
failed, and its exit status.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-04-18 12:17:00 +01:00
Colin Walters
e4d42be31f
Merge pull request #2343 from cgwalters/cargo-workspace
build-sys: Add toplevel workspace Cargo.toml
2021-04-15 19:32:11 -04:00
Colin Walters
09c5578a15
Merge pull request #2346 from cgwalters/release-2021-2
Release 2021.2
2021-04-15 15:40:30 -04:00
Colin Walters
3595414c86 configure: post-release version bump 2021-04-15 13:02:48 -04:00
Colin Walters
6a72674ec6 Release 2021.2 2021-04-15 13:02:48 -04:00
Colin Walters
7310203ce9 tests/inst: Make nondestructive tests runnable as unit tests
Ideally in the future we change more of our unit tests to
support running installed; we've tried this in the past with
https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests

I'd like to pick that back up again.  This takes a step
towards that by having our Rust tests.

To make this even easier, add a `tests/run-installed`
which runs the installed tests (uninstalled, confusingly
but conveniently for now).
2021-04-15 12:33:34 -04:00
Colin Walters
46a0911c6d build-sys: Add toplevel workspace Cargo.toml
rust-analyzer is happier with this because it understands
the project structure out of the box.

We aren't actually again adding a dependency on Rust/cargo in the core,
this is only used to make `cargo build` work out of the box to build
the Rust test code.
2021-04-15 12:33:34 -04:00
Luca Bruno
36693f064c
Merge pull request #2336 from bgilbert/edit
docs: fix "Edit this page on GitHub" links
2021-04-13 12:14:52 +00:00
Jonathan Lebon
3991009c93
Merge pull request #2335 from cgwalters/write-regfile-api 2021-04-12 16:48:27 -04:00
Benjamin Gilbert
388764f1ed docs: fix "Edit this page on GitHub" links 2021-04-09 23:00:33 -04:00
Colin Walters
9b4bffc454 repo: Ensure we set the size for regfile inline
Need to clean up the internal APIs for this.
2021-04-09 22:29:55 +00:00
Colin Walters
2cc11b7808 tests: More tests for inline writing
I'm hitting a bug, add more tests.
2021-04-09 21:54:44 +00:00
Colin Walters
6f84aff0ae repo: Add ostree_repo_write_regfile
This API is push rather than pull, which makes it much more
suitable to use cases like parsing a tar file from external
code.

Now, we have a large mess in this area internally because
the original file writing code was pull based, but static
deltas hit the same problem of wanting a push API, so I added
this special `OstreeRepoBareContent` just for writing regular
files from a push API.

Eventually...I'd like to deprecate the pull based API,
and rework things so that for regular files the push API
is the default, and then `write_content_object()` would
be split up into archive/bare cases.

In this world the `ostree_repo_write_content()` API would
then need to hackily bridge pull to push and it'd be
less efficient.

Anyways for now due to this bifurcation, this API only
works on non-archive repositories, but that's fine for
now because that's what I want for the `ostree-ext-container`
bits.
2021-04-09 21:54:44 +00:00
Jonathan Lebon
020f6cb652
Merge pull request #2334 from cgwalters/write-symlink
repo: Add ostree_repo_write_symlink
2021-04-09 13:35:06 -04:00
Colin Walters
fce69cdf70 repo: Add ostree_repo_write_symlink
Continuation of the addition of `ostree_repo_write_regfile_inline()`.
This will be helpful for ostree-rs-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.
2021-04-08 21:10:00 +00:00
Colin Walters
9332955b5f
Merge pull request #2327 from cgwalters/writing-apis
repo: Add ostree_repo_write_regfile_inline
2021-04-08 17:09:11 -04:00
Colin Walters
84a7ae506b
Merge pull request #2333 from dbnicholson/travis-32bit
Add back a travis 32 bit build
2021-04-08 14:23:22 -04:00
Dan Nicholson
ed16443c82 travis: Add back a 32 bit i386 build
Although people are likely not deploying on 32 bit x86 anymore,
deploying on 32 bit armv7 is still often used. Add back an i386 build on
debian to try to catch 32 bit bugs in CI.
2021-04-08 11:49:19 -06:00