Commit Graph

5245 Commits

Author SHA1 Message Date
Dan Nicholson
f1b900340f ci: Disable fail-fast in GitHub Tests workflow
Don't cancel all the jobs if one distro config fails. The jobs are
mostly independent, so we do want to let the others continue in case
the failure is isolated to that particular distro configuration.
2021-06-21 12:27:01 -06:00
Dan Nicholson
b6a55ab8f3
Merge pull request #2379 from dbnicholson/gh-actions-tests
Replace Travis CI with GitHub Actions
2021-06-21 10:46:51 -06:00
Colin Walters
7ff848f101
Merge pull request #2377 from smcv/libtest-core
libtest-core: Add some improvements from bubblewrap
2021-06-21 10:12:40 -04:00
Colin Walters
a8eee7b27f
Merge pull request #2380 from dbnicholson/no-systemd-units
Don't fail build when systemd unit path not defined
2021-06-19 17:31:12 -04:00
Dan Nicholson
00f7c88a2d ci: So long, Travis CI
travis-ci.org stopped running builds on June 15, 2021. Since this
organization is very unlikely to switch to travis-ci.com, just drop the
setup. The new GitHub Actions tests completely replace it.
2021-06-18 15:30:06 -06:00
Dan Nicholson
a0012ae0ee ci: Add GitHub Actions workflow for test suite
This runs the test suite in various distros. The intention is to use
this to replace the Travis CI setup since it often has rate limit
failures.

Each configuration in the matrix runs in a Docker container, installs
system dependencies and then builds and tests ostree. The scripts are
basically copy and paste of the travis ones with some of the lesser used
features pruned out.

Some differences from the travis setup:

* OS details are gathered from `/etc/os-release` instead of being passed
  in as environment variables.

* The scripts always assume the user is root and don't try to use
  `sudo`.

* The `installcheck` test has been removed since ostree doesn't actually
  use that. It could be added to run the installed tests or
  `gnome-desktop-testing-runner` could just be called directly.

There should be enough flexibility to run other distros like Fedora,
Arch or Alpine. Another option would be to use the other build scripts
in ci/.
2021-06-18 15:30:06 -06:00
Dan Nicholson
9ff141fe80 ci: Rename GitHub Actions rust workflow metadata file
This workflow is specific to using rust and not just the general test
suite.
2021-06-18 15:24:47 -06:00
Dan Nicholson
d9ef76a598 Don't fail build when systemd unit path not defined
In configure the systemd unit path is optional, but in the code it's
assumed to be defined. Add an `#ifdef` that throws an error when it's
not defined like the handling of `HAVE_LIBMOUNT` below it.
2021-06-18 12:35:41 -06:00
Jonathan Lebon
35c5433d71
Merge pull request #2366 from pwithnall/freenode
docs: Change IRC channel to libera.chat from freenode
2021-06-18 10:27:45 -04:00
Jonathan Lebon
c9a318faf4
Merge pull request #2375 from cgwalters/generator-remount
Use generator to enable ostree-remount.service and ostree-finalize-staged.path
2021-06-18 10:25:35 -04:00
Simon McVittie
14d6e75752 libtest-core: Mention bubblewrap as a user of this file
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-18 10:14:01 +01:00
Simon McVittie
c0157f96b2 libtest-core: Update URL of rpm-ostree
Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-18 10:13:38 +01:00
Simon McVittie
d7cdde5d34 libtest-core: On failure, make it clearer what has happened
If we fail as a result of `set -x`, It's often not completely obvious
which command failed or how. Use a trap on ERR to show the command that
failed, and its exit status.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2021-06-18 09:49:53 +01:00
Alexander Larsson
c934fc8f99 libtest-core: Add assert_files_equal
[Originally from bubblewrap commits c5c999a7 "tests: test --userns"
and 3e5fe1bf "tests: Better error message if assert_files_equal fails";
separated into this commit by Simon McVittie.]
2021-06-18 09:46:31 +01:00
Colin Walters
73e3ccc401 Use generator to enable ostree-remount.service and ostree-finalize-staged.path
We struggled for a long time with enablement of our "internal units",
trying to follow the philosophy that units should only be enabled
by explicit preset.

See https://bugzilla.redhat.com/show_bug.cgi?id=1451458
and https://github.com/coreos/rpm-ostree/pull/1482
etc.

And I just saw chat (RH internal on a proprietary system sadly) where
someone hit `ostree-remount.service` not being enabled in CentOS8.

Thinking about this more, I realized we've shipped a systemd generator
for a long time and while its only role until now was to generate `var.mount`,
but by using it to force on our internal units, we don't require
people to deal with presets anymore.

Basically we're inverting things so that "if ostree= is on the kernel
cmdline, then enable our units" and not "enable our units, but have
them use ConditionKernelCmdline=ostree to skip".

Drop the weird gyrations we were doing around `ostree-finalize-staged.path`
too; forking `systemctl start` is just asking for bugs.

So after this, hopefully we won't ever again have to think about
distribution presets and our units.
2021-06-16 09:40:28 -04:00
Dan Nicholson
91286808a2
Merge pull request #2374 from gicmo/small_typo
Fix small typo in ostree-sysroot.c
2021-06-15 16:16:19 -06:00
Christian Kellner
f653545833 Fix small typo in ostree-sysroot.c 2021-06-15 23:02:10 +02:00
Jonathan Lebon
f8496098ee
Merge pull request #2373 from cgwalters/warn-nonempty-var
deploy: Warn if we find content in the deployment's /var
2021-06-10 11:12:30 -04:00
Colin Walters
edf7477ee9 deploy: Warn if we find content in the deployment's /var
This will be ignored, so let's make it very clear
people are doing something wrong.  Motivated by a bug
in a build pipeline that injected `/var/lib/rpm` into an ostree
commit which ended up crashing rpm-ostree because it was an empty db
which it wasn't expecting.

It *also* turns out rpm-ostree is incorrectly dumping content in the
deployment `/var` today, which is another bug.
2021-06-10 07:33:17 -04:00
Dan Nicholson
3d66db2bae
Merge pull request #2370 from dbnicholson/auto-lock-no-gslice
repo: Use g_new for OstreeRepoAutoLock
2021-06-07 15:04:17 -06:00
Dan Nicholson
0cd9dfe815 repo: Use g_new for OstreeRepoAutoLock
GSlice is effectively deprecated and has little to no advantage over
using the system allocator on Linux.
2021-06-07 13:14:05 -06:00
Colin Walters
5523aee082
Merge pull request #2348 from dbnicholson/mt-locking
Improve multi-threaded locking
2021-06-05 11:34:09 -04:00
Dan Nicholson
06bb56be6d tests: Add single process repo locking tests
The semantics of multiple process locking are covered by
test-concurrency.py, but the semantics of the repository locking from a
single process aren't handled there.

This checks how the repository locking is handled from a single thread
with one OstreeRepo, a single thread with multiple OstreeRepos, and
multiple threads sharing an OstreeRepo.
2021-06-05 09:15:34 -06:00
Dan Nicholson
055b263dee test-concurrency: Lower lock timeout
If there's a locking issue in this test, then it's likely not going to
resolve after a few seconds of serializing access. Lower the default 30
second lock timeout to 5 seconds to prevent the test from hanging
unnecessarily.
2021-06-05 09:15:34 -06:00
Dan Nicholson
89f4ce2c1d repo: Make locking precondition failures fatal
Use `g_error` and `g_assert*` rather than `g_return*` when checking the
locking preconditions so that failures result in the program
terminating. Since this code is protecting filesystem data, we'd rather
crash than delete or corrupt data unexpectedly.

`g_error` is used when the error is due to the caller requesting an
invalid transition like attempting to pop a lock type that hasn't been
taken. It also provides a semi-useful message about what happened.
2021-06-05 09:15:34 -06:00
Dan Nicholson
ccef9784d7 repo: Make locking per-OstreeRepo
Previously each thread maintained its own lock file descriptor
regardless of whether the thread was using the same `OstreeRepo` as
another thread. This was very safe but it made certain multithreaded
procedures difficult. For example, if a main thread took an exclusive
lock and then spawned worker threads, it would deadlock if one of the
worker threads tried to acquire the lock.

This moves the file descriptor from thread local storage to the
`OstreeRepo` structure so that threads using the same `OstreeRepo` can
share the lock. A mutex guards against threads altering the lock state
concurrently.

Fixes: #2344
2021-06-05 09:15:32 -06:00
Dan Nicholson
eb09207e1a build-sys: Bump required GLib to 2.44
This will allow usage of `GMutexLocker`. This should be available on
many older distros:

* RHEL 7 - 2.56.1
* RHEL 8 - 2.56.4
* Debian 9 stretch (oldstable) - 2.50.3
* Debian 10 buster (stable) - 2.58.3
* Ubuntu 16.04 xenial - 2.48.2
* Ubuntu 18.04 bionic - 2.56.4
2021-06-05 09:12:14 -06:00
Dan Nicholson
c3ada6fa7a repo: Require lock type in ostree_repo_lock_pop
This simplifies the lock state management considerably since the
previously pushed type doesn't need to be tracked. Instead, 2 counters
are kept to track how many times each lock type has been pushed. When
the number of exclusive locks drops to 0, the lock transitions back to
shared.
2021-06-05 09:07:39 -06:00
Colin Walters
0f36d8c221 repo: Make locking APIs public
Doing anything even somewhat sophisticated requires this;
turns out our own `ostree prune` CLI wants this, e.g.
https://github.com/ostreedev/ostree/issues/2337

Closes: https://github.com/ostreedev/ostree/issues/2286
2021-06-05 09:00:21 -06:00
Dan Nicholson
a7b590f7ae
Merge pull request #2367 from dbnicholson/gpg-2.3.0-expire-test-regression
tests/gpg: Don't assert subkey expiration when only primary expired
2021-06-03 16:56:07 -06:00
Philip Withnall
c848b71cf2 docs: Change IRC channel to libera.chat from freenode
Freenode appears to be imploding.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2021-06-01 10:21:24 +01:00
Dan Nicholson
9c20d162a4 tests/gpg: Don't assert subkey expiration when only primary expired
In gnupg 2.3.0[1], if a primary key is expired and a subkey does not
have an expiration or its expiration is older than the primary key, the
subkey's expiration will be reported as the primary's. Previously a
subkey without an expiration would not report one regardless of the
primary key's expiration.

This caused a regression in a test setting an expiration on a primary
key. The test was checking that the subkey was not expired by asserting
that there was no `Key expired` line in the signature verification
output. With gnupg 2.3.0+, it will show as expired, causing the test to
fail.

Remove the assertion since it's not consistent across gnupg versions. In
practice we don't care whether the subkey is considered expired or not
as long as the signature verification fails when the primary key is
expired.

1. https://dev.gnupg.org/T3343

Fixes: #2359
2021-05-28 15:35:58 -06:00
Colin Walters
5b8ce9e614
Merge pull request #2364 from cgwalters/fix-ci-f34
ci: Fix staged-delay to work with newer systemd
2021-05-26 08:58:27 -04:00
Colin Walters
08fc6d92e7
Merge pull request #2365 from dbnicholson/apidoc-permissions
workflow/docs: Give token write permission to push gh-pages
2021-05-25 16:37:07 -04:00
Colin Walters
6664ee4ed4 ci: Fix staged-delay to work with newer systemd
Yeah, we should stop parsing the text; I need to dig at that
at some point.
2021-05-25 16:28:32 -04:00
Dan Nicholson
bd42df4c9b workflow/docs: Give token write permission to push gh-pages
The ostree repo has read permissions set for workflows, which prevents
the documentation job from pushing the built docs to the gh-pages
branch. Raise the job's permissions to write for repo contents to allow
that.
2021-05-25 14:25:28 -06:00
Colin Walters
27a3af6b1b
Merge pull request #2363 from dbnicholson/publish-api-docs
Publish libostree API docs
2021-05-25 15:10:47 -04:00
Dan Nicholson
e19840a252 docs: Copy in API docs and add link
Make a copy of `apidoc/html` to `docs/reference` and then tell Jekyll to
include it verbatim. This will include the gtk-doc API docs on the
static site. A link is added to the main index.

A script is added to do the copy (a symlink won't do) and is setup to
run before Jekyll in the GitHub workflow. Ideally this would be a local
Jekyll plugin to make the process automatic, but the github-pages gem
doesn't allow that.
2021-05-21 10:46:49 -06:00
Dan Nicholson
19a306ecef docs: Add github workflow for building and publishing docs
This uses the Jekyll Actions GitHub action to push the rendered docs to
the gh-pages branch rather than GitHub's automated docs flow. That will
allow greater control over how the docs are generated. Pushing to the
gh-pages branch only happens on pushes to main. For pull requests, the
docs are only built.
2021-05-21 10:45:08 -06:00
Dan Nicholson
3c7449397a docs: Provide bundler setup for building site locally
This mimics the GitHub Pages environment so that you can build and serve
the site locally for testing. It's will also be required later for using
Jekyll Actions[1] instead of the automated GitHub Pages flow.

1. https://github.com/marketplace/actions/jekyll-actions
2021-05-21 10:13:15 -06:00
Dan Nicholson
e4105a0366 docs: Fix CONTRIBUTING link
This returns a 404 since the site is already generated from the docs
directory. Furthermore, the `CONTRIBUTING.md` markdown file isn't in the
generated site, just the HTML.

Instead, use jekyll's `link` tag to create the link. Unfortunately,
before jekyll 4.0 (github-pages uses 3.9), you have to prepend the base
URL.
2021-05-20 16:45:12 -06:00
Colin Walters
6c904c630f
Merge pull request #2358 from travier/rename
*: rename master branch to main
2021-05-11 11:45:38 -04:00
Timothée Ravier
a709d4f846 *: rename master branch to main (external repos) 2021-05-07 16:55:03 +02:00
Timothée Ravier
02527f115e *: rename master to main in tests & examples 2021-05-07 16:55:03 +02:00
Timothée Ravier
b8cca6cef1 *: rename master branch to main 2021-05-07 16:55:03 +02:00
Timothée Ravier
135675243a packit: update for F34, rawhide branch & master rename 2021-05-07 16:54:57 +02:00
Colin Walters
733093461c
Merge pull request #2356 from miabbott/usretc
docs: typo fix for /usr/etc
2021-05-03 13:10:09 -04:00
Micah Abbott
7893e1907b docs: typo fix for /usr/etc 2021-05-03 10:34:01 -04:00
Dan Nicholson
af723f9a2a
Merge pull request #2355 from jlebon/pr/etc-merge-moar-docs
docs: Add more details about 3-way merge
2021-04-30 10:14:52 -06:00
Jonathan Lebon
788e171b7c docs: Add more details about 3-way merge
This came up a few times so let's go into more details in the docs.
2021-04-30 10:50:15 -04:00