Commit Graph

6290 Commits

Author SHA1 Message Date
Colin Walters
42aad34e0b
Merge pull request #2831 from nanonyme/larger-buffer
Increase buffer size for create_regular_tmpfile_linkable_with_content
2023-03-15 08:32:55 -04:00
Seppo Yli-Olli
931a85d3f1 Increase buffer size for create_regular_tmpfile_linkable_with_content
The small buffer size results in really bad performance under any
FUSE-based filesystems with round-trips.
2023-03-15 12:04:02 +02:00
Colin Walters
c0427ec1ea
Merge pull request #2814 from tintou/tintou/ostree-coverage
libostree: Enhance the annotation coverage
2023-03-14 17:35:19 -04:00
Colin Walters
e7c414cbff
Merge pull request #2821 from smcv/sbin-path
tests: Ensure non-root users have access to libcap tools
2023-03-13 17:29:21 -04:00
Colin Walters
751cca707a
Merge pull request #2829 from pwithnall/fetcher-wakeup
lib/fetcher-util: Wake up main context when a request is complete
2023-03-10 14:19:18 -05:00
Colin Walters
e0961b750b
Merge pull request #2824 from cgwalters/regen-bindings
rust-bindings: Regenerate for 2023.1 && rust-bindings: `Sysroot` is `Send`
2023-03-10 14:17:09 -05:00
Colin Walters
1d71602361
Merge pull request #2828 from travier/docs-theme-update
docs: Use upstream theme & update to 0.4.1
2023-03-10 14:16:56 -05:00
Philip Withnall
2600d5c06d lib/fetcher-util: Wake up main context when a request is complete
Since the value of `data->done` is not watched by the main context, the
context doesn’t know to wake up from `g_main_context_iteration()` when
that value is changed. The code currently relies on something else
happening to wake the main context up shortly after `data->done` is set.

That doesn’t seem very reliable, so wake the main context up explicitly.

Spotted this while reading the code while trying to debug a stall with
backtrace:
```
5  0x00007ffff68bbbfb in g_main_context_iteration (context=0x60f000136900, may_block=1) at ../../source/glib/glib/gmain.c:4343
6  0x00007fffdc0e50db in _ostree_fetcher_mirrored_request_to_membuf_once (fetcher=0x604001138c10, mirrorlist=0x603001df18b0, filename=0x7fffdc1049e7 "summary.sig", flags=OSTREE_FETCHER_REQUEST_OPTIONAL_CONTENT, if_none_match=0x602002f3b7f0 "\"640a49ff-250\"", if_modified_since=1678395903, out_contents=0x7fffdadd0e80, out_not_modified=0x7fffdadd0e38, out_etag=0x7fffdadd0e28, out_last_modified=0x7fffdadd0e00, max_size=10485760, cancellable=0x6060004bd720, error=0x7fffdadd0ca0) at src/libostree/ostree-fetcher-util.c:95
7  0x00007fffdc0e52e0 in _ostree_fetcher_mirrored_request_to_membuf (fetcher=0x604001138c10, mirrorlist=0x603001df18b0, filename=0x7fffdc1049e7 "summary.sig", flags=OSTREE_FETCHER_REQUEST_OPTIONAL_CONTENT, if_none_match=0x602002f3b7f0 "\"640a49ff-250\"", if_modified_since=1678395903, n_network_retries=5, out_contents=0x7fffdadd0e80, out_not_modified=0x7fffdadd0e38, out_etag=0x7fffdadd0e28, out_last_modified=0x7fffdadd0e00, max_size=10485760, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-fetcher-util.c:155
8  0x00007fffdc08d937 in _ostree_preload_metadata_file (self=0x61600057bd80, fetcher=0x604001138c10, mirrorlist=0x603001df18b0, filename=0x7fffdc1049e7 "summary.sig", is_metalink=0, if_none_match=0x602002f3b7f0 "\"640a49ff-250\"", if_modified_since=1678395903, n_network_retries=5, out_bytes=0x7fffdadd0e80, out_not_modified=0x7fffdadd0e38, out_etag=0x7fffdadd0e28, out_last_modified=0x7fffdadd0e00, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-repo-pull.c:3329
9  0x00007fffdc099712 in ostree_repo_remote_fetch_summary_with_options (self=0x61600057bd80, name=0x6020007f4fd0 "eos-apps", options=0x0, out_summary=0x7fffdadd0f88, out_signatures=0x7fffdadd0f80, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-repo-pull.c:6675
10 0x00007fffdc06887f in ostree_repo_remote_fetch_summary (self=0x61600057bd80, name=0x6020007f4fd0 "eos-apps", out_summary=0x7fffdadd0f88, out_signatures=0x7fffdadd0f80, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at src/libostree/ostree-repo.c:2706
11 0x00007fffdc18f5de in flatpak_dir_remote_fetch_summary (self=0x60c00577e640, name_or_uri=0x6020007f4fd0 "eos-apps", only_cached=0, out_summary=0x7fffdadd10f0, out_summary_sig=0x7fffdadd10e8, cancellable=0x6060004bd720, error=0x7fffdadd10c0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:12235
12 0x00007fffdc1918cb in _flatpak_dir_get_remote_state (self=0x60c00577e640, remote_or_uri=0x6020007f4fd0 "eos-apps", optional=1, local_only=0, only_cached=0, opt_summary_is_index=0, opt_summary=0x0, opt_summary_sig=0x0, cancellable=0x6060004bd720, error=0x7fffdadd15a0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:12789
13 0x00007fffdc19206d in flatpak_dir_get_remote_state_optional (self=0x60c00577e640, remote=0x6020007f4fd0 "eos-apps", only_cached=0, cancellable=0x6060004bd720, error=0x7fffdadd15a0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:12953
14 0x00007fffdc1784f2 in flatpak_dir_update_appstream (self=0x60c00577e640, remote=0x6020007f4fd0 "eos-apps", arch=0x7fffdc239f30 "x86_64", out_changed=0x0, progress=0x6110007479c0, cancellable=0x6060004bd720, error=0x7fffdadd15a0) at /opt/gnome/source/flatpak/common/flatpak-dir.c:5182
```

I don’t think it’ll fix it (calling `g_main_context_wakeup()` on the
context in the debugger didn’t help), but it can’t hurt.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2023-03-10 11:54:34 +00:00
Timothée Ravier
375bef3a4d docs: Use upstream theme & update to 0.4.1
Use a fixed tag for the theme so that we can directly pull it from
upstream and skip vendoring the theme in the coreos org.
2023-03-10 12:17:16 +01:00
Colin Walters
b00f27d476 rust-bindings: Sysroot is Send
I want to spawn tokio worker threads referencing sysroot objects
in bootc.

Just like the repo, there's nothing thread-local about it.
2023-03-04 10:54:48 -05:00
Colin Walters
be0dbf02d7 rust-bindings: Regenerate for 2023.1
Still need to automate this...
2023-03-04 10:54:04 -05:00
Simon McVittie
3500ad0443 tests: Ensure non-root users have access to libcap tools
On Debian systems, by default only root has /{usr/,}sbin in PATH.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2023-02-27 12:40:05 +00:00
Joseph Marrero Corchado
a43b143811
Merge pull request #2818 from cgwalters/release-rust
Release rust/ bindings 0.17.1
2023-02-20 17:13:44 -05:00
Jonathan Lebon
4f87e8b6ef
Merge pull request #2755 from cgwalters/use-new-selinux-arg 2023-02-17 16:47:04 -05:00
Colin Walters
31d5f9e13e Release rust/ bindings 0.17.1 2023-02-17 13:06:14 -05:00
Jonathan Lebon
c2974d0b35
Merge pull request #2817 from cgwalters/release 2023-02-17 12:33:12 -05:00
Colin Walters
15b2c4f45c configure: post-release version bump 2023-02-17 11:31:27 -05:00
Colin Walters
cd100df413 Release 2023.1 2023-02-17 11:31:27 -05:00
Corentin Noël
fa750fba80 libostree: Enhance the annotation coverage
Increase the coverage of the GObject Introspection annotation and most notably
replace the use of "allow-none" with "out" parameters to "nullable" or "optional"
as the previous annotation is deprecated and ambiguous.
2023-02-09 16:28:53 +01:00
Colin Walters
8025a83b6d
Merge pull request #1946 from dbnicholson/ostree-metadata-api
ostree-metadata commit API
2023-02-09 08:24:58 -05:00
Colin Walters
cff0a1730a
Merge pull request #2811 from dbnicholson/cli-polish
Various CLI improvements
2023-02-09 08:10:36 -05:00
Luca Bruno
51f3d03d9b
Merge pull request #2792 from lucab/ups/annotation-nullable
lib/sysroot-upgrader: add some 'nullable' annotations
2023-02-08 17:27:45 +00:00
Luca BRUNO
23fe6ada31
lib/sysroot-upgrader: add some 'nullable' annotations 2023-02-08 16:28:09 +00:00
Dan Nicholson
70ec92e1cd bin/remote-summary: Add options to show metadata 2023-02-07 22:59:30 -07:00
Dan Nicholson
b49f93aa95 bin/summary: Add options to show metadata 2023-02-07 22:59:30 -07:00
Dan Nicholson
d0f2c5d361 ostree/dump: Add support for summary metadata keys
Like with commit metadata, it's useful to list and print metadata keys
are in a summary file. This adds helpers to do that.
2023-02-07 22:59:30 -07:00
Dan Nicholson
86701f0f84 bin/show: Add options to list metadata keys
While `--print-metadata-key` is very useful, it's not that helpful if
you don't know what the keys are.
2023-02-07 22:59:30 -07:00
Dan Nicholson
008c5a9540 bin/remote-refs: Add option to print revisions
The only other way to get the remote ref revision from the CLI is to
scrape the output of `ostree remote summary` or pull the commit. The
revision is already there in the summary's ref map, so might as well add
an option to show it.
2023-02-07 22:59:30 -07:00
Dan Nicholson
90dd45fb27 bin/refs: Add option to print revisions
Allow printing the revision along with the ref. This is very convenient
for looping over the refs in a shell as well as for quickly seeing which
refs are pointed to the same commit.
2023-02-07 22:59:30 -07:00
Dan Nicholson
6b0f42ae37 bin/refs: Sort listing
The output is much more readable sorted. I can't think of any reason
you'd want it unsorted (which is essentially dentry order).
2023-02-07 22:59:30 -07:00
Dan Nicholson
2b8707b010 bin: Cleanup documentation of --cache-dir option
In all cases documentation was missing from the manual and the bash
completion was incorrectly assigning it as a boolean option.
2023-02-07 22:59:20 -07:00
Dan Nicholson
6f8669331d repo: Prevent publishing summary without matching signature
Use a temporary directory for the summary and signature file in
`ostree_repo_regenerate_metadata` so that the summary file isn't
published if signing fails. This prevents publishing a summary without a
signature file or leaving a mismatched signature file in place.
2023-02-07 14:50:47 -07:00
Dan Nicholson
6cc75a6c1e lib: Provide internal summary signing with separate directory
Refactor the summary signing APIs to use internal versions where the
directory fd containing the summary can be found. The existing signing
APIs still uses the repo directory fd, but this will allow using a
temporary directory for the summary and signature in the new metadata
generating API.
2023-02-07 14:50:47 -07:00
Dan Nicholson
b889481801 repo: Create metadata commit in ostree_repo_regenerate_metadata
Rather than creating the `ostree-metadata` commit in the summary
builtin, do it in the new `ostree_repo_regenerate_metadata` API. The
commit contents are unchanged and the commit is generated before the
summary as before. To keep from triggering an extra summary update,
automatic summary updating is disabled in the transaction.

Since the summary builtin was already using the new API, it will
continue to generate the `ostree-metadata` commit when the repo has a
collection ID. However, the `ostree_repo_regenerate_summary` API will
still only generate the summary file as before.
2023-02-07 14:50:47 -07:00
Dan Nicholson
b2f02337f8 commit: Allow skipping automatic summary generation
If a commit is being made during summary generation, then it would
trigger the summary to be generated again. That's either unwanted busy
work or could result in an infinite loop. Add a boolean in
`OstreeRepoTxn` to disable automatic summary generation as seen fit.
2023-02-07 14:50:47 -07:00
Dan Nicholson
582d7071d2 bin/summary: Use ostree_repo_regenerate_metadata
Call `ostree_repo_regenerate_metadata` with the provided GPG and sign
keys to generate and sign the summary in one call. This changes the
handling when GPGME support is disabled but GPG keys are supplied.
Instead of silently ignoring the option, it will now error. IMO that's
better as callers would otherwise not know that the summary is not GPG
signed.
2023-02-07 14:50:47 -07:00
Dan Nicholson
1384267973 repo: Add API to regenerate and sign repo metadata
Currently this is just a wrapper around regenerating and signing the
summary in one call, but later it will be used to also generate the
`ostree-metadata` commit if the repo has a collection ID.
2023-02-07 14:50:47 -07:00
Dan Nicholson
b54f486489 Strip trailing whitespace on all C files
My editor started following the configuration in .editorconfig and is
applying this rule to many files I'm editing. Let's just get this over
with and strip everything. This was done like so:

  git ls-files | grep '\.[ch]$' | xargs sed -ri 's/\s+$//'
2023-02-07 14:50:08 -07:00
Colin Walters
9cf98599c0
Merge pull request #2810 from tmdh/fix-docs
docs: Fix link to ostree-rs
2023-02-05 08:46:44 -05:00
Tareque Md Hanif
ef1277023c docs: Fix link to ostree-rs 2023-02-05 11:49:41 +06:00
Colin Walters
8616c4da56
Merge pull request #2808 from jlebon/pr/lock-prune
ostree/prune: Calculate reachability under exclusive lock
2023-02-01 08:48:14 -05:00
Jonathan Lebon
fb63f7aba8 ostree/prune: Calculate reachability under exclusive lock
When we calculate the reachability set in `ostree prune`, we do this
without any locking. This means that between the time we build the set
and when we call `ostree_repo_prune_from_reachable`, new content
might've been added. This then causes us to immediately prune that
content since it's not in the now outdated set.

Fix this by calculating the set under an exclusive lock.

I think this is what happened in
https://github.com/fedora-silverblue/issue-tracker/issues/405. While
the pruner was running, the `new-updates-sync` script[1] was importing
content into the repo. The newly imported commits were immediately
deleted by the many `ostree prune --commit-only` calls the pruner does,
breaking the refs.

[1] https://pagure.io/fedora-infra/ansible/blob/35b35127e444/f/roles/bodhi2/backend/files/new-updates-sync#_18
2023-01-30 15:08:27 -05:00
Colin Walters
af505336c2
Merge pull request #2795 from cgwalters/fix-docsrs-build
bindings: Use default for uninitialized fields in checkout opts
2023-01-27 08:18:06 -05:00
Colin Walters
f913e53c2e bindings: Consistently check for dox feature
This fixes the build on docs.rs:

https://docs.rs/crate/ostree-ext/0.10.2/builds/699039
```
[INFO] running `Command { std: "docker" "start" "-a" "7d90466f040d987cbcc393bc89f5f0688f052f49e2339c354dd731fdaf0f9667", kill_on_drop: false }`
[INFO] [stderr]     Checking ostree v0.17.0
[INFO] [stderr] error[E0063]: missing field `process_passthrough_whiteouts` in initializer of `repo_checkout_at_options::RepoCheckoutAtOptions`
[INFO] [stderr]   --> /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/ostree-0.17.0/rust-bindings/src/repo_checkout_at_options/mod.rs:61:9
[INFO] [stderr]    |
[INFO] [stderr] 61 |         RepoCheckoutAtOptions {
[INFO] [stderr]    |         ^^^^^^^^^^^^^^^^^^^^^ missing `process_passthrough_whiteouts`
```
2023-01-26 14:34:32 -05:00
Colin Walters
350809db42
Merge pull request #2807 from dbnicholson/hyper-staticfile-yanked
tests/inst: Update hyper-staticfile to 0.9.4
2023-01-25 19:46:34 -05:00
Dan Nicholson
4fd36337b1 tests/inst: Update hyper-staticfile to 0.9.4
Due to some vulnerabilities[1][2], all versions prior to 0.9.4 have been
yanked from crates.io. There have been some API changes since 0.6.0, but
none of them affect the simple `hyper_staticfile::Static` usage here.

1. https://rustsec.org/advisories/RUSTSEC-2022-0069.html
2. https://rustsec.org/advisories/RUSTSEC-2022-0072.html
2023-01-24 13:20:10 -07:00
Colin Walters
15a7ed50c5
Merge pull request #2793 from ericcurtin/aboot-bootloader-support
bootloader: Add an aboot (Android) bootloader backend
2023-01-05 13:26:22 -05:00
Colin Walters
139f736f79
Merge pull request #2799 from jeamland/replace-radix64
Replace the radix64 crate with base64
2023-01-03 10:46:25 -05:00
Benno Rice
b4667c1b31 Replace the radix64 crate with base64
The radix64 crate was last updated over 3 years ago. On the other
hand the base64 crate appears to be far more actively maintained,
supports all the needed features and has a few orders of magnitude
more users.
2023-01-03 12:02:57 +11:00
Colin Walters
2d4ee168d7
Merge pull request #2798 from akiernan/support-musl-erange
tests: Handle musl's ERANGE mapping
2023-01-02 12:23:59 -05:00