Commit Graph

6552 Commits

Author SHA1 Message Date
Colin Walters
f56131976e Add clang formatting infrastructure
Prep for reformatting the codebase.
2023-05-01 15:37:56 -04:00
Colin Walters
d333f20b4a build-sys: Squash automake conditional warning re .PHONY 2023-05-01 14:21:37 -04:00
Colin Walters
919212d73b
Merge pull request #2847 from jlebon/pr/calculate-and-cleanup
lib/sysroot-deploy: Add experimental support for automatic early prune
2023-05-01 13:25:25 -04:00
Jonathan Lebon
c561e6179e lib/sysroot-deploy: Add experimental support for automatic early prune
During the early design of FCOS and RHCOS, we chose a value of 384M
for the boot partition. This turned out to be too small: some arches
other than x86_64 have larger initrds, kernel binaries, or additional
artifacts (like device tree blobs). We'll likely bump the boot partition
size in the future, but we don't want to abandon all the nodes deployed
with the current size.[[1]]

Because stale entries in `/boot` are cleaned up after new entries are
written, there is a window in the update process during which the bootfs
temporarily must host all the `(kernel, initrd)` pairs for the union of
current and new deployments.

This patch determines if the bootfs is capable of holding all the
pairs. If it can't but it could hold all the pairs from just the new
deployments, the outgoing deployments (e.g. rollbacks) are deleted
*before* new deployments are written. This is done by updating the
bootloader in two steps to maintain atomicity.

Since this is a lot of new logic in an important section of the
code, this feature is gated for now behind an environment variable
(`OSTREE_ENABLE_AUTO_EARLY_PRUNE`). Once we gain more experience with
it, we can consider turning it on by default.

This strategy increases the fallibility of the update system since one
would no longer be able to rollback to the previous deployment if a bug
is present in the bootloader update logic after auto-pruning (see [[2]]
and following). This is however mitigated by the fact that the heuristic
is opportunistic: the rollback is pruned *only if* it's the only way for
the system to update.

[1]: https://github.com/coreos/fedora-coreos-tracker/issues/1247
[2]: https://github.com/ostreedev/ostree/issues/2670#issuecomment-1179341883

Closes: #2670
2023-05-01 12:12:03 -04:00
Joseph Marrero Corchado
8b2ba4f401
Merge pull request #2853 from cgwalters/bump-msrv
rust: Bump MSRV to 1.64
2023-04-30 13:42:43 -04:00
Colin Walters
1b43b39574 rust: Bump MSRV to 1.64
Since a dependency `winnow` bumped to this in a recent update.
2023-04-29 17:11:58 -04:00
Colin Walters
b4dcfd7ede
Merge pull request #2849 from pwithnall/request-debugging
lib/fetcher: Add some debugging messages to the libsoup request path
2023-04-26 16:46:43 -04:00
Philip Withnall
77e334e5b4 lib/fetcher: Add some debugging messages to the libsoup request path
This will help with debugging stalled requests in future, such as
issue #605.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>

Helps: #605
2023-04-26 15:44:03 -04:00
Joseph Marrero Corchado
9c548d832c
Merge pull request #2850 from cgwalters/ci-no-error-deprecated-declarations
ci: Turn off errors for deprecated-declarations
2023-04-26 10:50:50 -04:00
Colin Walters
64aa5af236 ci: Drop workaround for fedora-release-container
I think this isn't necessary anymore, and is now actively broken
with f38.
2023-04-26 09:46:57 -04:00
Colin Walters
3391d8f3f0 ci: Turn off errors for deprecated-declarations
Having `-Werror` on in CI only by default has generally worked OK,
but I don't think it's worth trying to immediately scramble to port
when they deprecate APIs.

Motivated in this case by
```
 src/libostree/ostree-fetcher-curl.c: In function 'initiate_next_curl_request':
src/libostree/ostree-fetcher-curl.c:876:3: error: 'CURLOPT_PROTOCOLS' is deprecated: since 7.85.0. Use CURLOPT_PROTOCOLS_STR [-Werror=deprecated-declarations]
  876 |   rc = curl_easy_setopt (req->easy, CURLOPT_PROTOCOLS, (long)(CURLPROTO_HTTP | CURLPROTO_HTTPS | CURLPROTO_FILE));
      |   ^~
```
2023-04-26 09:37:22 -04:00
Jonathan Lebon
fd3304e987 tests/kolainst: Add make localinstall
When hacking and testing locally with `cosa build-fast` and `kola run`,
I prefer to leave testing framework stuff within the work directory
rather than installed in my pet container. Add a `localinstall` target
for this which puts the tests in `tests/kola`. Then a simple `kola run`
will pick it up.
2023-04-14 15:19:17 -04:00
Jonathan Lebon
771deb55d1 tests/kola: delete unused .gitignore
AFAICT, I don't see how `runkola.sh` or the Makefile in `tests/kolainst`
can create files in `tests/kola` since it's geared towards installing
under `/usr`.
2023-04-14 15:19:17 -04:00
Jonathan Lebon
45772ede3e lib/sysroot-deploy: Nuke finalize-failure.stamp on successful finalization
In the unusual case where one is manually finalizing staged deployments,
as can happen in testing, we expect a successful finalization to remove
the failure stamp file.
2023-04-14 15:19:17 -04:00
Colin Walters
90f7c7bd9f
Merge pull request #2848 from jlebon/pr/calculate-and-cleanup-prep
Prep patches for automatic early prune
2023-04-14 14:44:38 -04:00
Jonathan Lebon
462d413d2e libotutil: add utility functions for calculating directory size
Prep for future patch.
2023-04-14 09:57:16 -04:00
Jonathan Lebon
a0681cd201 lib/sysroot-cleanup: Make bootfs cleanup function global
Prep for future patch.
2023-04-14 09:57:16 -04:00
Jonathan Lebon
bf974e32fc lib/sysroot-cleanup: Factor out bootfs cleanup
Crawling through the bootfs and the deployment dirs was already mostly
separate. The only inefficiency here is that we now iterate over the
array of active deployments twice when building the hash tables. No
functional change otherwise.

Prep for future patch.
2023-04-14 09:57:16 -04:00
Jonathan Lebon
49eb8c04d5 lib/sysroot-cleanup: Drop dead code
We weren't actually using this `stbuf` anywhere.
2023-04-14 09:57:16 -04:00
Jonathan Lebon
898b1fdc70 lib/sysroot-cleanup: Make some static utility functions global
Prep for future patch.
2023-04-14 09:57:16 -04:00
Jonathan Lebon
91b6e5ded6 lib/sysroot-cleanup: Convert bootdir listing to dfd-relative
No functional change.
2023-04-14 09:55:38 -04:00
Dan Nicholson
e509b24a01
Merge pull request #2547 from q66/soup3
port to libsoup3
2023-04-13 10:05:01 -06:00
Dan Nicholson
a0407225f1 fetcher/soup3: Rewrite without threads
soup3 works best using only the async API from a single thread[1].
Rework the fetcher to stop using worker threads. In order to maximize
session usage across requests, sessions will be reused for each main
context.

1. https://libsoup.org/libsoup-3.0/client-thread-safety.html
2023-04-13 09:00:23 -06:00
Dan Nicholson
945f0a8602 ci: Add test configuration with soup3
This needs to be on Debian testing for now since bullseye doesn't have
soup3.
2023-04-12 22:34:23 -06:00
Daniel Kolesa
d0ea2db430 fetcher: add libsoup3 backend
The default is still soup2, you can use --with-soup3 to enable
the soup3 backend instead.
2023-04-12 22:33:08 -06:00
Dan Nicholson
3ec7b5db18 tests: Ensure real GIO backends aren't used
Set a few environment variables during tests to ensure fake GIO backends
are used. This is particularly important with the soup fetcher backend
as it can cause strange test errors in containerized test environments.
Upstream soup has been setting these 3 environment variables for their
tests since 2015.
2023-04-12 21:42:33 -06:00
Colin Walters
c9271423b1
Merge pull request #2839 from cgwalters/treegen-race
treegen: Require at least one mutation
2023-03-23 17:55:36 -04:00
Colin Walters
cafe12cf1f treegen: Require at least one mutation
Since a later assertion would otherwise trigger.  We saw
this happen in CI.
2023-03-23 16:48:55 -04:00
Colin Walters
71902465f7
Merge pull request #2842 from cgwalters/release
Release 2023.2
2023-03-22 13:07:43 -04:00
Colin Walters
61dc678cb3 configure: post-release version bump 2023-03-22 09:58:54 -04:00
Colin Walters
29106eb939 Release 2023.2 2023-03-22 09:57:57 -04:00
Colin Walters
a63febcb02
Merge pull request #2823 from jlebon/pr/testiso-update
ci: update for new kolaTestIso()
2023-03-17 14:22:50 -04:00
Colin Walters
4cd97c62b5
Merge pull request #2838 from cgwalters/more-binding-fixes
More binding fixes
2023-03-17 14:08:18 -04:00
Colin Walters
17b6f4cdc2 rust-bindings: Regenerate
This picks up more of tintou's recent changes.
2023-03-17 11:54:14 -04:00
Colin Walters
00c41eacd2 Revert "repo: Fix nullability for remote options"
This reverts commit c4d03d2850.
2023-03-17 11:53:20 -04:00
Colin Walters
d49632faa6
Merge pull request #2812 from tintou/tintou/g-ir-fix
build: Do not include private headers in the introspection
2023-03-17 11:36:22 -04:00
Colin Walters
28731ab0e9
Merge pull request #2813 from tintou/tintou/asyncprogress-typedef
libostree: Ignore new_and_connect in the introspection
2023-03-17 11:32:18 -04:00
Jonathan Lebon
90ffc149cb ci: update for new kolaTestIso()
The `skipMetal4k` and `skipMultipath` parameters no longer do anything.
Instead use `--denylist-test` which is interpreted directly by the
new `kola testiso` code.

This will run more tests than before. We'll likely cut down tests we
consider redundant soon, but that should be transparent to this code.
2023-03-17 11:29:19 -04:00
Colin Walters
233cc8ce85
Merge pull request #2836 from smcv/file-info-size
Cope with GLib 2.76 being more strict about GFileInfo standard::size
2023-03-17 11:28:18 -04:00
Colin Walters
7b258b2499
Merge pull request #2837 from smcv/g-steal-fd
Use g_steal_fd() in preference to glnx_steal_fd()
2023-03-17 11:27:03 -04:00
Joseph Marrero Corchado
4f0c13fb8b
Merge pull request #2705 from cgwalters/always-boot
Add `sysroot.bootprefix` option
2023-03-17 11:09:19 -04:00
Jonathan Lebon
7e93837e67
Merge pull request #2791 from cgwalters/enable-trust-return-nullability 2023-03-17 09:37:40 -04:00
Simon McVittie
7b02fdfdf8 Use g_steal_fd() in preference to glnx_steal_fd()
g_steal_fd() exists in GLib since 2.70, and libglnx has a backport for
older GLib versions, equivalent to the libglnx-specific glnx_steal_fd().

Signed-off-by: Simon McVittie <smcv@collabora.com>
2023-03-17 13:32:26 +00:00
Simon McVittie
1b29ab7d90 write_content_object: Don't assume file info has standard::size
The file info object for symlinks might validly not have this attribute.
If not, behave as though it was 0, matching what happened with older
versions of GLib.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2023-03-17 12:56:39 +00:00
Simon McVittie
9e116c0f79 _ostree_zlib_file_header_new: Default size to 0
Similar to the previous commit, but for
ostree_raw_file_to_archive_z2_stream() and similar public APIs.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2023-03-17 12:54:33 +00:00
Simon McVittie
722a4179f3 ostree_raw_file_to_content_stream: Make size default to 0
Some existing code calls into ostree_raw_file_to_content_stream() with
file objects that do not have the standard::size attribute. Since GLib
2.76.0, attempting to access the size of such an object raises a
critical warning. Handle this more gracefully by defaulting the size
to 0, like earlier versions of GLib did.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2023-03-17 12:53:42 +00:00
Colin Walters
61a4a83f19 rust: Renerate bindings
To pick up the latest introspection changes.
2023-03-17 08:26:09 -04:00
Colin Walters
c4d03d2850 repo: Fix nullability for remote options
Unfortunately, the nullability of the output value here is
dependent on whether the `default_value` parameter is provided.  There's
no way to express this in introspection or Rust.
2023-03-17 08:25:23 -04:00
Colin Walters
cbec85f2d0 lib: Fix two nullable annotations
- commit parents are optional
- remote URLs are optional
2023-03-17 08:25:23 -04:00
Colin Walters
3f27cff6cb rust: Bump semver
Since we did a bunch of API changes due to nullability and
other introspection cleanups.
2023-03-17 08:13:13 -04:00