mirror of
https://github.com/ostreedev/ostree.git
synced 2025-01-09 01:18:35 +03:00
tree-wide: Replace various uses of archive-z2
→ archive
The `-z2` is annoying now since it's really a legacy; we've long since supported typing `archive`. Convert the docs fully and explain that. Also do some (but not all) of the tests just to encourage newer tests to use `archive` too. Closes: #980 Approved by: jlebon
This commit is contained in:
parent
8d586a9da0
commit
90e0d56332
@ -67,7 +67,7 @@ _installed_or_uninstalled_test_scripts = \
|
||||
tests/test-help.sh \
|
||||
tests/test-libarchive.sh \
|
||||
tests/test-parent.sh \
|
||||
tests/test-pull-archive-z.sh \
|
||||
tests/test-pull-archive.sh \
|
||||
tests/test-pull-commit-only.sh \
|
||||
tests/test-pull-depth.sh \
|
||||
tests/test-pull-mirror-summary.sh \
|
||||
|
@ -21,11 +21,11 @@ side, but aiming to do client side too).
|
||||
## Initializing
|
||||
|
||||
For this initial discussion, we're assuming you have a single
|
||||
`archive-z2` repository:
|
||||
`archive` repository:
|
||||
|
||||
```
|
||||
mkdir repo
|
||||
ostree --repo=repo init --mode=archive-z2
|
||||
ostree --repo=repo init --mode=archive
|
||||
```
|
||||
|
||||
You can export this via a static webserver, and configure clients to
|
||||
@ -107,7 +107,7 @@ equivalent to RPMs/debs.
|
||||
|
||||
Further, in order to make things fast, we will need a separate
|
||||
`bare-user` repository in order to perform checkouts quickly via
|
||||
hardlinks. We'll then export content into the `archive-z2` repository
|
||||
hardlinks. We'll then export content into the `archive` repository
|
||||
for use by client systems.
|
||||
|
||||
```
|
||||
@ -161,7 +161,7 @@ checksum).
|
||||
|
||||
Now that we have content in our `build-repo` repository (in
|
||||
`bare-user` mode), we need to move the `exampleos/x86_64/standard`
|
||||
branch content into the repository just named `repo` (in `archive-z2`
|
||||
branch content into the repository just named `repo` (in `archive`
|
||||
mode) for export, which will involve zlib compression of new objects.
|
||||
We likely want to generate static deltas after that as well.
|
||||
|
||||
|
@ -14,22 +14,23 @@ primary advantages are security and compute efficiency. Services like
|
||||
Amazon S3 and CDNs are a canonical target, as well as a stock static
|
||||
nginx server.
|
||||
|
||||
## The archive-z2 format
|
||||
## The archive format
|
||||
|
||||
In the [repo](repo) section, the concept of objects was introduced,
|
||||
where file/content objects are checksummed and managed individually.
|
||||
(Unlike a package system, which operates on compressed aggregates).
|
||||
|
||||
The archive-z2 format simply gzip-compresses each content object.
|
||||
The `archive` format simply gzip-compresses each content object.
|
||||
Metadata objects are stored uncompressed. This means that it's easy
|
||||
to serve via static HTTP.
|
||||
to serve via static HTTP. Note: this format used to be called `archive-z2`
|
||||
for historical reasons.
|
||||
|
||||
When you commit new content, you will see new `.filez` files appearing
|
||||
in `objects/`.
|
||||
|
||||
## archive-z2 efficiency
|
||||
## archive efficiency
|
||||
|
||||
The advantages of `archive-z2`:
|
||||
The advantages of `archive`:
|
||||
|
||||
- It's easy to understand and implement
|
||||
- Can be served directly over plain HTTP by a static webserver
|
||||
@ -47,7 +48,7 @@ such that large data that changes infrequently (e.g. graphic images)
|
||||
are stored separately from small frequently changing data (application
|
||||
code).
|
||||
|
||||
Other disadvantages of `archive-z2`:
|
||||
Other disadvantages of `archive`:
|
||||
|
||||
- It's quite bad when clients are performing an initial pull (without HTTP/2),
|
||||
- One doesn't know the total size (compressed or uncompressed) of content
|
||||
@ -55,7 +56,7 @@ Other disadvantages of `archive-z2`:
|
||||
|
||||
## Aside: the bare and bare-user formats
|
||||
|
||||
The most common operation is to pull from an `archive-z2` repository
|
||||
The most common operation is to pull from an `archive` repository
|
||||
into a `bare` or `bare-user` formatted repository. These latter two
|
||||
are not compressed on disk. In other words, pulling to them is
|
||||
similar to unpacking (but not installing) an RPM/deb package.
|
||||
@ -72,7 +73,7 @@ client systems are expected to update regularly. However, many OS vendors
|
||||
would like to supply content that's updated e.g. once a month or less often.
|
||||
|
||||
For this model, we can do a lot better to support batched updates than
|
||||
a basic `archive-z2` repo. However, we still want to preserve the
|
||||
a basic `archive` repo. However, we still want to preserve the
|
||||
model of "static webserver only". Given this, OSTree has gained the
|
||||
concept of a "static delta".
|
||||
|
||||
|
@ -300,7 +300,7 @@ Docker (natively) only shares storage via layering.
|
||||
|
||||
The biggest feature OSTree has over Docker though is support for
|
||||
(static) deltas, and even without pre-configured static deltas, the
|
||||
archive-z2 format has "natural" deltas. Particularly for a "base
|
||||
`archive` format has "natural" deltas. Particularly for a "base
|
||||
operating system", one really wants on-wire deltas. It'd likely be
|
||||
possible to extend Docker with this concept.
|
||||
|
||||
|
@ -66,8 +66,8 @@ is reverted to use zero again.
|
||||
|
||||
# Repository types and locations
|
||||
|
||||
Also unlike git, an OSTree repository can be in one of three separate
|
||||
modes: `bare`, `bare-user`, and `archive-z2`. A bare repository is
|
||||
Also unlike git, an OSTree repository can be in one of four separate
|
||||
modes: `bare`, `bare-user`, `bare-user-only`, and `archive`. A bare repository is
|
||||
one where content files are just stored as regular files; it's
|
||||
designed to be the source of a "hardlink farm", where each operating
|
||||
system checkout is merely links into it. If you want to store files
|
||||
@ -88,7 +88,7 @@ information is not applied anyway. The main advantage of `bare-user-only` is
|
||||
that repos can be stored on filesystems which do not support extended
|
||||
attributes, such as tmpfs.
|
||||
|
||||
In contrast, the `archive-z2` mode is designed for serving via plain
|
||||
In contrast, the `archive` mode is designed for serving via plain
|
||||
HTTP. Like tar files, it can be read/written by non-root users.
|
||||
|
||||
On an OSTree-deployed system, the "system repository" is
|
||||
|
@ -29,14 +29,14 @@ It's very common to want to perform a full or partial mirror, in
|
||||
particular across organizational boundaries (e.g. an upstream OS
|
||||
provider, and a user that wants offline and faster access to the
|
||||
content). OSTree supports both full and partial mirroring of the base
|
||||
`archive-z2` content, although not yet of static deltas.
|
||||
`archive` content, although not yet of static deltas.
|
||||
|
||||
To create a mirror, first create an `archive-z2` repository (you don't
|
||||
To create a mirror, first create an `archive` repository (you don't
|
||||
need to run this as root), then add the upstream as a remote, then use
|
||||
`pull --mirror`.
|
||||
|
||||
```
|
||||
ostree --repo=repo init --mode=archive-z2
|
||||
ostree --repo=repo init --mode=archive
|
||||
ostree --repo=repo remote add exampleos https://exampleos.com/ostree/repo
|
||||
ostree --repo=repo pull --mirror exampleos:exampleos/x86_64/standard
|
||||
```
|
||||
@ -66,7 +66,7 @@ unlikely most of your content consumers (i.e. not developers/testers)
|
||||
will be interested in all of it.
|
||||
|
||||
The original vision of OSTree was to fulfill this "dev" role, and in
|
||||
particular the "archive-z2" format was designed for it.
|
||||
particular the "archive" format was designed for it.
|
||||
|
||||
Then, what you'll want to do is promote content from "dev" to "prod".
|
||||
We'll discuss this later, but first, let's talk about promotion
|
||||
@ -138,7 +138,7 @@ We'll have other business requirements such as writing release notes
|
||||
|
||||
In [Build Systems](buildsystem-and-repos.md) we saw how the
|
||||
`pull-local` command can be used to migrate content from the "build"
|
||||
repository (in `bare-user` mode) into an `archive-z2` repository for
|
||||
repository (in `bare-user` mode) into an `archive` repository for
|
||||
serving to client systems.
|
||||
|
||||
Following this section, we now have three repositories, let's call
|
||||
@ -146,7 +146,7 @@ them `repo-build`, `repo-dev`, and `repo-prod`. We've been pulling
|
||||
content from `repo-build` into `repo-dev` (which involves gzip
|
||||
compression among other things since it is a format change).
|
||||
|
||||
When using `pull-local` to migrate content between two `archive-z2`
|
||||
When using `pull-local` to migrate content between two `archive`
|
||||
repositories, the binary content is taken unmodified. Let's go ahead
|
||||
and generate a new commit in our prod repository:
|
||||
|
||||
@ -178,7 +178,7 @@ default.
|
||||
|
||||
## Derived data - static deltas and the summary file
|
||||
|
||||
As discussed in [Formats](formats.md), the `archive-z2` repository we
|
||||
As discussed in [Formats](formats.md), the `archive` repository we
|
||||
use for "prod" requires one HTTP fetch per client request by default.
|
||||
If we're only performing a release e.g. once a week, it's appropriate
|
||||
to use "static deltas" to speed up client updates.
|
||||
|
@ -446,7 +446,7 @@ checkout_one_file_at (OstreeRepo *repo,
|
||||
/* But only under these conditions */
|
||||
if (is_bare || is_archive_z2_with_cache)
|
||||
{
|
||||
/* Override repo mode; for archive-z2 we're looking in
|
||||
/* Override repo mode; for archive we're looking in
|
||||
the cache, which is in "bare" form */
|
||||
_ostree_loose_path (loose_path_buf, checksum, OSTREE_OBJECT_TYPE_FILE, OSTREE_REPO_MODE_BARE);
|
||||
if (!checkout_file_hardlink (current_repo,
|
||||
@ -488,7 +488,7 @@ checkout_one_file_at (OstreeRepo *repo,
|
||||
g_autoptr(GInputStream) input = NULL;
|
||||
g_autoptr(GVariant) xattrs = NULL;
|
||||
|
||||
/* Ok, if we're archive-z2 and we didn't find an object, uncompress
|
||||
/* Ok, if we're archive and we didn't find an object, uncompress
|
||||
* it now, stick it in the cache, and then hardlink to that.
|
||||
*/
|
||||
if (can_cache
|
||||
|
@ -512,7 +512,7 @@ find repo3/objects -name '*.filez' > file-objects
|
||||
if test -s file-objects; then
|
||||
assert_not_reached "prune didn't delete all objects"
|
||||
fi
|
||||
echo "ok prune in archive-z2 deleted everything"
|
||||
echo "ok prune in archive deleted everything"
|
||||
|
||||
cd ${test_tmpdir}
|
||||
rm -rf test2-checkout
|
||||
@ -730,7 +730,7 @@ $OSTREE commit ${COMMIT_ARGS} -b test2 -s "Unfsynced commit" --fsync=false
|
||||
if test "$(id -u)" != "0"; then
|
||||
cd ${test_tmpdir}
|
||||
rm -f expected-fail error-message
|
||||
$OSTREE init --mode=archive-z2 --repo=repo-noperm
|
||||
$OSTREE init --mode=archive --repo=repo-noperm
|
||||
chmod -w repo-noperm/objects
|
||||
$OSTREE --repo=repo-noperm pull-local repo 2> error-message || touch expected-fail
|
||||
chmod +w repo-noperm/objects
|
||||
|
@ -50,7 +50,7 @@ echo "ok pull contents"
|
||||
|
||||
cd ${test_tmpdir}
|
||||
mkdir mirrorrepo
|
||||
ostree_repo_init mirrorrepo --mode=archive-z2
|
||||
ostree_repo_init mirrorrepo --mode=archive
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo remote add --set=gpg-verify=false origin $(cat httpd-address)/ostree/gnomerepo
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo pull --mirror origin main
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo fsck
|
||||
@ -63,7 +63,7 @@ ${CMD_PREFIX} ostree --repo=ostree-srv/gnomerepo commit -b otherbranch --tree=di
|
||||
${CMD_PREFIX} ostree --repo=ostree-srv/gnomerepo summary -u
|
||||
rm mirrorrepo -rf
|
||||
# All refs
|
||||
ostree_repo_init mirrorrepo --mode=archive-z2
|
||||
ostree_repo_init mirrorrepo --mode=archive
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo remote add --set=gpg-verify=false origin $(cat httpd-address)/ostree/gnomerepo
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo pull --mirror origin
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo fsck
|
||||
@ -73,7 +73,7 @@ done
|
||||
echo "ok pull mirror (all refs)"
|
||||
|
||||
rm mirrorrepo -rf
|
||||
ostree_repo_init mirrorrepo --mode=archive-z2
|
||||
ostree_repo_init mirrorrepo --mode=archive
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo remote add --set=gpg-verify=false origin $(cat httpd-address)/ostree/gnomerepo
|
||||
# Generate a summary in the mirror
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo summary -u
|
||||
@ -152,7 +152,7 @@ echo "ok pull commit metadata only (should not apply deltas)"
|
||||
|
||||
cd ${test_tmpdir}
|
||||
mkdir mirrorrepo-local
|
||||
ostree_repo_init mirrorrepo-local --mode=archive-z2
|
||||
ostree_repo_init mirrorrepo-local --mode=archive
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo-local remote add --set=gpg-verify=false origin file://$(pwd)/ostree-srv/gnomerepo
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo-local pull --mirror origin main
|
||||
${CMD_PREFIX} ostree --repo=mirrorrepo-local fsck
|
||||
@ -170,7 +170,7 @@ echo "ok pull detached metadata"
|
||||
|
||||
cd ${test_tmpdir}
|
||||
mkdir parentpullrepo
|
||||
ostree_repo_init parentpullrepo --mode=archive-z2
|
||||
ostree_repo_init parentpullrepo --mode=archive
|
||||
${CMD_PREFIX} ostree --repo=parentpullrepo remote add --set=gpg-verify=false origin file://$(pwd)/ostree-srv/gnomerepo
|
||||
parent_rev=$(ostree --repo=ostree-srv/gnomerepo rev-parse main^)
|
||||
rev=$(ostree --repo=ostree-srv/gnomerepo rev-parse main)
|
||||
|
@ -21,6 +21,6 @@ set -euo pipefail
|
||||
|
||||
. $(dirname $0)/libtest.sh
|
||||
|
||||
setup_fake_remote_repo1 "archive-z2"
|
||||
setup_fake_remote_repo1 "archive"
|
||||
|
||||
. ${test_srcdir}/pull-test.sh
|
@ -43,7 +43,7 @@ test_data_init (TestData *td)
|
||||
if (!td->repo)
|
||||
goto out;
|
||||
|
||||
if (!ot_test_run_libtest ("setup_fake_remote_repo1 archive-z2", error))
|
||||
if (!ot_test_run_libtest ("setup_fake_remote_repo1 archive", error))
|
||||
goto out;
|
||||
|
||||
if (!g_file_get_contents ("httpd-address", &http_address, NULL, error))
|
||||
|
Loading…
Reference in New Issue
Block a user