IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
If the average object size is greater than 4GiB, let's assume we're
dealing with opposite endianness. I'm fairly confident no one is
going to be shipping peta- or exa- byte size ostree deltas, period.
Past the gigabyte scale you really want bittorrent or something.
This is a more flexible version of the previous
ostree_repo_write_archive_to_mtree() which took a file reference.
This has an extensible options structure, and in particular
now supports `ignore_unsupported_content`.
I plan to use this for importing Docker images which contain device
nodes. (There's no reason for container images to have those, so
we'll just ignore them).
Also here, just like the export variant, the caller is responsible for
setting up libarchive.
At the moment I'm looking at using rpm-ostree to manage RPM inputs
which can then be converted into Docker images. It's most convenient
if we can stream directly out of libostree rather than doing a
checkout + tar combination.
There are also backup/debugging etc. reasons to implement `export` as
well.
While it's not strictly tied to OSTree, let's move
https://github.com/cgwalters/rofiles-fuse in here because:
- It's *very* useful in concert with OSTree
- It's tiny
- We can reuse OSTree's test, documentation, etc. infrastructure
One thing to consider also is that at some point we could experiment
with writing a FUSE filesystem for OSTree. This could internalize a
better equivalent of `--link-checkout-speedup`, but on the other hand,
the cost of walking filesystem trees for these types of operations is
really quite small.
But if we did decide to do more FUSE things in OSTree, this is a step
towards that too.
I was working on a different test, and ended up being very confused at
the behavior where removing the last deployment didn't remove the last
`ostree/X/X/X` ref pointing to its commit.
There's no reason to special case the last undeployment AFAIK, and the
existing code handles this.
test-lzma builds a copy of the compressor and decompressor directly, so
the compiler needs access to the LZMA headers and the linker needs to
link the program with liblzma.
This test uses gpgme directly to verify the signatures, so it needs to
find the gpgme headers and link with gpgme to ensure the linker can
resolve the symbols.
Having undefined (but in practice rarely changing) ordering for
`readdir()` ended up screwing us over for bootloader config
generation; see https://bugzilla.redhat.com/show_bug.cgi?id=1226520
Let's make things significantly more likely to fail more quickly in
the future if similar bugs are introduced. We accomplish this by
introducing a little `LD_PRELOAD` library that randomizes the results
of `readdir()`.
Having undefined ordering (but in practice rarely changing)
ordering for `readdir()` ended up screwing us over with respect
to bootloader config file read ordering.
Let's make things significantly more likely to fail more quickly in
the future if similar bugs are introduced. We accomplish this by
introducing a little `LD_PRELOAD` library that randomizes the results
of `readdir()`.
An OSTree user noticed that `ostree fsck` would produce `missing
object` errors in the case of interrupted pulls.
It's possible to do e.g. `ostree pull --subpath=/usr/share/rpm ...`,
which gets you just that portion of the commit. The use case for this
was being able to see what changes would appear in an update before
actually downloading all of it.
(I think this would be better covered by static deltas, but those
aren't final yet, and `--subpath` predates it)
Further, `.commitpartial` is used as a successor to the `transaction`
symlink for more precise knowledge in the case where a pull was
interrupted that we needed to resume scanning.
So it makes sense for `ostree fsck` to be aware of it.
If a system administrator happens to type `ostree admin upgrade`
multiple times, currently that will lead to a potentially corrupted
system.
I originally attempted to do locking *internally* in `libostree`, but
that didn't work out because currently a number of the commands
perform multi-step operations that all need to be serialized. All of
the current code in `ostree admin deploy` is an example.
Therefore, allow callers to perform locking, as most of the higher
level logic is presently implemented there.
At some point, we can revisit having internal locking, but it will be
difficult. A more likely approach would be similar to Java's approach
with concurrency on iterators - a "fail fast" method.
First, git doesn't do this, and whatever Linus thinks is right or
something.
Second specifically to OSTree, it's quite common to not have
intermediate commits. If one wants to reset a ref in order to prune
data after a deployment, the parentage check will fail.
Closes: https://github.com/GNOME/ostree/pull/87
Wrappers a referenced gpgme_verify_result_t so detailed verify results
can be examined independently of executing a verify operation.
_ostree_gpg_verifier_check_signature() now returns this object instead
of a single valid/invalid boolean, but the idea is for OstreeRepo to also
return this object for commit signature verification so it can be utilized
at the CLI layer (and possibly by other programs).
This does an rsync-style prepared delta basically. On my test data,
it shaves ~6MB of uncompressed data. Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
This creates a archive-z2 repo, pull-locals it to bare-user and then
again back to archive-z2 making sure things fsck along the way.
Then it checks out all repos and makes sure each one reproduces
the same result.
Unfortunately we can't install this as a real test because
it doesn't work in the test-runner because tmpfs doesn't support
user xattrs.
https://bugzilla.gnome.org/show_bug.cgi?id=741125
This just does whatever test-basic.sh does, but on a bare-user
repo.
This works standalone, but unfortunately it breaks in
gnome-desktop-testing-runner as /tmp doesn't support
xattrs, so it is not installed atm.
https://bugzilla.gnome.org/show_bug.cgi?id=741125
fixes a coredump when using a command like:
$ ostree --repo=repo checkout -U --subpath=/usr/lib/passwd \
fedora-atomic/rawhide/x86_64/docker-host usrlib-new
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Some operating systems may come with external tools for subscription
management that drive access to the content. In that case, the origin
file may not be useful (for example, it could refer to an installer
ISO).
This patch will allow OS installers to inject that state, with a
useful error message, directing the system administrator to an
external tool.
See: https://github.com/projectatomic/rpm-ostree/issues/31https://bugzilla.gnome.org/show_bug.cgi?id=737686
Now that we have a summary file, we can use it to allow a simple:
ostree pull --mirror
To download the latest commit on every branch. Also, for a case I'm
dealing with there's only one branch, but I don't want mirror users to
have to hardcode it.
https://bugzilla.gnome.org/show_bug.cgi?id=737807
Recursive over ostree and all subcommands, and check that --help
is supported, properly outputs to standard out, and exits
with a 0 exit status. Check that for commands with subcommands,
they produce the help output to standard error when run with no arguments.
https://bugzilla.gnome.org/show_bug.cgi?id=737194
For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.
In order to make metalink work though, it needs *one* file which can
be checksummed. (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).
This commit adds a new command:
$ ostree summary -u
To regenerate the summary file. Can only be run by one process at a
time.
After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.
https://bugzilla.gnome.org/show_bug.cgi?id=729585
Changes the pull API to allow pulling only a single directory instead
of the whole deployment. This option is utilized by the check-diff
option in rpm-ostree.
Add a new state directory to hold <checksum>.commitpartial files, so
we know that we've only downloaded partial state.
If fetching GPG-signed commits over plain HTTP, a MitM attacker can
fill up the drive of targets by simply returning an enormous stream
for the commit object.
Related to this, an attacker can also cause OSTree to perform large
memory allocations by returning enormous GVariants in the metadata.
This helps close that attack by limiting all metadata objects to 10
MiB, so the initial fetch will be truncated.
But now the attack is only slightly more difficult as the attacker
will have to return a correctly formed commit object, then return a
large stream of < 10 MiB dirmeta/dirtree objects.
https://bugzilla.gnome.org/show_bug.cgi?id=725921
We don't want to allow MITM attackers to intercept upgrade requests
and provide clients with older OS versions vulnerable to security
flaws.
Only "ostree admin upgrade" gets this behavior for now - whether we
want to do it for "ostree admin switch" is another question.
These GPG tests were failing for me on EL7 - it appears to be because
we had only one directory for both private and public keys, and we
were giving that to ostree for verification, which passed them onto
gpgv.
In EL7 beta at least, gpgv now barfs if it finds a private key where
it is just expecting to find public keys.
Fix this by splitting out the public trusted directory from the
private key directory. Except now for signing, we still need the
public key there, so symlink it. Whee!
This has a very basic level of functionality (deltas can be generated,
and applied offline). There is only some stubbed out pull code to
fetch them via HTTP.
But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
This test had some nondeterminism because we chose a random
object to corrupt, but because there were multiple commits, it
was possible that we chose an object that was not being pulled.
Fix this by writing some custom GJS code to find an explicitly random
object that exists in a given ref, an change a random byte offset.
This adds a lot more randomness to the testing too.
The test-admin-deploy-1 was copied into -uboot at some point. But
really they should be testing exactly the same thing, except for
the bootloader backend.
Unify these tests by extracting a common test core.
The official way to add bootloader arguments to the current deployment
is to redeploy with --karg. However, doing so tripped up an
optimization made inside the deployment code to just swap the
bootlinks if we're keeping the same "bootcsum".
Change this optimization to look at the pair of (bootcsum, options).
This large patch moves the core xattr logic down into libgsystem,
which allows the gs_shutil_cp_a() API to copy them. In turn, this
allows us to just use that API instead of rolling our own recursive
copy here.
As noted in the new comment though, one case that we are explicitly
regressing is where the new /etc removes a parent directory that's
needed by a modified file. This seems unlikely for most vendors now,
but let's do that as a separate bug.
https://bugzilla.gnome.org/show_bug.cgi?id=711058
Add a --generate-sizes option to commit to add size information to the
commit metadata. This will be used by higher level code which wants
to determine the total size necessary for downloading.
Adapted from Google protobufs. For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.
This will be used for static deltas and sizes metadata.
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default. The keyring can be overridden by specifying OSTREE_GPG_HOME.
Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.
Change all of the existing tests to disable GPG verification.
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).
The commit command gains a --gpg-sign=<key-id> argument. Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
Use a consistent temporary filename to download uri's.
Check for downloaded files before fetching from uri.
Download to hash.part file, then copy/move to hash.done when complete.
Add argument support to setup_fake_remote_repo1 function.
Add test for pull resume.
To implement this, pass --force-range-requests into the trivial-httpd,
which will only serve half of the objects to clients at a time.
https://bugzilla.gnome.org/show_bug.cgi?id=706344
We'll always have "bare" mode for keeping files-as-hardlinks as root.
But "archive" was my second attempt at a format for non-root file
storage, used by the gnome-ostree buildsystem which runs as non-root.
It was really handy to have a "tar" like mode where I can create
tarballs as a user, that contain files owned by root for example.
The "archive" mode stored content files as two pieces in the
filesystem; ".file" contained metadata, and ".filecontent" was the
actual content, uncompressed. The nice thing about this was that to
check out a tree as non-root, you could just hardlink into the repo.
However, archive was fairly bad for serving via HTTP; it required
*two* HTTP requests per content object, greatly magnifing the already
inefficient fetch process. So "archive-z2" was introduced.
To allow gnome-ostree to still check out trees as a user, the
"uncompressed-object-cache" was introduced, and that's how things have
been working for a while.
So we should just be able to kill this code. Specifically note just
how much better the stage_object() function became.
https://bugzilla.gnome.org/show_bug.cgi?id=706057
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.
More interesting things are diffs between commits, object statistics,
etc.
These corruption tests could be a lot better...like randomly try
single bit flips, range flips. Better, content-aware fuzzing. But
this is useful for now.
See https://wiki.gnome.org/OSTree/DeploymentModel2
This is a major rework of the on-disk filesystem layout, and the boot
process. OSTree now explicitly supports upgrading kernels, and these
upgrades are also atomic.
The core concept of the new model is the "deployment list", which is
an ordered list of bootable operating system trees. The deployment
list is reflected in the bootloader configuration; which has a kernel
argument that tells the initramfs (dracut) which operating system root
to use.
Invidiual notable changes that come along with this:
1) Operating systems should now come with their etc in usr/etc; OSTree
will perform a 3-way merge at deployment time, and place etc in
the actual root. This avoids the need for a bind mount, and is
just a lot cleaner.
2) OSTree no longer bind mounts /root, /home, and /tmp. It is expected
that the the OS/ has these as symbolic links into /var.
At the moment, OSTree only supports managing syslinux; other
bootloader backends will follow.
