IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Allow users to pass `<remote>:` to list all refs we have locally
belonging to `<remote>`. Also (re-)allow the similar `<remote>:.` syntax
for backwards compatibility with flatpak.
Closes: #1500
Approved by: cgwalters
Let's make our `run.sh` generically support any playbook. This is prep for
writing further tests in Ansible. Along with that, rework the Ansible so that
`tests.yml` is a playbook, and then the other bits are just task lists. It's
easier to read.
I also started to add a `use_git_build` variable with the idea that we'll be
able to run these same tests against an upstream image by setting that variable
off.
Closes: #1493
Approved by: jlebon
We noticed this in a recent PR. While I'm here, also only do
the `find` once, add `-type l` for good measure, and use our
built in `libtest.sh` assertion functions.
Closes: #1494
Approved by: giuseppe
Reusing the way `standard-test-roles` has support for booting
a qcow2 actually gets us to the "VM-in-container" flow. Plus
Ansible over shell script is sometimes nicer.
https://fedoraproject.org/wiki/CI/Tests#Testing_an_Atomic_Host
It's better than what we were doing before for installed tests,
and moreover using Ansible more broadly for testing is going
to align us better with Fedora's CI.
As part of this I split off a "libpaprci" which I intend to maintain
as a "copylib" for a little bit between ostree/rpm-ostree, and then
we'll figure out how to expand from there (maybe some of the patterns
get "baked in" to PAPR for example).
Note the `FAH27-insttests` context moves to the top since it's now
of primary importance, and I expect that we start expanding it.
Closes: #1462
Approved by: jlebon
When a new object is added to the repository, create a
$PAYLOAD-SHA256.payload-link symlink file as well. The target of the
symlink is the checksum of the object that was added the repository.
Whenever we add a new object file, in addition to lookup if the file is
already present with the same checksum we also check if an object with
the same payload is in the repository.
If a file with the same payload is already present in the repository, we
copy it with `glnx_regfile_copy_bytes` that internally attempts to
create a reflink (ioctl (..., FICLONE, ..)) to the target file if the
file system supports it. This enables to have objects that share the
payload but have a different inode and xattrs.
By default the payload-link-threshold value is G_MAXUINT64 that disables
the feature.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1443
Approved by: cgwalters
Add some "function global" prefixing in line with what we do in
other places now, and drop the "manual filename" prefixing that
is no longer necessary since
23f7df1500
Closes: https://github.com/ostreedev/ostree/issues/1467Closes: #1485
Approved by: jlebon
The original changes here apparently had the *idea* that `--static-deltas-only`
would be useful in general, but we never implemented that. The current
situation where it's ignored unless `--delete-commit` is specified is
very misleading and I can easily see it leading to data loss for people.
Let's error out until we have a chance to make it actually useful.
Related: https://github.com/ostreedev/ostree/issues/1479Closes: #1482
Approved by: giuseppe
I think this got changed in a refactor. We definitely want
to total up the amount of space that *would* be freed even
with `--no-prune` AKA `OSTREE_REPO_PRUNE_FLAGS_NO_PRUNE`.
It's actually a bit terrifying this is apparently the first test case for
the `--no-prune` option...
Closes: https://github.com/ostreedev/ostree/issues/1480Closes: #1483
Approved by: jlebon
Example user story: Jane rebases her OS to a new major version N, and wants to
keep around N-1 even after a few upgrades for a while so she can easily roll
back. I plan to add `rpm-ostree rebase --pin` to opt-in to this for example.
Builds on the new `libostree-transient` group to store pinning state there.
Closes: https://github.com/ostreedev/ostree/issues/1460Closes: #1464
Approved by: jlebon
This is a little clearer than a strcmp()-style negative/zero/positive
return, and also works in Python 2.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #1457
Approved by: cgwalters
Python 3 is pickier about this. Python 2.7 has Python 3-compatible
semantics for division when the division feature is imported from the
future.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #1457
Approved by: cgwalters
range in Python 3 does what xrange did in Python 2. This still works in
Python 2, it just uses a bit more memory.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #1457
Approved by: cgwalters
There are enough fixes here, and there are some potentially larger patches
incoming like wmanley's checkout speedups and the payload link that will need
soak time in master.
Closes: #1455
Approved by: jlebon
There are a few cases for knowing whether a commit has identical
content to another commit. Some people want to do a "promotion workflow",
where the content of a commit on a tesitng branch is then "promoted"
to a production branch with `ostree commit --tree=ref`.
Another use case I just hit in rpm-ostree deals with
[jigdo](https://github.com/projectatomic/rpm-ostree/issues/1081) where we're
importing RPMs on both the client and server, and will be using the
content checksum, since the client/server cases inject different metadata
into the commit object.
Closes: https://github.com/ostreedev/ostree/issues/1315Closes: #1449
Approved by: jlebon
Having the `uncompressed-object-cache` directory in `archive` repos by default
is clutter; the functionality should be considered deprecated.
Now we only create the directory if we're doing a checkout with the cache
enabled.
Closes: #1446
Approved by: jlebon
This is analogous to the filtering support for the commit API: we allow
library users to skip over checking out specific files. This is useful
in some tricky situations where we *know* that the files to be checked
out will conflict with existing files in subtle ways.
One such example is in rpm-ostree support for multilib. There, we want
to allow checking out a package onto an existing tree, but skipping over
files that are not coloured to our preferred value (e.g. not overwriting
an i686 version of `ldconfig` if we already have the `x86_64` version).
See https://github.com/projectatomic/rpm-ostree/pull/1227 for details.
Closes: #1441
Approved by: cgwalters
Lifted from rpm-ostree. Makes iterating on a single test much faster.
Example use:
TESTS=label-selinux ./ostree/tests/installed/run.sh
Closes: #1442
Approved by: cgwalters
When we changed around the kernel location in rpm-ostree, we
started installing the kernel into `/boot` as `modules_object_t`,
and the current policy didn't permit that. For maximum compatibility,
relabel installed kernel/initramfs/dtb as `boot_t`.
https://bugzilla.redhat.com/show_bug.cgi?id=1536991Closes: #1444
Approved by: jlebon
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.
Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
license for their code is respected
See http://spdx.org for further reading.
Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Closes: #1439
Approved by: cgwalters
Currently we were parsing `opt_filename` twice...I dug through
the history a bit and it looks like it may have been an accident
from refactoring.
What we're fixing here concretely is that using relative subdirectories
like `--filename somesubdir/foo` broke because we were incorrectly
passing the `somesubdir/` again.
Closes: #1423Closes: #1427
Approved by: jlebon
Much like the (optional) initramfs at
`/usr/lib/ostree-boot/initramfs-<SHA256>` or
`/usr/lib/modules/$kver/initramfs` you can now optionally include a
flattened devicetree (.dtb) file alongside the kernel at
`/usr/lib/ostree-boot/devicetree-<SHA256>` or
`/usr/lib/modules/$kver/devicetree`.
This is useful for embedded ARM systems which need the devicetree file
loaded by the bootloader for the kernel to discover and initialise
hardware. See https://en.wikipedia.org/wiki/Device_tree for more
information.
This patch was mostly produced by copy-pasting code for initramfs handling
and renaming `s/initramfs/devicetree/g`. It's not beautiful, but it is
fairly straightforward.
It may be useful to extend device-tree support in a number ways in the
future. Device trees dependant on many details of the hardware they
support. This makes them unlike kernels, which may support many different
hardware variants as long as the instruction-set matches. This means that
a ostree tree created with a device-tree in this manner will only boot on
a single model of hardware. This is sufficient for my purposes, but may
not be for others'.
I've tested this on my NVidia Tegra TK1 device which has u-boot running
in syslinux-compatible mode.
Closes: #1411
Approved by: cgwalters
In particular I'd like to get the `--copyup` changes out for an rpm-ostree
release that will use them. But there are other good changes here, and let's
keep up a regular release train 🚄 in general.
Closes: #1413
Approved by: jlebon
Previously we were doing e.g. `ot_util_filename_validate()` specifically inline
in dirtree objects, but only *after* writing them into the staging directory (by
default). In (non-default) cases such as not using a transaction, such an object
could be written directly into the repo.
A notable gap here is that `pull-local --untrusted` was *not* doing
this verification, just checksums. We harden that (and also the
static delta writing path, really *everything* that calls
`ostree_repo_write_metadata()` to also do "structure" validation
which includes path traversal checks. Basically, let's try hard
to avoid having badly structured objects even in the repo.
One thing that sucks in this patch is that we need to allocate a "bounce buffer"
for metadata in the static delta path, because GVariant imposes alignment
requirements, which I screwed up and didn't fulfill when designing deltas. It
actually didn't matter before because we weren't parsing them, but now we are.
In theory we could check alignment but ...eh, not worth it, at least not until
we change the delta compiler to emit aligned metadata which actually may be
quite tricky. (Big picture I doubt this really matters much right now
but I'm not going to pull out a profiler yet for this)
The pull test was extended to check we didn't even write a dirtree
with path traversal into the staging directory.
There's a bit of code motion in extracting
`_ostree_validate_structureof_metadata()` from `fsck_metadata_object()`.
Then `_ostree_verify_metadata_object()` builds on that to do checksum
verification too.
Closes: #1412
Approved by: jlebon
While we do protect against path traversal during pull, let's also validate
during checkout; it's a cheap operation and provides good last-mile protection.
Closes: #1412
Approved by: jlebon
I was reading about a recent security issue with both EMC and VMWare:
https://arstechnica.com/information-technology/2018/01/emc-vmware-security-bugs-throw-gasoline-on-cloud-security-fire/
It's a classic path traversal problem, and that made me think more about our
handling of this in libostree. Fortunately of course, not being new to
this rodeo, long ago I *did* consider path traversal. Inside the pull
code, we call `ot_util_filename_validate()`. Also, `fsck` does this too.
I have further followups here, but let's add some test cases for this. I crafted
a repository with a `../` in a dirtree object by patching libostree to inject
it, and that's included as a tarball.
This patch covers the two cases where we do already have checks; pulling
via HTTP, and in `fsck`.
Closes: #1412
Approved by: jlebon
`grep` supports checking multiple fixed strings separated by newlines,
but it's mostly just easier to pass them as separate arguments, so let's
support that. This is now at parity with the similar
`assert_file_has_content`.
Closes: #1409
Approved by: cgwalters
All the current uses of the find-remotes command in the tests use it to
find configured remotes or mounted (USB) remotes, so using
--finders=config and --finders=mount in the tests respectively shouldn't
affect the correctness of the tests. It does however allow the tests to
be run in an environment that doesn't have an Avahi daemon.
Closes: #1407
Approved by: cgwalters
Previously when initramfs-* was not found in a deployment's
boot directory, it was assumed that rootfs is prepared for
ostree booting by a kernel patch.
With this patch, the behaviour changes to be - if initramfs-*
is not found, assume that system is using a static
ostree-prepare-root as init process. Booting without initramfs
is a common use case on embedded systems. This approach is
also more convenient, than having to patch the kernel.
Closes: #1401
Approved by: cgwalters
Clients of libostree such as rpm-ostree make extensive use of the
`ostree commit -b foo --tree=ref=foo` pattern in their tests, e.g. to
simulate an update.
What I'm trying to solve here is that it's often the case that we want
to keep metadata from the previous commit without having to be too
verbose (i.e. reading from the parent, then passing it as an argument).
The new `--keep-metadata` switch makes this really easy. I intend to use
this in the rpm-ostree testsuite to make sure we always carry over the
`source-title` metadata as well as during set up for tests that require
`rpmostree.rpmdb.pkglist` metadata.
I initially implemented this in a small wrapper script that uses the API
directly, though we make use of so many other `ostree commit` functions
that it'd require re-implementing a lot of it.
Closes: #1402
Approved by: cgwalters
This tripped up the `docbook-dtds` `%post` in my experiments
with doing rpm-ostree for buildroots.
I cloned and built [xfstests](https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git)
but haven't yet investigated actually running it.
In the meantime let's do the obvious fix here; we need to distinguish
between "copyup enabled" and "actually did a copyup" in the open path
at least, since if we didn't do a copyup we don't need to re-open.
Closes: #1396
Approved by: jlebon
Nothing guarantees that each user has a group containing only
themselves. Even if they do, nothing guarantees that its group ID
equals the user ID, particularly if another user earlier in the same
range was created without a corresponding group or vice versa.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Closes: #1390
Approved by: cgwalters
Let's do a new release with the locking preview, the http2 disable options and
other misc bugfixes to close out the year.
Closes: #1386
Approved by: jlebon
We had this basically forced on in the CLI; down the line I'd really like to
make this an API option to commit or so, but given that we found a use case in
the rpm-ostree test suite for "unbound" commits, let's support creating them
from the cmdline.
See: https://github.com/ostreedev/ostree/pull/1379Closes: #1380
Approved by: jlebon
Today the rpm-ostree test suite uses `refs --create` to save
commits. I think this is a legitimate use case, and other
people may be doing something similar.
On the other hand, I think we should probably be changing the rpm-ostree test
suite to create "unbound" commits. But let's be maximially compatible here since
we hit a real-world case where something needed to change.
Closes: #1379
Approved by: pwithnall
I want some time to play with this more with different callers and work through
test scenarios. Let's disable the locking by default for now, but make it easy
to enable.
Closes: #1375
Approved by: jlebon
libtest-core.sh tries to clear the locale to a UTF-8 supporting C
locale, either by setting it to C.UTF-8 (preferred) or just C.
Some systems, like Fedora 26, use the locale name C.utf8, rather than
C.UTF-8. Support that too.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1347
Approved by: cgwalters
Mostly adding this for use in test cases; it allows us to add e.g.
integers, and we need to deal with byteswapping those.
Someone mind also find it useful to add fully structured metadata, although most
of those users should be using a real language and not shell script.
Closes: #1372
Approved by: jlebon
A while ago I did `truncate -s 0 /path/to/repo/00/123.commit`, and expected a
checksum error, but I actually got a validation error due to us loading the
commit into a variant and trying to parse out the parent checksum, etc.
I first started by changing the `load_and_fsck_one_object()` function to
checksum before loading, but the problem is that we do a traverse of all objects
first. Fixing this is going to require an `OSTREE_REPO_COMMIT_TRAVER_FLAG_FSCK`
or something.
In the meantime at least though, let's add a public API to fsck a single object
which *does* checksum cleanly before parsing the object, and change the `fsck`
command to use it.
We then change the fsck binary to do this while iterating over the refs
and finding the commit object. This way we'll at least get a checksum
first for commit objects, even if not dirtree/dirmeta.
Closes: #1364
Approved by: jlebon
This commit fixes an infinite loop that happens if you try to list the
remotes of a repo that has a parent repo set. It also adds a unit test
to ensure the right behavior, which is that both the child remotes and
parent remotes are listed.
Closes: #1366
Approved by: cgwalters
