IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
ENOTSUP and EOPNOTSUPP are numerically equal on most Linux ports,
but inexplicably differ on PA-RISC (hppa) and possibly other
rare architectures.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Closes: #1275
Approved by: cgwalters
There's a subtle issue going on with the way we use `UNION_IDENTICAL`
now in rpm-ostree. Basically, the crux of the issue is that we checkout
the whole tree from the system repo, but then overlay packages by
checking out from the pkgcache repo. This is an easy way to break the
assumption that we will be merging hardlinks from the same repo.
This ends up causing issues like:
https://github.com/projectatomic/rpm-ostree/issues/1047
There, `vim-minimal` is already part of the host and has an object for
`/usr/share/man/man1/ex.1.gz`. `vim-common` has that same file, but
because it's unpacked in the pkgcache repo first, the hardlinks are not
the same.
There are a few ways we *could* work around this in rpm-ostree itself,
e.g. by re-establishing hardlinks when we do the content pull into the
system repo, but it still felt somewhat hacky. Let's just do this the
proper way and fall back to checksumming the target file if needed,
which is what librpm does as well in this case. Note that we only
checksum if they're not hard links, but they're the same size.
Closes: #1258
Approved by: cgwalters
In case a filename contains invalid UTF-8 characters, libostree will
pass it to g_variant_builder_add() in create_tree_variant_from_hashes()
anyway, which leads to a critical warning from glib and an invalid
commit. This commit makes ostree print a useful error and exit instead.
Closes: #1271
Approved by: cgwalters
This is like `ostree_checksum_file` but fd-relative. This will be used
by https://github.com/ostreedev/ostree/pull/1258.
AFAICT, we actually didn't have any tests that check the `checksum` CLI.
Add a basic one here to test the old code as well as the new code.
Closes: #1263
Approved by: cgwalters
This ends up a lot better IMO. This commit is *mostly* just
`s/glnx_close_fd/glnx_autofd`, but there's also a number of hunks like:
```
- if (self->sysroot_fd != -1)
- {
- (void) close (self->sysroot_fd);
- self->sysroot_fd = -1;
- }
+ glnx_close_fd (&self->sysroot_fd);
```
Update submodule: libglnx
Closes: #1259
Approved by: jlebon
For many cases of commit, we can actually optimize things by simply "adopting"
the object rather than writing a new copy. For example, in rpm-ostree package
layering.
We can only make that optimization though if we take ownership of the file. This
commit hence adds an API where a caller tells us to do so. For now, that just
means we `unlink()` the files/dirs as we go, but we can now later add the
"adopt" optimization.
Closes: #1255
Approved by: jlebon
I was reading the pull code for the last release, and spotted
a bug in commit f923c2e1ea - in
the case where the ref doesn't exist, we don't set an error,
tripping an assertion in the main code.
The previous code wanted the ref to always exist, so just flip back the boolean
for "ignore noent". I moved the `g_strchomp()` just into the HTTP path - if a
local repo is corrupted in this way it's something to fix in that repo.
Closes: #1238
Approved by: pwithnall
This is the new way of publishing repository metadata, rather than as
additional-metadata in the summary file. The use of an ostree-metadata
ref means that the metadata from multiple upstream collections is not
conflated when doing P2P mirroring of many repositories.
The new ref is only generated if the repository has a collection ID set.
The old summary file continues to be generated for backwards
compatibility (and because it continues to be the canonical ref →
checksum map for the repository).
The new code is only used if configured with --enable-experimental-api.
Includes unit tests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1158
Approved by: cgwalters
Instead of checking for overlayfs, let's explicitly check for our
ability to relabel files since we now have a `libtest` function to do
this. Also port that logic to `libostreetest`.
Note that overlayfs *does* allow manipulating user xattrs. So ideally,
we should break down `OSTREE_NO_XATTRS` further to distinguish between
tests that use bare repos from other modes.
We check the current directory instead of `/` so that developers can
just point `TEST_TMPDIR` to a non-overlayfs mount point when hacking
from a container.
Closes: #1170
Approved by: cgwalters
We can't use the cache if the file we want to commit has been modified
by the client through the file info or xattr modifiers. We would
prematurely look into the cache in `write_dfd_iter_to_mtree_internal`,
regardless of whether any filtering applied.
We remove that path there, and make sure that we only use the cache if
there were no modifications. We rename the `get_modified_xattrs` to
`get_final_xattrs` to reflect the fact that the xattrs may not be
modified.
One tricky bit that took me some time was that we now need to store the
st_dev & st_ino values in the GFileInfo because the cache lookup relies
on it. I'm guessing we regressed on this at some point.
This patch does slightly change the semantics of the xattr callback.
Previously, returning NULL from the cb meant no xattrs at all. Now, it
means to default to the on-disk state. We might want to consider putting
that behind a flag instead. Though it seems like a more useful behaviour
so that callers can only override the files they want to without losing
original on-disk state (and if they don't want that, just return an
empty GVariant).
Closes: #1165Closes: #1170
Approved by: cgwalters
Test an end-to-end flow of pulling refs from an online
repository → local OS repository → create a USB stick of
them → pull to a local OS repository on another machine.
This is a manual test, as it requires a throwaway USB stick which the
test can format as ext4 or vfat to test the flow works with both file
systems.
Run it as:
MOUNT_INTEGRATION_DEV=/dev/sdb1 make check \
TESTS=tests/test-repo-finder-mount-integration.sh
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1203
Approved by: cgwalters
I was trying to debug this while working on another PR. Add `echo ok` lines and
also use `assert_file_has_content` more directly; there's no reason to use an
intermediate `grep` since if it fails it won't print the original file.
Closes: #1220
Approved by: jlebon
Followup for recent work in commits:
- 8a7a359709
- 1a9a473580
Keep track of how many objects we imported, and print that for `ostree
pull-local` (also do this even if noninteractive, like we did for `pull`).
In implementing this at first I used separate variables for import
from repo vs import from localcache, but that broke some of the
tests that checked those values.
It's easier to just merge them; we know from looking at whether or not
`remote_repo_local` is set whether or not we were doing a "HTTP pull with
localcache" versus a true `pull-local` and can use that when rendering status.
Closes: #1219
Approved by: jlebon
This can be used to put OSTree repositories on USB sticks in a format
recognised by OstreeRepoFinderMount.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1182
Approved by: cgwalters
Previously, collection–refs could only be pulled from a repository if it
had a summary file (which listed them). There was no way to pull from a
local repository which doesn’t have a summary file, and where the refs
were stored as refs/remotes/$remote/$ref, with a config section linking
that $remote to the queried collection ID.
Fix that by explicitly supporting pull_data->remote_repo_local in
fetch_ref_contents().
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #1182
Approved by: cgwalters
Our CI uses default Docker, which has SELinux labeling but is rather
evil in returning `EOPNOTSUPP` to any attempts to set `security.selinux`,
even if to the same value.
The previous fire 🔥 for this was: https://github.com/ostreedev/ostree/pull/759
The `bare` repo mode really only makes sense as uid 0, so our installed
test framework is a good match for this. However, the unit tests *do*
work in a privileged container even as non-root, and *also* should
work on SELinux-disabled systems. So let's teach the test framework
how to skip in those situations.
I tested this both in a priv container (my default builder) and an unpriv
container (like our CI).
At the same time, start executing the `test-basic.sh` from an installed test,
so we get better coverage than before.
This is just the start - all of the sysroot tests really need the
same treatment.
Closes: #1217
Approved by: jlebon
I now think commit fab1e113db was a mistake;
because it breaks the mental model that at least I'd built up that "local repos
don't have checksums verified, HTTP does".
For example, a problem with this is (with that mental model in place) it's easy
for people who set up mirrors like this to then do local pulls, and at that
point we've done a deployment with no checksum verification.
Further, since then we did PR #671 AKA commit 3d38f03 which is really most of
the speed hit.
So let's switch the default even for this case to doing checksum verification,
and add `ostree pull --http-trusted`. People who are in situations where they
know they want this can find it and turn it on.
Closes: https://github.com/ostreedev/ostree/issues/1211Closes: #1212
Approved by: jlebon
This fixes up the last of the embarassing bits I saw from
the stack trace in:
https://github.com/ostreedev/ostree/issues/1184
We had a hardlink fast path, but that doesn't apply across
devices, which occurs in two notable cases:
- Installer ISO with local repo
- Tools like pungi that copy the repo to a local snapshot
Obviously there are a lot of subtleties here around things like the
bare-user-only conversions as well as exactly what data we copy. I think to get
better test coverage we may want to add `pull-local --no-hardlink` or so.
Closes: #1197
Approved by: jlebon
We use `setfattr` to determine whether the filesystem we're on supports
xattrs, but we need to check that `setfattr` itself is available. We
just make it a hard requirement but only if trying to run tests that ask
about xattr support.
Closes: #1207
Approved by: cgwalters
We added a `.dir-locals.el` in commit: 9a77017d87
There's no need to have it per-file, with that people might think
to add other editors, which is the wrong direction.
Closes: #1206
Approved by: jlebon
Add a hash function for OstreeRepo instances, which relies on the repo
being open, and hence being able to hash the device and inode of its
root directory.
Add unit tests for this and ostree_repo_equal().
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/ostreedev/ostree/issues/1191Closes: #1205
Approved by: cgwalters
Conceptually `ostree-repo-pull.c` should be be written using
just public APIs; we theoretically support building without HTTP
for people who just want to use the object store portion and
do their own fetching.
We have some nontrivial behaviors in the pull layer though; one
of those is the "bareuseronly" verification. Make a new internal
API that accepts flags, move it into `commit.c`. This
is prep for further work in changing object import to support
reflinks.
Closes: #1193
Approved by: jlebon
In the `O_RDONLY` case, we were calling `openat` without a mode
argument. However, it's perfectly legal (albeit unusual) to do
`open(O_RDONLY|O_CREAT)`. One such application that makes use of this is
`flock(1)`.
This was actually caught by `_FORTIFY_SOURCE=2`, and once we run
`rofiles-fuse` with `-f`, the message is clear:
```
*** invalid openat64 call: O_CREAT or O_TMPFILE without mode ***:
rofiles-fuse terminated
======= Backtrace: =========
/lib64/libc.so.6(+0x7c8dc)[0x7f36d9f188dc]
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f36d9fbfaa7]
/lib64/libc.so.6(+0x10019a)[0x7f36d9f9c19a]
rofiles-fuse[0x401768]
...
```
Without `_FORTIFY_SOURCE`, the file gets created, but its mode is
completely random.
I ran into this while investigating
https://github.com/projectatomic/rpm-ostree/pull/1003.
Closes: #1200
Approved by: cgwalters
There are use cases for not syncing at all; think build cache repos, etc. Let's
be consistent here and make sure if fsync is disabled we do no sync at all.
I chose this opportunity to add tests using the shiny new strace fault
injection. I can forsee using this for a lot more things, so I made
the support for detecting things generic.
Related: https://github.com/ostreedev/ostree/issues/1184Closes: #1186
Approved by: jlebon
Make the equivalent of a `GOnce` or Rust `lazy_static!` detecting this and share
it between the two callers.
Prep for a future similar patch for strace fault injection.
Closes: #1186
Approved by: jlebon
In test-basic-root.sh we make assertions about the permissions
of files like baz/cow, which were created without an explicit chmod.
We can't do that unless we control the permissions.
For some reason the "debomatic" autobuilder used to do some Debian
archive rebuilds does the entire build including build-time tests
as uid 0 with umask 002, which broke those assertions. This seems
a weird thing to do, and I've opened a bug, but it also seems
reasonable to fix this test.
This also lets us remove a couple of existing workarounds for the
same issue.
Bug-Debian: https://bugs.debian.org/876138
Signed-off-by: Simon McVittie <smcv@collabora.com>
Closes: #1192
Approved by: cgwalters
As a GNU extension, LANGUAGE takes precedence over LC_ALL for
gettext(3) whenever the locale is not C, causing tests that grep for
specific English strings to fail when run in non-English locales.
The upstream glibc proposal for C.UTF-8 would give C.UTF-8 the same
special case as C here, but the implementation in Debian does not
currently have this, so we have to unset LANGUAGE too.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Closes: #1188
Approved by: jlebon
See issue #1174 for the rationale behind this. In summary:
• It required two lists of collection–refs to be maintained: one in the
repository, and one pointing to the repository.
• It didn’t automatically work for live USBs of OSs based on OSTree
(where there’s always a repository at /ostree/repo).
• It was unnecessarily complex.
The new scheme allows a list of repositories to be searched, but without
needing a layer of indirection through their collection–refs. It adds
/ostree/repo and /.ostree/repo as well-known repository locations which
are always checked on a mounted volume (if they exist).
Update the unit tests accordingly.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
https://github.com/ostreedev/ostree/issues/1174Closes: #1179
Approved by: cgwalters
In 5c940987e7 / #646 we
added `--retain-branch-depth`; this adds a symmetric
`--only-branch` for the case where a repo owner just
wants to prune a specific branch.
The implementation here is pretty straightforward; we
just walk all refs and inject the equivalent of
`--retain-branch-depth=$ref=-1` if they're *not* in
`--only-branch`.
Closes: https://github.com/ostreedev/ostree/issues/1115Closes: #1127
Approved by: jlebon
Update libglnx, which is mostly port the repo stagedir code
to the new tmpdir API. This turned out to require some
libglnx changes to support de-allocating the tmpdir ref while
still maintaining the on-disk dir.
Update submodule: libglnx
Closes: #1172
Approved by: jlebon
Because it runs a binary in ${libexecdir}, it will only work if
libostree was already installed, which makes the build-time test fail.
It also doesn't produce the output we expect: its usage mechanism
mentions "ostree-trivial-httpd", not "ostree trivial-httpd".
Signed-off-by: Simon McVittie <smcv@collabora.com>
Closes: #1177
Approved by: jlebon
In recent gjs, you can't declare a variable with "let" multiple times.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Closes: #1178
Approved by: cgwalters
It turns out that librpm automatically merges identical files between
distinct packages, and this occurs in practice with Fedora today between
`chkconfig` and `initscripts` for exmaple.
Since we added this for rpm-ostree, we basically want to do what librpm does,
let's change the semantics to do a merge. While we're here rename
to `UNION_IDENTICAL`.
Closes: #1156
Approved by: jlebon
If the new configuration passed to ostree_write_config () tries to
update options for a remote defined in a separate config file, return an
error. Without this, the full configuration would contain duplicate
remote specifications, which would raise an error the next time the repo
is opened.
Closes: #1159
Approved by: cgwalters
The `sed` expression wasn't actually matching the main output, so we
weren't recursing into the subcommands. Update the syntax to match the
current output and add a check so we don't miss that happening again.
Add a check that the help output is only printed once in all
circumstances. Also add a check for proper handling of non-existent
commands.
Closes: #1126
Approved by: cgwalters
This option allows a repo to explicitly opt out of adding new remotes in
a remotes configuration directory. This currently defaults to true for
system repos and false for non-system repos to maintain legacy behavior
that non-system repos don't add remotes in a configuration directory.
That would be problematic for flatpak, which specifies a remotes config
dir but adds remotes in ways that are incompatible with it.
So, what this really does is allow system repos to control whether they
want to add remotes in the config dir or not. That's important if your
flatpak repo is the system repo like at Endless.
Closes: #1134Closes: #1155
Approved by: cgwalters
If you lchown("symlink") then we were incorrectly trying to chown the
symlink target, rather than the symlink itself. In particular, this cause
cp -a to fail for a broken symlink. Additionally, it was using the
symlink target when verifying writability, rather than the symlink
itself.
To fix this, we need pass AT_SYMLINK_NOFOLLOW in these cases.
In general, the kernel itself will always resolve any symlinks for us
before calling into the fuse backend, so we should really never do any
symlink following in the fuse fs itself. So, we pro-actively add
NOFOLLOW flags to a few other places:
truncate:
In reality this will never be hit, because
the kernel will resolve symlinks before calling us.
access:
It seems the current fuse implementation never calls this
(faccessat w/AT_SYMLINK_NOFOLLOW never reaches the fuse fs)
but if this ever is implemented this is the correct behaviour.
We would ideally do `chmod` but this is not implemented on current kernels.
Because we're not multi-threaded, this is OK anyways.
Further, our write verification wasn't correctly handling the case of hardlinked
symlinks, which can occur for `bare` checkouts but *not* `bare-user` which the
tests were using. Change to `bare` mode to verify that.
Closes: #1137
Approved by: alexlarsson
Steal some code from flatpak for this, which allows porting a few more things to
new style. I started on a public API version of this but was trying to roll some
other things into it and it snowballed. Let's do this version since it's easy
for now.
While here I changed things so that `generate_deployment_refs()` now just uses
`_set_ref_immediate()` rather than requring a txn.
Also, AFAICS there was no test coverage of `generate_deployment_refs()`; I tried
commenting it out and at least `admin-test.sh` passed. Add some coverage of this
- I verified that with this commenting out bits of that function cause the test
to fail.
Closes: #1132
Approved by: jlebon
Before commit e0346c1, a non-system repo could specify
remotes-config-dir and have remotes read from there. However, adding
remotes would only be done in the config dir for a system repo. Restore
that by respecting remotes-config-dir when no sysroot is found and
adding back the ostree_repo_is_system() check when adding remotes.
Closes: #1133Closes: #1151
Approved by: cgwalters
Include non-default deployments in the uEnv.txt file imported by
U-Boot. All the configurations beside the defaults will have
numerical suffix E.G. "kernel_image2" or "bootargs2".
Those U-Boot environment variables may be used from interactive boot
prompt or from "altbootcmd" script.
Closes: #1138
Approved by: cgwalters
