IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Rather than manually starting the `ostree-finalize-staged.service` unit,
we can leverage systemd's path units for this. It fits quite nicely too,
given that we already have a path we drop iif we have a staged
deployment.
To give some time for the preset to make it to systems, we don't yet
drop the explicit call to `systemctl start`. Though we do make it
conditional based on a DEBUG env var so that we can actually test it in
CI for now. Once we're sure this has propagated, we can drop the
`systemctl start` path and the env var together.
Closes: #1740
Approved by: cgwalters
This commit disables searching on the local network for refs, unless
explicitly requested by the user either by changing the value of the
"core.repo-finders" config option, or by passing an OstreeRepoFinderAvahi to
ostree_repo_find_remotes_async() / ostree_repo_finder_resolve_async(),
or by specifying "lan" in the --finders option of the find-remotes
command.
The primary reason for this is that ostree_repo_find_remotes_async()
takes about 40% longer to complete with the LAN finder enabled, and that
API is used widely (e.g. in every flatpak operation). It's also probable
that some users don't want ostree doing potentially unexpected traffic
on the local network, even though everything pulled from a peer is GPG
verified.
Flathub will soon deploy collection IDs to everyone[1] so these code
paths will soon see a lot more use and that's why this change is being
made now.
Endless is the only potential user of the LAN updates feature, and we
can revert this patch on our fork of ostree. For it to be used outside
Endless OS we will need to upstream eos-updater-avahi and
eos-update-server into ostree.
[1] https://github.com/flathub/flathub/issues/676Closes: #1758
Approved by: cgwalters
Currently libostree essentially has two modes when it's pulling refs:
the "legacy" code paths pull only from the Internet, and the code paths
that are aware of collection IDs try to pull from the Internet, the
local network, and mounted filesystems (such as USB drives). The problem
is that while we eventually want to migrate everyone to using collection
IDs, we don't want to force checking LAN and USB sources if the user
just wants to pull from the Internet, since the LAN/USB code paths can
have privacy[1], security[2], and performance[3] implications.
So this commit implements a new repo config option called "repo-finders"
which can be configured to, for example, "config;lan;mount;" to check
all three sources or "config;mount;" to disable searching the LAN. The
set of values mirror those used for the --finders option of the
find-remotes command. This configuration affects pulls in three places:
1. the ostree_repo_find_remotes_async() API, regardless of whether or
not the user of the API provided a list of OstreeRepoFinders
2. the ostree_repo_finder_resolve_async() /
ostree_repo_finder_resolve_all_async() API
3. the find-remotes command
This feature is especially important right now since we soon want to
have Flathub publish a metadata key which will have Flatpak clients
update the remote config to add a collection ID.[4]
This effectively fixes https://github.com/flatpak/flatpak/issues/1863
but I'll patch Flatpak too, so it doesn't pass finders to libostree only
to then have them be removed.
[1] https://github.com/flatpak/flatpak/issues/1863#issuecomment-404128824
[2] https://github.com/ostreedev/ostree/issues/1527
[3] Based on how long the "ostree find-remotes" command takes to
complete, having the LAN finder enabled slows down that step of the
pull process by about 40%. See also
https://github.com/flatpak/flatpak/issues/1862
[4] https://github.com/flathub/flathub/issues/676Closes: #1758
Approved by: cgwalters
Previously, we were preparing the root very late in the boot process;
right before we switch root. The issue with that is that most services
in the initrd that run `After=initrd-root-fs.target` expect that
`/sysroot` already points to the rootfs we'll be pivoting to. Running
this late violates that assumption.
This patch fixes this by making `ostree-prepare-root.service` instead
run right after `sysroot.mount` (the physical sysroot mounted by
systemd) but still before `initrd-root-fs.target` (which is the target
signalling that `/sysroot` is now valid and ready).
This should make it easier to integrate OSTree with other initrd
services such as Ignition.
Related: https://github.com/dustymabe/ignition-dracut/issues/20Closes: #1759
Approved by: cgwalters
For the same reasons as #1697. This is especially important in services
that are likely to be used as an `After/Before=` target in other units.
`ostree-prepare-root.service` is one such service.
Closes: #1759
Approved by: cgwalters
See https://github.com/projectatomic/rpm-ostree/issues/1568
Basically for people on e.g. rotational media, the default 90
second timeout can be too small.
We're in a tough situation here, because delaying shutdown
can be problematic too if the user is trying to shut down their
laptop to put in a backpack, etc.
There's potential optimizations here to make; I think we
could pre-copy the kernel/initramfs for example.
I suspect for some people the grub2 os-prober is a factor here too,
if that tries to e.g. inspect attached USB rotational hard drives.
But hopefully we'll get rid of that soon.
Closes: #1755
Approved by: jlebon
I was debugging some rpm-ostree work and saw:
`openat: No such file or directory`
and it wasn't immediately obvious it was stderr from `rofiles-fuse`.
Use the `err` API which is better in many ways; in this case
it automatically prefixes with `argv0`.
Closes: #1747
Approved by: jlebon
The idea is that if the process is running as root, it can change
ownership of newly written files to match the owner of the repo.
Unfortunately, it currently applies in the other direction, too - a
non-root user writing to a root owned repository. If the repo is
writable by the user but owned by root, it can still create files and
directories there, but it can't change ownership of them.
This feature comes from
https://bugzilla.gnome.org/show_bug.cgi?id=738954. As it turns out, this
feature was never completed. It only works on content objects and not
metadata objects, refs, deltas, summaries, etc. Rather than try to fix
all of those, remove the feature until someone has interest in
completing it.
Closes: #1754
Approved by: cgwalters
Actually testing the patch to add `--force-copy-zerosized` to
rpm-ostree tripped over the fact that it uses `--union-identical`,
and we just hit an assertion failure with that combination.
Fix this by copying over the logic we have for the hardlink case.
Closes: #1753
Approved by: jlebon
In rpm-ostree we've hit a few cases where hardlinking zero-sized
files causes us problems. The most prominent is lock files in
`/usr/etc`, such as `/usr/etc/selinux/semanage.LOCK`. If there
are two zero-sized lock files to grab, but they're hardlinked,
then locking will fail.
Another case here is if one is using ostree inside a container
and don't have access to FUSE (i.e. `rofiles-fuse`), then the
ostree hardlinking can cause files that aren't ordinarily hardlinked
to become so, and mutation of one mutates all. An example where
this is concerning is Python `__init__.py` files.
Now, these lock files should clearly not be in the tree to begin
with, but - we're not gaining a huge amount by hardlinking these
files either, so let's add an option to disable it.
Closes: #1752
Approved by: jlebon
Deleting a ref with aliases makes them dangling. In such
cases, display an error message to the user.
Fixes#1597
Signed-off-by: Sinny Kumari <sinny@redhat.com>
Closes: #1749
Approved by: cgwalters
Write to the journal when starting to finalize a staged deployment.
Combined with the "Transaction completed" message we already emit, this
makes it easy later on to determine whether the operation was successful
by inspecting the journal. This will be used by `rpm-ostree status`.
Closes: #1750
Approved by: cgwalters
It's a neat way to point folks to the documentation (of course, better
would be to have man pages for each of those services). Also
consistently use Title Case everywhere.
Closes: #1750
Approved by: cgwalters
This commit removes the recommendation in the create-usb man page for
the user to update the summary in the source repo before using the
create-usb command. I'm not sure where I got the idea that create-usb
depends on a summary in the source repo. I went back to the first commit
that introduced the create-usb command and even using that a summary
isn't required, so it seems unlikely that this changed recently.
This is good news because the exclusive lock that's taken for summary
updates has been causing problems on Endless (due to other processes
having a lock for the duration of the 30 second acquire time out
period).
Closes: #1746
Approved by: cgwalters
There are use cases for libostree as a local content store
for content derived or delivered via other mechanisms (e.g. OCI
images, RPMs, etc.). rpm-ostree today imports RPMs into OSTree
branches, and puts the RPM header value as commit metadata.
Some of these can be quite large because the header includes
permissions for each file. Similarly, some OCI metadata is large.
Since there's no security issues with this, support committing
such content.
We still by default limit the size of metadata fetches, although
for good measure we make this configurable too via a new
`max-metadata-size` value.
Closes: https://github.com/ostreedev/ostree/issues/1721Closes: #1744
Approved by: jlebon
This commit fixes the bash tab completion handling of the "--repo"
argument. Before this commit, the completion only works if "--repo"
comes after the main command. After this commit, you can use "--repo"
directly after "ostree" in the command line, as is natural.
Closes: #1745
Approved by: jlebon
2018.7 started failing `test-ucontainer.sh`. I don't have the cycles to
look more deeply into what was going on there, but bumping to 2018.8
fixes it at least. (And of course, it's passing in rpm-ostree too.)
Closes: #1728
Approved by: cgwalters
Currently the locking code checks if the value -1 was set for the config
key "lock-timeout-secs" and if so, a thread trying to acquire a lock
will block indefinitely. Positive values specify how long to attempt to
acquire a lock in a non-blocking way (the attempt is made once every
second). But when the value is read from the config file,
g_ascii_strtoull() is used, which converts it to an unsigned integer.
This commit makes libostree use g_ascii_strtoll() instead, so that it's
possible to set that key to -1 as intended.
Closes: #1737
Approved by: jlebon
Currently on Endless OS, the OSTree ref for the operating system is
something like os/eos/amd64/eos3, so that's what gets passed to `ostree
create-usb` when copying the OS to a USB drive (for offline updates).
However, when eos-updater checks for updates it pulls the metadata for a
candidate commit and in so doing updates that eos3 ref to point to the
partial commit being examined as a potential update rather than the
deployed commit. This causes `ostree create-usb` to fail with an error
like "No such metadata object
7fb045cb2d1f1f3a81bfc157c6128ff443eb56350315b9536bdb56aee0659863.dirtree".
OSTree creates deployment refs that look like "ostree/1/1/0" to maintain
a pointer to the deployed commit, but create-usb can't use these because
it shows up in the summary as just a ref, not a collection-ref.
So this commit adds a --commit option to the create-usb command, so we
can use the appropriate ref but copy the deployed commit rather than a
(potentially partial) update commit.
Closes: #1735
Approved by: cgwalters
Copying the xattrs on metadata objects is wrong in general, we
don't "own" them. Notably this would fail in the situation of
doing a pull from e.g. a `bare-user` source to a destination
that was on a different mount point (so we couldn't hardlink),
and the source had e.g. a `security.selinux` attribute.
Closes: #1734Closes: #1736
Approved by: jlebon
Change the create-usb command so that it always creates the destination
repository using the "archive" mode, rather than using archive mode when
xattrs aren't supported and bare-user otherwise. This has a few
advantages:
1. The archive mode works with FAT filesystems, which is what most
USB drives are, and which doesn't support xattrs.
2. At least in some quick testing I did, archive mode is about
twice as performant as bare-user mode, in terms of how long it takes for
the create-usb command to complete.
3. This ensures that a tool can safely change the permissions on
".ostree/repo" and subdirectories after create-usb completes, which is
important for Endless since otherwise you can't use `ostree create-usb`
as root and then `flatpak create-usb` as a non-root user on the same USB
drive (or in other words copy OS updates and apps to the same USB).
Closes: #1733
Approved by: cgwalters
Just include the whole URL that failed if libcurl failed with something
elementary like CURLE_COULDNT_CONNECT or CURLE_COULDNT_RESOLVE_HOST.
Closes: #1731Closes: #1732
Approved by: cgwalters
There's some subtlety to this, we don't handle all cases.
But the 99% cases are using `--sysroot deploy` to create an
initial deployment, and then doing upgrades from inside
a booted deployment.
It was only the latter case that didn't work with a separate `/var`.
Fixing all of them would probably require libostree to learn
how to e.g. look at `/etc/fstab` (or worse, systemd mount units?)
and handle the mounting. I don't think we want to do anything
like that right now, since there are no active drivers for the
use case.
Closes: https://github.com/ostreedev/ostree/issues/1729Closes: #1730
Approved by: akiernan
Earlier, the actual reserved space (in blocks) were calculated inside the
transaction codepath ostree_repo_prepare_transaction(). However, while
reworking on ostree_repo_get_min_free_space_bytes() API, it was realized that
this calculation can be done independently from the transaction's codepaths, hence
enabling the usage for ostree_repo_get_min_free_space_bytes() API irrespective
of whether there is an ongoing transaction or not.
https://github.com/ostreedev/ostree/issues/1720Closes: #1722
Approved by: pwithnall
This commit defines a metadata key that tells clients to update their
remote config to add a collection ID. This functionality is currently
implemented in Flatpak for the key "xa.collection-id", but there are two
good reasons for moving the key to OSTree:
1) Servers such as Flathub shouldn't set xa.collection-id in their
metadata now or in the medium term future, because many users are still
using old versions of Flatpak and OSTree[1] which would hit various
bugs[2][3][4] on the P2P code paths that are enabled by collection IDs.
Defining a new key means that only clients running recent
(as-yet-unreleased) versions of Flatpak and OSTree will pay attention to
it and deploy the collection ID, leaving the users on old versions
unaffected.
2) OSTree is as "invested" in collection IDs as Flatpak, so there's no
reason the key should be defined in Flatpak rather than here. According
to Philip Withnall, the reason the key was put in Flatpak originally was
that at the time there was uncertainty about tying OSTree to collection
IDs.
[1] https://ahayzen.com/direct/flathub.html#downloadsbyflatpakstacked
[2] https://github.com/ostreedev/ostree/commit/e4e6d85ea
[3] https://github.com/flatpak/flatpak/commit/5813639f
[4] https://github.com/flatpak/flatpak/commit/5b21a5b7Closes: #1726
Approved by: pwithnall
There is no API method to remove a file or subdirectory from a MutableTree
besides directly manipulating the GHashTable returned by _get_files or
_get_subdirs. This isn't possible from an introspection binding that transforms
the returned GHashTable, and may also leave the tree checksum in an invalid
state. Introduce a new method so that removing files or subdirectories is
safe, and possible from bindings.
Closes: #1724
Approved by: jlebon
This fixes typos and grammar in the docs for OstreeRepo, and copies the
information about OSTREE_REPO_MODE_BARE_USER_ONLY from ostree-core.h
Closes: #1725
Approved by: jlebon
It started failing with:
```
ERROR: tests/check/test-ucontainer.sh - too few tests run (expected 2, got 0)
tap-driver.sh: internal error getting exit status
tap-driver.sh: fatal: I/O or internal error
make[4]: *** [Makefile:4353: tests/check/test-ucontainer.sh.log] Error 1
make[4]: *** Waiting for unfinished jobs....
```
And the artifacts are not being saved for some reason:
```
+ cleanup
+ mv test-suite.log /var/tmp/checkout
mv: cannot stat 'test-suite.log': No such file or directory
+ true
+ mv vmcheck /var/tmp/checkout
mv: cannot stat 'vmcheck': No such file or directory
+ true
```
Let's just disable this for now so that some other pending patches can
go in while we investigate.
Closes: #1727
Approved by: cgwalters
In the OstreeRepoFinderAvahi implementation,
ostree_avahi_service_build_repo_finder_result() is where the DNS-SD
records are processed and turned into OstreeRepoFinderResult objects.
Each result object is supposed to have a hash table mapping refs to
checksums, so this is accomplished by first adding a placeholder (a ref
mapping to a NULL checksum) for each ref matched by the bloom filter,
and later filling in the checksums using the remote's summary file,
which happens in get_checksums(). The problem is that there's no
guarantee all the checksums will be resolved (non-NULL), so the
ostree_repo_finder_result_new() call then hits an assertion failure in
is_valid_collection_ref_map() leading to a crash (in the case that one
or more refs had NULL checksums).
There are at least two situations where the ref checksum might not be
found in the peer remote's summary file:
1) The bloom filter match was a false positive. This is going to happen
sometimes by design.
2) The peer remote's summary is out of sync with its DNS-SD records.
This shouldn't normally happen but it's still good to be robust to the
possibility; in Endless OS nothing guarantees the atomicity of updating
the summary and DNS-SD records.
This commit changes libostree to be robust to the possibility of refs
missing from the peer remote's summary, by removing any that still have
a NULL checksum associated with them after the summary has been fetched
and processed.
The other OstreeRepoFinder implementations don't have this issue because
they use summary files directly and therefore always have access to the
checksum.
Closes: #1717
Approved by: pwithnall
when converted to bytes
In a subsequent commit, we add a public API to read the value of
min-free-space-* value in bytes. The value for free space check
is enforced in terms of block size instead of bytes. Therefore,
for consistency we check while preparing the transaction that the
value doesn't overflow when converted to bytes.
https://phabricator.endlessm.com/T23694Closes: #1715
Approved by: cgwalters
Debian and Debian-derived systems have their GRUB configuration file in
/boot/grub/grub.cfg, rather than /boot/grub2/grub.cfg. Detecting this
file is necessary to correctly generate GRUB boot configuration on
Debian systems.
Closes: #1714
Approved by: cgwalters
This adds the Contributing Tutorial (contributing-tutorial.md) to
the pages setting of Mkdocs, so that the tutorial will render
in the readthedocs.io documentation.
Closes: #1711
Approved by: jlebon
