Commit Graph

4047 Commits

Author SHA1 Message Date
Robert Fairley
7baf167881 ostree/pull: Add network-retries command line option
This exposes a way to specify from the command line the number
of times to retry each download after a network error. If a negative
value is given, then the default number of retries (5) is used. If 0
is given, then errors are returned without retrying.

closes #1659

Closes: #1669
Approved by: jlebon
2018-07-05 17:59:09 +00:00
Robert Fairley
4f096c8f11 tests: Move assert_fail function to tests/libtest.sh
This moves the assert_fail function definition which was defined
and called in tests/test-remote-headers.sh. Done in preparation
for use of the assert_fail function in other test files.

Closes: #1669
Approved by: jlebon
2018-07-05 17:59:09 +00:00
Alex Kiernan
3696e1cfdd build: Use ostree_prepare_root_CPPFLAGS for ostree-prepare-root
Swap from AM_CPPFLAGS to ostree_prepare_root_CPPFLAGS when compiling
ostree-prepare-root statically. This fixes a problem when you have
systemd and libmount, but only ostree_prepare_root_CPPFLAGS includes
-DHAVE_SYSTEMD_AND_LIBMOUNT=1.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>

Closes: #1670
Approved by: jlebon
2018-07-04 19:45:50 +00:00
Jonathan Lebon
2cb2571127 tests/installed: Add NOTE when re-using RPMs
One gotcha here is that we don't invalidate the RPMs if we're not
sitting on the same commit anymore. Shouldn't be too hard to fix, though
let's at least make a note of it for now.

Closes: #1668
Approved by: cgwalters
2018-07-04 16:06:38 +00:00
Colin Walters
61ba4e7e5a tests/installed: Add var-mount.yml destructive test
Closes: #1668
Approved by: cgwalters
2018-07-04 16:06:38 +00:00
Jonathan Lebon
a13ea6497e switchroot: Fix regression for separately mounted /var
I made a logical error in #1617 which resulted in the exact *opposite*
behaviour we want when `/var` is a separate mount.

Split this out and lower the number of negations to make it more obvious
that it's correct.

Closes: #1667

Closes: #1668
Approved by: cgwalters
2018-07-04 16:06:38 +00:00
Marcus Folkesson
8af389b758 build: add ostree-soup-* to build process when configured with avahi
Avoid getting these link errors:
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_path'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_new'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_free'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_scheme'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_to_string'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_host'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_port'
    collect2: error: ld returned 1 exit status

Reproduce with:
    ./configure --with-avahi --without-soup

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1666
Approved by: cgwalters
2018-07-03 19:42:49 +00:00
Alex Kiernan
a0527e7086 boot: Use emergency.target, not emergency.service
Follow systemd units in using emergency.target, not emergency.service
(which is the sole unit, by default, in emergency.target) so we can
easily reconfigure the units which are actived when entering
emergency mode.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>

Closes: #1665
Approved by: cgwalters
2018-07-03 13:38:10 +00:00
Umang Jain
0c8b86ea09 lib/repo: Minor fixes around min-free-space
Summary:
* Remove a useless if condition in prepare_transaction()
* Fix glnx_throw error propagation
* Integer overflow check while parsing min-free-space-size config
* Documentation fixes

Closes: #1663
Approved by: jlebon
2018-07-03 12:59:26 +00:00
William Manley
488365f9bf OstreeMutableTree: Invalidate parent contents checksum when metadata changes
This bug has existed before the previous commit, but thanks to the previous
commit it is now easy to fix.

Closes: #1655
Approved by: cgwalters
2018-06-29 21:31:08 +00:00
William Manley
5b0dd1002e OstreeMutableTree: Refactor: Add parent pointer
This implements a TODO item from
`ostree_mutable_tree_get_contents_checksum`.  We now no-longer invalidate
the dirtree contents checksum at `get_contents_checksum` time - we
invalidate it when the mtree is modified.  This is implemented by keeping
a pointer to the parent directory in each `OstreeMutableTree`.  This gives
us stronger invariants on `contents_checksum`.

For even stronger guarantees about invariants we could make
`ostree_repo_write_mtree` or similar a member of `OstreeMutableTree` and
remove `ostree_mutable_tree_set_metadata_checksum`.

I think I've fixed a bug here too.  We now invalidate parent's contents
checksum when our metadata checksum changes, whereas we didn't before.

Closes: #1655
Approved by: cgwalters
2018-06-29 21:31:08 +00:00
Colin Walters
f2e71c60ec ci/flatpak: Fix to use built ostree version
Noticed as part of a random failure in this PR:
https://github.com/ostreedev/ostree/pull/1655
that we weren't actually testing the version of ostree
we built in git.  Probably we should be generating RPMs but...later.

Closes: #1662
Approved by: jlebon
2018-06-29 20:54:42 +00:00
Philip Withnall
abff8b8cfa lib/repo-commit: Abort a transaction if preparing it fails
If ostree_repo_prepare_transaction() fails, we should reset the
repository’s state so that the failed call was essentially idempotent.
Do that by calling ostree_repo_abort_transaction() on the failure path.

Typically, the way for preparing a transaction to fail is for its
GCancellable to be triggered, rather than because any of the operations
involved in preparing a transaction are particularly failure prone.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1647
Approved by: cgwalters
2018-06-29 19:32:44 +00:00
Marcus Folkesson
be018ed70c ci: exclude 'lib' from libsoup configure option
The option used by configure script is actually
--with-soup/--without-soup.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1661
Approved by: jlebon
2018-06-29 13:16:05 +00:00
Colin Walters
7ead3c1aa8 sysroot: Reject attempts to pin the staged deployment
From https://github.com/projectatomic/rpm-ostree/pull/1434#discussion_r198936674

To support it we'd have to actually write it to disk, which...let's
not try that right now.

Closes: #1660
Approved by: jlebon
2018-06-29 01:52:30 +00:00
William Manley
bab3b2bd4c tests: Save corefiles back to tests/ directory if one exists
Makes it easier to debug failures from the tests.

Closes: #1657
Approved by: cgwalters
2018-06-28 15:07:14 +00:00
Matthew Leeds
05d8ade563 create-usb: Tweak docs for --destination-repo
Make it show up in the help output as --destination-repo=DEST so it's
clear that it takes an argument.

Closes: #1656
Approved by: jlebon
2018-06-28 13:03:09 +00:00
Umang Jain
d686056254 lib/repo: Cleanup current boot's staging dir min-free-space-* checks are hit
min-free-space-* act as a gating condition whether to we want hold onto caches in
repo/tmp. If it is found that the free-disk space is going below this threshold,
we flag it as an error and cleanup current boot's staging directory.

Closes: #1602
Approved by: jlebon
2018-06-27 19:02:02 +00:00
Umang Jain
1074668ede lib/repo: cleanup_tmpdir should be executed after releasing lock file
Here's a subtle bug in abort_transaction():
One of the policies of cleaning up is to skip the current boot's staging
directory. The responsible function for this is cleanup_tmpdir() which tries
to lock each of the tmpdir before deleting it. When it comes to the current
boot's staging dir, it tries to lock the directory(again!) but fails as there
is already a lockfile present. Just because the current boot's staging dir was
meant to be skipped, the bug never surfaced up and wasn't catastrohpic.

if (!_ostree_repo_try_lock_tmpdir (dfd, path, &lockfile, &did_lock, error))
  return FALSE;
if (!did_lock)
  return TRUE; /* Note early return */
...
if (g_str_has_prefix (path, self->stagedir_prefix))
  return TRUE; /* Note early return */

The actual check for skipping staging dir for current boot was never reached
because the function returned at did_lock failure.

Therefore, execute cleanup_tmpdir() after releasing the lockfile in
abort_transaction() so that cleanup_tmpdir gets a chance to lock current boot's
staging directory and succeed.

Closes: #1602
Approved by: jlebon
2018-06-27 19:02:02 +00:00
Javier Martinez Canillas
9f48e212a3 deploy: Change BootLoaderSpec filenames so they can be used for sorting
Currently the BLS snippets are named ostree-$ID-$VARIANT_ID-$index.conf,
but the BLS config files are actually sorted by using the version field
which is the inverse of the index.

In most places, _ostree_sysroot_read_boot_loader_configs() is used to
get the BLS files and this function already returns them sorted by the
version field. The only place where the index trailing number is used is
in the ostree-grub-generator script that lists the BLS files to populate
the grub config file.

But for some bootloaders the BLS filename is the criteria for sorting by
taking the filename as a string version. So on these bootloaders the BLS
entries will be listed in the reverse order.

To avoid that, change the BLS snippets filename to have the version field
instead of the index and also to have the version before deployment name.

Make the filenames to be of the form ostree-$version-$ID-$VARIANT_ID.conf
so the version is before the deployment name.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>

Closes: #1654
Approved by: cgwalters
2018-06-27 18:08:28 +00:00
Simon McVittie
47ae4f5c7e OstreeRepoFinderConfig: Fix guint/gsize confusion
If a function has a guint "out argument", passing a pointer to a gsize
is not, in general, valid. On an ILP64 platform there is no problem
since guint and gsize are identical, but on an LP64 platform it will
overwrite only the first word of the gsize, leaving the second word
unaffected. On little-endian machines, if the second word is
zero-initialized (as it is here), the result is numerically equal to
the guint, but on big-endian machines the result is around 4 billion
times what it should be, resulting in
ostree_repo_finder_config_resolve_async() reading past the end of
the array and causing undefined behaviour.

In practice this caused assertion failures (and consequently test
failures) on Debian's s390x (z/Architecture), ppc64 (64-bit PowerPC)
and sparc64 (64-bit SPARC) ports.

Closes: #1640
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1641
Approved by: cgwalters
2018-06-27 15:24:18 +00:00
Javier Martinez Canillas
25e17e0b62 ostree-grub-generator: sort BLS files by version instead of alphabetically
The ostree-grub-generator populates the grub.cfg menu entries using the
BLS config files. But it uses the ls command that by default sorts the
entries alphabetically, so the order won't be correct if there are more
than 10 deployments, i.e:

$ ls -1 /boot/loader/entries/
ostree-fedora-workstation-0.conf
ostree-fedora-workstation-10.conf
ostree-fedora-workstation-1.conf
...

So instead the -v option should be used to make ls use version sorting:

$ ls -1 -v /boot/loader/entries/
ostree-fedora-workstation-0.conf
ostree-fedora-workstation-1.conf
...
ostree-fedora-workstation-10.conf

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>

Closes: #1653
Approved by: cgwalters
2018-06-27 14:20:29 +00:00
Jonathan Lebon
8dff04601b tests/installed: Wait a bit more for http.server
And also print out the output if it still didn't start up in case there
are error messages hidden in there.

This should hopefully help with diagnosing the flakes we've been seeing
in starting it up.

Closes: #1652
Approved by: cgwalters
2018-06-27 13:29:55 +00:00
Colin Walters
087ad1167a ci: Workaround getfedora.org/atomic_qcow2_latest being 404
Closes: #1652
Approved by: cgwalters
2018-06-27 13:29:55 +00:00
Matthew Leeds
f1133a2a0a man/ostree.repo-config: Document collection-id
The collection-id option in the core section was recently made public
but not documented.

Closes: #1646
Approved by: cgwalters
2018-06-26 12:52:49 +00:00
William Manley
5d031ae78b Add test for composing trees in different ways
In preparation for adding `ostree commit` optimisations.

Closes: #1645
Approved by: jlebon
2018-06-25 16:22:22 +00:00
William Manley
5190f1df42 OstreeMutableTree: Document ostree_mutable_tree_ensure_dir
Closes: #1645
Approved by: jlebon
2018-06-25 16:22:21 +00:00
William Manley
ca8571a49b OstreeMutableTree: Document each private member of OstreeMutableTree
A prelude to my understanding.  Unfortunately `OstreeMutableTree` provides
little encapsulation, as each member has setters† so it's difficult to come
up with a list of invariants.

† `files` and `subdirs` only have getters, but the getters return mutable
  references to the internals, so we still can't reason about invariants.

Closes: #1645
Approved by: jlebon
2018-06-25 16:22:21 +00:00
Simon McVittie
e120a6b119 avahi: Fail immediately if we can't talk to D-Bus or Avahi
We special-case AVAHI_ERR_NO_DAEMON to not cause warnings, but if
we pass AVAHI_CLIENT_NO_FAIL to avahi_client_new, we never actually
see AVAHI_ERR_NO_DAEMON. Instead, we will get AVAHI_ERR_BAD_STATE
when we try to use the client.

Closes: #1618
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1639
Approved by: cgwalters
2018-06-23 14:54:39 +00:00
Umang Jain
8d97b55241 tests: Add tests for space checks during deltas codepath
Closes: #1614
Approved by: jlebon
2018-06-22 21:01:56 +00:00
Umang Jain
095376efa2 lib/repo: Enforce min-free-space-* size check for regfiles in deltas
During the pull, there is an explicit check for free space on disk
vs. the size of uncompressed delta; But while writing the new content
objects that are generated, they have to honor min-free-space-* checks
too. We enforce this check in _bare_content_commit as that is where
we can know the final size of the new content object.

Closes: #1614
Approved by: jlebon
2018-06-22 21:01:56 +00:00
Colin Walters
1174d9f5ba lib/repo: Fix 32 bit format string error 2018-06-21 11:33:23 -04:00
Jonathan Lebon
603c1258cc Post-release version bump 2018-06-21 11:23:40 -04:00
Colin Walters
31a356dca9 Release 2018.6 2018-06-19 15:48:14 -04:00
Colin Walters
0f88a2a72d tests/installed: Add a free-space success path test
We implicitly test the success percent path a lot, but not the
absolute path.

Closes: #1632
Approved by: jlebon
2018-06-19 18:29:31 +00:00
Colin Walters
5e9d382811 lib/repo: Do free space math under lock in error path
We were referencing the txn bits outside of the lock in the error
path. Generally shouldn't matter, but e.g. Rust wouldn't let us do this, and
race detector tooling will warn about it.

Closes: #1632
Approved by: jlebon
2018-06-19 18:29:31 +00:00
Colin Walters
acab2c1ac6 lib/repo: Rename free_space_size variable to free_space_mb
I generally like having variables include their units where applicable;
timer variables having `_secs` or `_ms`, etc.

Closes: #1632
Approved by: jlebon
2018-06-19 18:29:31 +00:00
Colin Walters
a2b08f9342 lib/repo: Fix double-set-error in min-free-space-size code
We need to pass `NULL` as the error, we only care if the key exists;
otherwise we'll try to set the error twice.

Closes: #1632
Approved by: jlebon
2018-06-19 18:29:31 +00:00
Colin Walters
cc5254ac34 lib/archive: Tell g-ir-scanner to ignore the private libarchive bits
Squashes this warning:
```
src/libostree/ostree-libarchive-private.h:46: syntax error, unexpected typedef-name in '  g_autoptr(OtAutoArchiveRead) a = archive_read_new ();' at 'OtAutoArchiveRead'
```

Closes: #1629
Approved by: jlebon
2018-06-18 16:40:39 +00:00
Dan Nicholson
ce58307757 deploy: Delete .updated file from /etc and /var on new deployments
Systemd units using ConditionNeedsUpdate run if the mtime of .updated in
the specified directory is newer than /usr. Since /usr has an mtime of
0, there's no way to have an older .updated file. Systemd units
typically specify ConditionNeedsUpdate=/etc or ConditionNeedsUpdate=/var
to support stateless systems like ostree.

Remove the file from the new deployment's /etc and the OS's /var
regardless of where they came from to ensure that these systemd units
run when booting new deployments. This will provide a method to run
services only on upgrade.

Closes: #1628
https://bugzilla.gnome.org/show_bug.cgi?id=752950

Closes: #1631
Approved by: cgwalters
2018-06-18 13:21:52 +00:00
Matthew Leeds
7dc3e45b3a lib/prune: Don't modify dirent->d_name in place
Currently when I run `ostree prune` it hits a seg fault when the
hash_func is used (in this case g_str_hash) from the call stack
_ostree_repo_prune_tmp() -> g_hash_table_contains() ->
g_hash_table_lookup_node(). So the key, in this case dent->d_name, must
be corrupt in some way.

glnx_dirfd_iterator_next_dent() uses readdir() to get the dirent struct.
And according to the man page for readdir(3), "POSIX.1 explicitly notes
that this field should not be used as an lvalue" (in reference to
d_name). So this commit avoids modifying d_name in place and copies it
instead. This seems to avoid the seg fault.

Closes: #1627
Approved by: jlebon
2018-06-15 19:01:46 +00:00
Alexander Larsson
677e181025 ostree_repo_resolve_rev: Resolve refs set in the transaction
This allows you to get at the current commit for a ref pending in the transaction.

Closes: #1624
Approved by: cgwalters
2018-06-14 17:58:47 +00:00
Philip Withnall
2d2f218669 lib/repo-commit: Delay propagation of errors from abort_transaction()
If there’s a problem while aborting a transaction, store the error but
don’t report it until the end of the function — do a best effort at
clearing the rest of the transaction state first (since most of it
cannot fail).

If cleanup_tmpdir() fails (which, arguably, should not be a
showstopper), this allows a caller to recover and start a new
transaction in future.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1626
Approved by: jlebon
2018-06-14 17:13:43 +00:00
Umang Jain
31809d32f2 lib/repo: Add min-free-space-size option
Similar to min-free-space-percent but it supports specific sizes
(in MB, GB or TB). Also, making min-free-space-percent and -size
mutually exclusive.

min-free-space-percent does not give a fine tuning of the free disk
space that a user might decide to keep. It can translate to very large
size (e.g. 1% = ~10GB on 1TB HDD) or very small (e.g. 1% = ~330MB on 32GB
system like Endless devices). Hence, it makes sense to introduce a config
option to honor specific size as per the user.

Closes: #1616
Approved by: jlebon
2018-06-13 18:57:37 +00:00
Matthew Leeds
c767f7b739 admin: Fix list of subcommands in help and manpage
This adds subcommands that were missing from the ostree-admin man page,
and makes cosmetic fixes there and in the --help output to ensure
alphabetical order and remove trailing whitespace.

Closes: #1621
Approved by: jlebon
2018-06-12 14:36:24 +00:00
Jonathan Lebon
ecdebeb20e switchroot: Allow letting ostree-prepare-root mount /var
In some scenarios, it might make sense to let `ostree-prepare-root` do
the `/var` mount from the state root as before. For example, one may
want to do some system configuration before the switch root. This of
course comes at the expense of supporting `/var` as a mount point in
`/etc/fstab`.

Closes: #1617
Approved by: cgwalters
2018-06-07 21:41:32 +00:00
Matthew Leeds
6f3b5620de tests: Add test for including ostree.h
Closes: #1615
Approved by: jlebon
2018-06-07 17:28:49 +00:00
Matthew Leeds
6ea21696a3 Fix building against old glib versions
We need to include libglnx.h in places where ostree-autocleanups.h is
included, so that we get backports of G_DEFINE_AUTOPTR_CLEANUP_FUNC and
friends.

Closes: #1615
Approved by: jlebon
2018-06-07 17:28:49 +00:00
Matthew Leeds
1d830c1792 Revert "lib: Fix building against old glib versions"
This reverts commit f1d9196076.

Since libglnx.h does not get installed, it can't be included in
ostree-autocleanups.h, which is included by ostree.h.

Closes: #1615
Approved by: jlebon
2018-06-07 17:28:49 +00:00
Matthew Leeds
8fbf19c9f5 Make P2P API public (no longer experimental)
Currently the API that allows P2P operations (e.g. pulling an ostree ref
from a LAN or USB source) is hidden behind the configure flag
--enable-experimental-api. This commit makes the API public and makes
that flag essentially a no-op (leaving it in place in case we want to
use it again in the future). The P2P API has been tested over the last
several months and proven to work.

This means that since we're no longer using the "experimental" feature
flag, P2P builds of Flatpak will fail when using versions of OSTree from
this commit onwards, until Flatpak is patched in the near future. If you
want to build Flatpak < 0.11.8 with P2P enabled and link against OSTree
2018.6, you'll have to patch Flatpak.  However, since Flatpak won't yet
have a hard dependency on OSTree 2018.6, it needs a new way to determine
if the P2P API in OSTree is available, so this commit adds a "p2p"
feature flag. This way the feature set is more semantically correct than
if we had continued to use the "experimental" feature flag.

In addition to making the P2P API public, this commit makes the P2P unit
tests run by default, removes the f27-experimental CI instance that's no
longer needed, changes a few man pages to reflect the changes, and
updates the bash completion script to accept the new commands and
options.

Closes: #1596
Approved by: cgwalters
2018-06-04 19:20:10 +00:00