1034 Commits

Author SHA1 Message Date
Sinny Kumari
c70526841e src/ostree: Don't delete refs having aliases
Deleting a ref with aliases makes them dangling. In such
cases, display an error message to the user.

Fixes #1597

Signed-off-by: Sinny Kumari <sinny@redhat.com>

Closes: #1749
Approved by: cgwalters
2018-10-11 13:41:32 +00:00
Colin Walters
2c55bc6997 Only verify OSTREE_MAX_METADATA_SIZE for HTTP fetches
There are use cases for libostree as a local content store
for content derived or delivered via other mechanisms (e.g. OCI
images, RPMs, etc.).  rpm-ostree today imports RPMs into OSTree
branches, and puts the RPM header value as commit metadata.
Some of these can be quite large because the header includes
permissions for each file.  Similarly, some OCI metadata is large.

Since there's no security issues with this, support committing
such content.

We still by default limit the size of metadata fetches, although
for good measure we make this configurable too via a new
`max-metadata-size` value.

Closes: https://github.com/ostreedev/ostree/issues/1721

Closes: #1744
Approved by: jlebon
2018-10-01 13:23:50 +00:00
Colin Walters
c141fe610b lib/commit: Don't copy xattrs for metadata objects
Copying the xattrs on metadata objects is wrong in general, we
don't "own" them.  Notably this would fail in the situation of
doing a pull from e.g. a `bare-user` source to a destination
that was on a different mount point (so we couldn't hardlink),
and the source had e.g. a `security.selinux` attribute.

Closes: #1734

Closes: #1736
Approved by: jlebon
2018-09-25 14:49:22 +00:00
Umang Jain
5d2e62affb tests: Update tests for ostree_repo_get_min_free_space_bytes()
https://github.com/ostreedev/ostree/issues/1720

Closes: #1722
Approved by: pwithnall
2018-09-21 15:09:12 +00:00
Umang Jain
a70d2f6731 Add tests for ostree_repo_get_min_free_space_bytes
https://phabricator.endlessm.com/T23694

Closes: #1715
Approved by: cgwalters
2018-09-04 21:31:34 +00:00
Colin Walters
7aa242c34c Release 2018.8
Closes: #1705
Approved by: jlebon
2018-08-22 13:53:24 +00:00
Sinny Kumari
dde3f1c0fb src/ostree: Add --group option to ostree config
Fetching value from a repo config using 'ostree config
get SECTIONNAME.KEYNAME' didn't work in some cases like
when having dots in Group Name entry.
As per Desktop entry file specification, Group Name
may contain all ASCII characters except for [ and ]
and control characters.
Link - https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-1.1.html

Having --group option will help user to clearly specify
Group Name and get desired result.

It also adds test for ostree config get|set and bash
completion for --group option

Fixes https://github.com/ostreedev/ostree/issues/1565

Closes: #1696
Approved by: cgwalters
2018-08-20 14:31:15 +00:00
Dan Nicholson
4e6b13e8b6 repo: Add OSTREE_REPO_TEST_ERROR=invalid-cache env var
Add an invalid-cache test error flag to ensure that the code that checks
for and recovers from a corrupted summary cache is hit. This helps make
sure that the recovery path is actually used without resorting to
G_MESSAGES_DEBUG.

Closes: #1698
Approved by: cgwalters
2018-08-14 13:38:11 +00:00
Dan Nicholson
1b5cb52da2 tests: Test for recovery from corrupted summary cache
Check that recovery from a corrupted summary cache (cached summary
doesn't match cached signature) works.

Closes: #1698
Approved by: cgwalters
2018-08-14 13:38:11 +00:00
Dan Nicholson
ce4eb12ffb tests: Add tests for remote summary update races
There have been subtle bugs in the past when a client pulls while the
remote server is updating the summary. The client may get the old
summary and new signature or vice versa. Add tests to simulate this
behavior to make sure there aren't regressions in the future.

Closes: #1698
Approved by: cgwalters
2018-08-14 13:38:11 +00:00
Jonathan Lebon
786ee6bdec lib/config: Rename change-update-summary to auto-...
Mildly bikeshed, though I find the name `auto-update-summary` to be
easier to grok than `change-update-summary`. I think it's because it can
be read as "verb-verb-noun" rather than "noun-verb-noun".

Closes: #1693
Approved by: mwleeds
2018-08-01 19:59:07 +00:00
Matthew Leeds
6869bada49 config: Add a core/change-update-summary option
This commits adds and implements a boolean repo config option called
"change-update-summary" which updates the summary file every time a ref
changes (additions, updates, and deletions).

The main impetus for this feature is that the `ostree create-usb` and
`flatpak create-usb` commands depend on the repo summary being up to
date. On the command line you can work around this by asking the user to
run `ostree summary --update` but in the case of GNOME Software calling
out to `flatpak create-usb` this wouldn't work because it's running as a
user and the repo is owned by root. That strategy also means flatpak
can't update the repo metadata refs for fear of invalidating the
summary.

Another use case for this relates to LAN updates. Specifically, the
component of eos-updater that generates DNS-SD records advertising ostree
refs depends on the repo summary being up to date.

Since ostree_repo_regenerate_summary() now takes an exclusive lock, this
should be safe to enable. However it's not enabled by default because of
the performance cost, and because it's more useful on clients than
servers (which likely have another mechanism for updating the summary).

Fixes https://github.com/ostreedev/ostree/issues/1664

Closes: #1681
Approved by: jlebon
2018-07-30 17:19:12 +00:00
Umang Jain
21318bbc1f Release 2018.7
Request via flatpak: mainly to port min-free-space-size

Closes: #1683
Approved by: cgwalters
2018-07-20 19:43:18 +00:00
Colin Walters
7468600029 deploy: Retain staged by default
For `rpm-ostree ex livefs` we have a use case of pushing a rollback
deployment.  There's no reason this should require deleting the staged
deployment (and doing so actually breaks livefs which tries to access
it as a data source).

I was initially very conservative here, but I think it ends up
being fairly easy to retain the staged deployment.  We need to handle
two cases:

First, when the staged is *intentionally* deleted; here, we just need
to unlink the `/run` file, and then everything will be sync'd up after
reloading.

Second, (as in the livefs case) where we're retaining it,
e.g. adding a deployment to the end.  What I realized here is that
we can have the code keep `new_deployments` as view without staged,
and then when we do the final reload we'll end up re-reading it from
disk anyways.

Closes: #1672
Approved by: jlebon
2018-07-06 15:23:52 +00:00
Robert Fairley
10c2fc33f6 tests: Run network retries test for many retries directly
This runs a test involving many retries for the --network-retries
option directly rather than inside a conditional statement, so that
the command does not silently fail and allow the test to continue
running.

Closes: #1673
Approved by: jlebon
2018-07-05 20:11:42 +00:00
Robert Fairley
7baf167881 ostree/pull: Add network-retries command line option
This exposes a way to specify from the command line the number
of times to retry each download after a network error. If a negative
value is given, then the default number of retries (5) is used. If 0
is given, then errors are returned without retrying.

closes #1659

Closes: #1669
Approved by: jlebon
2018-07-05 17:59:09 +00:00
Robert Fairley
4f096c8f11 tests: Move assert_fail function to tests/libtest.sh
This moves the assert_fail function definition which was defined
and called in tests/test-remote-headers.sh. Done in preparation
for use of the assert_fail function in other test files.

Closes: #1669
Approved by: jlebon
2018-07-05 17:59:09 +00:00
Jonathan Lebon
2cb2571127 tests/installed: Add NOTE when re-using RPMs
One gotcha here is that we don't invalidate the RPMs if we're not
sitting on the same commit anymore. Shouldn't be too hard to fix, though
let's at least make a note of it for now.

Closes: #1668
Approved by: cgwalters
2018-07-04 16:06:38 +00:00
Colin Walters
61ba4e7e5a tests/installed: Add var-mount.yml destructive test
Closes: #1668
Approved by: cgwalters
2018-07-04 16:06:38 +00:00
William Manley
488365f9bf OstreeMutableTree: Invalidate parent contents checksum when metadata changes
This bug has existed before the previous commit, but thanks to the previous
commit it is now easy to fix.

Closes: #1655
Approved by: cgwalters
2018-06-29 21:31:08 +00:00
Colin Walters
7ead3c1aa8 sysroot: Reject attempts to pin the staged deployment
From https://github.com/projectatomic/rpm-ostree/pull/1434#discussion_r198936674

To support it we'd have to actually write it to disk, which...let's
not try that right now.

Closes: #1660
Approved by: jlebon
2018-06-29 01:52:30 +00:00
William Manley
bab3b2bd4c tests: Save corefiles back to tests/ directory if one exists
Makes it easier to debug failures from the tests.

Closes: #1657
Approved by: cgwalters
2018-06-28 15:07:14 +00:00
Javier Martinez Canillas
9f48e212a3 deploy: Change BootLoaderSpec filenames so they can be used for sorting
Currently the BLS snippets are named ostree-$ID-$VARIANT_ID-$index.conf,
but the BLS config files are actually sorted by using the version field
which is the inverse of the index.

In most places, _ostree_sysroot_read_boot_loader_configs() is used to
get the BLS files and this function already returns them sorted by the
version field. The only place where the index trailing number is used is
in the ostree-grub-generator script that lists the BLS files to populate
the grub config file.

But for some bootloaders the BLS filename is the criteria for sorting by
taking the filename as a string version. So on these bootloaders the BLS
entries will be listed in the reverse order.

To avoid that, change the BLS snippets filename to have the version field
instead of the index and also to have the version before deployment name.

Make the filenames to be of the form ostree-$version-$ID-$VARIANT_ID.conf
so the version is before the deployment name.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>

Closes: #1654
Approved by: cgwalters
2018-06-27 18:08:28 +00:00
Jonathan Lebon
8dff04601b tests/installed: Wait a bit more for http.server
And also print out the output if it still didn't start up in case there
are error messages hidden in there.

This should hopefully help with diagnosing the flakes we've been seeing
in starting it up.

Closes: #1652
Approved by: cgwalters
2018-06-27 13:29:55 +00:00
William Manley
5d031ae78b Add test for composing trees in different ways
In preparation for adding `ostree commit` optimisations.

Closes: #1645
Approved by: jlebon
2018-06-25 16:22:22 +00:00
Umang Jain
8d97b55241 tests: Add tests for space checks during deltas codepath
Closes: #1614
Approved by: jlebon
2018-06-22 21:01:56 +00:00
Colin Walters
31a356dca9 Release 2018.6 2018-06-19 15:48:14 -04:00
Colin Walters
0f88a2a72d tests/installed: Add a free-space success path test
We implicitly test the success percent path a lot, but not the
absolute path.

Closes: #1632
Approved by: jlebon
2018-06-19 18:29:31 +00:00
Dan Nicholson
ce58307757 deploy: Delete .updated file from /etc and /var on new deployments
Systemd units using ConditionNeedsUpdate run if the mtime of .updated in
the specified directory is newer than /usr. Since /usr has an mtime of
0, there's no way to have an older .updated file. Systemd units
typically specify ConditionNeedsUpdate=/etc or ConditionNeedsUpdate=/var
to support stateless systems like ostree.

Remove the file from the new deployment's /etc and the OS's /var
regardless of where they came from to ensure that these systemd units
run when booting new deployments. This will provide a method to run
services only on upgrade.

Closes: #1628
https://bugzilla.gnome.org/show_bug.cgi?id=752950

Closes: #1631
Approved by: cgwalters
2018-06-18 13:21:52 +00:00
Umang Jain
31809d32f2 lib/repo: Add min-free-space-size option
Similar to min-free-space-percent but it supports specific sizes
(in MB, GB or TB). Also, making min-free-space-percent and -size
mutually exclusive.

min-free-space-percent does not give a fine tuning of the free disk
space that a user might decide to keep. It can translate to very large
size (e.g. 1% = ~10GB on 1TB HDD) or very small (e.g. 1% = ~330MB on 32GB
system like Endless devices). Hence, it makes sense to introduce a config
option to honor specific size as per the user.

Closes: #1616
Approved by: jlebon
2018-06-13 18:57:37 +00:00
Matthew Leeds
6f3b5620de tests: Add test for including ostree.h
Closes: #1615
Approved by: jlebon
2018-06-07 17:28:49 +00:00
Matthew Leeds
6ea21696a3 Fix building against old glib versions
We need to include libglnx.h in places where ostree-autocleanups.h is
included, so that we get backports of G_DEFINE_AUTOPTR_CLEANUP_FUNC and
friends.

Closes: #1615
Approved by: jlebon
2018-06-07 17:28:49 +00:00
Matthew Leeds
1d830c1792 Revert "lib: Fix building against old glib versions"
This reverts commit f1d9196076d4aea1f64e0d2cbd17bfa2891b8c4c.

Since libglnx.h does not get installed, it can't be included in
ostree-autocleanups.h, which is included by ostree.h.

Closes: #1615
Approved by: jlebon
2018-06-07 17:28:49 +00:00
Matthew Leeds
f1d9196076 lib: Fix building against old glib versions
This commit includes libglnx.h in ostree-autocleanups.h, so we get the
g_autoptr backports wherever they're needed. Also, remove the "#include
libglnx.h" lines elsewhere that are no longer needed.

Closes: #1596
Approved by: cgwalters
2018-06-04 19:20:09 +00:00
Jan Tojnar
f200efdb8a tests: Fix locale detection
When a locale with C.utf8 in its name (e.g. es_EC.utf8) was installed
on a system, the C.utf8 locale was chosen, even when it was not available.

This patch fixes the grep pattern to match whole lines returned by locale -a.

See: #1592

Closes: #1611
Approved by: cgwalters
2018-06-03 21:25:07 +00:00
Philip Withnall
c9619a4904 tests: Test pull behaviour when network timeouts occur
Extend test-pull-repeated.sh to test error 408 as well as error 500, to
ensure that the new retry-on-network-timeout code in ostree-repo-pull.c
correctly retries.

Rather than the 200 iterations needed for the error 500 tests, only do 5
iterations. The pull code internally does 5 retries (by default), which
means a full iteration count of 25. That seems to be sufficient to make
the tests reliably pass, in my testing — we can always bump it up to 200
/ 5 = 40 in future if needed (to put it in parity with the error 500
tests).

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1594
Approved by: jlebon
2018-05-30 16:23:57 +00:00
Matthew Leeds
f31087137e Fix building P2P code against old glib versions
When building the OstreeBloom code against old versions of glib, we have
to have the libglnx headers included so that it defines
G_DEFINE_AUTOPTR_CLEANUP_FUNC and friends for us.

This is similarly true for test-repo-finder-mount.c which indirectly
includes ostree-autocleanups.h.

Closes: #1605
Approved by: cgwalters
2018-05-30 14:19:25 +00:00
Alexander Larsson
5f82503a01 fsck: Add --all to print all corrupted object
Stopping on the first error is nice if you just want to know if everything is ok, but
if you want to figure out all that is wrong its nice to be able to continue and
print all corruptions.

Closes: #1591
Approved by: cgwalters
2018-05-24 21:00:52 +00:00
Alexander Larsson
ef86abe228 fsck: Include the commits that contain the corruption in errors
This makes it easier to figure out what the corruption affects.

Closes: #1591
Approved by: cgwalters
2018-05-24 21:00:52 +00:00
Colin Walters
986f176e7c Release 2018.5
Lots of stuff queued up.

Closes: #1585
Approved by: jlebon
2018-05-11 13:06:19 +00:00
Colin Walters
6873650cae lib/sysroot: Add OSTREE_EX_STAGE_DEPLOYMENTS environment variable
I feel like I'm drowning in a pile of experimental-but-almost-stable
features...

Anyways, since we made the feature opt-in in rpm-ostree in
https://github.com/projectatomic/rpm-ostree/pull/1352
let's mirror that a bit here with an environment variable so people
can play with it more easily.

The tests needed some tweaks; specifically we need to reload the
status fact after making changes.  I'm still a bit uncertain
about the Ansible-as-tests.

But we add an upgrade test that uses the new environment variable.

Closes: #1583
Approved by: jlebon
2018-05-11 12:39:17 +00:00
Colin Walters
605d313206 tests/installed: Disable all rpmmd repos
For the same reason we do in the rpm-ostree tests.  This also
made sure the test run worked when I was offline on a plane.

Closes: #1583
Approved by: jlebon
2018-05-11 12:39:17 +00:00
Colin Walters
a3295ee584 ci: Drop str hotpatch
The change we want is in the current Fedora repos, and git master
is broken:

> qemu-system-x86_64: -vnc :1: Failed to start VNC server: Failed to bind socket: Address already in use

 https://pagure.io/standard-test-roles/pull-request/186#comment-52440

Closes: #1582
Approved by: jlebon
2018-05-08 12:57:35 +00:00
Colin Walters
579faf92fd tests: Fix installed tests more
OK so I noticed that something was failing and we were missing
`set -xeuo pipefail` in our shells.  That of course revealed
the ansible tests didn't actually work - my only defense
here is spending so much time fighting to get it through CI
and trying something new.

Anyways, to make the staged-deploy tests work we need a task
that actually uses `rpm-ostree override` rather than `usroverlay`.

Let's make this a bit saner and have a clean split between
tests that are "shell-script+usroverlay" and "ansible+override".

Closes: #1577
Approved by: jlebon
2018-05-04 19:25:32 +00:00
Colin Walters
956618a188 tests: Tweak staged-deploy test to be faster
Work around https://github.com/ostreedev/ostree/issues/1569

Closes: #1573
Approved by: jlebon
2018-05-04 13:06:37 +00:00
Colin Walters
5337ba51b2 lib/deploy: Do post-ops when removing staged commit
These are further fixes based on running more of the rpm-ostree
test suite.

When dropping the staged deployment, we do need to do the
"post operations" such as bumping the sysroot mtime, so that
clients know something changed.  We also need to regenerate
the deployment refs.  And of course do a sysroot reload.

Also, add a "base cleanup" after creating a staged deployment
which also regenerates the refs.

Closes: #1570
Approved by: jlebon
2018-05-02 16:22:34 +00:00
Colin Walters
25ba8db987 lib/deploy: Throw an error if trying to stage when not ostree-booted
There's no reason to do this.  I didn't actually hit this problem,
but it's a corner case that just occurred to me while working on
the code.

I think callers should be adapted to skip trying to use staging
if there's no booted deployment.

Closes: #1568
Approved by: jlebon
2018-05-02 13:27:01 +00:00
Colin Walters
36ac768fb7 repo: Add checksum to error message opening unreadable object
This would have debugged trying to do a pull as non-root from
a sysroot repository.  See for example:
https://github.com/ostreedev/ostree/issues/1562

Closes: #1564
Approved by: jlebon
2018-05-01 19:22:10 +00:00
Colin Walters
8c1542134c lib/repo: Enable locking by default, but drop external API
The code has been sitting around for a while but since I disabled
it by default, I doubt anyone is really using it or relying on it.

This patch and turns on locking by default, and also drops the
API which was only public in the experimental API builds.
Conceptually these are two distinct things, and we
may actually want to split up the patches.

I don't think this will break anyone, but it's hard to say for sure.
It's also going to be hard to find out until we actually release
I suspect...

But anyone who is broken should be able to add `locking=false` into
their repo config.  On the flip side Endless has been shipping with
this enabled and it is reported to help.

The reason to drop the APIs: I'm a bit concerned about the interactions over time
between libostree's use of the API and any apps that start using it.
For example, if an app specifies a SHARED lock in their code, then
later internally we decide to temporarily grab an `EXCLUSIVE`, but the
app had a second thread/process that was `EXCLUSIVE` already, and
that process was waiting on the first bit of code, then we could
deadlock. I can't think of a real world situation where this would happen
yet though.

We are likely to in the future have say `fsck` take an external lock,
`checkout` grab a shared one, etc.

Closes: #1555
Approved by: jlebon
2018-04-30 17:24:51 +00:00
Colin Walters
06d39efcb5 bin/deploy: Avoid loading merge deployment kargs unless necessary
The fact that `ostree admin deploy` always itself loaded the
merge kargs masked a bug in the core.  Let's change our tests
to not pass any kernel arguments to ensure we cover this.

The new logic in the CLI is a bit subtle, but if you read
carefully is a lot clearer I believe.  Basically we have one
of a few "starting points" in the first section, which can
then be further augmented.

Closes: #1558
Approved by: jlebon
2018-04-27 17:48:21 +00:00