Commit Graph

3235 Commits

Author SHA1 Message Date
David Shea
22e753176e lib/repo: Fix annotations for out parameters
Change the annotation of the out parameters on ostree_repo_load_file
from `(allow-none)` to `(optional) (nullable)`. `allow-none` is
ambiguous, since these parameters can be both NULL on input and set to
NULL on return.

Closes: #939
Approved by: cgwalters
2017-06-19 13:15:19 +00:00
Alexander Larsson
07dc33ca4a static delta apply: Work on bare-user-only repos
Flatpak make check is failing when applying a static delta
to a bare-user-only repo due to an assert. The fix is to add
bare-user-only to the assert check.

Closes: #940
Approved by: giuseppe
2017-06-19 09:09:17 +00:00
Colin Walters
fb2c3c1db3 tests: Fix previous commit for selinux testing
I only checked the test passed, I didn't read the output closely, and
made it succeed without testing anything.

Fix the absolute/relative `/etc` references.

Closes: #937
Approved by: jlebon
2017-06-16 15:46:07 +00:00
Alexander Larsson
73ba3eb686 pull: When mirroring, only replace summary if we're doing a full mirror
We're hitting this in flathub, where we have a bunch of local builds,
but we also mirror a few refs from the gnome runtime repo into it.
Its fixable by re-doing the summary, but for a short time the
wrong version is visible.

Fixes https://github.com/ostreedev/ostree/issues/846

Closes: #935
Approved by: cgwalters
2017-06-16 15:08:39 +00:00
Colin Walters
371b4a5e7e checkout: Fix SELinux policy labeling when recursing
The code here tried to truncate the string to the previous length,
but that doesn't work when recursing, since further calls change the
length.

What actually ended up happening was the string would get corrupted
after the first level of recursion.

Closes: #936
Approved by: jlebon
2017-06-16 14:54:29 +00:00
Colin Walters
2bab43fb22 lib: Split symbol versioning into -released and -devel
So far a lot of submitted PR have added symbols into the first
section.  Split the file into `-released` and `-devel` to make
this more obvious.

To further enforce things, we hardcode a checksum of the `-released`
file in `test-symbols.sh`.  Only release commits should update that
checksum.

Did you notice I like checksums?

Closes: #931
Approved by: pwithnall
2017-06-16 09:31:16 +00:00
Tristan Van Berkom
3e3a0f0766 ostreee-version.h.in: Added Since: version annotations
This is especially interesting for the versioning symbols themselves,
as it is an indicator of when applications using introspection information
can start to use a symbol in the library to check if they have a
recent enough version of OSTree to use.

Closes: #932
Approved by: cgwalters
2017-06-15 12:59:59 +00:00
Colin Walters
9529e8d435 lib/pull: Extend BAREUSERONLY_FILES flag to HTTP requests
For the flatpak PR: https://github.com/flatpak/flatpak/pull/849

It's really more convenient if this works for HTTP pulls as well, since flatpak
does various types of pulling, and we can just set the flag everywhere.

Further, we might as well reject the content as early as possible.

Closes: #930
Approved by: alexlarsson
2017-06-15 07:10:22 +00:00
Colin Walters
0e6d23835b lib/sysroot: Add some g_prefix_error() for ostree_sysroot_cleanup()
We saw this fail in a CI run. We've been trying to add strategic error prefixing
as a general rule, and this specific instance may help debug.

Closes: #929
Approved by: pwithnall
2017-06-15 00:08:29 +00:00
Colin Walters
584735b1c9 build-sys: post-release version bump
Per request by flatpak.

Closes: #928
Approved by: alexlarsson
2017-06-14 14:51:25 +00:00
Anton Gerasimov
64ab8334b7 lib/sysroot: Add API to get pending/rollback for given stateroot
This imports a function that is used in rpm-ostree, and it's also intended for
use by https://github.com/advancedtelematic/aktualizr to display
what deployment we're going to boot next after the reboot.

Updated-by: Colin Walters <walters@verbum.org>

Closes: #897
Approved by: OYTIS
2017-06-14 09:56:01 +00:00
Colin Walters
0635fcbfd9 lib/checkout: Add bareuseronly_dirs option
This is a continuation of https://github.com/ostreedev/ostree/pull/926
for directories instead of files.

See: https://github.com/flatpak/flatpak/issues/845

This option suppresses mode bits outside of `0775` for directory
checkouts.  I think most people should start doing this by default,
and use explicit overrides for e.g. `/tmp` if doing a recommit based
on a checkout.

Closes: #927
Approved by: alexlarsson
2017-06-13 20:05:31 +00:00
Colin Walters
6ed824bf00 lib/pull: Add OSTREE_REPO_PULL_FLAGS_BAREUSERONLY_FILES
This is an option which is intended mostly for flatpak;
see: https://github.com/flatpak/flatpak/issues/845

We're adding an option for pulling into *all*
repo modes that has an effect similar to the `bare-user-only`
change from https://github.com/ostreedev/ostree/pull/909

This way one can pull content into e.g. a root-owned `bare` repository and
ensure that there aren't any setuid or world-writable files.

Closes: #926
Approved by: alexlarsson
2017-06-13 18:44:28 +00:00
Colin Walters
74e3581ed6 lib/repo: Support hardlink conversions from bare-user to bu-only
Thinking about the problem of flatpak converting from `bare-user` to `bare-user-only`
"in place" by creating a new repo and doing a `pull-local`, I realized
that we can optimize this process by doing hardlinks for both metadata
and regular files.  The repo formats are *almost* compatible, the
exception being symlinks.

An earlier patch caused us to do hardlinks for metadata, this patch takes things
to the next step and special cases this specific conversion. In this case we
need to parse the source object to determine whether or not it's a symlink.

Closes: #922
Approved by: alexlarsson
2017-06-13 12:02:12 +00:00
Colin Walters
b614c65eab lib/repo: Import metadata via hardlink even for distinct repo modes
Our previous logic for import-via-hardlink only tried if the repo modes match,
but we *can* hardlink metadata between e.g. `archive` and `bare-user` repos, and
that's quite useful thing to do. Our documentation encourages converting to/from
those repo modes locally for build systems.

Closes: #922
Approved by: alexlarsson
2017-06-13 12:02:12 +00:00
Colin Walters
695771667c lib/repo: Skip import via hardlink if repo owners don't match
Before this, if one had repos of matching mode but different owners,
which could happen if one e.g. makes a `bare` non-root repo in
`/ostree/deploy/$stateroot/var/tmp`, every time we tried to call `linkat()`
we'd get `EPERM` and fall back to a copy.

Fix this by saving the repo owner uid, and avoid trying to call `linkat()` if we
know it's going to fail. Of course most commonly in this scenario we'll
immediately fail trying to `chown` the files to `0`, but this is prep for a
future patch to improve `bare-user` → `bare-user-only` imports where we'll be a
bit more sophisticated.

Closes: #922
Approved by: alexlarsson
2017-06-13 12:02:12 +00:00
Philip Withnall
21eec96bfd lib/pull: Fix construction of a refspec to use the correct separator
This code looks like it was supposed to build a refspec, but it used a
slash as a separator rather than a colon. The following code does
recover by supporting prefix matching with slashes, but it seems like
this was perhaps not the intention.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #912
Approved by: cgwalters
2017-06-13 01:03:49 +00:00
Alexander Larsson
7159bed8e1 lib/repo: Always look in staging directory for objects
Its often the case that we want to look at objects inside a commit,
before the objects the transaction is finished. For instance:
  https://github.com/flatpak/flatpak/pull/837
Which tries to verify the file permissions before committing the
transaction.

And:
  1e5ffa926a
Which collects the storage size of the objects so that we can
put the total download size in the commit metadata.

I tried to find all the places where we did reads from the
object directories, and in particular this fixes:

 - `ostree_repo_load_file()` for `bare` repos (`archive` was already working).
 - `ostree_repo_query_object_storage_size()`
 - Applying deltas that reference not-yet-commited objects

Closes: #916
Approved by: cgwalters
2017-06-13 00:47:42 +00:00
Colin Walters
848b7c0201 lib/repo: Refactor object copy import function
This came up in: https://github.com/ostreedev/ostree/pull/881

Basically doing streaming for metadata is dumb. Split up the metadata/content
paths so we pass metadata around as `GVariant`. This drops the last internal
caller of `ostree_repo_write_metadata_stream_trusted()` which was the dumb
function mentioned.

Closes: #923
Approved by: jlebon
2017-06-12 21:13:23 +00:00
Colin Walters
8edb5161db lib/checkout: Ignore world-writable dirs for bare-user-only checkout
See https://github.com/ostreedev/ostree/pull/909 for more information on the
rationale. Basically there's no reason for flatpak (which uses `bare-user-only`)
to have world-writable dirs. Particularly with the presence of the system
helper.

An approach I considered instead was to parse and validate directory metadata
objects at commit time. We still may do that in addition; for file objects we *had*
to do it that way because the actual files would be laid down suid.  But directories
live only as inert `.dirmeta` objects until we do a checkout (i.e. `mkdir()`), so
we can solve the problem at checkout time.

Closes: #914
Approved by: alexlarsson
2017-06-12 14:24:22 +00:00
Colin Walters
18ae8e5267 lib/commit: Drop some conditionals/clarify code in content path
Both callers of `commit_loose_object_trusted()` were passing
`OSTREE_OBJECT_TYPE_FILE`, so drop that parameter.  This in turn
allows us to drop lots of checking of that inside the function.

Add a doc comment, and rename to `commit_loose_content_object()` for clarity.

Closes: #914
Approved by: alexlarsson
2017-06-12 14:24:22 +00:00
Colin Walters
aed8a6b09a lib/commit: Port final object writing function to new code style
I noticed my previous patches incorrectly started doing `return glnx_throw*`
inside a `goto out;` function. Fix this by porting forward consistently to new
style. We just do the error prefixing in the caller.

Closes: #914
Approved by: alexlarsson
2017-06-12 14:24:22 +00:00
Colin Walters
5de201df26 repo: Fix leak of superblock fds when generating summary
Related: https://github.com/ostreedev/ostree/issues/920

Closes: #921
Approved by: alexlarsson
2017-06-12 14:15:14 +00:00
Colin Walters
e0e07e2a48 ci: Add CentOS 7 build
For similar reasons as https://github.com/projectatomic/rpm-ostree/pull/824

Closes: #919
Approved by: jlebon
2017-06-09 18:30:15 +00:00
Colin Walters
a98faa9113 ci: Update to match current rpm-ostree
This copies the `ci/` directory from rpm-ostree, with much the same rationale;
among other things we don't want to depend on the Docker hub.

The specific reason I'm doing this is that I want to add a CentOS7 build, but
that means we can't use `projectatomic/ostree-tester`, and at that point we
might as well unwind it all.

Closes: #917
Approved by: jlebon
2017-06-08 21:08:31 +00:00
Alexander Larsson
2a3f17c7aa repo: After renaming in all loose objects, ensure metadata is stable
When a transaction is finished and we have moved all the staged loose
objects into the repo we fsync all the object directory, to ensure the
filenames are stable before we update the refs files to point to the
new commits.

With out this an unclean shutdown after the transaction is finished
could result in a refs file that points to an incomplete commit.

https://bugzilla.gnome.org/show_bug.cgi?id=759442

Closes: #918
Approved by: cgwalters
2017-06-08 20:03:18 +00:00
Colin Walters
c81252c1e0 repo/commit: Support group-writable files for bare-user-only
These exist in the wild for flatpak, and aren't really a problem. The canonical
permissions are still either `0755` or `0644`, we just support the additional
writable bit for the group (i.e. extend the set to include `0775` and `0664`)
now to avoid breaking some flatpak content.

Closes: #913
Approved by: alexlarsson
2017-06-08 06:58:54 +00:00
Colin Walters
81e1f7761f tests: Add a test for bare-user-only failing to commit suid content
We didn't have coverage of this before, and adding the test infrastructure will
help ensure we have coverage for more changes here.

Closes: #913
Approved by: alexlarsson
2017-06-08 06:58:54 +00:00
Colin Walters
5913b22944 lib/repo: For bare-user, mask content object modes with 0775
Having every object in a bare-user repo (and checkouts) be executable
is ugly.  I can't think of a good reason to do that; they should only
be executable if their input is.  This does
for `bare-user` what we did for `bare-user-only` in
https://github.com/ostreedev/ostree/pull/909
It's also a stronger version of what we do with `checkout -U` in suppressing
suid - here we also strip world-writable files and the sticky bit (even though
that's meaningless today, it might not be in the future).

Closes: https://github.com/ostreedev/ostree/issues/907

Closes: #908
Approved by: alexlarsson
2017-06-08 06:50:16 +00:00
Colin Walters
d3900f90f4 lib/repofile: Follow symlinks for g_file_read()
This avoids `ostree cat /path/to/symlink` crashing, a longstanding embarassing
issue.

Closes: #915
Approved by: jlebon
2017-06-07 20:54:32 +00:00
Colin Walters
807a804b16 lib/repofile: Port mostly to new code style
Prep for a bugfix.

Closes: #915
Approved by: jlebon
2017-06-07 20:54:32 +00:00
Colin Walters
6b402e53f4 builtins/cat: Port to new code style
Definitely better.  Prep for another fix.

Closes: #915
Approved by: jlebon
2017-06-07 20:54:32 +00:00
Philip Withnall
4418ab7fa9 lib/fetcher: Add cleanup function for OstreeFetcher
This is only used internally (the header is not public), so it doesn’t
have to go in ostree-autocleanups.h. It will be used in some following
commits.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:36 +00:00
Philip Withnall
59ffce73c8 lib/repo: Omit deltas from the summary file if there are none
If there are no deltas to be listed in the summary file, don’t bother
including the key for them in the additional metadata section of the
file. This saves a few bytes in some cases.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Philip Withnall
7ce6777028 ostree/dump: Improve formatting for well-known commit metadata keys
This follows on from commit a946c3d4, which added formatting for
well-known summary metadata keys. This commit adds it for commits.

Currently, the only well-known commit metadata key is
ostree.commit.timestamp. Formatting this correctly is especially
important, since it’s a big-endian uint64, which is completely unusable
for mere mortals when presented as a number rather than a date.

Non-formatted output can still be retrieved using the OSTREE_DUMP_RAW
flag, and the non-formatted key name is always printed for clarity.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Philip Withnall
87413ee63b lib/pull: Fix an over-indented block
Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Philip Withnall
c968c12ec3 lib/pull: Drop some trailing whitespace
Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Philip Withnall
14082e6b03 lib/pull: Simplify a for-loop initialisation
It’s a bit neater to initialise the loop iterator and maximum in the
same place.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Philip Withnall
52ede69df8 lib/pull: Fix a typo in a documentation comment
Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Philip Withnall
12479d8b05 lib/repo: Reindent some code in regenerate_summary() for clarity
This makes it a bit more easily separable from the rest of the code in
the function. No functional changes.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #911
Approved by: cgwalters
2017-06-07 16:55:35 +00:00
Colin Walters
0c4b3a2b6d Canonicalize bare-user-only perms with 0755 mask
For the flatpak use case where bare-user-only was introduced, we actually
don't want to support s{u,g} id files in particular.

Actually, I can't think of a reason to have anything outside of the
`0755 i.e. (u=rwx,g=rx,o=rx)` mask, so that's what we do here.

This will have the effect of treating existing `bare-user-only` repositories as
corrupted if they have files outside of that mask, but I think we should do this
now; most of the flatpak users will still be on `bare-user`, and we haven't
changed the semantics of that mode yet.

Note that in this patch we will also *reject* file content that doesn't
match this.  This is somewhat asymmetric, since we aren't similarly rejecting
e.g. directory metadata.  But, this will close off the biggest source
of the problem for flatpak (setuid binaries).

See: https://github.com/ostreedev/ostree/pull/908
See: https://github.com/flatpak/flatpak/pull/837

Closes: #909
Approved by: alexlarsson
2017-06-07 15:13:55 +00:00
Owen W. Taylor
25696b3fb0 lib/repo: Don't copy xattrs when manipulating the GPG keyring
Copying xattrs when manipulating the GPG keyring for a repository
causes errors when the underlying filesystem doesn't support writing
xattrs - overlayfs is a common example. It also causes the selinux
attributes of the keyring files to be copied from the temporary
location instead of properly inherited from the destination directory
(ending up, for example, as unconfined_u:object_r:user_tmp_t:s0, rather
than unconfined_u:object_r:data_home_t:s0)

Closes: #910
Approved by: cgwalters
2017-06-07 13:43:51 +00:00
Colin Walters
e18cacb06e Don't install trivial-httpd man page if not enabled
I just noticed this scroll by in a file listing.

Closes: #905
Approved by: jlebon
2017-06-02 19:36:56 +00:00
Jonathan Lebon
f813ae74ad basic-test.sh: explicitly check for uncompressed objects
It's not enough to check that the dir exists, since that's done by
default when we open the repo. We want to actually check that
uncompressed objects were cached (i.e. the opposite of the earlier error
path).

Closes: #903
Approved by: cgwalters
2017-06-02 17:46:16 +00:00
Jonathan Lebon
ab7c3fd800 manual: document bare-user-only repo mode
Closes: #903
Approved by: cgwalters
2017-06-02 17:46:16 +00:00
Jonathan Lebon
a32c6d2c70 checkout: also chmod in the user checkout case
When falling back to copying, we previously would only chmod checked out
files in the non-user-checkout mode. Fix this by always doing chmod.
The file_mode was being prepared but never actually applied.

Add a basic test in the archive-z2 --> usermode checkout case in which
we're guaranteed to always fall back to copy mode.

Closes: #633

Closes: #903
Approved by: cgwalters
2017-06-02 17:46:16 +00:00
Jonathan Lebon
3ec2b5773e checkout: don't apply SELinux labeling in user mode
If the user requested a user checkout, we don't want to set the SELinux
label xattr.

Closes: #903
Approved by: cgwalters
2017-06-02 17:46:16 +00:00
Brian C. Lane
c651982929 Remove the OSTREE_MAX_RECURSION limit on metadata depth
This was making it impossible to pull or mirror a large ostree repo, and
according to Colin is no longer necessary. It works fine with a test
against a repo with 2741 commit and 451468 objects in it.

Closes: #899

Closes: #904
Approved by: jlebon
2017-06-02 16:18:28 +00:00
Colin Walters
cad42d9601 Revert "Add a notion of "physical" sysroot, use for remote writing"
This reverts commit 1eff3e8343. There
are a few issues with it.  It's not a critical thing for now, so
let's ugly up the git history and revisit when we have time to
debug it and add more tests.

Besides the below issue, I noticed that the simple `ostree remote add`
now writes to `/ostree/repo/config` because we *aren't* using the
`--sysroot` argument.

Closes: https://github.com/ostreedev/ostree/issues/901

Closes: #902
Approved by: mike-nguyen
2017-06-02 14:11:58 +00:00
Krzesimir Nowak
2fdbdd4b2f lib/sysroot: Document the NO_CLEAN flag
Closes: #900
Approved by: jlebon
2017-06-02 13:19:51 +00:00