Commit Graph

3450 Commits

Author SHA1 Message Date
Colin Walters
138c4d7aae libarchive: Add support for translating paths during commit
For rpm-ostree, I want to move RPM files in `/boot` to `/usr/lib/ostree-boot`.
This is currently impossible without forking the libarchive code.  Supporting
this is pretty straightforward; we already had pathname translation in
the libarchive code, we just need to expose it as an option.

On the command line side, I chose to wrap this as a regexp. That should be good
enough for a lot of use cases; sophisticated users should as always be making
use of the API. Note that this required some new `#ifdef LIBARCHIVE` bits to use
the new API. Following previous patterns here, we use the new API only if a
relevant option is enabled, ensuring unit test coverage of both paths.

For the test cases, I ended up changing the accounting to avoid having to
multiply the test count.

Closes: #1105
Approved by: jlebon
2017-08-30 14:30:30 +00:00
Simon McVittie
355e8516b0 Remove shebang from bash completions
bash completions are to be sourced. It makes little sense to
execute them.

Detected by Debian's Lintian tool, which warns about non-executable
files that appear to be #! scripts.

Signed-off-by: Simon McVittie <smcv@collabora.com>

Closes: #1119
Approved by: cgwalters
2017-08-29 19:19:56 +00:00
Simon McVittie
3fcd81df86 test-basic-user-only: Skip final step if no user xattrs
We don't want to skip the entire test, because the whole point of
bare-user-only is that it works in the absence of xattrs; but we do
need to skip this last stage, which explicitly uses a bare-user
repository.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1120
Approved by: cgwalters
2017-08-29 19:08:59 +00:00
Simon McVittie
41e6871e51 basic-test: Skip explicit uses of bare-user if no user xattrs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1120
Approved by: cgwalters
2017-08-29 19:08:59 +00:00
Simon McVittie
a7686a0c40 libtest: Allow skipping single checks without user xattrs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1120
Approved by: cgwalters
2017-08-29 19:08:59 +00:00
Simon McVittie
1d755977c8 test-pull-bareuser.sh: This test uses bare-user, hence needs xattrs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #1120
Approved by: cgwalters
2017-08-29 19:08:59 +00:00
Simon McVittie
760ff2f66e test-libarchive-import: Skip if extended attributes are unsupported
This is the case at build-time on some (all?) Debian autobuilders.

Signed-off-by: Simon McVittie <smcv@collabora.com>

Closes: #1120
Approved by: cgwalters
2017-08-29 19:08:59 +00:00
Matthew Leeds
5f8e339e45 lib/repo-refs: Fix typos in last commit
This fixes a whitespace error and a mistake that made it into the last
commit, 7ed881baa, at the last minute.

Closes: #1112
Approved by: cgwalters
2017-08-25 16:54:15 +00:00
Colin Walters
5e30bbd1b9 bin/commit: Add --selinux-policy option
This was really straightforward to implement, and is useful
for dev/test scenarios mainly like we have in rpm-ostree at least.

Closes: https://github.com/ostreedev/ostree/issues/1113

Closes: #1114
Approved by: jlebon
2017-08-25 16:39:53 +00:00
Jonathan Lebon
9342be6e34 ostree-sysroot: make simple_write_deployment smarter
This is a follow-up to https://github.com/ostreedev/ostree/pull/1097.
We make simple_write_deployment smart enough so that it can be used for
rpm-ostree's purposes. This is mostly an upstreaming of logic that
already existed there.

Notably we correctly append NOT_DEFAULT deployments *after* the booted
deployment and we now support RETAIN_PENDING and RETAIN_ROLLBACK flags
to have more granularity on deployment pruning.

Expose these new flags on the CLI using new options (as well as expose
the previously existing NOT_DEFAULT flag as --not-as-default).

I couldn't add tests for --retain-pending because the merge deployment
is always the topmost one. Though I did check that it worked in a VM.

Closes: #1110
Approved by: cgwalters
2017-08-25 01:02:15 +00:00
Jonathan Lebon
d0f40a6af8 ostree-sysroot: convert function to new style
Also convert ot-admin-builtin-deploy.c.
Prep for more work there.

Closes: #1110
Approved by: cgwalters
2017-08-25 01:02:15 +00:00
Matthew Leeds
7ed881baa7 lib/repo-refs: Include remote refs when using collections
When working with collections it can be useful to see remote refs rather
than just local and mirrored ones. This commit changes the "ostree refs
-c" output to include remote refs, and includes remote refs with
collection IDs in summary file generation as well. The former behavior
is consistent with how "ostree refs" works, and the latter behavior is
useful in facilitating P2P updates even when mirrors haven't been
configured.

To accomplish this, OstreeRepoListRefsExtFlags was extended with an
EXCLUDE_REMOTES flag. This was done rather than an INCLUDE_REMOTES flag
so that existing calls to ostree_repo_list_refs_ext continue to have the
same behavior. This flag was added to ostree_repo_list_collection_refs
(which is an experimental API break).

Also, add unit tests for the "refs -c" and summary file behavior, and
update relevant tests.

Closes: #1069
Approved by: cgwalters
2017-08-24 19:57:33 +00:00
Colin Walters
95bac299e5 lib/repo: Add some assertions for ABI sizes
Things like https://sourceware.org/libabigail/manual/abidiff.html
look interesting but in a brief look I couldn't work out
how to conveniently use them for quick ABI sanity checking without
doing a diff from a previous build (which we could do but would be
more involved).

This way will at least catch struct ABI breaks on x86_64 which
I think we'd be most likely to do accidentally when trying
to use one of the previous unused values.

I found the hole values via gdb's `pahole` command.

Closes: #1108
Approved by: jlebon
2017-08-24 14:58:51 +00:00
Colin Walters
882fa906e2 bin/main: Remove duplicated usage output on unknown commands
It's been this way for a long time, though not forever; I went
back to v2014.11 as a random choice and it worked then.  Not
going to bother doing a full archive search for this though.
Anyone who wants more context can help themselves.

Closes: https://github.com/ostreedev/ostree/issues/1096

Closes: #1106
Approved by: jlebon
2017-08-23 17:51:17 +00:00
Colin Walters
89801dd481 build/maint.mk: Comment out setting of LC_ALL
This triggers obscure bugs; really we shouldn't be overriding
the global locale here.  In practice, production build systems
will be using fixed locales anyways.

Also, we only use a small subset of this file (`make syntax-check`),
which appears to work OK without this.

I will probably try to work out where to submit this as at
least an issue report for upstream gnulib.

Closes: https://github.com/ostreedev/ostree/issues/1101

Closes: #1107
Approved by: heftig
2017-08-23 17:41:06 +00:00
Colin Walters
eb6f7c6db8 lib/commit: Honor commit filter for libarchive --tar-autocreate-parents
This makes `ostree commit --tree=tar` honor `--owner-uid` and `--owner-gid`
for the root directory.

Prep for further commit filtering work, although mostly for the unit test cases;
this ensures we can use `ostree checkout` after autocreating a root directory.

Closes: #1104
Approved by: jlebon
2017-08-23 14:48:12 +00:00
Colin Walters
556e2deb93 lib/commit: Remove duplicated function for filter processing
The wrapping here is unnecessary, since `_ostree_repo_commit_modifier_apply()`
already does what this function did. Further, the return type was wrong.

Saw this while doing some libarchive work.

Closes: #1104
Approved by: jlebon
2017-08-23 14:48:12 +00:00
Jonathan Lebon
11fdca2919 pull: better description for --mirror
Describe the behaviour of --mirror a bit better.

Closes: #1100

Closes: #1099
Approved by: dustymabe
2017-08-23 01:23:38 +00:00
Philip Withnall
80eab25f8a build: Ensure ostree-tmpfiles.conf is distributed
Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1103
Approved by: cgwalters
2017-08-22 17:37:48 +00:00
Philip Withnall
e3a4049c24 build: Add distcheck configure flags to fix systemd and bash-completion
When running distcheck, the systemd system-generator and bash-completion
scripts are installed in absolute paths (/usr and /lib) as looked up
from their pkg-config files. This breaks distcheck. Use a
${prefix}-relative path for both of them when configuring for distcheck.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1103
Approved by: cgwalters
2017-08-22 17:37:48 +00:00
Jonathan Lebon
19429b1a26 tests: fix admin upgrade timestamp file check
In #1055, I tried to be helpful but that didn't work out all too well.
We need to recompute the file path since one is in archive mode and the
other in bare mode.

Closes: #1098
Approved by: cgwalters
2017-08-21 19:30:58 +00:00
Colin Walters
618617d68b lib/pull: Add support for timestamp-check option, use in upgrader
For both flatpak and ostree-as-host, we really want to verify up front during
pulls that we're not being downgraded. Currently both flatpak and
`OstreeSysrootUpgrader` do this before deployments, but at that point we've
already downloaded all the data, which is annoying.

Closes: https://github.com/ostreedev/ostree/issues/687

Closes: #1055
Approved by: jlebon
2017-08-21 18:24:07 +00:00
Guy Shapiro
ca61a2bd9d lib/sysroot: fix placement for not-default deployment
When using the
OSTREE_SYSROOT_SIMPLE_WRITE_DEPLOYMENT_FLAGS_NOT_DEFAULT flag, the
deployment is said to be added after the booted or merge deployment.
Fix the condition to do so instead of adding it in the second place.

Closes: #1097
Approved by: cgwalters
2017-08-21 16:52:13 +00:00
Colin Walters
1e5b06be5b lib/deploy: Add .img to end of initramfs in /usr/lib/modules
Follow up to <https://github.com/ostreedev/ostree/pull/1079>; I was working on
the rpm-ostree updates for this, and I think it's more consistent if we have
`.img` here, since that's a closer match to the "remove $kver" that results in
`vmlinuz`. Also just best practice to have file suffix types where they make
sense.

The astute reader might notice this sneaks in a change where we'd crash if the
legacy bootdir didn't have an initramfs...yeah, should probably have test
coverage of that.

Closes: #1095
Approved by: jlebon
2017-08-21 16:41:50 +00:00
Colin Walters
7b3e55a3f4 docs/build: s/libOSTree/libostree/
I find "libOSTree" awkward to type and really to look at. Let's be nicer on
people's pinky fingers and eyes and drop it all down to lowercase.

Closes: #1093
Approved by: jlebon
2017-08-18 17:51:09 +00:00
Colin Walters
3ab0d5e664 lib/sysroot: Support /usr/lib/modules/$kver for kernel/initramfs
This is the new Fedora kernel standard layout; it has the advantage
of being in `/usr` like `/usr/lib/ostree-boot`, but it's not OSTree
specific.

Further, I think in practice forcing tree builders to compute the checksum is an
annoying stumbling block; since we already switched to e.g. computing checksums
always when doing pulls, the cost of doing another checksum for the
kernel/initramfs is tiny. The "bootcsum" becomes more of an internal
implementation detail.

Now, there is a transition; my current thought for this is that rpm-ostree will
change to default to injecting into both `/usr/lib/ostree-boot` and
`/usr/lib/modules`, and stop doing `/boot`, then maybe next year say we drop the
`/usr/lib/ostree-boot` by default.

A twist here is that the default Fedora kernel RPM layout (and what's in
rpm-ostree today) includes a kernel but *not* an initramfs in
`/usr/lib/modules`. If we looked only there, we'd just find the kernel. So we
need to look in both, and then special case this - pick the legacy layout if we
have `/usr/lib/modules` but not an initramfs.

While here, rework the code to have an `OstreeKernelLayout` struct which makes
dealing with all of the variables nicer.

Closes: #1079
Approved by: jlebon
2017-08-18 17:34:36 +00:00
Colin Walters
984d22303d bin: Squash some -Wuninit warnings with porting to new style
I noticed this with a local build of an RPM:

```
/usr/include/glib-2.0/glib/glib-autocleanups.h:28:3: warning: 'help' may be used uninitialized in this function [-Wmaybe-uninitialized]
   g_free (*pp);
   ^~~~~~~~~~~~
src/ostree/ot-main.c:82:20: note: 'help' was declared here
   g_autofree char *help;
                    ^~~~
```

Closes: #1091
Approved by: jlebon
2017-08-17 21:04:04 +00:00
Colin Walters
e6a4203c52 boot: Add a tmpfiles.d snippet to clean up /var/tmp/ostree-ovl.XXX
This is simplest for now.  Compare with similar logic from
`/usr/lib/tmpfiles.d/tmp.conf`:
```
R! /tmp/systemd-private-*
```

Closes: https://github.com/ostreedev/ostree/issues/393

Closes: #1090
Approved by: jlebon
2017-08-17 18:38:45 +00:00
Colin Walters
bd9e931b33 build-sys: Post-release version bump
Closes: #1089
Approved by: jlebon
2017-08-17 18:18:21 +00:00
Colin Walters
395676917b Release 2017.10
Closes: #1089
Approved by: jlebon
2017-08-17 18:18:21 +00:00
Colin Walters
c4148d4ddd lib: Fix v2017.10 symbols to inherit from v2017.8
I actually don't quite know what the version inheritance really does, but let's
be safe and fix this. I'm being conservative here and fixing it to inherit from
2017.8, skipping .9 since that doesn't have a parent.

Related: https://github.com/ostreedev/ostree/issues/1087

Closes: #1088
Approved by: jlebon
2017-08-17 16:57:33 +00:00
Colin Walters
ceeefa69f5 docs/related-projects: Tweak client side snapshot text, add casync
The latter came up on the list.

Closes: #1086
Approved by: jlebon
2017-08-17 16:48:25 +00:00
Colin Walters
6709aa83a8 lib/sysroot: Port a few functions to new style
Not sure why we didn't do this earlier. Just noticed them when looking at the
code for a different reason.

Closes: #1085
Approved by: jlebon
2017-08-17 14:37:49 +00:00
Colin Walters
40a64bcfe9 lib/deploy: Ignore errors from FITHAW
In the production case since we used `daemon()` our stderr is `/dev/null`¹
there's not much use in logging errors from `FITHAW` or `exit(1)`, and doing so
breaks the test suite which checks the return from `waitpid()`. There's nothing
we can really do if `FITHAW` fails, and in most of those cases `EINVAL`,
`EOPNOTSUPP`, we *shouldn't* do anything anyways.

¹ Though perhaps we should set up the systemd journal, but let's not
  go there right now.

Closes: #1084
Approved by: jlebon
2017-08-16 18:01:04 +00:00
Colin Walters
ba28684ac2 lib/deploy: Really close testing race condition
I added `waitpid()`, but that didn't actually help because we were
`daemon()`izing. Don't daemonize if we're testing so that we can `waitpid()`.

Note I still haven't reproduced this race locally, but I'm pretty sure this will
fix it.

While here, actually check the return value from `waitpid()` just in case
something goes wrong there.

Closes: #1084
Approved by: jlebon
2017-08-16 18:01:04 +00:00
Colin Walters
eb35821a51 build-sys: Move bash completions to /usr/share/ by default
This is in line with the "/etc is for sysadmins", "/usr is OS" model;
e.g. systemd's bash completions go there.

Making this change since I was looking at the required spec file changes.

Closes: #1083
Approved by: mbarnes
2017-08-16 15:14:31 +00:00
Jonathan Lebon
76fc1ba476 ot-main.c: fix signal callback signature
Signal callbacks take a void* as their final parameter, which we don't
use in this case.

Closes: #1082
Approved by: cgwalters
2017-08-16 13:17:14 +00:00
Jonathan Lebon
c0e20861d5 pull: mention libcurl in NOT_SUPPORTED pull path
Since it's now possible to build without libsoup but still have HTTP
functionality.

Closes: #1082
Approved by: cgwalters
2017-08-16 13:17:14 +00:00
Jonathan Lebon
fa3a31af92 ostree-deployment.c: simplify equality check
Just a random cozy patch I made while perusing the codebase. When
determining if two OstreeDeployment objects are the same, rather than
just checking for NULL, we can just directly check for equality of
pointers to also catch the trivial case.

Closes: #1082
Approved by: cgwalters
2017-08-16 13:17:14 +00:00
Colin Walters
6063bdb013 Update libglnx
This is mostly the `copy_file_range` changes plus the Coverity files.

```
Colin Walters (4):
      localalloc: Abort on EBADF from close() by default
      local-alloc: Remove almost all macros like glnx_free, glnx_unref_variant
      console: Fix Coverity NULL deref warning
      fdio: Merge systemd code to use copy_file_range(), use FICLONE

Jonathan Lebon (1):
      console: trim useless check

Matthew Leeds (1):
      dirfd: Fix typo in comment

Philip Withnall (1):
      glnx-console: Add missing NULL check before writing out text
```

Update submodule: libglnx

Closes: #1081
Approved by: jlebon
2017-08-16 12:56:48 +00:00
Matthew Barnes
b669bcafe5 ostree: Add naggy comments to help keep options in sync
/* ATTENTION:
 * Please remember to update the bash-completion script (bash/ostree) and
 * man page (man/ostree-$COMMANDNAME.xml) when changing the option list.
 */

Closes: #1080
Approved by: cgwalters
2017-08-15 17:02:26 +00:00
Matthew Barnes
a11d696b4c bash: Add bash completion
Completes commands, options, commit checksums, ref names, remotes, and file paths.

Closes: #1077
Approved by: jlebon
2017-08-15 15:51:34 +00:00
Colin Walters
fd98bda3c7 repo: Introduce ostree_repo_open_at() and ostree_repo_create_at()
This essentially completes our fd-relative conversion.

While here, I cleaned up the semantics of `ostree_repo_create()` and
`ostree_repo_create_at()` to be more atomic - basically various scripts were
testing for the `objects` subdirectory, so let's formalize that.

Closes: #820
Approved by: jlebon
2017-08-15 12:35:10 +00:00
Jonathan Lebon
64b7c42025 pull: mark commits from local cache as partial
If one of the localcache repos has the exact same commit we resolved
from the remote, then we need to make sure to mark it as partial so that
we download the full tree.

Closes: #1074

Closes: #1076
Approved by: cgwalters
2017-08-14 12:04:42 +00:00
Philip Withnall
2522db0164 lib/repo-finder-config: Add some more debug output
This makes diagnosing false negatives a little easier.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1075
Approved by: jlebon
2017-08-11 18:07:29 +00:00
Jonathan Lebon
d3a5d26176 man: document configured branches
Closes: #1072

Closes: #1073
Approved by: guyshapiro
2017-08-10 14:54:03 +00:00
Colin Walters
95c832b4e2 lib/deploy: Close test suite race condition
Saw this in a PR result; we need to wait for the child to have written its
result to stderr before we exit, otherwise the test suite may not read it in
time.

Closes: #1070
Approved by: jlebon
2017-08-10 14:42:58 +00:00
Matthew Leeds
6c7a6519dd man: Update ostree-refs manpage
Update the ostree-refs manpage to document recently added options, and
fix some minor mistakes.

Closes: #1068
Approved by: cgwalters
2017-08-10 14:31:49 +00:00
Colin Walters
de153dea30 lib/sysroot: Add journal-msg signal
This will allow us to drop the awful hack in rpm-ostree where we watch our own
stdout. In general, libraries shouldn't write to stdout.

Also we can kill the systemd journal wrapper code. There's some duplication at
each call site now...but it's easier than trying to write a `sd_journal_send()`
wrapper.

I was originally going to have this emit all of the structured data too as a
`GVariant` but decided it wasn't worth it right now.

Closes: #1052
Approved by: jlebon
2017-08-10 14:20:00 +00:00
Philip Withnall
75bce24cb9 lib/gpg-verify: Add an OstreeGpgError error domain
Add a new error domain for GPG signing/verification errors, and use it
throughout libostree for describing verification errors. This replaces
various uses of G_IO_ERROR_FAILED, and one instance of
G_IO_ERROR_NOT_FOUND (for which some code in ot-builtin-show.c had to be
changed to ensure it was still handled correctly).

The use of a separate error domain allows failures in GPG operations to
be handled separately from network failures (where the summary file
could not be found to be downloaded, for example) or timeouts.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1064

Closes: #1071
Approved by: mbarnes
2017-08-10 13:38:40 +00:00