Commit Graph

3425 Commits

Author SHA1 Message Date
Colin Walters
3ab0d5e664 lib/sysroot: Support /usr/lib/modules/$kver for kernel/initramfs
This is the new Fedora kernel standard layout; it has the advantage
of being in `/usr` like `/usr/lib/ostree-boot`, but it's not OSTree
specific.

Further, I think in practice forcing tree builders to compute the checksum is an
annoying stumbling block; since we already switched to e.g. computing checksums
always when doing pulls, the cost of doing another checksum for the
kernel/initramfs is tiny. The "bootcsum" becomes more of an internal
implementation detail.

Now, there is a transition; my current thought for this is that rpm-ostree will
change to default to injecting into both `/usr/lib/ostree-boot` and
`/usr/lib/modules`, and stop doing `/boot`, then maybe next year say we drop the
`/usr/lib/ostree-boot` by default.

A twist here is that the default Fedora kernel RPM layout (and what's in
rpm-ostree today) includes a kernel but *not* an initramfs in
`/usr/lib/modules`. If we looked only there, we'd just find the kernel. So we
need to look in both, and then special case this - pick the legacy layout if we
have `/usr/lib/modules` but not an initramfs.

While here, rework the code to have an `OstreeKernelLayout` struct which makes
dealing with all of the variables nicer.

Closes: #1079
Approved by: jlebon
2017-08-18 17:34:36 +00:00
Colin Walters
984d22303d bin: Squash some -Wuninit warnings with porting to new style
I noticed this with a local build of an RPM:

```
/usr/include/glib-2.0/glib/glib-autocleanups.h:28:3: warning: 'help' may be used uninitialized in this function [-Wmaybe-uninitialized]
   g_free (*pp);
   ^~~~~~~~~~~~
src/ostree/ot-main.c:82:20: note: 'help' was declared here
   g_autofree char *help;
                    ^~~~
```

Closes: #1091
Approved by: jlebon
2017-08-17 21:04:04 +00:00
Colin Walters
e6a4203c52 boot: Add a tmpfiles.d snippet to clean up /var/tmp/ostree-ovl.XXX
This is simplest for now.  Compare with similar logic from
`/usr/lib/tmpfiles.d/tmp.conf`:
```
R! /tmp/systemd-private-*
```

Closes: https://github.com/ostreedev/ostree/issues/393

Closes: #1090
Approved by: jlebon
2017-08-17 18:38:45 +00:00
Colin Walters
bd9e931b33 build-sys: Post-release version bump
Closes: #1089
Approved by: jlebon
2017-08-17 18:18:21 +00:00
Colin Walters
395676917b Release 2017.10
Closes: #1089
Approved by: jlebon
2017-08-17 18:18:21 +00:00
Colin Walters
c4148d4ddd lib: Fix v2017.10 symbols to inherit from v2017.8
I actually don't quite know what the version inheritance really does, but let's
be safe and fix this. I'm being conservative here and fixing it to inherit from
2017.8, skipping .9 since that doesn't have a parent.

Related: https://github.com/ostreedev/ostree/issues/1087

Closes: #1088
Approved by: jlebon
2017-08-17 16:57:33 +00:00
Colin Walters
ceeefa69f5 docs/related-projects: Tweak client side snapshot text, add casync
The latter came up on the list.

Closes: #1086
Approved by: jlebon
2017-08-17 16:48:25 +00:00
Colin Walters
6709aa83a8 lib/sysroot: Port a few functions to new style
Not sure why we didn't do this earlier. Just noticed them when looking at the
code for a different reason.

Closes: #1085
Approved by: jlebon
2017-08-17 14:37:49 +00:00
Colin Walters
40a64bcfe9 lib/deploy: Ignore errors from FITHAW
In the production case since we used `daemon()` our stderr is `/dev/null`¹
there's not much use in logging errors from `FITHAW` or `exit(1)`, and doing so
breaks the test suite which checks the return from `waitpid()`. There's nothing
we can really do if `FITHAW` fails, and in most of those cases `EINVAL`,
`EOPNOTSUPP`, we *shouldn't* do anything anyways.

¹ Though perhaps we should set up the systemd journal, but let's not
  go there right now.

Closes: #1084
Approved by: jlebon
2017-08-16 18:01:04 +00:00
Colin Walters
ba28684ac2 lib/deploy: Really close testing race condition
I added `waitpid()`, but that didn't actually help because we were
`daemon()`izing. Don't daemonize if we're testing so that we can `waitpid()`.

Note I still haven't reproduced this race locally, but I'm pretty sure this will
fix it.

While here, actually check the return value from `waitpid()` just in case
something goes wrong there.

Closes: #1084
Approved by: jlebon
2017-08-16 18:01:04 +00:00
Colin Walters
eb35821a51 build-sys: Move bash completions to /usr/share/ by default
This is in line with the "/etc is for sysadmins", "/usr is OS" model;
e.g. systemd's bash completions go there.

Making this change since I was looking at the required spec file changes.

Closes: #1083
Approved by: mbarnes
2017-08-16 15:14:31 +00:00
Jonathan Lebon
76fc1ba476 ot-main.c: fix signal callback signature
Signal callbacks take a void* as their final parameter, which we don't
use in this case.

Closes: #1082
Approved by: cgwalters
2017-08-16 13:17:14 +00:00
Jonathan Lebon
c0e20861d5 pull: mention libcurl in NOT_SUPPORTED pull path
Since it's now possible to build without libsoup but still have HTTP
functionality.

Closes: #1082
Approved by: cgwalters
2017-08-16 13:17:14 +00:00
Jonathan Lebon
fa3a31af92 ostree-deployment.c: simplify equality check
Just a random cozy patch I made while perusing the codebase. When
determining if two OstreeDeployment objects are the same, rather than
just checking for NULL, we can just directly check for equality of
pointers to also catch the trivial case.

Closes: #1082
Approved by: cgwalters
2017-08-16 13:17:14 +00:00
Colin Walters
6063bdb013 Update libglnx
This is mostly the `copy_file_range` changes plus the Coverity files.

```
Colin Walters (4):
      localalloc: Abort on EBADF from close() by default
      local-alloc: Remove almost all macros like glnx_free, glnx_unref_variant
      console: Fix Coverity NULL deref warning
      fdio: Merge systemd code to use copy_file_range(), use FICLONE

Jonathan Lebon (1):
      console: trim useless check

Matthew Leeds (1):
      dirfd: Fix typo in comment

Philip Withnall (1):
      glnx-console: Add missing NULL check before writing out text
```

Update submodule: libglnx

Closes: #1081
Approved by: jlebon
2017-08-16 12:56:48 +00:00
Matthew Barnes
b669bcafe5 ostree: Add naggy comments to help keep options in sync
/* ATTENTION:
 * Please remember to update the bash-completion script (bash/ostree) and
 * man page (man/ostree-$COMMANDNAME.xml) when changing the option list.
 */

Closes: #1080
Approved by: cgwalters
2017-08-15 17:02:26 +00:00
Matthew Barnes
a11d696b4c bash: Add bash completion
Completes commands, options, commit checksums, ref names, remotes, and file paths.

Closes: #1077
Approved by: jlebon
2017-08-15 15:51:34 +00:00
Colin Walters
fd98bda3c7 repo: Introduce ostree_repo_open_at() and ostree_repo_create_at()
This essentially completes our fd-relative conversion.

While here, I cleaned up the semantics of `ostree_repo_create()` and
`ostree_repo_create_at()` to be more atomic - basically various scripts were
testing for the `objects` subdirectory, so let's formalize that.

Closes: #820
Approved by: jlebon
2017-08-15 12:35:10 +00:00
Jonathan Lebon
64b7c42025 pull: mark commits from local cache as partial
If one of the localcache repos has the exact same commit we resolved
from the remote, then we need to make sure to mark it as partial so that
we download the full tree.

Closes: #1074

Closes: #1076
Approved by: cgwalters
2017-08-14 12:04:42 +00:00
Philip Withnall
2522db0164 lib/repo-finder-config: Add some more debug output
This makes diagnosing false negatives a little easier.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1075
Approved by: jlebon
2017-08-11 18:07:29 +00:00
Jonathan Lebon
d3a5d26176 man: document configured branches
Closes: #1072

Closes: #1073
Approved by: guyshapiro
2017-08-10 14:54:03 +00:00
Colin Walters
95c832b4e2 lib/deploy: Close test suite race condition
Saw this in a PR result; we need to wait for the child to have written its
result to stderr before we exit, otherwise the test suite may not read it in
time.

Closes: #1070
Approved by: jlebon
2017-08-10 14:42:58 +00:00
Matthew Leeds
6c7a6519dd man: Update ostree-refs manpage
Update the ostree-refs manpage to document recently added options, and
fix some minor mistakes.

Closes: #1068
Approved by: cgwalters
2017-08-10 14:31:49 +00:00
Colin Walters
de153dea30 lib/sysroot: Add journal-msg signal
This will allow us to drop the awful hack in rpm-ostree where we watch our own
stdout. In general, libraries shouldn't write to stdout.

Also we can kill the systemd journal wrapper code. There's some duplication at
each call site now...but it's easier than trying to write a `sd_journal_send()`
wrapper.

I was originally going to have this emit all of the structured data too as a
`GVariant` but decided it wasn't worth it right now.

Closes: #1052
Approved by: jlebon
2017-08-10 14:20:00 +00:00
Philip Withnall
75bce24cb9 lib/gpg-verify: Add an OstreeGpgError error domain
Add a new error domain for GPG signing/verification errors, and use it
throughout libostree for describing verification errors. This replaces
various uses of G_IO_ERROR_FAILED, and one instance of
G_IO_ERROR_NOT_FOUND (for which some code in ot-builtin-show.c had to be
changed to ensure it was still handled correctly).

The use of a separate error domain allows failures in GPG operations to
be handled separately from network failures (where the summary file
could not be found to be downloaded, for example) or timeouts.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1064

Closes: #1071
Approved by: mbarnes
2017-08-10 13:38:40 +00:00
Colin Walters
f1102763df lib/sysroot-deploy: Refactor kernel layout parsing
I'd like to move the new canonical kernel directory to `/usr/lib/modules/$kver`,
as Fedora has done. The `get_kernel_from_tree()` function now abstracts over
parsing the data (src vs destination filenames, as well as checksum) in
preparation for adding the new case.

In preparation for this, let's change the current test suite to use the
*current* directory of `/usr/lib/ostree-boot`, and also add coverage of `/boot`.

Closes: #1053
Approved by: jlebon
2017-08-09 13:46:17 +00:00
Colin Walters
8b60f63f58 lib/sysroot-deploy: Port a kernel finding logic to new style
Prep for more work here.

Closes: #1053
Approved by: jlebon
2017-08-09 13:46:17 +00:00
Lisa St. John
596fc2967e Update introduction.md
Added missing "is" in the first sentence.
Closes: #1067
Approved by: cgwalters
2017-08-09 13:15:44 +00:00
Philip Withnall
8c148eb7e1 lib/repo-finder: Emit gpg-verify-summary=false in dynamic remote config
When returning results from finding repos, set gpg-verify-summary=false
in their configs, since any pulls from such remotes will necessarily
involve collection IDs, and hence should be using the unsigned summary
support. In the intended deployment mode for P2P transmission of OSTree
refs, summaries *cannot* be signed, so setting gpg-verify-summary=true
would cause all the pulls to fail.

The unsigned summary support is the move of repository metadata from
the summary file (not spliceable) to the well-known ostree-metadata ref
(spliceable, as it can exist for multiple collection IDs in the same
repository).

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1066
Approved by: cgwalters
2017-08-08 18:08:05 +00:00
Colin Walters
8642ef5ab3 lib/deploy: Use a FIFREEZE/FITHAW cycle for /boot
See: http://marc.info/?l=linux-fsdevel&m=149520244919284&w=2

XFS doesn't flush the journal on `syncfs()`. GRUB doesn't know how to follow the
XFS journal, so if the filesystem is in a dirty state (possible with xfs
`/boot`, extremely likely with `/`, if the journaled data includes content for
`/boot`, the system may be unbootable if a system crash occurs.

Fix this by doing a `FIFREEZE`+`FITHAW` cycle.  Now, most people
probably would have replaced the `syncfs()` invocation with those two
ioctls.  But this would have become (I believe) the *only* place in
libostree where we weren't safe against interruption.  The failure
mode would be ugly; nothing else would be able to write to the filesystem
until manual intervention.

The real fix here I think is to land an atomic `FIFREEZETHAW` ioctl
in the kernel.  I might try a patch.

In the meantime though, let's jump through some hoops and set up
a "watchdog" child process that acts as a fallback unfreezer.

Closes: https://github.com/ostreedev/ostree/issues/876

Closes: #1049
Approved by: jlebon
2017-08-08 16:09:04 +00:00
Colin Walters
9f8f351cd4 lib: Port gpg verification for remotes to fd-relative
This was the last use of `repo->repodir` internally, and will help finally add
`ostree_repo_open_at()`.

Closes: #1034
Approved by: jlebon
2017-08-08 15:50:25 +00:00
Philip Withnall
1672e2eee0 lib/repo: Fix handling of missing summary files when downloading
The API for downloading a summary file can legitimately return NULL for
the summary file contents when it returns TRUE (success). This indicates
an error 404 — the summary file was not found.

Two call sites were not handling that correctly, which was causing later
assertion failures.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1061

Closes: #1065
Approved by: cgwalters
2017-08-08 15:33:45 +00:00
Philip Withnall
86dce4b252 ostree/parse-datetime: Ensure tm structs are initialised
Otherwise tm.tm_wday remains uninitialised and gets propagated
elsewhere.

Spotted by Coverity as issue #209265.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1059
Approved by: cgwalters
2017-08-08 14:11:07 +00:00
Philip Withnall
e9b9123bae lib/sepolicy: Drop duplicate assignment
Spotted by Coverity as issue #1452619.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1059
Approved by: cgwalters
2017-08-08 14:11:07 +00:00
Philip Withnall
4f187b576d lib/repo-commit: Drop unreachable conditional branch
(remaining > 0) is asserted by the loop condition, and remaining is not
modified between that check and the G_UNLIKELY — so the condition in the
G_UNLIKELY will always be true.

Spotted by Coverity as issue #1452617.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1059
Approved by: cgwalters
2017-08-08 14:11:07 +00:00
Philip Withnall
baa9534fda lib/repo-finder-avahi: Drop redundant conditional
summary_timestamp is checked for non-NULL-ness above, and the function
bails if it’s NULL.

Fixes Coverity issue #1452616.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1059
Approved by: cgwalters
2017-08-08 14:11:06 +00:00
Philip Withnall
f35b409077 lib/repo-refs: Add ostree_repo_remote_list_collection_refs() API
This parallels ostree_repo_remote_list_refs(), but returns a map of
OstreeCollectionRef → checksum, and includes refs from collection IDs
other than the remote repository’s main collection ID.

Use this in OstreeRepoFinderConfig to ensure that refs are matched
against even if they’re stored in the repository summary file’s
collection map, rather than its main ref map. This fixes false negatives
when searching for refs in some situations.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1058
Approved by: cgwalters
2017-08-08 13:59:58 +00:00
Philip Withnall
11e165b154 lib/repo-pull: Add a missing precondition
This catches a few failure modes in the pull code a little earlier,
before the incorrectly-NULL repo makes its way into a closure and a
worker thread, where the cause of the problem is harder to track down.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1058
Approved by: cgwalters
2017-08-08 13:59:58 +00:00
Philip Withnall
0d2a9a7909 lib/repo-finder: Avoid a potential unref-of-NULL crash
As the comment explains, it’s possible for a result to be freed while
ref_to_checksum is NULL, even though normally the data structure
guarantees it’s non-NULL. This was causing crashes when results were
filtered out of a find-remotes call. Guard against that.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1058
Approved by: cgwalters
2017-08-08 13:59:58 +00:00
Philip Withnall
be4832242d lib/repo-pull: Fix counting of latest commits when finding repos
The intended behaviour of ostree_repo_find_remotes() is to return
results which have the latest version of at least one of the requested
refs. Results which have some of the requested refs, but don’t have the
latest version of any of them, should be ignored. The logic to do this
was broken in the case that a result contained a positive number of the
requested refs, but none of them were the latest version. (It previously
worked when the result contained none of the requested refs.)

Fix the counting to work correctly in both cases.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1058
Approved by: cgwalters
2017-08-08 13:59:58 +00:00
Stef Walter
39b3c8e90a man: The min-free-space-percent item goes in [core] section
The documentation incorrectly indicates that min-free-space-percent
goes in the [remote "name"] section. It should go in [core] instead.

Closes: #1062
Approved by: cgwalters
2017-08-08 12:56:47 +00:00
Stef Walter
773154a513 autogen.sh: Fix running out of tree
The autogen.sh script should be runnable out of tree. It's mostly
already the case, just one little tweak to make it work.

   $ mkdir build
   $ cd build
   $ ../autogen.sh --prefix=/usr

Closes: #1063
Approved by: cgwalters
2017-08-08 12:44:17 +00:00
Colin Walters
6d861dd92d tree-wide: Remove trailing semicolon from autoptr declarations
It confuses `g-ir-scanner`, and isn't necessary.

Closes: #1056
Approved by: pwithnall
2017-08-07 17:42:32 +00:00
Colin Walters
48f8f32921 bin/pull: Fix @ override syntax when pulling multiple refs
Coverity spotted an infloop here since we were incrementing `i++`
instead of `j++`.  But adding a test revealed other bugs - we need
to keep the arrays in sync.

Coverity CID: 1452204

Closes: #1041
Approved by: pwithnall
2017-08-07 16:10:08 +00:00
Colin Walters
8eec337fee lib/deltas: Squash Coverity warning for div-by-zero in delta show
If a delta happens to have zero objects, we could end up doing
a divide-by-zero when inferring endianness.  In practice,
a zero-object delta isn't possible to generate I think, but
let's make sure the code is defensive all the same.

Spotted by Coverity.

Coverity CID: 1452208

Closes: #1041
Approved by: pwithnall
2017-08-07 16:10:08 +00:00
Guy Shapiro
b08a198f00 Documentation: static delta default from
Document that the default behavior of `ostree static-delta generate` if to generate the delta from the parent.
Closes: #1057
Approved by: cgwalters
2017-08-07 15:06:55 +00:00
Matthew Leeds
5682f4cc04 main: Fix subcommand usage output
This commit sets prgname correctly so that the "ostree subcommand
--help" output prints the subcommand rather than just "ostree".

This was removed in commit f0519e541f because it tripped the thread
sanitizer, but it's being added back conditionally so most users who
don't compile with -fsanitize=adress see proper help output.

Closes: #1054
Approved by: cgwalters
2017-08-05 00:55:36 +00:00
Colin Walters
4e068f3924 tree-wide: Fix the build with old glib (Ubuntu Trusty etc.)
This regressed with <https://github.com/ostreedev/ostree/pull/1040>
but currently the Travis builds aren't gating.

Closes: #1051
Approved by: jlebon
2017-08-03 16:23:41 +00:00
Colin Walters
d3897e52e7 lib: Define and use an autoptr cleanup for gpgme_key_t
Followup for previous patch, allows porting a bit to new code style.

Closes: #1039
Approved by: jlebon
2017-08-03 15:01:57 +00:00
Colin Walters
b929b620ae tree-wide: Use g_autoptr(Ostree*)
Part of cleaning up our usage of libglnx; we want to use what's in GLib where we
can.

Had to change a few .c files to `#include ostree.h` early on to pick up
autoptrs for the core types.

Closes: #1040
Approved by: jlebon
2017-08-03 13:48:12 +00:00