3014 Commits

Author SHA1 Message Date
Simon McVittie
53f1fabfbe ot_remote_builtin_show_url: autofree context
Leak found with valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #556
Approved by: cgwalters
2016-10-30 22:11:15 +00:00
Simon McVittie
1fc2a1202e ostree_repo_pull_with_options: clear dirs array
Leak found with valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #556
Approved by: cgwalters
2016-10-30 22:11:15 +00:00
Simon McVittie
4739709742 keyfile_set_from_vardict: free the string array
g_variant_get_strv is (transfer container): the caller is expected to
free the array, but not the individual strings.

Leak found with valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #556
Approved by: cgwalters
2016-10-30 22:11:15 +00:00
Simon McVittie
c8a6b037ef _ostree_sysroot_write_deployments_internal: stop leaking hash table
It appears the result of assign_bootserials() is never actually used,
but I haven't changed it to return void right now.

Leak found with valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #556
Approved by: cgwalters
2016-10-30 22:11:15 +00:00
Simon McVittie
ff28ac4a30 ostree_sysroot_upgrader_finalize: free new_revision
Leak found with valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #556
Approved by: cgwalters
2016-10-30 22:11:15 +00:00
Simon McVittie
4c32344b8e ostree_admin_option_context_parse: explicitly clean up when exiting early
The cleanup attribute doesn't clean up before calling a noreturn
function like exit(). Explicitly clean up the pointer variables
(but don't assume that a simple g_object_unref() would be OK either,
in case the behaviour of the cleanup attribute changes).

This isn't a real leak since we're about to exit anyway, but if we
don't fix it then valgrind memcheck will make the tests fail.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #559
Approved by: cgwalters
2016-10-30 21:37:38 +00:00
Simon McVittie
213d5013ce ostree_builtin_pull: consistently set free-function on refs_to_fetch
We are relying on the GPtrArray to free its contents, but we only
give it a free-function on one code path.

Found by valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #559
Approved by: cgwalters
2016-10-30 21:37:38 +00:00
Simon McVittie
22ed96d23c ot_admin_builtin_set_origin: don't leak options GVariant
Found by valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #559
Approved by: cgwalters
2016-10-30 21:37:38 +00:00
Simon McVittie
e757f736e7 _ostree_kernel_args_replace_take: don't leak when replacing
If !existed, then we add arg to kargs->order, where it will be freed
by that array's free-function. However, if the kernel argument did
already exist, we have to either free arg ourselves (and make sure
the old key is what appears in the hash table), or do a linear search
on kargs->order to replace the old key with the new.

Leak found by valgrind memcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #559
Approved by: cgwalters
2016-10-30 21:37:37 +00:00
Simon McVittie
a0e7d411c1 Distribute valgrind suppressions in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #557
Approved by: cgwalters
2016-10-29 18:41:56 +00:00
Simon McVittie
4eb55a687d Distribute test scripts even if we wouldn't run them
This fixes a "make dist" tarball produced on a minimal system and run on a
non-minimal system. Automake knows that files that are only conditionally
included in dist_whatever_WHATEVER are to be distributed, but it does not
do the same for files that are only conditionally included in EXTRA_DIST,
which is how glib-tap.mk's various variables like dist_test_scripts work.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #557
Approved by: cgwalters
2016-10-29 18:41:56 +00:00
Simon McVittie
7091d288f7 Force C.UTF-8 or C locale for tests
Otherwise several tests fail, for example in this build done in a French
locale by Debian's reproducible builds initiative, to check whether
the resulting binaries are identical to what was produced in an
English locale:

<https://tests.reproducible-builds.org/debian/logs/unstable/amd64/ostree_2016.11-1.build2.log.gz>

(test-basic)
# error: Cannot write to repository: Permission non accordée
...
File 'error-message' doesn't match regexp 'Permission denied'

(test-help)
# Utilisation :
#   ostree [OPTION...] COMMAND
...
File 'out' doesn't match regexp '[Uu]sage'

(test-pull-metalink)
# error: Erreur à la ligne 1, caractère 1 : Le document doit commencer avec un élément (par ex. <book>)
...
File 'err.txt' doesn't match regexp 'Document must begin with an element'

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #558
Approved by: cgwalters
2016-10-29 18:12:42 +00:00
Colin Walters
a15dc7f191 ci: Use -fsanitize=undefined by default
It's fast enough to use for CI testing by default, and it
can catch a lot of bad things.

Closes: #552
Approved by: jlebon
2016-10-28 17:47:50 +00:00
Colin Walters
05dc77d7e5 remote-refs: Add NULL terminator to options array
Caught by `-fsanitize=undefined`.

Closes: #552
Approved by: jlebon
2016-10-28 17:47:50 +00:00
Colin Walters
396563e7e9 libglnx: Bump to master (for -fsanitize fixes)
Closes: #552
Approved by: jlebon
2016-10-28 17:47:50 +00:00
Giuseppe Scrivano
c4c8937b20 static-delta: find a similar filename using what is before '.' or '-'
Improve the heuristic to use only the part before the first '.' when
looking for a similar file in the current directory.

last versions of dracut generate reproducible initramfs files, but we
still fallback to the full file download if there is any minimal
change that causes a different checksum and file name.

This change extends that case to deal better with similar files that
have a different suffix.

This is the difference generating a static delta from
fedora-atomic/f24/x86_64/docker-host to fedora-atomic/f24/x86_64/testing/docker-host

before the patch:

fallback for 111ec866aa7ce3688407fa4a1ae7c9fca93dcee0b851fc9434c59ff947830cc7 (47.0 MB)
fallback for c6a898265de22b02c89ea2f35d132628d0ee1c0a058052ed14fee5799c17904c (47.0 MB)
fallback for fbce656249ece77260887ed873e445561b9d43bcb28a32e759c0b1bab89e7137 (6.6 MB)
fallback for cfdb51457e47e0a0fe0bac38991a21279d2646ff2f019630c7b52a0cd3451397 (6.6 MB)
part 0 n:1972 compressed:11239809 uncompressed:33747412
part 1 n:1079 compressed:9683681 uncompressed:55641397
part 2 n:1507 compressed:15050265 uncompressed:44448838
part 3 n:101 compressed:1865881 uncompressed:31896086
part 4 n:278 compressed:2452585 uncompressed:52811323
part 5 n:18 compressed:67621 uncompressed:100220
uncompressed=218645276 compressed=40359842 loose=545102
rollsum=49 objects, 2117254 bytes
bsdiff=4067 objects

after the patch:

part 0 n:843 compressed:19844109 uncompressed:95443178
part 1 n:1223 compressed:11188609 uncompressed:33330401
part 2 n:990 compressed:15762905 uncompressed:61214132
part 3 n:1441 compressed:20614573 uncompressed:31534195
part 4 n:163 compressed:2734997 uncompressed:51356423
part 5 n:285 compressed:2480813 uncompressed:52902904
part 6 n:14 compressed:59125 uncompressed:75341
uncompressed=325856574 compressed=72685131 loose=533283
rollsum=51 objects, 57235332 bytes
bsdiff=4073 objects

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #554
Approved by: cgwalters
2016-10-28 17:04:25 +00:00
Jonathan Lebon
0333260559 delta: return valid enum member
If we can't figure out what endianness a delta is, we should just throw
ENDIAN_INVALID.

Resolves: #550

Closes: #553
Approved by: cgwalters
2016-10-27 22:16:32 +00:00
Colin Walters
7f2960db43 Define an initializer for GVariant{Builder,Dict}
So we build warning-free on GLib (< 2.50, >= 2.50).  This
is a band aid until we hard-require >= 2.50.

Closes: #547
Approved by: jlebon
2016-10-27 18:49:15 +00:00
Jonathan Lebon
cde7928496 .redhat-ci.yml: add clang
Clang has better detection for unused vars when using auto cleanup
functions. We should eventually just fold this back into the first
testsuite. But let's just turn it on for now, at least until it's
satisfied with the whole codebase.

Closes: #549
Approved by: cgwalters
2016-10-27 18:12:05 +00:00
Colin Walters
835d97d659 deploy: Suppress unused variable warning for fscreatecon cleanup
Fixes the clang build.

Closes: #551
Approved by: jlebon
2016-10-27 17:50:56 +00:00
Colin Walters
cbbfb5369f parse-datetime: Use labs() for long input value
Fixes a compliation warning with CLang, I didn't study it to
see whether this was a major issue or not.

Closes: #548
Approved by: jlebon
2016-10-27 17:02:01 +00:00
Colin Walters
21ca60f987 otutil: Note that ot_log_structured takes a printf format
This notably fixes compilation with CLang.

Closes: #548
Approved by: jlebon
2016-10-27 17:02:01 +00:00
Colin Walters
b77edf24a3 tree-wide: Remove unused variables detected by CLang
CLang finds these, whereas GCC treats having
`__attribute__((cleanup))` as a use.

This obsoletes https://github.com/ostreedev/ostree/pull/411

Closes: #548
Approved by: jlebon
2016-10-27 17:02:01 +00:00
Colin Walters
82a4f56593 tests: Skip libarchive/selinux tests if in container without SELinux
I'm doing builds and `make check` inside a Docker container, with
selinux on as a build-time option, but no policy in the container.
This currently aborts.  Let's not do that.

(This type of thing is why installed tests are a better model)

Closes: #546
Approved by: jlebon
2016-10-27 16:48:34 +00:00
Colin Walters
27f37b55bc docs: Link to releng-scripts
Now that the repo starts to implement some of this stuff.

Closes: #544
Approved by: jlebon
2016-10-27 16:28:55 +00:00
Alexander Larsson
d0e74cf3af Fix pruning of partial commits
If we have a partial commit it is not an error for a dirmeta to be
missing (in fact, that is likely), so instead of returning a not-found
error from ostree_repo_traverse_commit() we ignore the error and
continue.

In particular, this means we don't stop early at the first
missing dirmeta, which previously caused ostree_repo_prune() to
thing the dirmetas after that to be unreached and thus purged.

Also, we remove the special casing in ostree_repo_prune() to
not report errors for commitpartial, because these should not
be reported anymore.

This fixes https://github.com/ostreedev/ostree/issues/541

Closes: #542
Approved by: cgwalters
2016-10-24 17:48:19 +00:00
Colin Walters
d3f14f02e3 Release 2016.12
Closes: #540
Approved by: cgwalters
v2016.12
2016-10-21 19:24:31 +00:00
Alexander Larsson
67bddf76f7 detached metadata: Put these in transaction
If there is a transaction active, then we put writes to detached
metadata into the staging dir, and when reading it we look there
first. This allows transactions to be aborted half-way without
writing the detached metadata into the repository (possibly
overwriting any old metadata from there).

This fixes https://github.com/ostreedev/ostree/issues/526

Closes: #539
Approved by: giuseppe
2016-10-21 10:50:41 +00:00
Alexander Larsson
d43c121675 ostree_repo_read_commit_detached_metadata: Handle parent repo
If the detached metadata is not in the repo, try in the parent
repo if that is set.

Without this a commit will not gpg validate in the child repo

Closes: #539
Approved by: giuseppe
2016-10-21 10:50:41 +00:00
Alexander Larsson
cf6ec1bbbc Fix regression for symlinks in bare-user repos
Commit 1d4f1b8878addd28059c3a3928640491755cd615 started using hardlinks
checkouts of symlinks. However, symlinks are not stored as symlink in the
repo for bare-user repos, so this breaks user-mode checkouts of such repos.

We fix this by checking for !is_symlink in the bare-user case.

This fixes:
     https://github.com/ostreedev/ostree/issues/537

Closes: #538
Approved by: giuseppe
2016-10-21 09:24:27 +00:00
Jonathan Lebon
64568bc706 .redhat-ci.yml: use projectatomic/ostree-tester
Same Dockerfile, but automated to rebuild on pushes.

Closes: #536
Approved by: giuseppe
2016-10-20 08:35:13 +00:00
Jonathan Lebon
e62defa1a4 add .redhat-ci.yml and .redhat-ci.Dockerfile
Add a YAML file for the new Red Hat CI framework.

Rather than re-installing all the build deps everytime, which takes
time, I added a Dockerfile that we can wire up to the Docker Hub. For
now it lives at `jlebon/ostree-tester:rhci`, but we can move it under
the `projectatomic` org (or a new `ostree` org).

Closes: #535
Approved by: cgwalters
2016-10-19 20:35:23 +00:00
Colin Walters
1d4f1b8878 core: Do create hardlinks to symlinks for checkouts
I was noticing a recent performance issue with checkouts
which seemed to be mostly us going back to doing a `fsync()` on
directories.

Regardless, while looking at that, I saw we were spending time
creating new symlinks. Even though symlinks are small, it's still
better to hardlink them.

Going way back in time, the reason we weren't doing this is
because we were hitting `EMFILE` on ext4, but that was for
gnome-continuous which creates *many* build roots.  Even
there though, they're just a cache, and we handle `EMFILE`.

For ostree-for-host-system, we don't expect to have many roots (just 3
at most transiently), so hardlinking symlinks does make sense.

Closes: #521
Approved by: jlebon
2016-10-17 20:40:15 +00:00
Jonathan Lebon
a660e5650d OstreeFetcher: provide proxy credentials if needed
There seems to be an issue in libsoup which causes basic auth
credentials to not be passed to the proxy during requests. We thus have
to handle PROXY_UNAUTHORIZED responses and provide the auth ourselves.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1370558
Related: https://bugzilla.gnome.org/show_bug.cgi?id=772932

Closes: #529
Approved by: cgwalters
2016-10-14 16:06:08 +00:00
Jonathan Lebon
a0598cb494 ostree-sysroot-deploy.c: delete redundant check
Just noticed this while inspecting the code. The deployments retrieved
by `_ostree_sysroot_list_deployment_dirs_for_os` will forcibly already
have a matching osname since it indirectly uses that same variable to
construct them. Having a check there makes it look like there may be
subtle corner cases, when there aren't.

Closes: #529
Approved by: cgwalters
2016-10-14 16:06:08 +00:00
Jonathan Lebon
3943284dad docs: amend vmlinuz & initramfs naming convention
I was confused while reading the docs how this could work, since in at
least the Fedora/CentOS/RHEL distros, they're named e.g.
initramfs-`uname -r`-$checksum.

Closes: #529
Approved by: cgwalters
2016-10-14 16:06:08 +00:00
Alexander Larsson
ee484697cd pull: Support multiple specifications of --subpath
I need this in flatpak to avoid doing multiple pulls when doing
locale subsetting.

Closes: #523
Approved by: cgwalters
2016-10-11 16:41:03 +00:00
Alexander Larsson
0446cdb0d3 pull: Support inherit-transaction
If this is true, don't initiate, abort of commit a transaction, instead
it is assumed that the caller initiated the transaction, and that it
will eventually be commited.

This allows you to do multiple pulls or a combination of pulls and
commits in a single transaction.

Closes: #525
Approved by: cgwalters
2016-10-07 19:34:47 +00:00
Colin Walters
a0e1344cf8 Release 2016.11
Closes: #524
Approved by: cgwalters
v2016.11
2016-10-07 19:21:41 +00:00
Colin Walters
54621d9e53 libglnx: Update to latest
This pulls in a new compilation flag for wrpseudo compatibility.  Also
note we need to add some includes since glnx-libcontainer went away,
and with it some includes for `sys/mount.h` etc.

Closes: #522
Approved by: cgwalters
2016-10-06 11:54:39 +00:00
Colin Walters
524d2d5cb2 trivial-httpd: Port mostly to fd-relative
We were seeing some weird potential memory corruption in this code
when using it for `rpm-ostree-toolbox installer`, which is almost
certainly not its fault, but let's use it as an excuse to port
(mostly) to fd-relative and away from GFile.

Dropping the last GFile use here is a bit tricky as it does have a
nice high level wrapper around inotify.

Closes: #512
Approved by: jlebon
2016-10-05 21:48:41 +00:00
Colin Walters
a981e5fd76 checkout: Fix fsync defaults for new API to be off for real
My previous change in https://github.com/ostreedev/ostree/pull/425
actually broke things so we basically used the repository defaults =(

This is a subtle mess since we're only trying to flip things off
for the *new* API.

Clean this up so that the "default repo inheritance" lives only in one
place - in the compat layer for the old checkout API.  The new
checkout API defaults to off period, so the repository state is
irrelevant.

Closes: #520
Approved by: jlebon
2016-10-05 21:34:57 +00:00
Simon McVittie
2eae12220e Fix spelling of "repository"
Detected by Debian's Lintian tool.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #519
Approved by: cgwalters
2016-10-01 18:50:41 +00:00
Dan Nicholson
6c84fa4366 admin: Allow running status unlocked
It's useful to let non-root see the current system status.

Closes: #515

Closes: #516
Approved by: jlebon
2016-09-30 18:29:26 +00:00
Jonathan Lebon
056ca71a3b docs: add mention of rpm-ostree package layering
Closes: #514
Approved by: cgwalters
2016-09-20 18:45:44 +00:00
Jonathan Lebon
318430dc70 ostree_sysroot_init_osname: also create /var/log
/var/log is another one of those core directories that should be made
available and properly labeled during early boot before tmpfiles.d
starts up.

Related: https://bugzilla.redhat.com/show_bug.cgi?id=1265295

Closes: #513
Approved by: cgwalters
2016-09-20 18:03:57 +00:00
Colin Walters
450361d89b boot: Ensure we remount /var writable before systemd does journal flush
Otherwise, we may not get a persistent journal for the first boot.
https://bugzilla.redhat.com/show_bug.cgi?id=1265295

Closes: #511
Approved by: jlebon
2016-09-15 19:41:44 +00:00
Colin Walters
f2b6afd2df sysroot: Drop an unnecessary fsync
While looking at a slow update issue (which I'm guessing is
unpredictable I/O latency in an OpenStack instance), I noticed
in one of the traces we were inside a fsync here.

Dropping the fsync here is just another of a long series of unwinding
them - we `syncfs()` the sysroot fd and `/boot` and we have a big
`sync()` anyways.

Closes: #508
Approved by: jlebon
2016-09-14 19:14:46 +00:00
Colin Walters
4f736ac33e sysroot: Drop an fsync for origin file when writing deployments
More fsync pruning.  Since we have a public API for writing the origin
file and it did a fsync before, let's preserve that.  But when writing
deployments as part of a full transaction, we rely on the global
`syncfs()`, so add an internal function for origin file writing that
doesn't.

Closes: #509
Approved by: giuseppe
2016-09-14 10:26:39 +00:00
Colin Walters
fcffb73280 sysroot: Port origin writing code to fd-relative
Just preparatory cleanup for a next patch which makes
the fsyncing configurable.

Closes: #509
Approved by: giuseppe
2016-09-14 10:26:39 +00:00