2989 Commits

Author SHA1 Message Date
Colin Walters
704c7e57fa .travis.yml: Delete, we aren't using it right now
@smcv will re-add a new version.

Closes: #439
Approved by: smcv
2016-08-05 20:50:43 +00:00
Simon McVittie
0de51bfc45 Skip tests that use whiteouts under Docker/aufs
ostree's naming convention for whiteouts is similar to what is
done in aufs, which means we can't compose the trees to test this
feature under Docker with the aufs storage driver, as used on
travis-ci.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #437
Approved by: cgwalters
2016-08-05 20:32:57 +00:00
Simon McVittie
64e4525793 Update libglnx: Add missing files to libglnx distribution
This fixes distcheck.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #436
Approved by: cgwalters
2016-08-05 19:41:09 +00:00
Colin Walters
d55655b096 repo: Drop internal GFile config_file
The remote parsing code still uses GFiles but this is a start.

Closes: #432
Approved by: giuseppe
2016-08-05 08:26:07 +00:00
Colin Walters
18a576c390 lib: Drop GFiles for self->{objects_dir,state_dir}
Everything that used to reference these is now fd-relative.

Closes: #432
Approved by: giuseppe
2016-08-05 08:26:07 +00:00
Colin Walters
308c75e281 repo: Port metadata writing code to fd-relative
It was the last thing referencing `self->objects_dir`.

Closes: #432
Approved by: giuseppe
2016-08-05 08:26:07 +00:00
Colin Walters
cfd402e5f7 lib: Fix a compiler warning introduced from earlier patch
I think we'd end up closing stdin...yuck.  Need to investigate
having this be fatal, but CentOS 7 `gcc-4.8.5` doesn't understand
`-Werror=int-conversion`.

Closes: #430
Approved by: giuseppe
2016-08-05 07:38:09 +00:00
Colin Walters
30aa1ec668 lib: Use libglnx file replace API more consistently
We have a better API now, drop use of the internal helper, which also
depended on libgsystem.

This required bumping libglnx to pull in a fix.

Closes: #429
Approved by: giuseppe
2016-08-04 21:27:32 +00:00
Colin Walters
1ce633f158 Update libglnx for O_TMPFILE build time option
See: https://github.com/ostreedev/ostree/issues/421

Closes: #426
Approved by: gatispaeglis
2016-08-04 13:30:55 +00:00
Colin Walters
593effe2fb lib: Add padding booleans to OstreeRepoCheckoutAtOptions
Since this is a new API, and adding booleans is the most likely thing
we'll do, let's stick some explicit padding for them in here now.

We could use the `unused_ints[]` but it'd be out of order, and this
will more clearly remind people about the padding.  The efficiency hit
versus bitfields is annoying, but oh well, not a real world problem.

Closes: #427
Approved by: giuseppe
2016-08-04 12:18:28 +00:00
Colin Walters
18d826e3a5 repo: Flip the fsync default to off for new checkout API
Since we're adding a new API, we have the opportunity to fix
the defaults.  We expect clients to do a `syncfs()` or equivalent
on their own now, since it's way more efficient.

Flip the checkout fsync default to off.

Closes: #425
Approved by: giuseppe
2016-08-04 07:33:31 +00:00
Colin Walters
f66906c401 deltas: Port compilation to libglnx/fd-relative
This was the last use of libgsystem tmpfile APIs.  The change here is
a bit uglier than it needs to be because we support creating the delta
in place as well as in the repo, so we needed to abstract over
locations.

Closes: #424
Approved by: giuseppe
2016-08-03 08:10:27 +00:00
Colin Walters
a843d16de2 grub2: Port away from gs_file_sync_data()
Inlining this is uglier, but this code kind of sucks anyways =( We're
going to need to do some fd-relative porting in here at some point.

Closes: #424
Approved by: giuseppe
2016-08-03 08:10:27 +00:00
Colin Walters
c671c1dd3e lib: Port away from gs_file_rename()
In one case, we already had relative fds and hence this was
nicer.  Unfortunately the other areas got uglier.  More fd-relative
porting to do later.

Closes: #424
Approved by: giuseppe
2016-08-03 08:10:27 +00:00
William Manley
52acb9d8e3 ostree-prepare-root: Add test that overlayfs over /usr works.
Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
78f99d4780 ostree-prepare-root: Fix typo in error message
Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
6356edaaba ostree-prepare-root: Make error message capitalisation consistent
There seemed to be more lower case first letters so I've standardised
on that.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
485a374b21 ostree-prepare-root: Use pivot_root if real sysroot is already mounted at /
This allows ostree-prepare-root outside of the initramfs context where the
real rootfs is already mounted at /.  We can't use `mount --move` in this
case because we would be trying to move / into a subdirectory of itself.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
83c0fdc352 Refactor ostree-prepare-root: Perform chdir to deploy directory earlier
...for simplicity.  This way we don't need to keep concatenating
deploy_path to everything.  We can just refer relative to the current
working directory.

We need to do this after bind-mounting it over itself otherwise our cwd
is still on the non-bind-mounted filesystem below.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
47e7afab27 ostree-prepare-root: Refactor: Create /sysroot.tmp much later
Typically we have our ready made-up up root at
`/sysroot/ostree/deploy/.../` (`deploy_path`) and the real rootfs at
`/sysroot` (`root_mountpoint`).  We want to end up with our made-up root
at `/sysroot/` and the real rootfs under `/sysroot/sysroot` as systemd
will be responsible for moving `/sysroot` to `/`.

We need to do this in 3 moves to avoid trying to move `/sysroot` under
itself:

1. `/sysroot/ostree/deploy/...` -> `/sysroot.tmp`
2. `/sysroot` -> `/sysroot.tmp/sysroot`
3. `/sysroot.tmp` -> `/sysroot`

This is a refactoring to group all these operations together so I can
implement an alternative in terms of `pivot_root`.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
a0a4052365 ostree-prepare-root: Cope with /proc not being mounted
When trying to read kernel command-line.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
c9551dc231 ostree-prepare-root as init: exec init from deployment if run as PID1
This supports running ostree on embedded platforms without an initrd.
Specificially I'm trying to do bringup on an NVidia Tegra based Jetson TK1
dev board.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
1364e6e30d ostree-prepare-root: Refactor code to resolve deploy_path
I'll reuse this for a new ostree-init.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
William Manley
94e50ca5aa tests: Add basic tests for ostree-prepare-root
These tests use unshare and mount to prepare a fake initrd/early boot
directory structure so we can then test ostree-prepare-root.

Things that are tested:

* Running in an initrd environment
* Running without initrd
* /var and /sysroot being mounted correctly
* /usr being mounted read-only

Things not tested (yet):

* Running as init - this could be accomplished by unsharing the pid
  namespace too.
* mounting/unmounting `/proc` if `/proc/cmdline` isn't available
* Persistent overlayfs for `/usr`
* Probably other things

The tests are basic but can be extended in the future as we do more work
on `ostree-prepare-root`.

These tests must be run as root as they require the ability to `mount`
and to `unshare` the mount namespace. Perhaps in the future we can use
user namespaces for this test once they are more widely available.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
Colin Walters
24671f7d25 repo: Port to g_autoptr() rather than old style cleanup macros
This notably kills a few libgsystem uses here.

Closes: #423
Approved by: giuseppe
2016-08-01 21:00:12 +00:00
Colin Walters
421c902125 repo: Port sign_data() to libglnx tmpfile APIs
Part of dropping libgsystem.

Closes: #423
Approved by: giuseppe
2016-08-01 21:00:12 +00:00
Colin Walters
6f17237849 deploy: Replace a use of gs_file_enumerator with compat wrapper
More libglnx/glib porting.

Closes: #423
Approved by: giuseppe
2016-08-01 21:00:12 +00:00
Colin Walters
5334758ba7 repo: Make ostree_repo_create() nonfatal on existing repos
In general we want to support "idempotentcy" or "state
synchronization" across interruption.  If a repo is only partially
created due to a crash or whatever, it's hard for a user to know that.
Let's just make `ostree_repo_create()` idempotent. Since all we're
doing is a set of `mkdirat()` invocations, it's quite simple.

This also involved porting to fd-relative, which IMO makes the
code a lot clearer.

Closes: #422
Approved by: 14rcole
2016-08-01 15:12:14 +00:00
Dan Nicholson
a312d8fd8c tests: Add test for delta with empty parts
Closes: #420
Approved by: cgwalters
2016-07-31 00:01:52 +00:00
Dan Nicholson
512090785e deltas: Allow processing of empty delta parts
If a static delta is generated between 2 commits with the same content,
then the delta will contain 1 part with no checksums. While useless,
this is a valid delta that shouldn't raise an assertion. If the delta
part has no checksums, then there are no objects to recreate and the
processing can be skipped.

Closes: #420
Approved by: cgwalters
2016-07-31 00:01:52 +00:00
Giuseppe Scrivano
edecae06ab libostree, ostree: fix usage of ostree_repo_checkout_tree_at
it was deprecated, use ostree_repo_checkout_at.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Giuseppe Scrivano
0bac88e132 libostree: skip introspection for two functions
These two functions are not safe for gobject introspection, so annotate
them to be skipped:

1) ostree_repo_import_archive_to_mtree
2) ostree_repo_export_tree_to_archive

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Giuseppe Scrivano
8867693240 libostree: mark ostree_repo_checkout_tree_at as deprecated
and move its definition to a separate file.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Giuseppe Scrivano
30963766c7 libostree: new function ostree_repo_checkout_at
Provide a gobject introspection safe version for
`ostree_repo_checkout_tree_at'.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Colin Walters
6d310db1e7 libglnx porting: Migrate to new tempfile code
In general this is even cleaner now, though it was better after I
extracted a helper function for the "write tempfile with contents"
bits that were shared between metadata and regular file codepaths.

Closes: #369
Approved by: jlebon
2016-07-29 19:02:41 +00:00
Jonathan Lebon
1db0c8dcd6 static-delta: remove unused struct
Closes: #418
Approved by: cgwalters
2016-07-29 16:38:19 +00:00
Jonathan Lebon
e9f08d45f7 static-delta: fix command contexts
Closes: #418
Approved by: cgwalters
2016-07-29 16:38:19 +00:00
Colin Walters
6ffcb24d22 deltas: Handle untrusted checksums faster and more robustly
When reworking the ostree core [to use O_TMPFILE](https://github.com/ostreedev/ostree/pull/369),
I hit an issue in the way the untrusted delta codepath ends up trying
to re-open the file to checksum it.  That's not possible with
`O_TMPFILE` since the fd (which we opened `O_WRONLY`) is the only
accessible reference to the content.

Fix this by changing the delta processing code to update a checksum as
we're doing writes, which is also faster, and ends up simplifying the
code as well.

What would be an even larger simplification here is if we e.g. used a
separate thread calling `write_object()` or something like that; the
main issue I see there is somehow bridging the fact that function
wants a `GInputStream*` but the delta code is generating stream of
writes.

Closes: #392
Approved by: jlebon
2016-07-29 16:03:28 +00:00
Colin Walters
fb0bf27d10 tests/delta-crosscheck: Add missing --from
Otherwise we generate a static delta from the *previous* and get
confused.  To make doubly sure, add `--require-static-deltas` to pull.

Closes: #416
Approved by: jlebon
2016-07-29 14:30:42 +00:00
Colin Walters
fc25fa16e1 lib: Fix compiler warning from previous patch
I had this in a `fixup!` I thought.

Closes: #414
Approved by: giuseppe
2016-07-28 14:46:17 +00:00
Colin Walters
a38f2f41b2 lib: Fix leak in bootconfig parser
Pretty simple.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
27559c58a9 deploy: Fix leaks in parsing /etc/os-release
This one is a bit subtle; we're generating a hash that contains
pointers to the strings we parsed, so we need to carefully track
ownership.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
707ef93097 sysroot: Fix a leak in deployment dirpath API
This one is pretty obvious in retrospect.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
402020b4ca glib.supp: Suppress g_task -> thread leaks
There are variants of this, but right now because `GTask` has its
own pool which isn't necessarily cleaned up on exit, we need
to suppress all of this.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
ba21023d6c fetcher: Explicitly join thread if it's not self
This fixes a valgrind leak; see [this StackOverflow thread](http://stackoverflow.com/questions/17642433/why-pthread-causes-a-memory-leak).

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
544365f834 glib.supp: Suppress worker context -> thread bits
The worker context isn't cleaned up now.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
35008ead77 pull-local: Explicitly unref variant rather than relying on floating
This shows up as a leak in valgrind; the callee isn't sinking.  In
general through the power of cleanup attributes we can do explicit
cleanup rather than relying on floating refs.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
d13cf37b3f glib.supp: Suppress some dynamic type registrations too
These mirror the static ones, and we need the dynamic versions for
glib-networking -> gnutls stuff.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
7f335a8437 trivial-httpd: Fix leak of option context
Right now our valgrind runs also end up valgrinding the
`trivial-httpd` code, so while it doesn't matter, let's fix this leak
anyways.  We need to avoid calling `_exit()` since that won't run the
cleanup functions.

Closes: #410
Approved by: giuseppe
2016-07-28 10:10:17 +00:00
Colin Walters
bac1fa1839 pull: Don't execute static deltas when mirroring
We don't presently support this, since the static delta code assumes
it can just `mmap()` file objects.  We could at some point implement
this, but for now just skip executing deltas when doing
`archive -> archive` mirroring.

I noticed this when trying to mirror a repo in Jenkins in
[CAHC](https://wiki.centos.org/SpecialInterestGroup/Atomic/Devel).

Closes: #408
Approved by: giuseppe
2016-07-25 06:50:51 +00:00