Commit Graph

3815 Commits

Author SHA1 Message Date
Colin Walters
2b8d586c5a switchroot: Ensure /sysroot is set to "private" propagation
Downstream BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1498281

This came up as a problem with `oci-umount` which was trying to ensure some host
mounts like `/var/lib/containers` don't leak into privileged containers.  But
since our `/sysroot` mount wasn't private we also got a copy there.

We should have done this from the very start - it makes `findmnt` way, way less
ugly and is just the obviously right thing to do, will possibly create world
peace etc.

Closes: #1438
Approved by: rhvgoyal
2018-01-30 15:05:37 +00:00
Philip Withnall
785da8d5a6 lib/core: Expand documentation for ostree_parse_refspec()
The old documentation had outdated and incomplete annotations, and
didn’t make it very clear that out_remote could legitimately return
NULL.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1437
Approved by: cgwalters
2018-01-29 18:21:00 +00:00
Colin Walters
794d1d98d6 ci: Bump flatpak version (and build to f27)
Just keeping this updated.

Closes: #1400
Approved by: jlebon
2018-01-24 23:24:58 +00:00
Matthew Leeds
660bc588ee lib/pull: Allow specific commits in P2P updates
Currently users of the find_remotes_async()/pull_from_remotes_async()
functions have no way to specify a commit hash to use instead of the
latest one available. This commit implements an "override-commit-ids"
option analogous to the one used by ostree_repo_pull_with_options().
It's accomplished by returning OstreeRepoFinderResult objects pointing
to the given commit checksum(s) regardless of which ones were available
from the remotes, but in the future this implementation could be
improved to take into account the commits advertised by the remotes.

One effect of this is that flatpak will have the ability to downgrade
apps that use collection IDs
(https://github.com/flatpak/flatpak/issues/1309).

Closes: #1425
Approved by: pwithnall
2018-01-24 14:15:53 +00:00
Colin Walters
42eea23864 bin/delta: Fix compilation with relative subdirs --filename
Currently we were parsing `opt_filename` twice...I dug through
the history a bit and it looks like it may have been an accident
from refactoring.

What we're fixing here concretely is that using relative subdirectories
like `--filename somesubdir/foo` broke because we were incorrectly
passing the `somesubdir/` again.

Closes: #1423

Closes: #1427
Approved by: jlebon
2018-01-22 14:02:34 +00:00
Colin Walters
661ec2b1a4 ci: Run a subset ⊂ of rpm-ostree's tests
This is a quick hack to get us more than unit testing, albeit indirectly.

See: https://github.com/projectatomic/rpm-ostree/issues/662

Closes: #771
Approved by: jlebon
2018-01-19 14:16:51 +00:00
Colin Walters
6e4306ea10 ci/papr: Update most contexts to f27
Many of them actually already *were* because they
were inherting.

An exception is flatpak which is being worked on in
https://github.com/ostreedev/ostree/pull/1400

Closes: #1426
Approved by: jlebon
2018-01-19 07:41:18 +00:00
Colin Walters
654a9177d0 lib/pull: Port a few functions to new style
Prep for further work here. This diff is a bit noisy for the delta bits because
the identation was off originally as well.

Closes: #1424
Approved by: jlebon
2018-01-19 07:41:13 +00:00
William Manley
720e2ec9bc Add support for devicetree files alongside the kernel and initramfs
Much like the (optional) initramfs at
`/usr/lib/ostree-boot/initramfs-<SHA256>` or
`/usr/lib/modules/$kver/initramfs` you can now optionally include a
flattened devicetree (.dtb) file alongside the kernel at
`/usr/lib/ostree-boot/devicetree-<SHA256>` or
`/usr/lib/modules/$kver/devicetree`.

This is useful for embedded ARM systems which need the devicetree file
loaded by the bootloader for the kernel to discover and initialise
hardware.  See https://en.wikipedia.org/wiki/Device_tree for more
information.

This patch was mostly produced by copy-pasting code for initramfs handling
and renaming `s/initramfs/devicetree/g`.  It's not beautiful, but it is
fairly straightforward.

It may be useful to extend device-tree support in a number ways in the
future.  Device trees dependant on many details of the hardware they
support.  This makes them unlike kernels, which may support many different
hardware variants as long as the instruction-set matches.  This means that
a ostree tree created with a device-tree in this manner will only boot on
a single model of hardware.  This is sufficient for my purposes, but may
not be for others'.

I've tested this on my NVidia Tegra TK1 device which has u-boot running
in syslinux-compatible mode.

Closes: #1411
Approved by: cgwalters
2018-01-16 22:54:53 +00:00
William Manley
c5112c25e4 syslinux: Add support for DEVICETREE from bootloader spec
The bootloader spec says:

> `devicetree` refers to the binary device tree to use when executing the
> kernel. This also shall be a path relative to the `$BOOT` directory. This
> key is optional. Example:
> `6a9857a393724b7a981ebb5b8495b9ea/3.8.0-2.fc19.armv7hl/tegra20-paz00.dtb`

This is necessary for booting my NVidia Tegra TK1 device.  It uses u-boot
with syslinux compatibility.  In the syslinux files that come with the
device this is called `FDT`, but u-boot treats `FDT and `DEVICETREE` as
synonyms.

See also: [f43c401 in u-boot].

[f43c401 in u-boot]: http://git.denx.de/?p=u-boot.git;a=commit;h=f43c401b72bb0db43ab0b55c4a79e1f4889d3aa2

Closes: #1411
Approved by: cgwalters
2018-01-16 22:54:53 +00:00
William Manley
2098f75c2f ostree admin deploy: Add --no-prune option
If you want cleanup, but don't want to prune the repo.  Pruning can
be quite expensive so ostree admin deploy can be much faster without
pruning.

Closes: #1418
Approved by: cgwalters
2018-01-16 18:59:06 +00:00
William Manley
c5d6725d91 ostree admin deploy: Refactor bringing cleaning into main
In the next commit I will add --no-prune which will affect cleaning.  By
doing this refactor we avoid having to add a NO_PRUNE flag.

Closes: #1418
Approved by: cgwalters
2018-01-16 18:59:06 +00:00
Matthew Leeds
ebc104d3c7 find-remotes: Minor fixes to --finders code
This introduces no functional changes, only cleanups.

Closes: #1414
Approved by: jlebon
2018-01-16 14:15:10 +00:00
Matthew Leeds
4c2205276c man: Add man page for find-remotes
Closes: #1410
Approved by: pwithnall
2018-01-15 19:26:43 +00:00
Matthew Leeds
792b32eb96 man: Update ostree-summary.xml
Update the man page for the summary command to add the undocumented
options, make the syntax clear, and add examples.

Closes: #1416
Approved by: pwithnall
2018-01-15 17:33:06 +00:00
Colin Walters
52dd4f282c build-sys: Post-release version bump
I'm still doing release, then versionbump as separate PRs to ensure
the release commit is tested by itself.

Closes: #1417
Approved by: pwithnall
2018-01-15 15:18:59 +00:00
Colin Walters
d3fa95023e Release 2018.1
In particular I'd like to get the `--copyup` changes out for an rpm-ostree
release that will use them. But there are other good changes here, and let's
keep up a regular release train 🚄 in general.

Closes: #1413
Approved by: jlebon
2018-01-15 14:10:37 +00:00
Colin Walters
8e6e64a5ad lib: Validate metadata structure more consistently during pull
Previously we were doing e.g. `ot_util_filename_validate()` specifically inline
in dirtree objects, but only *after* writing them into the staging directory (by
default). In (non-default) cases such as not using a transaction, such an object
could be written directly into the repo.

A notable gap here is that `pull-local --untrusted` was *not* doing
this verification, just checksums.  We harden that (and also the
static delta writing path, really *everything* that calls
`ostree_repo_write_metadata()` to also do "structure" validation
which includes path traversal checks.  Basically, let's try hard
to avoid having badly structured objects even in the repo.

One thing that sucks in this patch is that we need to allocate a "bounce buffer"
for metadata in the static delta path, because GVariant imposes alignment
requirements, which I screwed up and didn't fulfill when designing deltas. It
actually didn't matter before because we weren't parsing them, but now we are.
In theory we could check alignment but ...eh, not worth it, at least not until
we change the delta compiler to emit aligned metadata which actually may be
quite tricky.  (Big picture I doubt this really matters much right now
but I'm not going to pull out a profiler yet for this)

The pull test was extended to check we didn't even write a dirtree
with path traversal into the staging directory.

There's a bit of code motion in extracting
`_ostree_validate_structureof_metadata()` from `fsck_metadata_object()`.

Then `_ostree_verify_metadata_object()` builds on that to do checksum
verification too.

Closes: #1412
Approved by: jlebon
2018-01-12 19:38:34 +00:00
Colin Walters
f3ae36ff43 lib/checkout: Validate pathnames during checkout
While we do protect against path traversal during pull, let's also validate
during checkout; it's a cheap operation and provides good last-mile protection.

Closes: #1412
Approved by: jlebon
2018-01-12 19:38:34 +00:00
Colin Walters
2b78df25f4 tests: Add a test case for path traversal in a dirtree
I was reading about a recent security issue with both EMC and VMWare:
https://arstechnica.com/information-technology/2018/01/emc-vmware-security-bugs-throw-gasoline-on-cloud-security-fire/

It's a classic path traversal problem, and that made me think more about our
handling of this in libostree.  Fortunately of course, not being new to
this rodeo, long ago I *did* consider path traversal.  Inside the pull
code, we call `ot_util_filename_validate()`.  Also, `fsck` does this too.

I have further followups here, but let's add some test cases for this. I crafted
a repository with a `../` in a dirtree object by patching libostree to inject
it, and that's included as a tarball.

This patch covers the two cases where we do already have checks; pulling
via HTTP, and in `fsck`.

Closes: #1412
Approved by: jlebon
2018-01-12 19:38:34 +00:00
Jonathan Lebon
854a823e05 tests/libtest-core: support multiple literal checks
`grep` supports checking multiple fixed strings separated by newlines,
but it's mostly just easier to pass them as separate arguments, so let's
support that. This is now at parity with the similar
`assert_file_has_content`.

Closes: #1409
Approved by: cgwalters
2018-01-11 21:30:22 +00:00
Colin Walters
fdf7e2c560 lib/fetcher: Add version to USER_AGENT string
This came up in allowing Fedora infrastructure to work around a libcurl bug with
HTTP2: https://pagure.io/atomic-wg/issue/405

Closes: https://github.com/ostreedev/ostree/issues/1405

Closes: #1406
Approved by: jlebon
2018-01-11 14:06:16 +00:00
Matthew Leeds
0be95ded99 tests: Use --finders option for find-remotes
All the current uses of the find-remotes command in the tests use it to
find configured remotes or mounted (USB) remotes, so using
--finders=config and --finders=mount in the tests respectively shouldn't
affect the correctness of the tests. It does however allow the tests to
be run in an environment that doesn't have an Avahi daemon.

Closes: #1407
Approved by: cgwalters
2018-01-11 02:19:07 +00:00
Matthew Leeds
2c932d9721 find-remotes: Add --finders option
It can be helpful to be able to choose which OstreeRepoFinder instances
to use when using the find-remotes command. For example, if the tests
need to run in an environment that can't have an Avahi daemon, this
allows you to disable the Avahi (LAN) finder. This commit adds the
--finders option for this purpose.

Closes: #1407
Approved by: cgwalters
2018-01-11 02:19:07 +00:00
William Manley
3318db548e Tests: test-no-initramfs: Test both legacy and new kernel locations
Closes: #1401
Approved by: cgwalters
2018-01-10 13:52:58 +00:00
Gatis Paeglis
3724692d9e ostree-grub-generator: update outdated comment
Closes: #1401
Approved by: cgwalters
2018-01-10 13:52:58 +00:00
Gatis Paeglis
4233b1db19 Support for booting without initramfs
Previously when initramfs-* was not found in a deployment's
boot directory, it was assumed that rootfs is prepared for
ostree booting by a kernel patch.

With this patch, the behaviour changes to be - if initramfs-*
is not found, assume that system is using a static
ostree-prepare-root as init process. Booting without initramfs
is a common use case on embedded systems. This approach is
also more convenient, than having to patch the kernel.

Closes: #1401
Approved by: cgwalters
2018-01-10 13:52:58 +00:00
Gatis Paeglis
652d9dd98a deploy: add --karg-none argument
If the current deployment has "rootwait root=/dev/sda2",
but the new deployment does not need "rootwait" anymore,
there is no way to clear this arg at the moment (as opposed
to "karg=root=", which overrides any earlier argument with
the same name). With "--karg-none" users can now clear all
the previous args and set new "root=":

ostree admin deploy --karg-none --karg=root=LABEL=rootfs

Closes: #1401
Approved by: cgwalters
2018-01-10 13:52:58 +00:00
Gatis Paeglis
62cb078973 ostree-prepare-root: enabler for simpler kernel arg
With the current approach, when ostree-prepare-root is used
on the kernel command line as init=, it always assumes that
the next value in the argument list is a path to the sysroot.
The code for falling back to a default path (if none is provided),
would only work if init= is the last arg in the argument list.
We can not rely on that and have to explicitly provide the
path to the sysroot. Which defeats the purpose of a default
path selection code.

To keep command line neater assume that sysroot is on / when
using ostree-prepare-root as init. This probably is what most
people want anyways. Also _ostree_kernel_args* API assumes
that args are space separated list. Which is problematic for:
"init=${ostree}/usr/lib/ostree/ostree-prepare-root /" as it
gets split in two.

Closes: #1401
Approved by: cgwalters
2018-01-10 13:52:58 +00:00
Jonathan Lebon
94bbbdf3ca bash/ostree: add missing --add-metadata option
Closes: #1402
Approved by: cgwalters
2018-01-10 01:42:56 +00:00
Jonathan Lebon
939791b4fa bin/commit: add --keep-metadata option
Clients of libostree such as rpm-ostree make extensive use of the
`ostree commit -b foo --tree=ref=foo` pattern in their tests, e.g. to
simulate an update.

What I'm trying to solve here is that it's often the case that we want
to keep metadata from the previous commit without having to be too
verbose (i.e. reading from the parent, then passing it as an argument).

The new `--keep-metadata` switch makes this really easy. I intend to use
this in the rpm-ostree testsuite to make sure we always carry over the
`source-title` metadata as well as during set up for tests that require
`rpmostree.rpmdb.pkglist` metadata.

I initially implemented this in a small wrapper script that uses the API
directly, though we make use of so many other `ostree commit` functions
that it'd require re-implementing a lot of it.

Closes: #1402
Approved by: cgwalters
2018-01-10 01:42:56 +00:00
Jonathan Lebon
95e574d09b bin/commit: move parent checking code higher up
No functional change. Prep for the next commit.

Closes: #1402
Approved by: cgwalters
2018-01-10 01:42:56 +00:00
Colin Walters
2c2e6799be grub2: Exit gracefully if there's no system ostree repository
Apparently there testing systems that literally install *all*
packages.  Having `ostree-grub2` currently causes grub2 to fail
on a non-ostree managed system.  Let's just gracefully exit
if there's no system repository.

https://bugzilla.redhat.com/show_bug.cgi?id=1532668

Closes: #1399
Approved by: jlebon
2018-01-09 18:30:59 +00:00
Will Thompson
9fe6ddbaef ostree-grub-generator: fix typo in comment
Closes: #1398
Approved by: jlebon
2018-01-09 14:22:30 +00:00
Anton Gerasimov
353fb175c6 build-sys: Allow building with curl, but without libsoup
Some people (particularly embedded) may find it simpler to
drop libsoup from the build dependency side, but still use libcurl.

Note though this currently neuters almost all of the tests.

Signed-off-by: Anton Gerasimov <anton.gerasimov@openmailbox.org>

Closes: #1397
Approved by: cgwalters
2018-01-08 15:38:07 +00:00
Colin Walters
c8d9da8d96 bin: Fix cookie builtin build with curl but no soup
Prep for supporting `--with-curl --without-soup`.

Closes: #1397
Approved by: cgwalters
2018-01-08 15:38:07 +00:00
Colin Walters
3b9304b5d7 rofiles: Fix --copyup when creating a new file
This tripped up the `docbook-dtds` `%post` in my experiments
with doing rpm-ostree for buildroots.

I cloned and built [xfstests](https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git)
but haven't yet investigated actually running it.

In the meantime let's do the obvious fix here; we need to distinguish
between "copyup enabled" and "actually did a copyup" in the open path
at least, since if we didn't do a copyup we don't need to re-open.

Closes: #1396
Approved by: jlebon
2018-01-08 15:21:29 +00:00
Colin Walters
46a841a062 rofiles: Add --copyup option
Sadly https://sourceware.org/bugzilla/show_bug.cgi?id=22089 is I think going to
actually force us to cave here. Even if we got the glibc patch in today, we need
to support the RHEL glibc. See also discussion about fish as part of the general
Fedora tracker.

This is basically needed to unblock rpm-ostree unified core 🌐:
https://github.com/projectatomic/rpm-ostree/issues/729

Closes: https://github.com/ostreedev/ostree/issues/1377

Closes: #1382
Approved by: jlebon
2018-01-05 21:04:39 +00:00
Simon McVittie
994cd66744 tests: Assert that byte-order is swapped on LE but not BE CPUs
Closes: #1392
Signed-off-by: Simon McVittie <smcv@collabora.com>

Closes: #1393
Approved by: cgwalters
2018-01-04 12:32:47 +00:00
Colin Walters
95f9b392a4 Revert "ci: Make rust build nonblocking for now"
This reverts commit 8ef18fd850.

Closes: #1391
Approved by: smcv
2018-01-04 12:23:06 +00:00
Colin Walters
1f832597fc build-sys: Link with -ldl for rust build
I didn't dive into this too much, it looks like something in rust changed that
broke our build. Probably libstd gained a dependency on `-ldl` or so, and that's
handled by cargo? Anyways linking against it isn't going to hurt.

Closes: #1391
Approved by: smcv
2018-01-04 12:23:06 +00:00
Simon McVittie
f63e62fbd2 tests: Don't assume uid == primary gid
Nothing guarantees that each user has a group containing only
themselves. Even if they do, nothing guarantees that its group ID
equals the user ID, particularly if another user earlier in the same
range was created without a corresponding group or vice versa.

Signed-off-by: Simon McVittie <smcv@collabora.com>

Closes: #1390
Approved by: cgwalters
2018-01-02 14:31:36 +00:00
Marcus Folkesson
8d3d14503b lib/pull: allways include ostree-repo-pull-private.h
Allways include ostree-repo-pull-private.h to get rid of the following
build error when HAVE_LIBCURL_OR_LIBSOUP is not defined:

src/libostree/ostree-repo-pull.c:1493:1: error: no previous prototype
for '_ostree_repo_verify_bindings' [-Werror=missing-prototypes]

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1389
Approved by: cgwalters
2017-12-21 22:47:06 +00:00
Philip Withnall
cac42bb6f5 build: Fix typo in -Wparentheses warning
GCC supports -Wparentheses, not -Wparenthesis.

https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wno-parentheses

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1388
Approved by: jlebon
2017-12-21 22:34:26 +00:00
Colin Walters
117d5c9f77 build-sys: Post-release version bump
Closes: #1387
Approved by: jlebon
2017-12-21 21:50:08 +00:00
Colin Walters
8ef18fd850 ci: Make rust build nonblocking for now
Will debug at some point but for now let's
unblock other things.

```
/usr/bin/ld: /var/tmp/checkout/target/release/libbupsplit_rs.a(bupsplit_rs-db7d02fa07221ce3.bupsplit_rs0.rust-cgu.o): undefined reference to symbol 'dladdr@@GLIBC_2.2.5'
```

Closes: #1387
Approved by: jlebon
2017-12-21 21:50:08 +00:00
Colin Walters
19d08dab61 Release 2017.15
Let's do a new release with the locking preview, the http2 disable options and
other misc bugfixes to close out the year.

Closes: #1386
Approved by: jlebon
2017-12-19 16:10:26 +00:00
Colin Walters
5a77b8dafe Bump libglnx, use "n items" progress for fsck
Sooo much nicer.  See also
https://github.com/projectatomic/rpm-ostree/pull/1143

Update submodule: libglnx

Closes: #1383
Approved by: jlebon
2017-12-15 15:50:34 +00:00
Colin Walters
8c42e81f12 build-sys: Use -fno-strict-aliasing by default
See discussion in https://bugzilla.gnome.org/show_bug.cgi?id=791622

This is what e.g. systemd, the Linux kernel, and lots of other projects do. It's
astonishingly hard to reliably get right; the optimization IMO only really
matters for truly high performance inner loops, but if you're doing
that kind of stuff today you're probably doing it on a GPU anyways.

Closes: #1384
Approved by: pwithnall
2017-12-15 14:52:38 +00:00
Colin Walters
b822f337b5 bin/refs: Disallow aliases to remote refs
It can't really work in general; the client and server would
have to agree on the name of the remote.

Closes: https://github.com/ostreedev/ostree/issues/1342

Closes: #1381
Approved by: jlebon
2017-12-14 22:22:39 +00:00