Commit Graph

151 Commits

Author SHA1 Message Date
Daniel Drake
c1ed9a15c1 Allow commits to mark refs as EOL, replaced by others
A commit can now include a "ostree.endoflife-rebase" metadata key
pointing to a new ref.

When updating, the sysroot upgrader will see this and proceed to
pull and deploy the new ref instead. The origin file in the new
deployment will point to the new ref.

This functionality is planned to be used in Endless OS. We will create
a lesser tested branch for brand new, cutting edge hardware support,
and ship that on hardware platforms that require the latest drivers.
However, once our slower-moving official release is later updated to
support the new hardware, we will use this functionality to migrate
those bleeding-edge users over to the official release.

Closes: #874
Approved by: cgwalters
2017-05-22 20:18:07 +00:00
Colin Walters
a195888b0f lib/checkout: Fix regression in subpath for regular files
This is what caused the merge of
https://github.com/projectatomic/rpm-ostree/pull/652
to blow up, since https://github.com/ostreedev/ostree/pull/848
landed right before we tried to merge it.

When I was writing that PR I remember having an uncertain feeling
since we were doing a `mkdirat` above, but at the time I thought
we'd have test suite coverage...turns out we didn't.

For backwards compatibility, we need to continue to do a `mkdirat` here of the
parent. However...I can't think of a reason anyone would *want* that behavior.
Hence, let's add a special trick - if the destination name is `.`, we skip
`mkdirat()`. That way rpm-ostree for example can open a dfd for `/etc` and avoid
the `mkdir`.

Fold the subpath tests into `test-basic.sh` since it's not worth a separate
file. Add a test case for checking out a file.

Closes: #854
Approved by: jlebon
2017-05-12 14:00:20 +00:00
Colin Walters
48d2637e98 tests: Migrate test-pull-many.sh to installed on FAH
`test-pull-many.sh` is was just too slow to be a unit test.  Generating
a bunch of files via shell is slow, the delta generation is slow, etc.
Every developer doesn't need to run it every time.

Somewhat address this by converting it into our installed test framework, which
moves it out of the developer fast paths.  Another advantage to this is
that we can simply reuse the FAH tree content rather than synthesizing
new bits each time.

Closes: #840
Approved by: jlebon
2017-05-09 15:08:26 +00:00
Philip Withnall
c9244b1bb2 build: Add --enable-experimental-api configure option for unstable APIs
There are currently no unstable APIs, but some will be added in
following commits. They will be built and exposed in the libostree
global symbol list iff configured with --enable-experimental-api.

Distributions should not package OSTree with --enable-experimental-api.
This is designed for previewing new APIs on controlled platforms; any of
the APIs hidden behind this option may be changed or removed at any
point.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Colin Walters
c7efe01520 Add --enable-installed-tests=exclusive, fix installed case
The major reason to do this is that running tests *both* installed
and uninstalled in our CI is a mostly pointless waste of time.
Particularly given we have a few expensive tests.

We *do* have tests that only run uninstalled (since they require
the source code) like `test-symbols.sh`.

Hence, add `--enable-installed-tests=exclusive` to mean *only* do installed for
most tests.

We'll still have uninstalled coverage via the Travis/Debian configs, and we
could perhaps do another build with a subset of uninstalled tests, but I'm not
really concerned about it.

I'd like to do a renewed push for the InstalledTests model since
I feel it's just fundamentally better.  (`g-d-t-r` kind of sucks,
but then so does the automake runner).

Also while we're here - fix the CI to use the correct context,
which started this mess.

Closes: #837
Approved by: dbnicholson
2017-05-08 18:34:10 +00:00
Dan Nicholson
1fe914755a tests: Install libtest-core.sh with installed tests
Without this, running the installed tests fails dramatically. Remove it
from EXTRA_DIST since dist_installed_test_data takes care of dist.

Closes: #837
Approved by: dbnicholson
2017-05-08 18:34:10 +00:00
Philip Withnall
f3cc0eb25a tests: Add a test for ostree summary --view
This includes a test of the new human-readable key names.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #826
Approved by: cgwalters
2017-05-08 00:55:25 +00:00
Colin Walters
654b0c4877 tests/installed: New installed, privileged tests using Fedora AH
Our container-driven tests can't e.g. test SELinux sanely, and
have to support being run as root *and* non-root too.

Use redhat-ci to provision a VM and run tests directly there. These are
installed tests too.

Closes: https://github.com/ostreedev/ostree/issues/806

Closes: #807
Approved by: jlebon
2017-04-25 15:15:06 +00:00
Colin Walters
511b31cfb5 checkout: Merge union/add logic for copies during checkout
We really have an astonishing variety of similar functions which write files and
symlinks. I was working on a different PR and the duplication between the
union-mode and add-mode/none-mode checkout functions bothered me.

I realized that the "handle EEXIST" tri-state maps directly to the
`GLnxLinkTmpfileReplaceMode`, so deduping things makes even more sense.

Closes: #801
Approved by: jlebon
2017-04-25 13:52:35 +00:00
Colin Walters
305db981d4 Add Coccinelle usage: one for blacklisting, one for patch collection
This is inspired by the [Coccinelle](http://coccinelle.lip6.fr/) usage
in systemd.  I also took it a bit further and added infrastructure
to have spatches which should never apply.  This acts as a blacklist.

The reason to do the latter is that coccinelle is *way* more powerful than the
regular expresssions we have in `make syntax-check`.

I started with blacklisting `g_error_free()` directly. The reason that's bad is
it leaves a dangling pointer.

Closes: #754
Approved by: jlebon
2017-03-30 19:19:54 +00:00
Alexander Larsson
7c8f95c86f Add basic tests for bare-user-only repo modes
This is somewhat complicated by such repos only properly supporting
some subset of file metadata (uid/gid 0, etc). We fix this by
always commiting with filters that make it work.

Closes: #750
Approved by: cgwalters
2017-03-27 13:48:41 +00:00
Colin Walters
36b28cb4d2 upgrade: Add support for --pull-only and --deploy-only
This makes it easier to script downloading updates in the background,
and only do deployments just before rebooting.

Partially addresses https://github.com/ostreedev/ostree/issues/640

Closes: #642
Approved by: jlebon
2017-02-27 20:57:23 +00:00
Colin Walters
d894f609db oxidation: Add implementation of bupsplit in Rust
This is an initial drop of "oxidation", or adding implementation
of components in Rust.  The bupsplit code is a good target - no
dependencies, just computation.

Translation into Rust had a few twists -

 - The C code relies a lot on overflowing unsigned ints, and
   also on the C promotion rules for e.g. `uint8_t -> int32_t`
 - There were some odd loops that I introduced bugs in while
   translating...in particular, the function always returns `len`,
   but I mistakenly translated to `len+1`, resulting in an OOB
   read on the C side, which was hard to debug.

On the plus side, an off-by-one array indexing in the Rust code paniced nicely.

In practice, we'll need a lot more build infrastructure to make this work, such
as using `cargo vendor` when producing build artifacts for example. Also, Cargo
is yet another thing we need to cache.

Where do we go with this? Well, I think we should merge this, it's not a lot of
code. We can just have it be an alternative CI target. Should we do a lot more
right now? Probably not immediately, but I find the medium/long term prospects
pretty exciting!

Closes: #656
Approved by: jlebon
2017-02-03 14:29:00 +00:00
Colin Walters
d1f4d94720 tests: Add a big (many objects) pull
This would be more likely to tickle things like
https://github.com/ostreedev/ostree/issues/601
reliably.

Also, while working on the curl backend, I hit on the fact that curl doesn't
queue (by default, you can enable) and will happily create 20000+ concurrent TCP
connections if you try. Having this test would have made that more likely to
fail.

Closes: #650
Approved by: giuseppe
2017-01-23 17:29:02 +00:00
Simon McVittie
9d94fc40c8 Make corrupt-repo-ref.js executable
Debian's Lintian packaging consistency check complains that it isn't
executable but has a #! line. In fact it's reasonable to run this
script directly, so make it executable, and put it in a _scripts
variable so it will be installed executable.

Closes: #652
Approved by: cgwalters
2017-01-19 13:54:59 +00:00
Simon McVittie
01fb30b839 Fix TAP syntax in test-basic-user.sh, and run it
In its initial commit, Alexander Larsson wrote

    This works standalone, but unfortunately it breaks in
    gnome-desktop-testing-runner as /tmp doesn't support
    xattrs, so it is not installed atm.

but we now (a) use /var/tmp, and (b) explicitly skip the test if
xattr support is unavailable. So it should be OK to run now.

Closes: #652
Approved by: cgwalters
2017-01-19 13:54:59 +00:00
Colin Walters
ced22f6c9b Split trivial-httpd into separate binary
Working on the libcurl backend, I hit the issue that the trivial-httpd program
depends on libsoup. I briefly considered having two versions, but libcurl is
client only, and moreover trivial-httpd is no longer trivial - it has various
features which are used by the test suite extensively.

Hence, what we'll do is build it as a separate binary which links to libsoup,
and use it during the tests. We *also* currently still provide `ostree
trivial-httpd` since some things use it like `rpm-ostree-toolbox` and the
Cockpit tests.

After those are ported to use some other webserver, I plan to add a build-time
option to drop it.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
5aefe17ee9 ci: Combine UBSAN and ASAN by default
I only recently realized this was possible.  While we're still seeing
leaks in the CI environment for some reason, adding ASAN gives us
use-after-free detection etc., which is obviously still very useful
even if we're not doing leak checking.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
359ea9b58f tests: Tweak installed tests to deal with ASAN
We need to disable readdir-rand there too.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
47b4dd1b38 Skip gjs-based tests if ASAN is enabled
Unfortunately, introspection uses dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.

Trying to inject LD_PRELOAD=libasan means the outer executable has to
be leak free...which, yeah, I'm not going to get into running ASAN
today on gjs or pygobject.

So, let's skip those tests - ideally, we still run them in some other
context without the sanitizers.  The coverage we have from them is
middling anyways.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
7f2830cb4e build: Make libsoup optional again
The "remote cookies" code broke this.  While I'm not sure anyone is
actually using ostree-without-http, it isn't too hard to keep the
build time conditional going.  Further, this work is preparatory for
libcurl porting.

Closes: #621
Approved by: jlebon
2016-12-08 16:04:16 +00:00
Colin Walters
555f1fb03d tests: Use G_DEBUG=fatal-warnings here too
I am trying to track down a warning I see in `test-keyfile-utils`
which turned out to be the installed case only, but let's inject
this here too.

(The GLib default is broken, but it's hard to fix upstream without
 breaking the world)

Closes: #610
Approved by: jlebon
2016-12-06 14:36:06 +00:00
Simon McVittie
8162463956 tests: prepend to an existing LD_LIBRARY_PATH, GI_TYPELIB_PATH
If we're using LD_LIBRARY_PATH for some locally-built library, we want
to search those after OSTree's own libraries.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #606
Approved by: cgwalters
2016-12-01 14:23:40 +00:00
Simon McVittie
a73fef3eaa build: clean up ostree-remount if building without systemd
This is necessary for "make distcheck" on Travis-CI.

Signed-off-by: Simon McVittie <simon.mcvittie@collabora.co.uk>

Closes: #600
Approved by: cgwalters
2016-11-29 14:11:50 +00:00
Colin Walters
37c07d2f1c pull: Add support for http-headers option
Some deployments may want to gate access to content based on things
like OAuth.  In this model, the client system would normally compute a
token and pass it to the server via an API.

We could theoretically support this in the remote config too, but
that'd be a bit weird for OAuth as the information is dynamic.
Therefore this cleans up the code a little bit to more clearly handle
the case that the fetcher is initialized from both remote config
data plus pull options.

Closes: #574
Approved by: giuseppe
2016-11-16 10:04:22 +00:00
Sjoerd Simons
6af8db6fc4 tests: Add test for the cookie jar handling
Closes: #531
Approved by: cgwalters
2016-11-05 17:34:09 +00:00
Simon McVittie
a0e7d411c1 Distribute valgrind suppressions in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #557
Approved by: cgwalters
2016-10-29 18:41:56 +00:00
Simon McVittie
4eb55a687d Distribute test scripts even if we wouldn't run them
This fixes a "make dist" tarball produced on a minimal system and run on a
non-minimal system. Automake knows that files that are only conditionally
included in dist_whatever_WHATEVER are to be distributed, but it does not
do the same for files that are only conditionally included in EXTRA_DIST,
which is how glib-tap.mk's various variables like dist_test_scripts work.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #557
Approved by: cgwalters
2016-10-29 18:41:56 +00:00
Jonathan Lebon
661e4636f5 tests: add tests for contenturl and mirrorlist
Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Colin Walters
056cc0543f build: Update .gitignore
The patch to add more symlink bits also needed CLEANFILES updates.

Closes: #446
Approved by: jlebon
2016-08-09 14:41:07 +00:00
William Manley
94e50ca5aa tests: Add basic tests for ostree-prepare-root
These tests use unshare and mount to prepare a fake initrd/early boot
directory structure so we can then test ostree-prepare-root.

Things that are tested:

* Running in an initrd environment
* Running without initrd
* /var and /sysroot being mounted correctly
* /usr being mounted read-only

Things not tested (yet):

* Running as init - this could be accomplished by unsharing the pid
  namespace too.
* mounting/unmounting `/proc` if `/proc/cmdline` isn't available
* Persistent overlayfs for `/usr`
* Probably other things

The tests are basic but can be extended in the future as we do more work
on `ostree-prepare-root`.

These tests must be run as root as they require the ability to `mount`
and to `unshare` the mount namespace. Perhaps in the future we can use
user namespaces for this test once they are more widely available.

Closes: #403
Approved by: cgwalters
2016-08-02 19:07:25 +00:00
Colin Walters
29451a3999 build: Make symlink depend on source
Closes: https://github.com/ostreedev/ostree/issues/389

Closes: #400
Approved by: smcv
2016-07-16 20:38:48 +00:00
Giuseppe Scrivano
307f4b2957 Makefile-tests.am: make check uses the built binaries
The tests suite was failing locally as it was using the installed
version of rofiles-fuse, instead of the built one.  Create the needed
symlinks in tests/ as we are already doing for the "ostree" binary.

ostree-prepare-root and ostree-remount added for completeness.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #395
Approved by: cgwalters
2016-07-14 13:26:48 +00:00
Colin Walters
d7629d33a4 tests: Add some test coverage of repeated pulls w/HTTP 500s
Systems like pulp may want to keep retrying in a loop if the server
throws a (hopefully transient) 500, and we need test coverage of
handling these errors versus our existing 404 and 206 coverage.

Closes: #383
Approved by: mbarnes
2016-07-08 18:38:11 +00:00
Dan Nicholson
b53fb92a9d tests: Remove gpg verification files from EXTRA_DIST
Follow on from 70a11189. These files are already disted.

Closes: #372
Approved by: cgwalters
2016-06-27 20:20:21 +00:00
Simon McVittie
0ed9f520da tests: fail the build if symlinking tests/ostree fails
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #367
Approved by: cgwalters
2016-06-26 21:31:35 +00:00
Simon McVittie
ca899ccfd3 tests: use our own generated libtool, not the one in $PATH
libtoolize creates a version of libtool for the right architecture
in $(top_builddir), which is guaranteed to be present, and is
guaranteed to match what we are compiling (even during
cross-compilation).

Packaging systems sometimes separate /usr/bin/libtool, which is
specific to one architecture, from the libtool development files
such as libtoolize and ltmain.sh, which are architecture-independent.
For example, in Debian, libtool_*_all.deb contains the files necessary
to libtoolize a package and is depended on by the dh-autoreconf package,
but libtool-bin_*_amd64.deb (or whatever architecture) contains
/usr/bin/libtool and is not normally necessary to depend on.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #367
Approved by: cgwalters
2016-06-26 21:31:35 +00:00
Colin Walters
71e768d230 build: Fix libreaddir-rand to honor global CFLAGS
It's unfortunate that in automake one has to explicitly include the globa
`$(AM_CFLAGS)` if one sets `CFLAGS`, and similarly for other variables.

I'm trying to use `-fsanitize=address`, and not including it was
causing linker failures.

We also weren't inheriting the global warnings etc., so I had to fix a
decl-after-statement.

Closes: #351
Approved by: jlebon
2016-06-17 14:43:44 +00:00
Simon McVittie
14ff4f94fb build: re-create tests/ostree-symlink-stamp if Makefile changes
The filename of the real ostree executable could be either .libs/ostree
or .libs/lt-ostree.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
70a1118955 build: distribute more test files
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
906575271c build: distribute TAP helper scripts in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
c337d4dab5 build: distribute test scripts in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:03 +00:00
Simon McVittie
b9e18b83fd Link libreaddir-rand to libdl
It uses dlsym(). There's no point in being extra-portable here
because OSTree only targets Linux anyway.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #336
Approved by: cgwalters
2016-06-13 00:12:55 +00:00
Colin Walters
70af1d26b1 tests: Modernize valgrind infrastructure
The recent memleak fixes motivated me to look at the bitrotted code to
run invocations of `ostree` in the test suite underneath valgrind.

There are a few things here.  First, update suppressions file from
libhif, since I recently worked on it.

When running *uninstalled* as we now support, we need
`libtool --mode=execute` in the mix so it expands out to
the uninstalled binary and we don't valgrind the intermediate shell.

However, it's harder than that because we chdir into a tmpdir,
which defeats the libtool logic.  AFAICS, the only fix for this
is to determine the realbin path before we chdir, and then unfortunately
we need to change every use of `ostree` to `${OSTREE}` =(

Then this immediately breaks for me on RHEL7 because my ancient
copy of `valgrind-3.10.0-16.el7.x86_64` is unaware of syscall 306, i.e.
`syncfs`.

But let's do this first before I dive into that.

Closes: #292
Approved by: krnowak
2016-06-09 21:10:35 +00:00
Colin Walters
7fb49037ab tests/test-pull-c: New test that runs through the pull API via C
We have had in the past issues with running `ostree_repo_pull()`
multiple times in the same process, embarassingly enough.  Nothing in
the current test suite covers this, so let's start.

Closes: #322
Approved by: jlebon
2016-06-07 15:58:37 +00:00
Jonathan Lebon
b717fd2c18 ot-builtin-commit.c: add --skip-list option
This was already supported by the commit modifier API, just needed to
expose it. This will also be used to test the libarchive API in a future
test.

Closes: #275
Approved by: cgwalters
2016-05-06 14:44:55 +00:00
Colin Walters
a56ba6081a repo: Clean up staging directory for previous boot IDs
We had a policy of cleaning up all files in `$repo/tmp` older
than one day, but we should really clean up previous bootid staging
directories too, as they can potentially take up a lot of disk space.

https://bugzilla.gnome.org/show_bug.cgi?id=760531

Closes: #170
Approved by: jlebon
2016-05-02 18:44:44 +00:00
Krzesimir Nowak
c2931583d3 tests: Add a test for ostree pull --url
Closes: #271
Approved by: cgwalters
2016-04-22 14:27:06 +00:00
Colin Walters
e9a1809c4d Rename test-abi to test-symbols, start verifying symbols are documented
We keep forgetting to update `apidoc/ostree-sections.txt`, so let's
start enforcing it.  Of course it turns out we had some bugs here
like symbols marked as public but never implemented, etc.  Those
are fixed in the prior commits.

Closes: #263
Approved by: giuseppe
2016-04-17 13:52:07 +00:00
Alexander Larsson
6aa447a917 Add test case for inheriting remote options
Closes: #236
Approved by: cgwalters
2016-04-07 20:02:46 +00:00