164 Commits

Author SHA1 Message Date
Matthew Barnes
06818ceddc tests: Add test-remote-gpg-import.sh 2015-05-13 13:08:49 -04:00
Giuseppe Scrivano
ae672c3c9f core: new function _ostree_parse_delta_name
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-05-07 21:58:04 +02:00
Colin Walters
5becd5ccad Teach fsck about partial commits
An OSTree user noticed that `ostree fsck` would produce `missing
object` errors in the case of interrupted pulls.

It's possible to do e.g. `ostree pull --subpath=/usr/share/rpm ...`,
which gets you just that portion of the commit.  The use case for this
was being able to see what changes would appear in an update before
actually downloading all of it.

(I think this would be better covered by static deltas, but those
 aren't final yet, and `--subpath` predates it)

Further, `.commitpartial` is used as a successor to the `transaction`
symlink for more precise knowledge in the case where a pull was
interrupted that we needed to resume scanning.

So it makes sense for `ostree fsck` to be aware of it.
2015-05-06 08:07:20 -04:00
Colin Walters
9ef98fd05a sysroot: Add an API to lock
If a system administrator happens to type `ostree admin upgrade`
multiple times, currently that will lead to a potentially corrupted
system.

I originally attempted to do locking *internally* in `libostree`, but
that didn't work out because currently a number of the commands
perform multi-step operations that all need to be serialized.  All of
the current code in `ostree admin deploy` is an example.

Therefore, allow callers to perform locking, as most of the higher
level logic is presently implemented there.

At some point, we can revisit having internal locking, but it will be
difficult.  A more likely approach would be similar to Java's approach
with concurrency on iterators - a "fail fast" method.
2015-05-05 08:52:44 -04:00
Matthew Barnes
97379ec38c libotutil: Add ot_gpgme_ctx_tmp_home_dir()
Currently used for signature verification, will also be used for
importing GPG keys.
2015-05-01 10:21:40 -04:00
Colin Walters
ab15eafe56 reset: Don't enforce parent commits
First, git doesn't do this, and whatever Linus thinks is right or
something.

Second specifically to OSTree, it's quite common to not have
intermediate commits.  If one wants to reset a ref in order to prune
data after a deployment, the parentage check will fail.

Closes: https://github.com/GNOME/ostree/pull/87
2015-04-15 07:12:20 -04:00
Giuseppe Scrivano
9e6ac6d822 config: add new parameter "commit-update-summary" to core section
When set to true, the summary file is automatically updated after
a commit.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-14 17:46:53 +02:00
Daniel Svensson
f01fceb5d7 tests: Missing linker flags for test-rollsum
Reproducable on Debian; Fedora lets it slide.
2015-04-06 10:29:07 -04:00
Matthew Barnes
4a2733f9e7 gpg: Add OstreeGpgVerifyResult
Wrappers a referenced gpgme_verify_result_t so detailed verify results
can be examined independently of executing a verify operation.

_ostree_gpg_verifier_check_signature() now returns this object instead
of a single valid/invalid boolean, but the idea is for OstreeRepo to also
return this object for commit signature verification so it can be utilized
at the CLI layer (and possibly by other programs).
2015-03-18 11:52:22 -04:00
Giuseppe Scrivano
3f3096a65c tests: Add tests for test-ot-tool-util
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-06 18:45:38 +01:00
Giuseppe Scrivano
c1c34c601a tests: add test for test-ot-opt-utils.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-06 18:45:38 +01:00
Giuseppe Scrivano
50c69c3392 keyfile-utils: add tests
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-06 18:45:37 +01:00
Giuseppe Scrivano
d414ee5852 tests: add tests for mutable tree.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-04 17:52:17 -05:00
Giuseppe Scrivano
aeafdfff05 tests: Remove some duplications from Makefile-tests.am
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 18:32:24 -05:00
Giuseppe Scrivano
6029f4d820 tests: add test for bsdiff
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 12:16:17 -05:00
Giuseppe Scrivano
ef53e0100b tests: do not run tests/test-rollsum as part of make check
it is not really a unit test.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Conflicts:
	Makefile-tests.am
2015-02-21 17:19:20 -05:00
Colin Walters
c2d439f115 tests: Restore accidentally deleted Makefile bit
Not sure how this got deleted...
2015-02-16 10:29:14 -05:00
Colin Walters
9aa7e30b38 deltas: Implement rollsums
This does an rsync-style prepared delta basically.  On my test data,
it shaves ~6MB of uncompressed data.  Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
2015-02-16 10:10:35 -05:00
Giuseppe Scrivano
cfc344fbb0 tests: Add tests for ot-unix-utils
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-02 17:42:41 +01:00
Giuseppe Scrivano
53122dd2f9 tests: Move test-varint and test-rollsum under "make check"
"make check" tests are always enabled and they do not require to be
installed.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-02 17:14:52 +01:00
Giuseppe Scrivano
505ce19972 Do not interleave spaces and tabs
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-30 15:27:36 +01:00
Alexander Larsson
dbf717ac4b Add local-pull archive-z2 <=> bare-user roundtrip test
This creates a archive-z2 repo, pull-locals it to bare-user and then
again back to archive-z2 making sure things fsck along the way.
Then it checks out all repos and makes sure each one reproduces
the same result.

Unfortunately we can't install this as a real test because
it doesn't work in the test-runner because tmpfs doesn't support
user xattrs.

https://bugzilla.gnome.org/show_bug.cgi?id=741125
2014-12-08 12:03:27 +01:00
Alexander Larsson
fcd3caf6dd Add test-basic-user.sh testing for bare-user repos
This just does whatever test-basic.sh does, but on a bare-user
repo.

This works standalone, but unfortunately it breaks in
gnome-desktop-testing-runner as /tmp doesn't support
xattrs, so it is not installed atm.

https://bugzilla.gnome.org/show_bug.cgi?id=741125
2014-12-08 10:39:44 +01:00
Alexander Larsson
a3422791d4 Split out basic tests from test-basic.sh
This will let us reuse them with other repo types

https://bugzilla.gnome.org/show_bug.cgi?id=741125
2014-12-08 10:39:44 +01:00
Colin Walters
bcf40b4902 pull: Add depth support
For mirroring in particular, we really want to be able to traverse
all history.

$ ostree --repo=repo pull --mirror --depth=-1

https://bugzilla.gnome.org/show_bug.cgi?id=739240
2014-10-28 11:16:55 -04:00
Giuseppe Scrivano
7973a58607 checkout: permit checkout of a single file
fixes a coredump when using a command like:

$ ostree --repo=repo checkout -U --subpath=/usr/lib/passwd \
  fedora-atomic/rawhide/x86_64/docker-host usrlib-new

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2014-10-24 11:18:45 -04:00
Colin Walters
b3ad113f78 Add "unconfigured-state" concept to origin files
Some operating systems may come with external tools for subscription
management that drive access to the content.  In that case, the origin
file may not be useful (for example, it could refer to an installer
ISO).

This patch will allow OS installers to inject that state, with a
useful error message, directing the system administrator to an
external tool.

See: https://github.com/projectatomic/rpm-ostree/issues/31

https://bugzilla.gnome.org/show_bug.cgi?id=737686
2014-10-03 14:03:55 -04:00
Colin Walters
63abc1b513 pull: Support full recursive mirrors of repositories with summary file
Now that we have a summary file, we can use it to allow a simple:

ostree pull --mirror

To download the latest commit on every branch.  Also, for a case I'm
dealing with there's only one branch, but I don't want mirror users to
have to hardcode it.

https://bugzilla.gnome.org/show_bug.cgi?id=737807
2014-10-03 13:56:40 -04:00
Owen W. Taylor
c3f8019c19 Add test case for 'admin instutil set-kargs'
Test out the newly added options to 'instutil set-kargs' along with
the existing functionality.

https://bugzilla.gnome.org/show_bug.cgi?id=731051
2014-09-26 11:55:11 -04:00
Owen W. Taylor
7fce7e0338 Add test for the behavior of --help
Recursive over ostree and all subcommands, and check that --help
is supported, properly outputs to standard out, and exits
with a 0 exit status. Check that for commands with subcommands,
they produce the help output to standard error when run with no arguments.

https://bugzilla.gnome.org/show_bug.cgi?id=737194
2014-09-25 14:28:58 -04:00
Colin Walters
f8f5da219e Add repository "summary" file and metalink support
For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.

In order to make metalink work though, it needs *one* file which can
be checksummed.  (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).

This commit adds a new command:

$ ostree summary -u

To regenerate the summary file.  Can only be run by one process at a
time.

After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.

https://bugzilla.gnome.org/show_bug.cgi?id=729585
2014-09-03 13:21:52 -04:00
Anne LoVerso
3742c32945 repo-pull: Allow pulling only one directory
Changes the pull API to allow pulling only a single directory instead
of the whole deployment.  This option is utilized by the check-diff
option in rpm-ostree.

Add a new state directory to hold <checksum>.commitpartial files, so
we know that we've only downloaded partial state.
2014-08-20 15:09:32 -04:00
Colin Walters
47610b45c2 Limit metadata to 10 MiB
If fetching GPG-signed commits over plain HTTP, a MitM attacker can
fill up the drive of targets by simply returning an enormous stream
for the commit object.

Related to this, an attacker can also cause OSTree to perform large
memory allocations by returning enormous GVariants in the metadata.

This helps close that attack by limiting all metadata objects to 10
MiB, so the initial fetch will be truncated.

But now the attack is only slightly more difficult as the attacker
will have to return a correctly formed commit object, then return a
large stream of < 10 MiB dirmeta/dirtree objects.

https://bugzilla.gnome.org/show_bug.cgi?id=725921
2014-05-27 14:15:27 -04:00
Colin Walters
efd3a75daa Revert "Disable test-sysroot.test"
This reverts commit 73868a96d14fa13e69d9df778d2363b89f41fef5.
2014-03-19 09:15:38 -04:00
Vadim Rutkovsky
73868a96d1 Disable test-sysroot.test
It hangs on test-sysroot.js:40 during GLib.spawn_command_line_sync - it seems it can't
handle sh -c and the inner process becomes a zombie
2014-03-19 13:08:37 +01:00
Colin Walters
f2e0162846 upgrade: Refuse chronologically older commits unless --allow-downgrade
We don't want to allow MITM attackers to intercept upgrade requests
and provide clients with older OS versions vulnerable to security
flaws.

Only "ostree admin upgrade" gets this behavior for now - whether we
want to do it for "ostree admin switch" is another question.
2014-02-20 18:25:56 -05:00
Colin Walters
371cebf258 build: make "sudo make install" over existing install work 2014-02-13 10:57:51 -05:00
Colin Walters
26d5db696b tests: Fix up GPG tests for more strict EL7 GPG
These GPG tests were failing for me on EL7 - it appears to be because
we had only one directory for both private and public keys, and we
were giving that to ostree for verification, which passed them onto
gpgv.

In EL7 beta at least, gpgv now barfs if it finds a private key where
it is just expecting to find public keys.

Fix this by splitting out the public trusted directory from the
private key directory.  Except now for signing, we still need the
public key there, so symlink it.  Whee!
2014-02-10 09:40:35 -05:00
Colin Walters
2d6374822b Initial basic static delta code drop
This has a very basic level of functionality (deltas can be generated,
and applied offline).  There is only some stubbed out pull code to
fetch them via HTTP.

But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
2014-02-04 10:31:44 -05:00
Colin Walters
844c5ea652 core: Import bup's "rollsum" code, add a test case
For static deltas, one strategy that will be employed is to split each
object into chunks, and only include changed chunks in the deltas.
2014-02-04 10:12:56 -05:00
Colin Walters
9c701b6ae5 tests: Only install test-corruption if gjs is found
It now uses a gjs-based helper.
2014-01-21 19:53:06 +00:00
Colin Walters
3802a0679b tests/pull-corruption: Ensure we corrupt an object to be pulled
This test had some nondeterminism because we chose a random
object to corrupt, but because there were multiple commits, it
was possible that we chose an object that was not being pulled.

Fix this by writing some custom GJS code to find an explicitly random
object that exists in a given ref, an change a random byte offset.
This adds a lot more randomness to the testing too.
2014-01-19 13:19:10 -05:00
Colin Walters
878a43411e admin/switch: New builtin to switch between trees
This is something I want to make easier, as it better showcases the
flexibility of OSTree.
2014-01-18 17:47:16 -05:00
Colin Walters
5bf6099a8b Unify uboot and syslinux test cases
The test-admin-deploy-1 was copied into -uboot at some point.  But
really they should be testing exactly the same thing, except for
the bootloader backend.

Unify these tests by extracting a common test core.
2014-01-15 09:48:04 -05:00
Colin Walters
54e58a51ca deploy: Write bootloader config even if just kernel arguments changed
The official way to add bootloader arguments to the current deployment
is to redeploy with --karg.  However, doing so tripped up an
optimization made inside the deployment code to just swap the
bootlinks if we're keeping the same "bootcsum".

Change this optimization to look at the pair of (bootcsum, options).
2014-01-15 09:19:32 -05:00
Colin Walters
a24afd68f0 Move xattr handling into libgsystem, fix sysroot to handle directory ownership/perms
This large patch moves the core xattr logic down into libgsystem,
which allows the gs_shutil_cp_a() API to copy them.  In turn, this
allows us to just use that API instead of rolling our own recursive
copy here.

As noted in the new comment though, one case that we are explicitly
regressing is where the new /etc removes a parent directory that's
needed by a modified file.  This seems unlikely for most vendors now,
but let's do that as a separate bug.

https://bugzilla.gnome.org/show_bug.cgi?id=711058
2013-11-28 23:28:13 -05:00
Jeremy Whiting
f583c4ab0b core: Add size information to commit metadata
Add a --generate-sizes option to commit to add size information to the
commit metadata.  This will be used by higher level code which wants
to determine the total size necessary for downloading.
2013-10-19 11:56:51 -04:00
Colin Walters
6500026ba7 trivial: Add missing files to dist 2013-10-15 16:39:22 -04:00
Colin Walters
3b700ccb50 core: Add code to read/write "varints"
Adapted from Google protobufs.  For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.

This will be used for static deltas and sizes metadata.
2013-10-11 12:21:37 -04:00
Colin Walters
650aab7628 tests: Add a simple test-sysroot.js that covers OSTree.Sysroot
This will be more interesting as a test case user of the API.
2013-10-02 20:18:06 -04:00