414 Commits

Author SHA1 Message Date
Matthew Barnes
4f33515316 Juggling libglnx.h includes 2015-05-06 21:50:06 -04:00
Colin Walters
5becd5ccad Teach fsck about partial commits
An OSTree user noticed that `ostree fsck` would produce `missing
object` errors in the case of interrupted pulls.

It's possible to do e.g. `ostree pull --subpath=/usr/share/rpm ...`,
which gets you just that portion of the commit.  The use case for this
was being able to see what changes would appear in an update before
actually downloading all of it.

(I think this would be better covered by static deltas, but those
 aren't final yet, and `--subpath` predates it)

Further, `.commitpartial` is used as a successor to the `transaction`
symlink for more precise knowledge in the case where a pull was
interrupted that we needed to resume scanning.

So it makes sense for `ostree fsck` to be aware of it.
2015-05-06 08:07:20 -04:00
Colin Walters
9ef98fd05a sysroot: Add an API to lock
If a system administrator happens to type `ostree admin upgrade`
multiple times, currently that will lead to a potentially corrupted
system.

I originally attempted to do locking *internally* in `libostree`, but
that didn't work out because currently a number of the commands
perform multi-step operations that all need to be serialized.  All of
the current code in `ostree admin deploy` is an example.

Therefore, allow callers to perform locking, as most of the higher
level logic is presently implemented there.

At some point, we can revisit having internal locking, but it will be
difficult.  A more likely approach would be similar to Java's approach
with concurrency on iterators - a "fail fast" method.
2015-05-05 08:52:44 -04:00
Giuseppe Scrivano
cd93780d97 show: add option --gpg-homedir
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-27 15:22:41 +02:00
Giuseppe Scrivano
8ab2e60edc test-auto-summary.sh properly quote arguments to assert_streq
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-24 15:46:28 +02:00
Matthew Barnes
9c449624f2 pull: Always request detached metadata for commits
Always request detached metadata for commit objects, even if we already
have the commit object.  This ensures we fetch any post facto detached
metadata updates such as new GPG signatures.

https://bugzilla.gnome.org/748220
2015-04-23 19:57:10 -04:00
Colin Walters
ab15eafe56 reset: Don't enforce parent commits
First, git doesn't do this, and whatever Linus thinks is right or
something.

Second specifically to OSTree, it's quite common to not have
intermediate commits.  If one wants to reset a ref in order to prune
data after a deployment, the parentage check will fail.

Closes: https://github.com/GNOME/ostree/pull/87
2015-04-15 07:12:20 -04:00
Giuseppe Scrivano
9e6ac6d822 config: add new parameter "commit-update-summary" to core section
When set to true, the summary file is automatically updated after
a commit.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-14 17:46:53 +02:00
Matthew Barnes
a1352fec9d tests: Fix root uid check in test-commit-sign.sh 2015-04-14 08:53:48 -04:00
Colin Walters
54f848a78c pull-metalink: Don't print error output when we expect failure
I think I added `cat err.txt` to debug, but it's not necessary now.
2015-04-10 17:51:13 -04:00
Colin Walters
c2aabcac3b ostree_repo_checkout_tree_at: New API for checkouts
rpm-ostree currently uses ostree_repo_checkout_tree(), which as a side
effect will use the uncompressed objects cache by default.  This is
rather annoying if you're using rpm-ostree on a server-side
repository, because if you then rsync the repo, you'll be syncing out
the uncompressed objects unless you exclude them.

We added the ability to disable the uncompressed cache in the
repository config to fix this, but it's better to allow application
control over this.  The uncompressed cache will in some future version
become opt in as well.

This new API further:
 - Drops the `GFile` usage in favor of `openat` APIs
 - Improves ergonomics by avoiding callers having to query the source
   `GFileInfo` (and carry around a copy of `OSTREE_GIO_FAST_QUERYINFO`)
 - Has a more extensible options structure

Per the comment, I rather crudely have the `ostree checkout` builtin
call both APIs to ensure some testing coverage.

However, I'd like to in the future have easier-to-set-up testing code
that calls `libtest.sh` to set up dummy data.
2015-04-07 15:12:16 -04:00
Colin Walters
1e501422e2 tests: Verify that the pull error was from interruption
While working on https://github.com/GNOME/ostree/pull/84 I wanted to
verify that nothing else was going wrong.
2015-04-06 14:33:16 -04:00
Giuseppe Scrivano
d3545b0661 tests: skip test-commit-sign.sh when not root
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-03 09:57:20 +02:00
Giuseppe Scrivano
12bc9a336a tests/basic-test.sh: enable repo-noperm test only for non-root user
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-03 09:57:20 +02:00
Sam Thursfield
d667ebe156 core: Fix possible crash in ostree_mutable_tree_walk()
If the starting index is beyond the end of the list, it's a programming
error. Previously, the code was trying to raise a runtime error, but
actually causing a segfault.

This was detected by test code in test-mutable-tree.c, which is removed
in this commit because it should now not be possible to crash here.

https://bugzilla.gnome.org/747032
2015-03-31 13:04:31 -04:00
Matthew Barnes
6b16aba5f9 tests: Add a test case for unwritable repos 2015-03-25 17:24:06 -04:00
Matthew Barnes
da918a74d5 gpg: Regenerate test data for test-gpg-verify-result
Turns out the expired signature case was failing because the signature
itself was corrupted.  Reconstructed the test data and updated the test
code.  Note, an expired signature is still counted as valid.

Also, handy debugging trick for setting a key or signature expiry: the
CLI makes it appear the shortest expiry is 1 day, but it also secretly
recognizes "seconds=N".
2015-03-19 12:43:04 -04:00
Matthew Barnes
9b8bad3c2e tests: Update test-gpg-signed-commit.sh
Utilize and test new CLI capabilities:

  - Signature count in 'ostree show' result
  - Duplicate signatures now rejected
  - Ability to delete signatures
2015-03-18 11:53:26 -04:00
Matthew Barnes
4a2733f9e7 gpg: Add OstreeGpgVerifyResult
Wrappers a referenced gpgme_verify_result_t so detailed verify results
can be examined independently of executing a verify operation.

_ostree_gpg_verifier_check_signature() now returns this object instead
of a single valid/invalid boolean, but the idea is for OstreeRepo to also
return this object for commit signature verification so it can be utilized
at the CLI layer (and possibly by other programs).
2015-03-18 11:52:22 -04:00
Giuseppe Scrivano
3f3096a65c tests: Add tests for test-ot-tool-util
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-06 18:45:38 +01:00
Giuseppe Scrivano
c1c34c601a tests: add test for test-ot-opt-utils.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-06 18:45:38 +01:00
Giuseppe Scrivano
50c69c3392 keyfile-utils: add tests
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-06 18:45:37 +01:00
Giuseppe Scrivano
d414ee5852 tests: add tests for mutable tree.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-04 17:52:17 -05:00
Giuseppe Scrivano
aea173b3f8 tests: enforce ${CMD_PREFIX} on all ostree processes
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 18:32:24 -05:00
Giuseppe Scrivano
6029f4d820 tests: add test for bsdiff
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 12:16:17 -05:00
Matthew Barnes
a5b002dae6 ostree: Add gpg-sign command
Signs a commit with one or more GPG keys.
2015-02-26 12:53:01 -05:00
Colin Walters
2f205331f4 tests: Move test gpg keyring into writable tmpdir
At least the version of gpgme in RHEL7 wants to create a lock file,
work around this by copying it into the writable test temporary
directory.
2015-02-23 15:21:23 -05:00
Daniel Drake
7c267d6089 libostree: set directory mtimes to 0 on checkout
We already set all file mtimes to 0 so that they are constant
over all checkouts, and can be made constant with a known value from
the system where the ostree was created.

However, this was not happening for directories. Zero their mtimes too.

This is important for shipping a fontconfig cache in the ostree;
the fontconfig cache files embed a directory mtime.
2015-02-18 18:27:18 -05:00
Colin Walters
08476ce254 deltas: Prune deltas when the corresponding "to" commit vanishes
We want prune to actually give you back disk space when using deltas.
2015-02-16 10:10:35 -05:00
Colin Walters
dbad2f6e19 deltas: Make syntax-check happy 2015-02-16 10:10:35 -05:00
Colin Walters
9aa7e30b38 deltas: Implement rollsums
This does an rsync-style prepared delta basically.  On my test data,
it shaves ~6MB of uncompressed data.  Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
2015-02-16 10:10:35 -05:00
Colin Walters
92cc3b5968 deltas: Use base64 for csums, add version to parts 2015-02-16 10:10:35 -05:00
Colin Walters
ab3bf493dd pull: Optimize file:/// URIs to skip libsoup and hardlink if possible
It's always been suboptimal to have both pull and pull-local; as we go
beyond the raw object data into things like deltas and summary files,
the logic to perform e.g. mirroring should only be in one place.

This will be used by Pulp's OSTree content plugin at least to perform
promotions.
2015-02-06 10:02:57 -05:00
Colin Walters
1fdecbd263 pull: Copy the upstream summary file when doing a pull --mirror
While it could be regenerated downstream, there might be other
metadata upstream, and the goal here is a mirror.

https://bugzilla.gnome.org/show_bug.cgi?id=739377
2015-02-05 21:24:21 -05:00
Colin Walters
fab1e113db When mirroring, write content directly, do not verify
When doing a pull --mirror from an archive-z2 repository into another
archive-z2 repository, currently we gunzip/checksum/gzip each content
object.  The re-gzip process in particular is fairly expensive.

This does assume that the upstream content is trusted and correct.
It'd be nice in the future to do at least a CRC check, if not the full
checksum.  (Could we append CRC data to the end of filez objects?)

We could also choose to only do this optimization if fetching over
TLS.

before: 1626 metadata, 20320 content objects fetched; 299634 KiB transferred in 62 seconds
after : 1626 metadata, 20320 content objects fetched; 299634 KiB transferred in 11 seconds
2015-02-05 21:24:21 -05:00
Giuseppe Scrivano
cfc344fbb0 tests: Add tests for ot-unix-utils
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-02 17:42:41 +01:00
Giuseppe Scrivano
6a3959c895 syntax-check: Remove empty lines at the end of file
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-02 15:07:56 +01:00
Giuseppe Scrivano
10642cd732 Replace "==" with "=" in shell script test
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-30 15:27:36 +01:00
Colin Walters
6bbfa5f85a admin: Add set-origin command
See projectatomic/rpm-ostree#42 for rationale. There are two high
level use cases:

 - If the OS comes unconfigured, this is a way to point it at a repo of your choice.
 - To switch between repositories while keeping the same branch easily.
2015-01-19 13:55:20 -05:00
Giuseppe Scrivano
f2ae405f9f static-delta: do not accept both --from=REV and --empty for generate
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-15 22:54:56 -05:00
Alexander Larsson
5b721a5b08 Allow creating and pulling static deltas starting from "empty"
You create these with something like:
  ostree static-delta generate --empty --to=master

These will be automatically used during pull if no previous revision
exists in the target repo.

These work very much like the normal static deltas except they
are named just by the "to" revision. I.e:

deltas/94/f7d2dc23759dd21f9bd01e6705a8fdf98f90cad3e0109ba3f6c091c1a3774d

for a from-scratch to 94f7d2dc23759dd21f9bd01e6705a8fdf98f90cad3e0109ba3f6c091c1a3774d delta.

https://bugzilla.gnome.org/show_bug.cgi?id=721799
2015-01-14 14:43:32 +01:00
Matthew Barnes
7727fe84d9 Require a PREFIX when deleting refs
Also fix the "ostree refs" help output to not give the impression that
the --delete option takes its own PREFIX argument.

https://bugzilla.gnome.org/show_bug.cgi?id=742454
2015-01-06 10:33:12 -05:00
Giuseppe Scrivano
ed2b56a430 fix --help for static-delta
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2014-12-18 12:48:47 +01:00
Colin Walters
5c47389243 test-rollsum: Process all input, print more statistics
Copying the bup code, we need to loop over all chunks even after
hitting the rollsum returning 0.

Also print more statistics.
2014-12-18 12:48:47 +01:00
Colin Walters
ca678224be Static deltas support
https://bugzilla.gnome.org/show_bug.cgi?id=721799
2014-12-18 12:48:47 +01:00
Matthew Barnes
e54d48be39 Add "ostree remote list" command
Lists available remote names.
2014-12-17 16:05:53 -05:00
Colin Walters
d3edda5edc basic-test: Fixup mtime check for bare-user
https://bugzilla.gnome.org/show_bug.cgi?id=741662
2014-12-17 11:34:10 -05:00
Colin Walters
9dadebb501 tests: Fix two bugs in tests revealed by new remote changes 2014-12-17 10:43:01 -05:00
Colin Walters
f6a6e68412 Add more flexible _remote_change() API , expose via 'ostree remote'
For Anaconda, I needed OSTREE_REPO_REMOTE_CHANGE_ADD_IF_NOT_EXISTS,
with the GFile *sysroot argument to avoid ugly hacks.  We want to
write the content provided via "ostreesetup" as a remote to the target
chroot only in the case where it isn't provided as part of the tree
content itself.

This is also potentially useful in idempotent systems management tools
like Ansible.

https://bugzilla.gnome.org/show_bug.cgi?id=741577
2014-12-15 21:28:09 -05:00
Alexander Larsson
e908ebd9b3 test-local-pull: Sort find output to make test robust
There is no guarantee that find will produce output in the same
order, so we need to sort the output to ensure we always
get the same output.

https://bugzilla.gnome.org/show_bug.cgi?id=741125
2014-12-08 20:28:09 +01:00