Commit Graph

173 Commits

Author SHA1 Message Date
Colin Walters
390b781d3a pull: Add remotename:ref syntax
This is really the common case.
2014-01-21 08:57:34 +00:00
Colin Walters
3802a0679b tests/pull-corruption: Ensure we corrupt an object to be pulled
This test had some nondeterminism because we chose a random
object to corrupt, but because there were multiple commits, it
was possible that we chose an object that was not being pulled.

Fix this by writing some custom GJS code to find an explicitly random
object that exists in a given ref, an change a random byte offset.
This adds a lot more randomness to the testing too.
2014-01-19 13:19:10 -05:00
Colin Walters
0a9f246016 tests: Small tweaks to pull corruption test
This one is failing here, I suspect it's the generic pull race
condition, but these fixes should make it slightly more reliable.
2014-01-19 11:48:27 -05:00
Colin Walters
1a20ab4420 test-sysroot: Update for API change 2014-01-18 17:50:22 -05:00
Colin Walters
878a43411e admin/switch: New builtin to switch between trees
This is something I want to make easier, as it better showcases the
flexibility of OSTree.
2014-01-18 17:47:16 -05:00
Colin Walters
5034bf3a9d commit: Reject non-regular/non-symlinks earlier with better error message
Also avoid _NOT_SUPPORTED as that triggers the --help behavior from
the commandline; just use _FAILED.

https://bugzilla.gnome.org/show_bug.cgi?id=722410
2014-01-17 10:39:36 -05:00
Colin Walters
b2d0ba7ac1 deploy: Rework kernel arguments, add --karg-append to "admin deploy"
The "ordered hash" code was really just for kernel arguments.  And it
turns out it needs to be a multihash (for e.g. multiple console=
arguments).

So turn the OstreeOrderedHash into OstreeKernelArgs, and move the bits
to split key=value and such into there.

Now we're not making this public API yet - the public OstreeSysroot
just takes char **kargs.  To facilitate code reuse between ostree/ and
libostree/, make it a noinst libtool library.  It'll be duplicated in
the binary and library, but that's OK for now.  We can investigate
making OstreeKernelArgs public later.

https://bugzilla.gnome.org/show_bug.cgi?id=721136
2014-01-16 15:07:55 -05:00
Colin Walters
5bf6099a8b Unify uboot and syslinux test cases
The test-admin-deploy-1 was copied into -uboot at some point.  But
really they should be testing exactly the same thing, except for
the bootloader backend.

Unify these tests by extracting a common test core.
2014-01-15 09:48:04 -05:00
Colin Walters
108546c08c admin: Add --karg-proc-cmdline argument
When installing a new tree inside an existing OS, this is a convenient
way to include the command line arguments one needs (such as root=).
2014-01-15 09:19:32 -05:00
Colin Walters
54e58a51ca deploy: Write bootloader config even if just kernel arguments changed
The official way to add bootloader arguments to the current deployment
is to redeploy with --karg.  However, doing so tripped up an
optimization made inside the deployment code to just swap the
bootlinks if we're keeping the same "bootcsum".

Change this optimization to look at the pair of (bootcsum, options).
2014-01-15 09:19:32 -05:00
Colin Walters
091523a3bd sysroot: Error out on deploy --os=<unknown> which has not been initialized
We should hard require "ostree admin os-init foo" before letting
deployments go there; it's too easy to typo the argument.
2014-01-08 18:30:38 -05:00
Colin Walters
39e3c1bb8a ostree/remote-add: Add --no-gpg-verify option
To more conveniently disable GPG verification.
2013-12-25 14:17:36 -05:00
Roddy Shuler
fe5ed36461 core: Set mtime of content objects to 0
This is necessary to satisfy tools such as guile and python, which
compare mtimes to determine whether or not source files need to be
compiled.

https://bugzilla.gnome.org/show_bug.cgi?id=720363
2013-12-15 11:32:34 -05:00
Colin Walters
a24afd68f0 Move xattr handling into libgsystem, fix sysroot to handle directory ownership/perms
This large patch moves the core xattr logic down into libgsystem,
which allows the gs_shutil_cp_a() API to copy them.  In turn, this
allows us to just use that API instead of rolling our own recursive
copy here.

As noted in the new comment though, one case that we are explicitly
regressing is where the new /etc removes a parent directory that's
needed by a modified file.  This seems unlikely for most vendors now,
but let's do that as a separate bug.

https://bugzilla.gnome.org/show_bug.cgi?id=711058
2013-11-28 23:28:13 -05:00
Daniel Narvaez
03aa10f17d main/remote: Add a show-url operation to the remote command
Useful to get the remote url in scripts.

https://bugzilla.gnome.org/show_bug.cgi?id=710967
2013-10-28 09:14:23 -04:00
Colin Walters
c65923e642 Add OstreeAsyncProgress, use it for ostree_repo_pull
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications.  This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.

From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.

PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.

So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.

Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
2013-10-24 14:27:13 -04:00
Jeremy Whiting
f583c4ab0b core: Add size information to commit metadata
Add a --generate-sizes option to commit to add size information to the
commit metadata.  This will be used by higher level code which wants
to determine the total size necessary for downloading.
2013-10-19 11:56:51 -04:00
Colin Walters
b2a98f4ca3 sysroot: Handle deleting directories in /etc
We need to use the full shutil_rm_rf() in order to actually delete
complete directories.

Test suite code based on a patch from Sjoerd Simons <sjored@luon.net>

https://bugzilla.gnome.org/show_bug.cgi?id=710097
2013-10-15 14:56:43 -04:00
Colin Walters
3b700ccb50 core: Add code to read/write "varints"
Adapted from Google protobufs.  For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.

This will be used for static deltas and sizes metadata.
2013-10-11 12:21:37 -04:00
Colin Walters
af1c9b8721 tests: Extend test-sysroot.js further
Now covers a lot more of the API.
2013-10-04 10:52:05 -04:00
Colin Walters
07904c2457 sysroot: Support more arbitrary deployment changes
This commit changes the sysroot API so that one can create arbitrary
new deployment checkouts, then commit them as one step.  This is to
enable things like an automatic bisection tool which say create 50
deployments at once, then when done clean them up.

This also moves some printfs from the library into src/ostree.
2013-10-02 20:18:06 -04:00
Colin Walters
650aab7628 tests: Add a simple test-sysroot.js that covers OSTree.Sysroot
This will be more interesting as a test case user of the API.
2013-10-02 20:18:06 -04:00
Jeremy Whiting
7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Jeremy Whiting
7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters
0f486105db remote-add: Add --set=KEY=VALUE option
This can be used to add a remote and set e.g. tls-permissive=true, or
gpgverify=false.
2013-09-28 12:00:16 -04:00
Colin Walters
7d72a4b941 pull: Also fetch detached commitmeta files
These will contain GPG signatures and the like in the future, so we
should fetch them now.
2013-09-26 14:36:06 -04:00
Javier Martinez Canillas
4df1d0644f tests: Fix U-Boot test case
commit 08b873457 ("deploy: Remove checksum from generated loader entries")
changed the generated loader entries file names thus making the U-Boot test
case to fail since this test parsed those files.

Fix test-admin-deploy-uboot.sh by looking to the updated files names.

https://bugzilla.gnome.org/show_bug.cgi?id=708511

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-21 11:05:01 -04:00
Colin Walters
298625d7f8 deploy: Correctly swap bootloader version with new boot checksums
If we had two deployments with different boot checksums, and were
trying to remove the one that was the same and add a new one (the
normal case), we'd end up assuming due to comparison with 0 that
we only needed to do the fast subbootversion swap.

Fix this by actually putting 1 where we really mean 1.

And update the tests to verify the fix; I have double-verified by
undoing the fix, and noting that the test fails.

https://bugzilla.gnome.org/show_bug.cgi?id=708351
2013-09-20 11:21:08 -04:00
Colin Walters
08b8734576 deploy: Remove checksum from generated loader entries
The actual deployment checksum shouldn't be in there, because we may
just swap bootlinks, rendering the name of the old bootloader entry
file invalid.  Thankfully nothing actually parsed the names of these
files, so let's just use the index.
2013-09-19 22:41:33 -04:00
Colin Walters
d1bc9e2acf deploy: Some minor assertion and error message improvements
To help debug https://bugzilla.gnome.org/show_bug.cgi?id=708351

Plus run a status consistently in the tests.
2013-09-19 22:23:19 -04:00
Colin Walters
b4082d5642 test-core.js: Add standard copyright header
Like everything else.
2013-09-18 16:34:57 -04:00
Colin Walters
58a8d6d6ef tests: Add a gjs-based test
This covers introspection, and in general is a much better way to get
API coverage tests.
2013-09-18 12:02:12 -04:00
Javier Martinez Canillas
3715b0cc82 admin: add U-Boot backend test case
OSTree now supports a backend for the U-Boot bootloader,
add a test case for this.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Javier Martinez Canillas
ae61abb029 tests: allow setup_os_repository to create U-Boot stub config
Currently OSTree supports two bootloader backends: syslinux
and U-Boot; allow to create a stub configuration for both.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Javier Martinez Canillas
f19ba12360 tests: don't create a syslinux stub unconditionally in libtest
libtest.sh has an setup_os_repository() helper function tha is
used by many tests to setup an OSTree initial repository.

This function creates an syslinux configuration unconditionally
but OSTree supports other bootloader backends besides syslinux.

So, is better to conditionally create a syslinux configuration
only when it is needed.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Colin Walters
ac2d61dd51 core: Add detached metadata, readd metadata to commits
Previously I thought we'd have to ditch the current commit
format to avoid a{sv} due to

See https://bugzilla.gnome.org/show_bug.cgi?id=673012

But I realized that we don't really have to care about
unpacking/repacking commit objects, so let's just re-expose the
existing metadata a{sv} in commits in the API.

Also, add support for "detached" metadata that can be updated at any
time post-commit.  This is specifically designed for GPG signatures.

https://bugzilla.gnome.org/show_bug.cgi?id=707379
2013-09-09 17:01:32 -04:00
Colin Walters
b2625268aa tests: Drop kernel modules from checksums
We can actually share these across different kernels.  Thanks to
Tobias Hunger <tobias.hunger@gmail.com> for pointing this out.
2013-08-30 18:12:31 -04:00
Colin Walters
c9d7050d3d main: Support passing bare directory names to commit
It segfaulted before if you passed non-options, and a single directory
name is probably what people want.
2013-08-28 17:32:21 -04:00
Jeremy Whiting
499df2a90b pull: Add support for resuming downloads via range requests
Use a consistent temporary filename to download uri's.
Check for downloaded files before fetching from uri.
Download to hash.part file, then copy/move to hash.done when complete.
Add argument support to setup_fake_remote_repo1 function.
Add test for pull resume.
To implement this, pass --force-range-requests into the trivial-httpd,
which will only serve half of the objects to clients at a time.

https://bugzilla.gnome.org/show_bug.cgi?id=706344
2013-08-28 14:35:54 -04:00
Jeremy Whiting
902848cd71 test: Improve pull corruption test
To verify pull with a second repo works.
2013-08-27 20:35:45 -04:00
Colin Walters
8abad0b452 admin: Write out correct version fields in boot/loader/entries files
Before, we were writing the "bootversion", which is either 0 or 1, for
all entries.  This is completely wrong; the idea of the "version"
field is to compare between entries.

Fix this by writing out the inverted index - internally, index 0 is
the *first* boot entry, so we give it the highest version number, and
index N is the last, so give it version 0.

Then fix the deployment sorting code to correctly reverse the version
number comparison, so we read back the right order.

In practice before this bug didn't matter because "normally" you only
have at most two deployments.

https://bugzilla.gnome.org/show_bug.cgi?id=706546
2013-08-22 05:46:11 -04:00
Javier Martinez Canillas
2033edc658 test: fix a trivial typo in libtests.sh
When running the test-admin-deploy-1.sh unit test,
cat shows the following error:

cat: boot/vmlinuz-3-6.0: No such file or directory

due a trivial typo in the kernel image file name.

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>

https://bugzilla.gnome.org/show_bug.cgi?id=706371
2013-08-20 06:46:52 -04:00
Colin Walters
1ec7c30408 core: Remove old "archive" mode
We'll always have "bare" mode for keeping files-as-hardlinks as root.
But "archive" was my second attempt at a format for non-root file
storage, used by the gnome-ostree buildsystem which runs as non-root.

It was really handy to have a "tar" like mode where I can create
tarballs as a user, that contain files owned by root for example.

The "archive" mode stored content files as two pieces in the
filesystem; ".file" contained metadata, and ".filecontent" was the
actual content, uncompressed.  The nice thing about this was that to
check out a tree as non-root, you could just hardlink into the repo.

However, archive was fairly bad for serving via HTTP; it required
*two* HTTP requests per content object, greatly magnifing the already
inefficient fetch process.  So "archive-z2" was introduced.

To allow gnome-ostree to still check out trees as a user, the
"uncompressed-object-cache" was introduced, and that's how things have
been working for a while.

So we should just be able to kill this code.  Specifically note just
how much better the stage_object() function became.

https://bugzilla.gnome.org/show_bug.cgi?id=706057
2013-08-16 22:56:42 -04:00
Stef Walter
f9b2c45fc0 Add 'ostree reset' command to undo a bad commit
Accepts the following arguments: ref checksum

Checks that the checksum is a parent of the ref before rewriting
the ref.

https://bugzilla.gnome.org/show_bug.cgi?id=705979
2013-08-16 17:28:43 +02:00
Stef Walter
3989e0d397 Add 'ostree log' command
Follows the parent of commits showing each in turn until it reaches
the top of the commit tree.

https://bugzilla.gnome.org/show_bug.cgi?id=705973
2013-08-15 07:01:30 +02:00
Stef Walter
790132a81a Intelligible display for 'ostree show'
Show something similar to git metadata display. Show raw variant
data when --raw is specified

https://bugzilla.gnome.org/show_bug.cgi?id=705973
2013-08-15 06:48:25 +02:00
Colin Walters
f68ac018c2 test-xattrs: Skip if current FS doesn't support user_xattr
tmpfs doesn't =(
2013-08-14 20:46:46 +02:00
Stef Walter
1904e3e0f4 Don't pass non-checksums to ostree_repo_has_object()
The 'ostree show' command passed non-checksum arguments
to the ostree_repo_has_object() function which led to
an assertion.

https://bugzilla.gnome.org/show_bug.cgi?id=705967
2013-08-14 16:08:18 +02:00
Colin Walters
8c31e6fda7 tests: Add xattr test case
Since it was broken up until now for multiple attributes =(
2013-08-14 15:07:14 +02:00
Colin Walters
760b866104 main: Assume / for ls if no path given
Since it just is kind of a sane default.
2013-07-27 20:12:07 -04:00