414 Commits

Author SHA1 Message Date
Colin Walters
54e58a51ca deploy: Write bootloader config even if just kernel arguments changed
The official way to add bootloader arguments to the current deployment
is to redeploy with --karg.  However, doing so tripped up an
optimization made inside the deployment code to just swap the
bootlinks if we're keeping the same "bootcsum".

Change this optimization to look at the pair of (bootcsum, options).
2014-01-15 09:19:32 -05:00
Colin Walters
091523a3bd sysroot: Error out on deploy --os=<unknown> which has not been initialized
We should hard require "ostree admin os-init foo" before letting
deployments go there; it's too easy to typo the argument.
2014-01-08 18:30:38 -05:00
Colin Walters
39e3c1bb8a ostree/remote-add: Add --no-gpg-verify option
To more conveniently disable GPG verification.
2013-12-25 14:17:36 -05:00
Roddy Shuler
fe5ed36461 core: Set mtime of content objects to 0
This is necessary to satisfy tools such as guile and python, which
compare mtimes to determine whether or not source files need to be
compiled.

https://bugzilla.gnome.org/show_bug.cgi?id=720363
2013-12-15 11:32:34 -05:00
Colin Walters
a24afd68f0 Move xattr handling into libgsystem, fix sysroot to handle directory ownership/perms
This large patch moves the core xattr logic down into libgsystem,
which allows the gs_shutil_cp_a() API to copy them.  In turn, this
allows us to just use that API instead of rolling our own recursive
copy here.

As noted in the new comment though, one case that we are explicitly
regressing is where the new /etc removes a parent directory that's
needed by a modified file.  This seems unlikely for most vendors now,
but let's do that as a separate bug.

https://bugzilla.gnome.org/show_bug.cgi?id=711058
2013-11-28 23:28:13 -05:00
Daniel Narvaez
03aa10f17d main/remote: Add a show-url operation to the remote command
Useful to get the remote url in scripts.

https://bugzilla.gnome.org/show_bug.cgi?id=710967
2013-10-28 09:14:23 -04:00
Colin Walters
c65923e642 Add OstreeAsyncProgress, use it for ostree_repo_pull
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications.  This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.

From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.

PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.

So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.

Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
2013-10-24 14:27:13 -04:00
Jeremy Whiting
f583c4ab0b core: Add size information to commit metadata
Add a --generate-sizes option to commit to add size information to the
commit metadata.  This will be used by higher level code which wants
to determine the total size necessary for downloading.
2013-10-19 11:56:51 -04:00
Colin Walters
b2a98f4ca3 sysroot: Handle deleting directories in /etc
We need to use the full shutil_rm_rf() in order to actually delete
complete directories.

Test suite code based on a patch from Sjoerd Simons <sjored@luon.net>

https://bugzilla.gnome.org/show_bug.cgi?id=710097
2013-10-15 14:56:43 -04:00
Colin Walters
3b700ccb50 core: Add code to read/write "varints"
Adapted from Google protobufs.  For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.

This will be used for static deltas and sizes metadata.
2013-10-11 12:21:37 -04:00
Colin Walters
af1c9b8721 tests: Extend test-sysroot.js further
Now covers a lot more of the API.
2013-10-04 10:52:05 -04:00
Colin Walters
07904c2457 sysroot: Support more arbitrary deployment changes
This commit changes the sysroot API so that one can create arbitrary
new deployment checkouts, then commit them as one step.  This is to
enable things like an automatic bisection tool which say create 50
deployments at once, then when done clean them up.

This also moves some printfs from the library into src/ostree.
2013-10-02 20:18:06 -04:00
Colin Walters
650aab7628 tests: Add a simple test-sysroot.js that covers OSTree.Sysroot
This will be more interesting as a test case user of the API.
2013-10-02 20:18:06 -04:00
Jeremy Whiting
7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Jeremy Whiting
7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters
0f486105db remote-add: Add --set=KEY=VALUE option
This can be used to add a remote and set e.g. tls-permissive=true, or
gpgverify=false.
2013-09-28 12:00:16 -04:00
Colin Walters
7d72a4b941 pull: Also fetch detached commitmeta files
These will contain GPG signatures and the like in the future, so we
should fetch them now.
2013-09-26 14:36:06 -04:00
Javier Martinez Canillas
4df1d0644f tests: Fix U-Boot test case
commit 08b873457 ("deploy: Remove checksum from generated loader entries")
changed the generated loader entries file names thus making the U-Boot test
case to fail since this test parsed those files.

Fix test-admin-deploy-uboot.sh by looking to the updated files names.

https://bugzilla.gnome.org/show_bug.cgi?id=708511

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-21 11:05:01 -04:00
Colin Walters
298625d7f8 deploy: Correctly swap bootloader version with new boot checksums
If we had two deployments with different boot checksums, and were
trying to remove the one that was the same and add a new one (the
normal case), we'd end up assuming due to comparison with 0 that
we only needed to do the fast subbootversion swap.

Fix this by actually putting 1 where we really mean 1.

And update the tests to verify the fix; I have double-verified by
undoing the fix, and noting that the test fails.

https://bugzilla.gnome.org/show_bug.cgi?id=708351
2013-09-20 11:21:08 -04:00
Colin Walters
08b8734576 deploy: Remove checksum from generated loader entries
The actual deployment checksum shouldn't be in there, because we may
just swap bootlinks, rendering the name of the old bootloader entry
file invalid.  Thankfully nothing actually parsed the names of these
files, so let's just use the index.
2013-09-19 22:41:33 -04:00
Colin Walters
d1bc9e2acf deploy: Some minor assertion and error message improvements
To help debug https://bugzilla.gnome.org/show_bug.cgi?id=708351

Plus run a status consistently in the tests.
2013-09-19 22:23:19 -04:00
Colin Walters
b4082d5642 test-core.js: Add standard copyright header
Like everything else.
2013-09-18 16:34:57 -04:00
Colin Walters
58a8d6d6ef tests: Add a gjs-based test
This covers introspection, and in general is a much better way to get
API coverage tests.
2013-09-18 12:02:12 -04:00
Javier Martinez Canillas
3715b0cc82 admin: add U-Boot backend test case
OSTree now supports a backend for the U-Boot bootloader,
add a test case for this.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Javier Martinez Canillas
ae61abb029 tests: allow setup_os_repository to create U-Boot stub config
Currently OSTree supports two bootloader backends: syslinux
and U-Boot; allow to create a stub configuration for both.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Javier Martinez Canillas
f19ba12360 tests: don't create a syslinux stub unconditionally in libtest
libtest.sh has an setup_os_repository() helper function tha is
used by many tests to setup an OSTree initial repository.

This function creates an syslinux configuration unconditionally
but OSTree supports other bootloader backends besides syslinux.

So, is better to conditionally create a syslinux configuration
only when it is needed.

https://bugzilla.gnome.org/show_bug.cgi?id=708069

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-15 12:46:39 -04:00
Colin Walters
ac2d61dd51 core: Add detached metadata, readd metadata to commits
Previously I thought we'd have to ditch the current commit
format to avoid a{sv} due to

See https://bugzilla.gnome.org/show_bug.cgi?id=673012

But I realized that we don't really have to care about
unpacking/repacking commit objects, so let's just re-expose the
existing metadata a{sv} in commits in the API.

Also, add support for "detached" metadata that can be updated at any
time post-commit.  This is specifically designed for GPG signatures.

https://bugzilla.gnome.org/show_bug.cgi?id=707379
2013-09-09 17:01:32 -04:00
Colin Walters
b2625268aa tests: Drop kernel modules from checksums
We can actually share these across different kernels.  Thanks to
Tobias Hunger <tobias.hunger@gmail.com> for pointing this out.
2013-08-30 18:12:31 -04:00
Colin Walters
c9d7050d3d main: Support passing bare directory names to commit
It segfaulted before if you passed non-options, and a single directory
name is probably what people want.
2013-08-28 17:32:21 -04:00
Jeremy Whiting
499df2a90b pull: Add support for resuming downloads via range requests
Use a consistent temporary filename to download uri's.
Check for downloaded files before fetching from uri.
Download to hash.part file, then copy/move to hash.done when complete.
Add argument support to setup_fake_remote_repo1 function.
Add test for pull resume.
To implement this, pass --force-range-requests into the trivial-httpd,
which will only serve half of the objects to clients at a time.

https://bugzilla.gnome.org/show_bug.cgi?id=706344
2013-08-28 14:35:54 -04:00
Jeremy Whiting
902848cd71 test: Improve pull corruption test
To verify pull with a second repo works.
2013-08-27 20:35:45 -04:00
Colin Walters
8abad0b452 admin: Write out correct version fields in boot/loader/entries files
Before, we were writing the "bootversion", which is either 0 or 1, for
all entries.  This is completely wrong; the idea of the "version"
field is to compare between entries.

Fix this by writing out the inverted index - internally, index 0 is
the *first* boot entry, so we give it the highest version number, and
index N is the last, so give it version 0.

Then fix the deployment sorting code to correctly reverse the version
number comparison, so we read back the right order.

In practice before this bug didn't matter because "normally" you only
have at most two deployments.

https://bugzilla.gnome.org/show_bug.cgi?id=706546
2013-08-22 05:46:11 -04:00
Javier Martinez Canillas
2033edc658 test: fix a trivial typo in libtests.sh
When running the test-admin-deploy-1.sh unit test,
cat shows the following error:

cat: boot/vmlinuz-3-6.0: No such file or directory

due a trivial typo in the kernel image file name.

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>

https://bugzilla.gnome.org/show_bug.cgi?id=706371
2013-08-20 06:46:52 -04:00
Colin Walters
1ec7c30408 core: Remove old "archive" mode
We'll always have "bare" mode for keeping files-as-hardlinks as root.
But "archive" was my second attempt at a format for non-root file
storage, used by the gnome-ostree buildsystem which runs as non-root.

It was really handy to have a "tar" like mode where I can create
tarballs as a user, that contain files owned by root for example.

The "archive" mode stored content files as two pieces in the
filesystem; ".file" contained metadata, and ".filecontent" was the
actual content, uncompressed.  The nice thing about this was that to
check out a tree as non-root, you could just hardlink into the repo.

However, archive was fairly bad for serving via HTTP; it required
*two* HTTP requests per content object, greatly magnifing the already
inefficient fetch process.  So "archive-z2" was introduced.

To allow gnome-ostree to still check out trees as a user, the
"uncompressed-object-cache" was introduced, and that's how things have
been working for a while.

So we should just be able to kill this code.  Specifically note just
how much better the stage_object() function became.

https://bugzilla.gnome.org/show_bug.cgi?id=706057
2013-08-16 22:56:42 -04:00
Stef Walter
f9b2c45fc0 Add 'ostree reset' command to undo a bad commit
Accepts the following arguments: ref checksum

Checks that the checksum is a parent of the ref before rewriting
the ref.

https://bugzilla.gnome.org/show_bug.cgi?id=705979
2013-08-16 17:28:43 +02:00
Stef Walter
3989e0d397 Add 'ostree log' command
Follows the parent of commits showing each in turn until it reaches
the top of the commit tree.

https://bugzilla.gnome.org/show_bug.cgi?id=705973
2013-08-15 07:01:30 +02:00
Stef Walter
790132a81a Intelligible display for 'ostree show'
Show something similar to git metadata display. Show raw variant
data when --raw is specified

https://bugzilla.gnome.org/show_bug.cgi?id=705973
2013-08-15 06:48:25 +02:00
Colin Walters
f68ac018c2 test-xattrs: Skip if current FS doesn't support user_xattr
tmpfs doesn't =(
2013-08-14 20:46:46 +02:00
Stef Walter
1904e3e0f4 Don't pass non-checksums to ostree_repo_has_object()
The 'ostree show' command passed non-checksum arguments
to the ostree_repo_has_object() function which led to
an assertion.

https://bugzilla.gnome.org/show_bug.cgi?id=705967
2013-08-14 16:08:18 +02:00
Colin Walters
8c31e6fda7 tests: Add xattr test case
Since it was broken up until now for multiple attributes =(
2013-08-14 15:07:14 +02:00
Colin Walters
760b866104 main: Assume / for ls if no path given
Since it just is kind of a sane default.
2013-07-27 20:12:07 -04:00
Colin Walters
187c75eae5 main: Support using repo from current directory
$ cd repo
$ ostree ls foo /
...

Can be a lot more convenient than typing --repo=repo a lot.
2013-07-24 14:05:12 -04:00
Colin Walters
c9b61cbfee Drop support for related objects and metadata in commit objects
While the actual commit object format is presently the same, for a
number of reasons we'd like to change it fairly radically.  Among
other things, we need to drop our a{sv} types in objects, to protect
against GVariant changing format.

Since now gnome-ostree now longer uses related objects, and nothing
ever used metadata, just drop them both.
2013-07-24 12:59:27 -04:00
Colin Walters
3b9da094d8 main: Drop log builtin
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.

More interesting things are diffs between commits, object statistics,
etc.
2013-07-23 18:19:14 -04:00
Colin Walters
3d7bff2d41 admin: Add an "undeploy" command
Otherwise it's really easy to keep accumulating deployments.  Also, we
may want to run this after rebooting, so we're back down to one
operating system.
2013-07-23 09:19:24 -04:00
Colin Walters
67823beb1f core: Don't strip setuid bits when creating files
This ugly regression occurred because I overlooked the fact that our
chown() invocation would strip off setuid.
2013-07-18 14:23:55 -04:00
Colin Walters
f0c5a5f334 main: Add --delete option to fsck
This is useful for clearing out all corrupted objects locally.
2013-07-18 12:09:44 -04:00
Colin Walters
a40b09310c tests: Add a test for link checkout speedup 2013-07-17 19:47:40 -04:00
Colin Walters
62a896350b Drop support for fifos and devices
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.

Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.

Less complexity and code here if we just support directories, regular
files, and symbolic links.
2013-07-16 09:35:44 -04:00
Colin Walters
fb93b95807 admin: Rename prune -> cleanup, avoid doing repo prune twice
Calling it "cleanup" is better since it does more than repo pruning.

We were also doing a prune twice; ot_admin_cleanup() already does one,
so drop the bits to do it in cleanup.c.
2013-07-15 16:13:12 -04:00