1470 Commits

Author SHA1 Message Date
Colin Walters
c65923e642 Add OstreeAsyncProgress, use it for ostree_repo_pull
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications.  This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.

From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.

PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.

So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.

Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
2013-10-24 14:27:13 -04:00
Colin Walters
9aaa29598e Update libgsystem
Just to test the new code.
2013-10-23 15:43:29 -04:00
Jeremy Whiting
f583c4ab0b core: Add size information to commit metadata
Add a --generate-sizes option to commit to add size information to the
commit metadata.  This will be used by higher level code which wants
to determine the total size necessary for downloading.
2013-10-19 11:56:51 -04:00
Colin Walters
b35d1499b8 Release 2013.7 v2013.7 2013-10-15 16:44:35 -04:00
Colin Walters
7c903eb889 repo: Remove set-but-unused variable
The gpgme examples use this, but from what I can tell we don't really
need to because we don't need detailed results; we only care whether
we signed it at all.
2013-10-15 16:44:19 -04:00
Colin Walters
6500026ba7 trivial: Add missing files to dist 2013-10-15 16:39:22 -04:00
Colin Walters
4e5b31b8fc trivial: Don't include config.h in headers
This should be included by each .c file.  This fixes using libostree
from a "plain" project without config.h.
2013-10-15 15:26:37 -04:00
Colin Walters
b2a98f4ca3 sysroot: Handle deleting directories in /etc
We need to use the full shutil_rm_rf() in order to actually delete
complete directories.

Test suite code based on a patch from Sjoerd Simons <sjored@luon.net>

https://bugzilla.gnome.org/show_bug.cgi?id=710097
2013-10-15 14:56:43 -04:00
Colin Walters
3b700ccb50 core: Add code to read/write "varints"
Adapted from Google protobufs.  For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.

This will be used for static deltas and sizes metadata.
2013-10-11 12:21:37 -04:00
Colin Walters
2b37abf7b6 build: Also make gjs-based tests conditional on introspection support
Since they use imports.gi.OSTree.
2013-10-11 12:04:24 -04:00
Jasper St. Pierre
3b31f2b4d1 Update libgsystem 2013-10-09 16:17:38 -04:00
Colin Walters
c62141004a libostree: Fix an introspection scanner warning 2013-10-09 12:53:12 -04:00
Colin Walters
2708124190 repo: Tweak traversal API
It's convenient for bindings if we have a version that doesn't mutate
the hash table, because they pass temporary hash tables as input.
2013-10-09 12:05:56 -04:00
Colin Walters
af1c9b8721 tests: Extend test-sysroot.js further
Now covers a lot more of the API.
2013-10-04 10:52:05 -04:00
Colin Walters
cb251ae5ca sysroot: Move ostree_sysroot_origin_new_from_refspec here
Rather than having it live in admin.  This is useful for other
consumers like the test suite.
2013-10-03 18:34:24 -04:00
Colin Walters
ae2234b183 sysroot: Ensure we create /boot/loader.%d/entries even if there are no deployments
Not doing so breaks things, and we should support this.
2013-10-03 18:34:24 -04:00
Colin Walters
1641ade728 sysroot: Avoid bad double-free
If a deployment is somehow in the list twice, the hash table will free
the *new* value with g_hash_table_insert which gets all broken.  Just
use g_hash_table_replace().
2013-10-03 18:34:24 -04:00
Colin Walters
2b2c6ead22 sysroot: Update some annotations
We were double-freeing before.
2013-10-03 18:34:04 -04:00
Colin Walters
07904c2457 sysroot: Support more arbitrary deployment changes
This commit changes the sysroot API so that one can create arbitrary
new deployment checkouts, then commit them as one step.  This is to
enable things like an automatic bisection tool which say create 50
deployments at once, then when done clean them up.

This also moves some printfs from the library into src/ostree.
2013-10-02 20:18:06 -04:00
Colin Walters
650aab7628 tests: Add a simple test-sysroot.js that covers OSTree.Sysroot
This will be more interesting as a test case user of the API.
2013-10-02 20:18:06 -04:00
Colin Walters
f9379b0ce3 repo: Only delete temp files older than a day
This is somewhat lame, but to do better we need a reliable
multiprocess synchronization mechanism.

https://bugzilla.gnome.org/show_bug.cgi?id=709115
2013-10-02 18:17:27 -04:00
Colin Walters
7b119370a2 repo: Enumerate objects using openat() too, make more efficient
This drops several calls to malloc()/g_object_new() per object
enumeration.  Just a followup to using openat() and friends in other
places.
2013-09-30 08:51:25 -04:00
Colin Walters
8592922c2c core: Make ostree_get_relative_object_path() private
I plan to rename all of these APIs to use the term 'loose', so that it
makes more sense after pack files are introduced.  External users
should not use them; instead use _load_variant() or _read_commit().
2013-09-29 20:17:18 -04:00
Jeremy Whiting
7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Colin Walters
b064581577 commit: Store detached metadata in normal form
Otherwise corrupted variants may crash us.
2013-09-29 14:00:53 -04:00
Jeremy Whiting
7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters
0f486105db remote-add: Add --set=KEY=VALUE option
This can be used to add a remote and set e.g. tls-permissive=true, or
gpgverify=false.
2013-09-28 12:00:16 -04:00
Colin Walters
38a5f6e5ed show: Handle keys of any type, and set an error if key doesn't exist
Previously we were just handling strings, and silently doing nothing
if the key didn't exist, which is pretty broken.
2013-09-27 12:57:01 -04:00
Colin Walters
2ef652439d repo-pull: Allocate with g_new0 rather than g_new.
Conflicts:

	src/libostree/ostree-repo-pull.c
2013-09-26 14:38:37 -04:00
Colin Walters
7d72a4b941 pull: Also fetch detached commitmeta files
These will contain GPG signatures and the like in the future, so we
should fetch them now.
2013-09-26 14:36:06 -04:00
Colin Walters
10cf4ccabc repo: When adding detached metadata, ensure the objdir exists
We may be creating the commitmeta file before the commit, so we need
to call mkdir(<first two checksum characters>) here too.
2013-09-26 13:59:08 -04:00
Colin Walters
edcfcd21a3 show: Print an error message if commit doesn't have detached metadata
...instead of segfaulting.
2013-09-26 13:57:59 -04:00
Colin Walters
7959ad9eaf fetcher: Fix previous commit
I was getting hangs in the test suite, and looking at the previous
commit, we were calling the async completion functions out of the
finalizer for the URI, which is weird.  I didn't analyze what's going
wrong, but what we really should be doing is processing our internal
queue after we've downloaded a file, and the request is about to be
finalized.

I suspect doing queue management from the finalizer created a circular
reference type situation.

This patch deduplicates the queue processing bits too.

https://bugzilla.gnome.org/show_bug.cgi?id=708126
2013-09-25 11:59:43 -04:00
Sjoerd Simons
5f310868f7 fetcher: Add a queue to limit requests sent to soup
On a large ostree repository pulling over http slows to a crawl. Pulling
from localhost results in:
 5944 metadata, 63734 content objects fetched; 850509 KiB transferred in
 1106 seconds
In other words about 800KiB/s. Some profiling shows that essentially
all of the CPU goes into libsoup doing its request bookkeeping instead
of into the actual downloading.

Adding a simple queue to limit to number of active request sent into
libsoup makes for a dramatic improvement:
 5944 metadata, 63734 content objects fetched; 850509 KiB transferred
 in 89 seconds
So around 9450 KiB/s.

https://bugzilla.gnome.org/show_bug.cgi?id=708126
2013-09-22 23:40:51 +02:00
Colin Walters
c044cdfb7d tests: Add u-boot
It needs to be installed so it'll be run by the InstalledTests scheme.
2013-09-21 11:06:20 -04:00
Javier Martinez Canillas
4df1d0644f tests: Fix U-Boot test case
commit 08b873457 ("deploy: Remove checksum from generated loader entries")
changed the generated loader entries file names thus making the U-Boot test
case to fail since this test parsed those files.

Fix test-admin-deploy-uboot.sh by looking to the updated files names.

https://bugzilla.gnome.org/show_bug.cgi?id=708511

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
2013-09-21 11:05:01 -04:00
Colin Walters
5c406132ce libostree: Document and annotate some methods to quash g-ir-scanner warnings
Since warnings are bad.  Introspection is good.
2013-09-20 14:00:31 -04:00
Colin Walters
b4bf5af5fa libostree: Remove private header file from ostree.h
Was breaking pkgsys-ostree.
2013-09-20 13:54:43 -04:00
Colin Walters
0f65d73581 upgrade: Don't segfault if there's no previous deployment
Obviously.
2013-09-20 11:21:08 -04:00
Colin Walters
298625d7f8 deploy: Correctly swap bootloader version with new boot checksums
If we had two deployments with different boot checksums, and were
trying to remove the one that was the same and add a new one (the
normal case), we'd end up assuming due to comparison with 0 that
we only needed to do the fast subbootversion swap.

Fix this by actually putting 1 where we really mean 1.

And update the tests to verify the fix; I have double-verified by
undoing the fix, and noting that the test fails.

https://bugzilla.gnome.org/show_bug.cgi?id=708351
2013-09-20 11:21:08 -04:00
Colin Walters
08b8734576 deploy: Remove checksum from generated loader entries
The actual deployment checksum shouldn't be in there, because we may
just swap bootlinks, rendering the name of the old bootloader entry
file invalid.  Thankfully nothing actually parsed the names of these
files, so let's just use the index.
2013-09-19 22:41:33 -04:00
Colin Walters
d03b8bbad9 deploy: Do a full system sync after writing bootloader config
This is just something I noticed on inspection; we should catch any
changes to /boot in the sync(), even though theoretically gio should
have done fdatasync().
2013-09-19 22:26:43 -04:00
Colin Walters
ae68b8380f deploy: Only query bootloader if we have to make a loader change
This will also help distinguish in the output when we're just swapping
bootlinks versus when we need to change the loader config.
2013-09-19 22:25:12 -04:00
Colin Walters
d1bc9e2acf deploy: Some minor assertion and error message improvements
To help debug https://bugzilla.gnome.org/show_bug.cgi?id=708351

Plus run a status consistently in the tests.
2013-09-19 22:23:19 -04:00
Colin Walters
9b31b526cc libostree: Add API to write metadata as stream
And use it in pull-local.  The goal here is to kill users of
ostree_repo_load_variant(), and move a bit more towards hiding the
variants.
2013-09-18 18:44:57 -04:00
Colin Walters
b4082d5642 test-core.js: Add standard copyright header
Like everything else.
2013-09-18 16:34:57 -04:00
Colin Walters
4c1a69f74e main: Print error: in red bold just for fun
It's easier to see.  Also drop old dead ot_main() function.
2013-09-18 16:34:14 -04:00
Colin Walters
58a8d6d6ef tests: Add a gjs-based test
This covers introspection, and in general is a much better way to get
API coverage tests.
2013-09-18 12:02:12 -04:00
Colin Walters
0818a462c2 sysroot: Look for booted deployment correctly
We need to look at the list we just gathered, not potentially NULL.
2013-09-17 10:53:41 -04:00
Colin Walters
8f1ea1b50a sysroot: Clean up API
Now that we have a real GObject for the sysroot, we have a convenient
place to keep track of 4 pieces of state:

* The current deployment list
* The current bootversion
* The current subbootversion
* The current booted deployment (if any)

Avoid requiring callers to pass all of this around and load it
piecemeal; instead the new thing is ostree_sysroot_load().
2013-09-16 18:32:13 -04:00