2323 Commits

Author SHA1 Message Date
John Hiesey
153f092d51 static-delta: assert on non-regular files
Now that the computed similar objects are all regular files,
get_unpacked_unlinked_content should never be called on any other
object type. Assert that this is true instead of silently succeeding.
2015-08-26 16:25:28 -04:00
John Hiesey
43d045309c static-delta: Ignore symlinks when computing similar objects
_ostree_delta_compute_similar_objects should not output symlinks.
Previously, a symlink in the "from" commit could be matched to a
real file in the "to" commit, since nothing was filtering symlinks
on the "from" side. This led to failures running the bzdiff
algorithm.
2015-08-26 16:25:28 -04:00
Dan Nicholson
4f57aa5b15 tests: Build test-lzma with LZMA flags
test-lzma builds a copy of the compressor and decompressor directly, so
the compiler needs access to the LZMA headers and the linker needs to
link the program with liblzma.
2015-08-26 15:20:42 -04:00
Jasper St. Pierre
8e8bfa8708 Update .gitignore 2015-08-26 12:16:29 -07:00
Jasper St. Pierre
20647edcbf repo-pull: Add a queue for scanning
On systems with slow disks, the recursive scanning of directories can
be expensive -- it takes upwards of 2 minutes on our systems. This can
block the main loop for such a long time that it allows the download to
time out...

As such, move all the scanning of objects to a queue, processed from
an idle, to make sure that we don't block the main loop when scanning.

https://bugzilla.gnome.org/show_bug.cgi?id=753336
2015-08-26 12:15:38 -07:00
Giuseppe Scrivano
863f5d8598 tests: Add tests for rollsum 2015-08-25 09:27:56 -04:00
Colin Walters
b89326ac41 rollsum: Fix assertion for CRC matches
It's possible for two blocks to have the same CRC but different
length.  Rather than asserting, treat them as not matching.
2015-08-25 09:26:22 -04:00
Giuseppe Scrivano
9b9f4b04b4 tests: rename test-rollsum to test-rollsum-cli
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-08-24 11:07:36 +02:00
Colin Walters
9f3d586993 pull: Stop using GMainLoop
First of all, what we were doing with having GMainLoop in the internal
APIs is wrong.  Synchronous APIs should always create their own main
context and not iterate the caller's.  Doing the latter creates
potential for evil reentrancy issues.  Sync API should block, async
API is for not blocking.

Now that's out of the way, fix the pull code to do the clean

```
while (termination_condition (state))
  g_main_context_iteration (mainctx, TRUE);
```

model for looping.  This is a lot easier to understand and ultimately
more reliable than having other code call `g_main_loop_quit()`, as the
loop condition is in exactly one place.

We can also remove the idle source which only fired once.

Note we have to add a hack here to discard the synchronous session and
create a new one which we only use async.

https://bugzilla.gnome.org/show_bug.cgi?id=753336
2015-08-13 22:02:00 -04:00
Colin Walters
5c20ea920e sysroot: Add an unload() API
This will be used by gnome-continuous at least to drop the reference
to the fd so that unmounting can proceed.  See
https://git.gnome.org/browse/gnome-continuous/commit/?id=95e9910ea288d302509ca667e0d190dd89377dd5
2015-08-13 12:43:14 -04:00
Giuseppe Scrivano
3861b79efd tests: add tests for LZMA compressor and decompressor
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-08-12 07:55:50 +02:00
Jasper St. Pierre
3c152fb38b Update .gitignore
https://bugzilla.gnome.org/show_bug.cgi?id=753336
2015-08-08 21:53:43 -04:00
Giuseppe Scrivano
75cdbb8e82 tests: skip tests using gjs/parallel if they are not installed
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes #131
2015-08-05 10:58:00 -04:00
Giuseppe Scrivano
8b9effea56 tests: add tests for --disable-bsdiff and --max-bsdiff-size
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-29 17:34:21 -04:00
Colin Walters
cd6eac2ce9 README: Attempt to flesh out more, start moving docs from wiki
I'd like to migrate content from the GNOME wiki, as frankly the wiki
is crap.  Markdown in git is better in every way.

Start by fleshing out the README.md to be more useful.
2015-07-28 18:39:43 -04:00
Jasper St. Pierre
8034a3842b Update .gitignore
https://bugzilla.gnome.org/show_bug.cgi?id=752950
2015-07-28 14:23:02 -04:00
Cosimo Cecchi
a5f266f25d repo: don't forget to abort the transaction when failed
ostree_repo_prepare_transaction() should always be matched with a call
to either ostree_repo_commit_transaction() or
ostree_repo_abort_transaction().

Since ostree_repo_pull_with_options() does not call
ostree_repo_abort_transaction() on errors, the OstreeRepo instance will
hit an assertion when it's re-used later for another attempt, such as
when the update is driven by an external component through libostree and
network temporarily goes down.

This commit simply always calls ostree_repo_abort_transaction() in the
exit path of ostree_repo_pull_with_options(), since the function is safe
to call even when we're not in a transaction, and that matches e.g. what
ostree-sysroot-cleanup.c does.
2015-07-28 11:35:42 -04:00
Cosimo Cecchi
939a7aebfb repo: fix an incorrect comment 2015-07-28 11:35:33 -04:00
Giuseppe Scrivano
b7063ed1ae static-delta: add max-bsdiff-size option
It allows to specify the maximum size for input files to attempt
bsdiff compression for.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-28 13:02:27 +02:00
Giuseppe Scrivano
cac92f62c4 static-delta: do not fail compilation with big files
Just skip the bsdiff compression instead of failing the operation.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-28 12:28:25 +02:00
Giuseppe Scrivano
11ba9056ea pull: new option --commit-metadata-only
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-24 12:43:35 -04:00
Giuseppe Scrivano
56f0ae6a4a repo: new function _ostree_preload_metadata_file
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-24 12:37:42 -04:00
Giuseppe Scrivano
783e83c62c repo: merge repo_remote_fetch_summary_{metalink,url}
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-24 12:37:42 -04:00
Giuseppe Scrivano
133fb5ffdc libostree: new API ostree_repo_remote_list_refs
The new API permits to query a remote repository summary file and
retrieve the list of available refs.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-24 12:37:42 -04:00
Giuseppe Scrivano
a917c96976 tests: do not commit from the working directory
It fixes this problem:

```
error: Not a regular file or symlink: S.gpg-agent
```

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-07-21 10:02:30 -04:00
Matthew Barnes
df0cae4375 tests: Export OSTREE_SYSROOT in setup_os_repository
Eliminates the need for constantly passing --sysroot=sysroot, but
also makes ostree place remote configs for sysroot/ostree/repo in
sysroot/etc/ostree/remotes.d where they should have been all along.
2015-07-16 12:49:34 -04:00
Matthew Barnes
254def3584 main: Fix UID check based on sysroot path
This is another place where we were hard-coding a default.  Create the
OstreeSysroot first and THEN check if its path is the root directory.
2015-07-16 12:49:28 -04:00
Matthew Barnes
708d923125 repo: Fix location of remote configs for system repos
Need to respect the repo's system root directory instead of assuming the
compile-time $(sysconfdir).
2015-07-16 12:49:18 -04:00
Matthew Barnes
74fb777edb sysroot: Pass the internal repo a system root path 2015-07-16 12:49:11 -04:00
Matthew Barnes
104b8bf6fb repo: Add a "sysroot-path" property
Adds ostree_repo_new_for_sysroot_path() to specify a system root path.
OstreeRepo otherwise uses _ostree_get_default_sysroot_path().
2015-07-16 12:48:56 -04:00
Matthew Barnes
26225cb198 sysroot: Use _ostree_get_default_sysroot_path()
Instead of hard-coding the root directory as default.
2015-07-16 12:48:44 -04:00
Matthew Barnes
f2ddf10d4e core: Add _ostree_get_default_sysroot_path()
Returns a GFile for the default system root, which is usually the root
directory unless overridden by the OSTREE_SYSROOT environment variable
(which is mainly intended for testing).
2015-07-16 12:48:11 -04:00
Colin Walters
31d16c9cce pull: Plug a memory leak 2015-06-29 21:57:44 -04:00
Colin Walters
889b86e96d pull: Avoid leaking signal handlers across fetch requests
libsoup will cache sessions, so it might be the case that we get a
reused session when pulling from the same repo multiple times in one
process.

In this case we were leaking signal connections, which caused
callbacks into freed memory with bad consequences.

Fix it by tying the signal connection to the object lifetime.
2015-06-29 21:56:03 -04:00
Jeremy Cline
f0a02fbf20 build: Make gtk-doc optional
This is similar to what's done in glib.
2015-06-29 16:08:51 -04:00
Colin Walters
530631376e tests: Check error messages instead of "expected-fail", handle old parallel 2015-06-29 13:35:07 -04:00
Colin Walters
c8c239f94c pull: Also fix misplaced remote name handling
We want to set the remote name only if we're operating on a remote
URL.
2015-06-29 11:20:41 -04:00
Colin Walters
536d6fb083 core: Fix inverted conditional in GPG checking 2015-06-29 11:09:00 -04:00
Colin Walters
720e3b5f83 pull: Error if gpg=true and summary is 404, add more tests
I did a quick audit pass through the pull code.  What I focused on the
most is the case where `gpg-verify-summary=true`, and in particular
where `gpg-verify=false` too.  This should be a valid and secure
configuration.

The primary change here is to error out very quickly if either
`summary` or `summary.sig` are 404.  Previously, we'd only error out
if we were processing deltas.

Expand the existing test case to cover this, plus invalid summary and
invalid sig.  (The test case was failing with current git master too).
2015-06-27 12:04:18 -04:00
Giuseppe Scrivano
3737b9851a ostree_repo_remote_fetch_summary: honor gpg-verify-summary
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-06-26 11:02:25 +02:00
Giuseppe Scrivano
0bd10eb6e2 tests: add test for check for remote add --set=gpg-verify-summary=true
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-06-26 11:02:25 +02:00
Giuseppe Scrivano
19ce011e1f pull: fail if GPG is enabled and the summary is not signed
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-06-26 11:02:25 +02:00
Giuseppe Scrivano
b09c9b7005 repo: new function ostree_repo_remote_get_gpg_verify_summary
It allows to specify whether GPG verification for the summary file is
enabled for a specific repository.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-06-26 11:02:25 +02:00
Giuseppe Scrivano
6bf067411d pull: verify summary signatures also when not mirroring
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-06-26 11:02:24 +02:00
Matthew Barnes
6284beb2b6 ostree: Add a "remote refs" command
Works like "ostree refs" but fetches refs from a remote repo.

This depends on the remote repo having a summary file, but any repo
being served over HTTP *ought* to have one.
2015-06-26 11:02:24 +02:00
Matthew Barnes
0dbf91484b repo: Add ostree_repo_remote_fetch_summary()
Reusable method for fetching a summary file and signatures.
2015-06-26 11:02:24 +02:00
Matthew Barnes
87851c7bbe metalink: Return requested file as a GBytes
This may not be the best idea for general usage, but the only use case
for metalinks currently is fetching a summary file and those are pretty
small.  Far more convenient to return the file content in a GBytes.
2015-06-26 11:02:24 +02:00
Matthew Barnes
ebef43eabe metalink: Allow NULL for "out" params in metalink requests
Caller may not be interested in all the outbound params, particularly
"fetching_sync_uri".
2015-06-26 11:02:24 +02:00
Matthew Barnes
3515e01f6a repo: Redo ostree_repo_remote_get_url()
Make it work like in ostree_repo_pull_with_options(), handling "file://"
remotes and inheriting the "url" option from parent repos if needed.
2015-06-26 11:02:24 +02:00
Matthew Barnes
d4111aeac0 repo: Handle "file" remotes in ostree_repo_remote_get_gpg_verify() 2015-06-26 11:02:24 +02:00