Commit Graph

3032 Commits

Author SHA1 Message Date
Simon McVittie
9d94fc40c8 Make corrupt-repo-ref.js executable
Debian's Lintian packaging consistency check complains that it isn't
executable but has a #! line. In fact it's reasonable to run this
script directly, so make it executable, and put it in a _scripts
variable so it will be installed executable.

Closes: #652
Approved by: cgwalters
2017-01-19 13:54:59 +00:00
Simon McVittie
9a3f82caae Sourced test snippets: remove shebang and make non-executable
They are installed non-executable, which makes Debian's Lintian
packaging consistency check complain that #! is only useful
in executable scripts. But in fact they are not useful to execute
directly (they rely on setup being done in the script that sources
them), so just chmod them -x.

Closes: #652
Approved by: cgwalters
2017-01-19 13:54:59 +00:00
Simon McVittie
01fb30b839 Fix TAP syntax in test-basic-user.sh, and run it
In its initial commit, Alexander Larsson wrote

    This works standalone, but unfortunately it breaks in
    gnome-desktop-testing-runner as /tmp doesn't support
    xattrs, so it is not installed atm.

but we now (a) use /var/tmp, and (b) explicitly skip the test if
xattr support is unavailable. So it should be OK to run now.

Closes: #652
Approved by: cgwalters
2017-01-19 13:54:59 +00:00
Colin Walters
b260fa764c libtest: Enable web server logs
Now that we're daemonizing, it's useful to have the logs.  I
wanted this while debugging cookies.

Closes: #651
Approved by: giuseppe
2017-01-19 10:47:15 +00:00
Colin Walters
686f91062f tests: Loosen error regexp
libcurl AFAICS doesn't have an API to convert HTTP code ➡️ error
string, so let's make the test regexp operate on both.

Closes: #651
Approved by: giuseppe
2017-01-19 10:47:15 +00:00
Colin Walters
56891f9d48 tests: Don't inject newline in URL
It turns out libsoup strips all whitespace even *inside* a URL. We could do that
for libcurl too but...really, people shouldn't do that. In this test we were
adding the trailing newline into the URL. If someone complains who is using the
libcurl code we can deal with it then.

Closes: #651
Approved by: giuseppe
2017-01-19 10:47:15 +00:00
Colin Walters
e6952de3d6 fetcher: Rework API to use strings for tls keys/db
This is prep for the libcurl porting. `GTlsCertificate/GTlsDatabase` are
abstract classes implemented in glib-networking for gnutls. curl's APIs take
file paths as strings, so it's easier to work on both if we move the GLib TLS
bits into the libsoup code.

Closes: #651
Approved by: giuseppe
2017-01-19 10:47:15 +00:00
Colin Walters
b28b785f01 pull: Fix theoretical checksum collision for metadata fetches
I was making some other changes in this code, and noticed that we were adding
checksums without object types into the same hash table for metadata. We should
*never* do this with both metadata content objects, since in theory a content
object could have the same hash as metadata.

I don't actually think it's possible in practice for pure metadata to collide,
since they have different structures, but let's do this anyways since it's
conceptually right.

Closes: #651
Approved by: giuseppe
2017-01-19 10:47:15 +00:00
Colin Walters
2f71136eec tests: Alias assert_not_reached() -> fatal()
We had a lot of copies of the "echo something 1>&2; exit 1" code even though
`assert_not_reached()` was it.  Hence, I think we need a shorter alias for that.

Doing this particularly since I noticed a missing `1` in an `exit 1` call in the
rpm-ostree copy of this.

Closes: #648
Approved by: jlebon
2017-01-18 14:28:29 +00:00
Colin Walters
5782e0a1d3 unlock: Fix description for --hotfix
Came up during an IRC discussion.  The text is accurate but
not very helpful.

Closes: #647
Approved by: dustymabe
2017-01-16 18:45:08 +00:00
Colin Walters
9a77017d87 .dir-locals.el: Standard Emacs indentation config
Better than having it per-file.

Closes: #644
Approved by: jlebon
2017-01-12 16:09:34 +00:00
Colin Walters
2a71afc507 trivial-httpd: Daemonize better
I was trying to debug `test-pull-c`, and typing `Ctrl-C` in gdb
ended up sending `SIGINT` to trivial-httpd as well, killing it.

Daemonize a bit more properly to avoid this. I also followed the standard
`/dev/null` guidelines.

Closes: #643
Approved by: jlebon
2017-01-12 15:26:01 +00:00
Colin Walters
d9f43cd2fb pull: Rework delta superblock fetches to be async
For the pending libcurl port, the backend is a bit more
sensitive to the main context setup.  The delta superblock
fetch here is a synchronous request in the section that's
supposed to be async.

Now, libsoup definitely supports mixing sync and async requests, but it wasn't
hard to help the libcurl port here by making this one async. Now fetchers are
either sync or async.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
ced22f6c9b Split trivial-httpd into separate binary
Working on the libcurl backend, I hit the issue that the trivial-httpd program
depends on libsoup. I briefly considered having two versions, but libcurl is
client only, and moreover trivial-httpd is no longer trivial - it has various
features which are used by the test suite extensively.

Hence, what we'll do is build it as a separate binary which links to libsoup,
and use it during the tests. We *also* currently still provide `ostree
trivial-httpd` since some things use it like `rpm-ostree-toolbox` and the
Cockpit tests.

After those are ported to use some other webserver, I plan to add a build-time
option to drop it.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
c051ee4a35 build-sys: Minor makefile tweaks
I'm introducing a new binary in a later patch, and it makes
sense to move more things to be common into the common section.

Also I noticed we were missing an inclusion of common `$(AM_LDFLAGS)`, though
AFAIK this doesn't break anything right now.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
21aca3fa83 fetcher: Split lowlevel API into file/membuf variants
The previous commit introduced a single low level API - however,
we can do things in a more optimal way for the curl backend if
we drop the "streaming API" variant.  Currently, we only use
it to synchronously splice into a memory buffer...which is pretty
silly when we could just do that in the backend.

The only tweak here is that we have an "add NUL character" flag that is
(possibly) needed when fetching into a membuf.

The code here ends up being better I think, since we avoid the double return
value for the `_finish()` invocation, and now most of the fetcher code (in the
soup case) writes to a `GOutputStream` consistently.

This will again make things easier for a curl backend.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
9d0d0a26db fetcher: Move high level functions into "fetcher-util"
Conceptually these now lay on top of the core API, and don't reference libsoup.
This is preparation for libcurl porting, but it's also just generally better.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
af560047a2 fetcher: Hoist core "mirrored request" API to public
This is in preparation for the libcurl port. We're basically making public what
we had internally. The next step here is to create `ostree-fetcher-util.[ch]`
that only operates in terms of this lower level API.

Also drop the `_mirrored` from the function name since it's
the default now.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:10 +00:00
Colin Walters
eceb165013 docs: Fix ostree.version -> version
I think originally I had envisioned this as `ostree.version`, but at the last
minute we changed it to just `version`. That's what all of the code uses, so
let's fix the docs.

Closes: #638
Approved by: paulvt
2016-12-23 15:24:56 +00:00
Paul van Tilburg
4d0b9be175 admin: Use execlp() to look for systemctl as the shell would
Closes: #637
Approved by: cgwalters
2016-12-23 12:18:27 +00:00
Dan Nicholson
300752e55a repo: Fix list_objects annotations
Without the element-type annotations, bindings don't know how to handle
the elements of the hash table. Since the table is created with destroy
functions, the caller does not own the elements, so transfer container
is used.

Closes: #635
Approved by: cgwalters
2016-12-22 16:07:52 +00:00
Dan Nicholson
dd3cda401b repo: Fix object list keys ownership
ostree_object_name_serialize returns a floating ref, so sink it before
adding it to the hash table so it can properly be freed later when the
hash table is destroyed.

This is particularly a problem for pygobject, which sinks the refs on
variants as it marshals them to native python types. If the ref isn't
already sunk, then the ref count won't increase and a critical warning
will be raised when both the hash table and pygobject try to unref it.

Closes: #635
Approved by: cgwalters
2016-12-22 16:07:52 +00:00
Dan Nicholson
169a629345 repo: Fix indentation
Closes: #635
Approved by: cgwalters
2016-12-22 16:07:52 +00:00
Mario Sanchez Prada
722e143ee1 static-delta: Pretend that world unreadable objects are new objects
This will prevent including in the delta the bits to update files that
are not world readable, so that we don't run into a permissions problem
when applying the deltas from a bare-user repository that has a bare
repository set as its parent.

This is the case for Endless when updating flatpak runtimes, as the
temporary directory created in ~/.local/share/flatpak/system-cache will
be of type bare-user with its parent set to /var/lib/flatpak which is a
bare repository in EOS, as it's shared with the one at /ostree/repo.

https://phabricator.endlessm.com/T14159

Closes: #634
Approved by: cgwalters
2016-12-16 14:52:09 +00:00
Colin Walters
c9d565a5a9 Release 2016.15
Closes: #632
Approved by: jlebon
2016-12-12 17:19:01 +00:00
Colin Walters
8bb8f0e3eb ci: Rebase to f25
This is now the devel target, plus I think this may fix some of the
ASAN issues I'm seeing; I mostly have been using f25 for local
testing.

Also remove the MAINTAINER line since the maintainers are defined by
`git log`.

Closes: #631
Approved by: jlebon
2016-12-12 16:00:45 +00:00
Colin Walters
2f86a5284c lib: Squash last use of GFile deltas_dir
I was having this thought today about making more of the OS readonly,
and ultimately if we got to the point where all ostree operations are
through the repo and sysroot dfds, we could have rpm-ostree be the
only process holding those fds open, and have a read-only bind mount
on top.

Anyways, we're not there, likely won't be soon, but this gets us
closer to being fully fd relative.

Closes: #628
Approved by: jlebon
2016-12-12 15:50:11 +00:00
Colin Walters
5aefe17ee9 ci: Combine UBSAN and ASAN by default
I only recently realized this was possible.  While we're still seeing
leaks in the CI environment for some reason, adding ASAN gives us
use-after-free detection etc., which is obviously still very useful
even if we're not doing leak checking.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
67ce5ec917 ci: Drop sudo installed tests
This conflicts with the ASAN work...and in general, I think I'd like
to make a new format for tests that require root, and have them be
defined to be in mutable containers or VMs.

Our coverage loss from this isn't much because some of these tests
already required `CAP_SYS_ADMIN` which we didn't have in Docker
anyways.

While we have the patient open, parallelize the regular installed
tests.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
359ea9b58f tests: Tweak installed tests to deal with ASAN
We need to disable readdir-rand there too.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
47b4dd1b38 Skip gjs-based tests if ASAN is enabled
Unfortunately, introspection uses dlopen(), which doesn't quite
work when the DSO is compiled with ASAN but the outer executable
isn't.

Trying to inject LD_PRELOAD=libasan means the outer executable has to
be leak free...which, yeah, I'm not going to get into running ASAN
today on gjs or pygobject.

So, let's skip those tests - ideally, we still run them in some other
context without the sanitizers.  The coverage we have from them is
middling anyways.

Closes: #622
Approved by: jlebon
2016-12-09 18:05:53 +00:00
Colin Walters
17f264a487 repo: Add unconfigured-state to remote config options
This is a migration from the origin version.  It's
nicer to have it in the remote, since that's what one
needs to change.  Then tools don't need to mess with
the origin file.o

In fact in this scenario one can keep the "media source" like
`file:///install/repo` or whatever, since conceptually that's where it
came from.  We're just providing a better error.

Closes: https://github.com/ostreedev/ostree/issues/626

Closes: #627
Approved by: jlebon
2016-12-09 17:46:54 +00:00
Colin Walters
ef438c8d60 build: Error if glib isn't found
This is a bit extracted from my work on ASAN.

Closes: #625
Approved by: jlebon
2016-12-09 16:35:52 +00:00
Dan Nicholson
640e92ef37 repo: Fix annotations for remote_fetch_summary functions
These are out parameters, so add the (out) annotation and switch
(nullable) to (optional) since the latter is used for the purpose of
optional out parameters.

Closes: #629
Approved by: cgwalters
2016-12-09 13:07:42 +00:00
Colin Walters
c19fae0282 tree-wide: Switch to autoptr for GOptionContext
We were leaking in a few places that I noticed in an ASAN run.  Also,
this was one of the last non-autoptr cleanup sections we have in
`out:` cleanup sections, making us a lot closer to a potential
full-tree rewrite to `return FALSE`.

Closes: #624
Approved by: jlebon
2016-12-08 16:48:20 +00:00
Colin Walters
5155662b57 build: Always do enum scanning now
Since we stopped including the libsoup headers in `ostree-fetcher.h`,
we can now unconditionally do enum scanning, and drop a build time
conditional.

Prep for libcurl porting.

Closes: #620
Approved by: jlebon
2016-12-08 16:34:06 +00:00
Colin Walters
58e318d678 [ASAN] sysroot: Squash a leak in lockfile acquisition
I installed `parallel` in my dev container, which got me
the sysroot locking tests, which caught this leak when
built with ASAN.

Closes: #623
Approved by: jlebon
2016-12-08 16:18:19 +00:00
Colin Walters
7f2830cb4e build: Make libsoup optional again
The "remote cookies" code broke this.  While I'm not sure anyone is
actually using ostree-without-http, it isn't too hard to keep the
build time conditional going.  Further, this work is preparatory for
libcurl porting.

Closes: #621
Approved by: jlebon
2016-12-08 16:04:16 +00:00
Colin Walters
7519457382 fetcher: Define an abstraction over SoupURI
This is preparatory work for a potential libcurl backend.

Closes: #616
Approved by: jlebon
2016-12-07 23:00:58 +00:00
Colin Walters
099576ee4a lib: Ensure we use _GNU_SOURCE in enum templates
Due to the way glib-mkenums runs the preprocessor itself, it
doesn't pick up the `AC_USE_SYSTEM_EXTENSIONS()` that we have in
`configure.ac`.

This blew up in an obscure way when I later wanted to `#include
"libglnx.h"` in one of the headers, since it needs the `basename()`
from `string.h` which is only available with `_GNU_SOURCE`.

Closes: #616
Approved by: jlebon
2016-12-07 23:00:58 +00:00
Colin Walters
a2d627352d build: Add more default errors
Newer gcc has `-Wincompatible-pointer-types`, hooray!
Add a few others that we pass today.

Closes: #618
Approved by: jlebon
2016-12-07 16:44:15 +00:00
Colin Walters
daf01b27d4 ci: Make all ci tests gating for Homu
See the rhci docs.

Closes: #617
Approved by: jlebon
2016-12-07 15:42:14 +00:00
Colin Walters
37965644ff tree-wide: Use g_hash_table_add() where applicable
Just noticed a few while reading some code, decided to do a quick
cleanup.  It's shorter and clearer.

Closes: #614
Approved by: jlebon
2016-12-06 17:45:23 +00:00
Colin Walters
247a8718bb lib: Remove unused ostree_metalink_get_uri()
While doing something else I noticed it was unused.

Closes: #615
Approved by: jlebon
2016-12-06 16:27:52 +00:00
Colin Walters
0fb3645fed pull: Write .commitpartial for local pulls first too
This is what we do for non-local (i.e. HTTP) pulls; we wnat to
correctly handle being interrupted during partial pulls.

Closes: https://github.com/ostreedev/ostree/issues/579

Closes: #613
Approved by: jlebon
2016-12-06 16:13:53 +00:00
Colin Walters
4c8fc92aa0 lib: Always checksum content in deltas
This is a follow up to conversation on list - in practice, if we're
backing away from summary signing, then it makes sense to remove the
special casing for checksums in deltas around summary signatures.

This is also related to the recent change to enable GPG checking for
commits in deltas - now we have a more coherent story between the
previous pull path and deltas.

I didn't do any performance checking, and while it's slightly annoying
that we're now doing sha256 on the delta content twice (once for the
part and once per object)...sha256 is pretty fast, I think most users
are I/O bound anyways, and it'd drop even farther if we started using
openssl.

Closes: #612
Approved by: jlebon
2016-12-06 15:59:35 +00:00
Colin Walters
66da1199f0 tests/keyfile-utils: Drop tests covering preconditions
The spam in stderr was bothering me, and further at some eventual
point in the future we want to annotate the functions with
`__attribute__((nonnull))` which would then cause tests like these to
become undefined behavior.

The coverage of this isn't worth the log spam basically.

Closes: #611
Approved by: jlebon
2016-12-06 14:50:08 +00:00
Colin Walters
555f1fb03d tests: Use G_DEBUG=fatal-warnings here too
I am trying to track down a warning I see in `test-keyfile-utils`
which turned out to be the installed case only, but let's inject
this here too.

(The GLib default is broken, but it's hard to fix upstream without
 breaking the world)

Closes: #610
Approved by: jlebon
2016-12-06 14:36:06 +00:00
Colin Walters
e1c48c8adb [ASAN] tests: Cleanup all current remaining leaks
We now run fully through ASAN here.

Closes: #609
Approved by: jlebon
2016-12-05 15:20:56 +00:00
Colin Walters
7bb0ff46a4 Define and use cleanup helpers for libarchive
This should fix some of the ASAN leaks around libarchive usage,
and is generally better.

Closes: #609
Approved by: jlebon
2016-12-05 15:20:56 +00:00