Commit Graph

70 Commits

Author SHA1 Message Date
Matthew Barnes
70cabcea0a gpg: Rewrite OstreeGpgVerifier to use GPGME
This sets the stage for more advanced signature management.

(Also, talking to GPG over pipes sucks.)

Previously we were spawning gpgv2 with a bunch of --keyring options
for /usr/share/ostree/trusted.gpg.d/ and whatever other keyring files
were explicitly added.  GPGME has no public API for multiple keyrings,
so we work around the issue by setting up a temp directory to serve as
a fake "home" directory for the crypto engine and then concatenate all
the keyring files into a single public keyring (pubring.gpg).

Unfortunately at present we do this on every signature verification.
There's a desire to cache this concatenation, but the problem is the
user may be unprivileged.  So it seems the cache would have to be per
user under $XDG_CACHE_HOME, which OSTree doesn't otherwise use.  I'm
open to suggestions.

We do at least clean up the temp directory when finished, and I have
further API changes planned to OstreeGpgVerifier to help mitigate the
performance impact.
2015-03-06 08:22:44 -05:00
Giuseppe Scrivano
a705d9cf29 Add bsdiff submodule
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 12:16:17 -05:00
Giuseppe Scrivano
0a7a70ed67 build: build libbupsplit separately
It avoids gobject-introspection warnings:

src/libostree/bupsplit.h:42: Warning: OSTree: symbol='bupsplit_find_ofs': Unknown namespace for symbol 'bupsplit_find_ofs'
src/libostree/bupsplit.h:43: Warning: OSTree: symbol='bupsplit_selftest': Unknown namespace for symbol 'bupsplit_selftest'
src/libostree/bupsplit.h:33: Warning: OSTree: symbol='BUP_BLOBBITS': Unknown namespace for symbol 'BUP_BLOBBITS'
src/libostree/bupsplit.h:34: Warning: OSTree: symbol='BUP_BLOBSIZE': Unknown namespace for symbol 'BUP_BLOBSIZE'
src/libostree/bupsplit.h:35: Warning: OSTree: symbol='BUP_WINDOWBITS': Unknown namespace for symbol 'BUP_WINDOWBITS'
src/libostree/bupsplit.h:36: Warning: OSTree: symbol='BUP_WINDOWSIZE': Unknown namespace for symbol 'BUP_WINDOWSIZE'

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-23 17:07:13 -05:00
Matthew Barnes
12c8ef1425 Add explicit zlib dependency
Needed for CRC32 computations.
2015-02-23 14:48:52 -05:00
Colin Walters
965a304a17 Use libglnx
Starting down the path of not using libgsystem.  The main win here
will be code sharing between ostree/rpm-ostree as well as going down
the path of not using GFile * for local files.
2015-02-22 21:02:27 -05:00
Colin Walters
c54df89771 deltas: Search for similar objects (possibly renamed across directories)
The previous diff algorithm was file tree based, and only looked
at modified files that lived at the same path.

However, components like the Linux kernel have versioned
subdirectories, e.g. /usr/lib/modules/$kver/.../ext4.ko.  We want to
be able to detect these "modified renames" so that we can compute
diffs (rollsum, bsdiff).
2015-02-16 10:10:35 -05:00
Colin Walters
9aa7e30b38 deltas: Implement rollsums
This does an rsync-style prepared delta basically.  On my test data,
it shaves ~6MB of uncompressed data.  Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
2015-02-16 10:10:35 -05:00
Colin Walters
3df8be0d92 deltas: Compute rollsum targets 2015-02-16 10:10:35 -05:00
Giuseppe Scrivano
7ebf7676cb Fix make distcheck
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-30 15:27:36 +01:00
Matthew Barnes
a4d904ee38 build: Include SELinux cflags with libostree build too
This is likely another issue with newer automake.

https://bugzilla.gnome.org/738875
2014-10-20 15:23:55 -04:00
Colin Walters
d546abfa2a libostree: Add initial GRUB2 support
In this approach, we drop a /etc/grub.d/15_ostree file which is a
hybrid of shell/C that picks up bits from the GRUB2 library (e.g. the
block device script generation), and then calls into libostree's
GRUB2 code which knows about the BLS entries.

This is admittedly ugly.  There exists another approach for GRUB2 to
learn the BLS specification.  However, the spec has a few issues:

https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html

This approach also gives a bit more control to the admin via the
naming of the 15_ostree symlink; they can easily disable it:

Or reorder the ostree entries ahead of 10_linux:

Also, this approach doesn't require patches for grub2, which is an
issue with the pressure to backport (rpm-)OSTree to EL7.
2014-10-16 14:15:00 -04:00
Colin Walters
1242704d68 build: Unify CPPFLAGS settings
The libostree core uses SYSCONFDIR now, so we should ensure it's used
consistently.  Someone else was seeing SYSCONFDIR not being defined
while compiling with a newer automake version, which may process
CPPFLAGS more precisely.
2014-09-08 11:47:58 -04:00
Colin Walters
f8f5da219e Add repository "summary" file and metalink support
For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.

In order to make metalink work though, it needs *one* file which can
be checksummed.  (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).

This commit adds a new command:

$ ostree summary -u

To regenerate the summary file.  Can only be run by one process at a
time.

After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.

https://bugzilla.gnome.org/show_bug.cgi?id=729585
2014-09-03 13:21:52 -04:00
Colin Walters
b4d21e9406 deploy: Set the immutable bit on the deployment root
This prevents people from creating new directories there and expecting
them to be persisted.  The OSTree model has all local state to be in
/etc and /var.

This introduces a compile-time dependency on libe2fsprogs.

We're only doing this for the root directory at the moment.

https://bugzilla.gnome.org/show_bug.cgi?id=728006
2014-06-03 13:41:48 -04:00
Colin Walters
7ccfc93f71 build: Add missing DESTDIR
Hooray for read-only bind mounts and building as non-root.
2014-05-09 09:07:00 -04:00
Colin Walters
f47a20fb81 Support /etc/ostree/remotes.d
For many OS install scenarios, one runs through an installer which may
come with embedded data, and then the OS is configured post-install to
receive updates.

In this model, it'd be nice to avoid the post-install having to rewrite
the /ostree/repo/config file.

Additionally, it feels weird for admins to interact with "/ostree" -
let's make the system feel more like Unix and have our important
configuration in /etc.

https://bugzilla.gnome.org/show_bug.cgi?id=729343
2014-05-08 18:59:24 -04:00
Colin Walters
25ad4a9f65 pull: Add tls-client-cert-{path,key} (if we have new enough libsoup)
This is an actually working version of client-side certificates.
Depends on:
See: https://bugzilla.gnome.org/show_bug.cgi?id=334021

We detect whether libsoup is new enough for this.

https://bugzilla.gnome.org/show_bug.cgi?id=729356
2014-05-01 17:13:13 -04:00
Colin Walters
006e8f0073 deltas: Link to liblzma, add internal API to use it
For future static deltas work, we'll be linking to liblzma.  Since
it's fairly widespread, let's just make it a hard dependency.
2014-04-29 10:59:57 -04:00
Colin Walters
1bdabda5f3 Use external libgsystem 2014.2
It's been split off for a while, let's kill the code duplication.

Among other things, this fixes the systemd detection for the journal
logging.
2014-04-04 16:52:37 -04:00
Colin Walters
7baa600e23 Add an OstreeSysrootUpgrader API
This moves some utility code from the ostree tool into the shared
library, which will make it easier to consume by external tools.
2014-03-24 18:08:22 -04:00
Colin Walters
3337334be5 libostree: Split off SELinux OstreeSePolicy class
It's better if this is independent from the OstreeSysroot; for
example, a policy is active in a given deployment root at once, not
for a sysroot globally.

We can also collect SELinux-related API in one place.

Unfortunately at the moment there can be only one instance of this
class per process.
2014-02-19 08:43:45 -05:00
Colin Walters
2d6374822b Initial basic static delta code drop
This has a very basic level of functionality (deltas can be generated,
and applied offline).  There is only some stubbed out pull code to
fetch them via HTTP.

But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
2014-02-04 10:31:44 -05:00
Colin Walters
c7bcfc1c61 build: Look for /usr/bin/gpgv2 vs /usr/bin/gpgv
For some reason, RHEL has gpgv, but Fedora doesn't.  We need to detect
which to use, since presumably Debian only has gpgv.
2014-01-30 19:32:59 -05:00
Colin Walters
ebe6207847 build: Install README-gpg in /usr/share/ostree/trusted.gpg.d
Since this is what the current code actually reads.
2014-01-30 15:28:13 -05:00
Colin Walters
b2d0ba7ac1 deploy: Rework kernel arguments, add --karg-append to "admin deploy"
The "ordered hash" code was really just for kernel arguments.  And it
turns out it needs to be a multihash (for e.g. multiple console=
arguments).

So turn the OstreeOrderedHash into OstreeKernelArgs, and move the bits
to split key=value and such into there.

Now we're not making this public API yet - the public OstreeSysroot
just takes char **kargs.  To facilitate code reuse between ostree/ and
libostree/, make it a noinst libtool library.  It'll be duplicated in
the binary and library, but that's OK for now.  We can investigate
making OstreeKernelArgs public later.

https://bugzilla.gnome.org/show_bug.cgi?id=721136
2014-01-16 15:07:55 -05:00
Colin Walters
c65923e642 Add OstreeAsyncProgress, use it for ostree_repo_pull
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications.  This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.

From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.

PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.

So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.

Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
2013-10-24 14:27:13 -04:00
Colin Walters
6500026ba7 trivial: Add missing files to dist 2013-10-15 16:39:22 -04:00
Colin Walters
3b700ccb50 core: Add code to read/write "varints"
Adapted from Google protobufs.  For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.

This will be used for static deltas and sizes metadata.
2013-10-11 12:21:37 -04:00
Jeremy Whiting
7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Jeremy Whiting
7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters
ac0154713d libostree: Make OstreeBootloader private
It was only temporarily public while functionality was being merged
down; that's done now.
2013-09-15 20:26:13 -04:00
Colin Walters
c6292942ff libostree: Nearly complete move of API into OstreeSysroot
Move the deployment code too.
2013-09-15 20:16:20 -04:00
Colin Walters
95f07d486a libostree: Move a lot more sysroot API here
OstreeBootloader is temporarily public API.
2013-09-15 18:08:06 -04:00
Colin Walters
35bab87691 Move Deployment and BootconfigParser into libostree
As part of moving admin functionality there.  While we are doing this,
rename OtConfigParser to OstreeBootConfig parser since it's a better
name.
2013-09-15 15:06:31 -04:00
Colin Walters
af0f888057 libostree: Add new OstreeSysroot class
At the moment, just a container for a path, but we will start moving
admin functionality here.
2013-09-15 14:33:57 -04:00
Jasper St. Pierre
c817217ad8 repo: Move commit code to another file
ostree-repo.c is a bit too big, and most of the commit code is
fairly standalone.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:31:11 -04:00
Jasper St. Pierre
52dd6b0b74 libostree: Make introspectable again
At some point, the variable name for the headers changed and the
introspection build line forgot to be updated.

https://bugzilla.gnome.org/show_bug.cgi?id=707228
2013-09-01 11:19:15 -04:00
Sjoerd Simons
d1babde95e Fix make distcheck
make distcheck was unhappy for various reasons:
  * headers aren't data, so use _HEADERS otherwise compilation fails
  * Mark the gir & typelib data as cleanfiles so they aren't left around
    after make clean
  * Don't nuke the .la file. This breaks make uninstall, leave it up to
    distributions to not install .la files if they don't want them.

https://bugzilla.gnome.org/show_bug.cgi?id=705850
2013-08-17 14:57:05 -04:00
Colin Walters
17560a57bf Add gtk-doc support
Yes, it's really me.  Colin Walters.  Writing documentation.  You
don't need to do a DNA test.
2013-08-16 22:56:12 -04:00
Colin Walters
a5d43bb959 Install a shared library
This required a fair bit of surgery because previously ostree.h
included otutil.h, but that's supposed to be a private library.
2013-07-26 19:25:07 -04:00
Colin Walters
6fc9483887 lib: Use -export-symbol-regex
Just a small step towards being a better library.
2013-07-25 17:09:00 -04:00
Colin Walters
2535f32c56 libostree: Move pull code into here
More library work.
2013-07-09 20:14:53 -04:00
Colin Walters
5dd0d5da40 libostree: Move prune into OstreeRepo namespace
More library work.
2013-07-09 20:05:31 -04:00
Colin Walters
2aa0251b1b libostree: Rename ostree-traverse.h into OstreeRepo namespace
This operates on the repo, it might as well be methods on it.
2013-07-09 19:59:39 -04:00
Colin Walters
cb6b69616c libostree: Split off -refs.c
Continuing to break up ostree-repo.c.
2013-07-09 19:49:00 -04:00
Colin Walters
3b6a94d272 libostree: Split off libarchive code 2013-07-09 19:11:37 -04:00
Colin Walters
dbffd31017 libostree: Start splitting up the monstrous ostree-repo.c
Checkout now lives in its own file.
2013-07-09 19:02:38 -04:00
Colin Walters
26cef497a6 Remove built in "triggers"
Originally, the idea was that clients would replicate "OS/tree"s from
a build server, but we'd run things like "ldconfig" on the client.
This was to allow adding e.g. the nVidia binary driver.

However, the triggers were the only thing in the system at the moment
that really had expected knowledge of the *contents* of the OS, like
the location of binaries.

For now, it's architecturally cleaner if we move the burden of
triggers to the tree builder (e.g. gnome-ostree or RPM).  Eventually
we may want OSTree to assist with this type of thing (perhaps
something like RPM %ghost), but this is the right thing to do now.
2013-07-07 14:37:59 -04:00
Colin Walters
7e882cc2cf dracut: Add ostree-remount
Linux creates a copy of the soure mount flags when creating a bind
mount; if the source is read-only, then the bind mount is.

The problem is that systemd will remount the rootfs read/write, but
each mount (/home, /var etc.) will still be read-only.  We need to
remount every bind mount except for /usr to read-write too.

This only "worked" with the old ostree-switch-root because it
effectively force mounted the rootfs read-write always, ignoring the
"ro" flag.
2013-06-04 15:59:52 -04:00
Colin Walters
cf31942a69 core: Refactor diff to be an internal API
For future use by other code.
2012-08-31 17:39:04 -07:00