Commit Graph

106 Commits

Author SHA1 Message Date
Colin Walters
2bab43fb22 lib: Split symbol versioning into -released and -devel
So far a lot of submitted PR have added symbols into the first
section.  Split the file into `-released` and `-devel` to make
this more obvious.

To further enforce things, we hardcode a checksum of the `-released`
file in `test-symbols.sh`.  Only release commits should update that
checksum.

Did you notice I like checksums?

Closes: #931
Approved by: pwithnall
2017-06-16 09:31:16 +00:00
Colin Walters
30705889cb Switch to using a systemd generator for /var
If one wants to set up a mount for `/var` in `/etc/fstab`, it
won't be mounted since `ostree-prepare-root` set up a bind mount for
`/var` to `/sysroot/ostree/$stateroot/var`, and systemd will take
the already extant mount over what's in `/etc/fstab`.

There are a few options to fix this, but what I settled on is parsing
`/etc/fstab` in a generator (exactly like `systemd-fstab-generator` does),
except here we look for an explicit mount for `/var`, and if one *isn't* found,
synthesize the default ostree mount to the stateroot. Another nice property is
that if an admin creates a `var.mount` unit in `/etc` for example, that will
also override our mount.

Note that today ostree doesn't hard depend on systemd, so this behavior only
kicks in if we're built with systemd *and* libmount support (for parsing
`/etc/fstab`).  I didn't really test that case though.

Initially I started writing this as a "pure libc" program, but at one point
decided to use `libostree.so` to find the booted deployment. That didn't work
out because `/boot` wasn't necessarily mounted and hence we couldn't find the
bootloader config. A leftover artifact from this is that the generator code
calls into libostree via the "cmd private" infrastructure. But it's an easy way
to share code, and doesn't hurt.

Closes: #859
Approved by: jlebon
2017-05-16 16:13:05 +00:00
Philip Withnall
6eac575f21 libostree: Make OstreeRemote a public and internal API
Previously it was static to ostree-repo.c. Make it usable throughout
libostree so it can be used by an upcoming commit, but also expose the
typedef and reference counting functions so that opaque OstreeRemote
pointers can be used by user code, in anticipation of exposing more of
its API publicly in future.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Philip Withnall
c9244b1bb2 build: Add --enable-experimental-api configure option for unstable APIs
There are currently no unstable APIs, but some will be added in
following commits. They will be built and exposed in the libostree
global symbol list iff configured with --enable-experimental-api.

Distributions should not package OSTree with --enable-experimental-api.
This is designed for previewing new APIs on controlled platforms; any of
the APIs hidden behind this option may be changed or removed at any
point.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Colin Walters
ffc178311e build: Dist ostree-sepolicy-private.h
Should fix the Travis builds which actually generate a legacy tarball via
Automake.

Closes: #755
Approved by: jlebon
2017-03-23 21:58:55 +00:00
Colin Walters
94c47cde6a build: Various fixes for openssl build
This actually worked before because `pkg-config --cflags openssl`
is empty, and the linker was satisfying `-lssl -lcrypto` from our
indirect dependencies.

Also, in fact we *currently* just want `pkg-config libcrypto` i.e.
`-lcrypto`, since we aren't actually using openssl to speak TLS.

This doesn't actually matter on Fedora at least since they're both in the
`openssl-libs` package, but they are separate for a reason.

Closes: #749
Approved by: jlebon
2017-03-21 21:16:04 +00:00
Colin Walters
4ebe1ee94b build: Quiet automake warning for bupsplit
Use `_SRCS` and not `_SOURCES`, the latter of which makes automake assume
there's something for it to do.

Closes: #749
Approved by: jlebon
2017-03-21 21:16:04 +00:00
Colin Walters
741911d0fc lib: Exclude soup header from introspection
We should fix this more reliably by defining a separate variable for
introspection (and gtk-doc) sources at some point, but this does the trick for
now.

Avoids a spam of warnings, and it's definitely not public API.

Closes: #734
Approved by: jlebon
2017-03-13 15:20:45 +00:00
Georges Basile Stavracas Neto
fda4a47cae libostree: add versioning macros
OSTree currently provides no way to inspect the versioning
information at run time, being only available at compile
time through pkg-config.

This is a problem for e.g. Flatpak, that needs to check
whether the 'update-frequency' option is available. Checking
at compile time isn't great since it's not looking for new
symbols, but only if an optional feature is present.

This commit, then, adds a new header that is generated
at compile time, exposing OSTree's versioning information.

Closes: #728
Approved by: cgwalters
2017-03-11 15:38:56 +00:00
Colin Walters
ba350982e8 build: Remove .PHONY for Rust shared library
I have no idea why I made the lib `.PHONY` originally; it's clearly wrong, and I
noticed because when I was doing `sudo make install`, we were doing a rebuild,
which in turn triggered other things to be built, and they'd be owned by root.

Closes: #682
Approved by: jlebon
2017-02-14 14:27:39 +00:00
Colin Walters
361aa449fb libcurl backend
For rpm-ostree, we already link to libcurl indirectly via librepo, and
only having one HTTP library in process makes sense.

Further, libcurl is (I think) more popular in the embedded space.  It
also supports HTTP/2.0 today, which is a *very* nice to have for OSTree.

This seems to be working fairly well for me in my local testing, but it's
obviously brand new nontrivial code, so it's going to need some soak time.

The ugliest part of this is having to vendor in the soup-url code. With
Oxidation we could follow the path of Firefox and use the
[Servo URL parser](https://github.com/servo/rust-url).  Having to redo
cookie parsing also sucked, and that would also be a good oxidation target.

But that's for the future.

Closes: #641
Approved by: jlebon
2017-02-09 16:37:45 +00:00
Colin Walters
9169268c31 rust: Support make dist -> cargo vendor
What we do here basically is set things up in a `dist-hook` so that our Rust
sources are vendored at `dist` time. This gives us a single tarball still, and
ideally should be transparent to downstream builders, as long as they have the
`cargo/rust` toolchain.

Closes: #669
Approved by: jlebon
2017-02-07 16:50:37 +00:00
Jonathan Lebon
0400df75d1 trusted.gpg.d: keep in the same location
With the package rename from ostree to libostree, the trusted.gpg.d/ dir
changed install location from /usr/share/ostree to /usr/share/libostree.
Let's keep the same dir to remain compatible with existing installations
that may already have keys there.

Closes: #668
Approved by: cgwalters
2017-02-03 19:28:41 +00:00
Colin Walters
d894f609db oxidation: Add implementation of bupsplit in Rust
This is an initial drop of "oxidation", or adding implementation
of components in Rust.  The bupsplit code is a good target - no
dependencies, just computation.

Translation into Rust had a few twists -

 - The C code relies a lot on overflowing unsigned ints, and
   also on the C promotion rules for e.g. `uint8_t -> int32_t`
 - There were some odd loops that I introduced bugs in while
   translating...in particular, the function always returns `len`,
   but I mistakenly translated to `len+1`, resulting in an OOB
   read on the C side, which was hard to debug.

On the plus side, an off-by-one array indexing in the Rust code paniced nicely.

In practice, we'll need a lot more build infrastructure to make this work, such
as using `cargo vendor` when producing build artifacts for example. Also, Cargo
is yet another thing we need to cache.

Where do we go with this? Well, I think we should merge this, it's not a lot of
code. We can just have it be an alternative CI target. Should we do a lot more
right now? Probably not immediately, but I find the medium/long term prospects
pretty exciting!

Closes: #656
Approved by: jlebon
2017-02-03 14:29:00 +00:00
Simon McVittie
ce10fd9261 libostree: Don't distribute generated enumtypes in tarballs
They are built at "make" time and cleaned up by "make clean", so there
is no need to distribute them.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #665
Approved by: cgwalters
2017-01-31 22:30:37 +00:00
Colin Walters
9d0d0a26db fetcher: Move high level functions into "fetcher-util"
Conceptually these now lay on top of the core API, and don't reference libsoup.
This is preparation for libcurl porting, but it's also just generally better.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
5155662b57 build: Always do enum scanning now
Since we stopped including the libsoup headers in `ostree-fetcher.h`,
we can now unconditionally do enum scanning, and drop a build time
conditional.

Prep for libcurl porting.

Closes: #620
Approved by: jlebon
2016-12-08 16:34:06 +00:00
Colin Walters
7bb0ff46a4 Define and use cleanup helpers for libarchive
This should fix some of the ASAN leaks around libarchive usage,
and is generally better.

Closes: #609
Approved by: jlebon
2016-12-05 15:20:56 +00:00
Giuseppe Scrivano
8867693240 libostree: mark ostree_repo_checkout_tree_at as deprecated
and move its definition to a separate file.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Colin Walters
56e652035b build-sys: Make libostree-1.so depend on the symbol file
Otherwise one changing it doesn't cause a symbol to be exported.

Closes: #365
Approved by: jlebon
2016-06-24 21:06:56 +00:00
Colin Walters
9e2763106b lib: Use sd_journal directly (optionally)
This was the last caller of libgsystem that isn't
`gs_file_get_path_cached()`.  I think the use case ostree has where
the same code can be called via command line and via a shared library
*and* via a daemon is rather unusual, so let's just copy the code for
logging from libgsystem into here.

For example rpm-ostree hard depends on a daemon mode, so it'll just
use `sd_journal` directly.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Simon McVittie
19cfe2d73a build: distribute libostree.sym in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:03 +00:00
Mathnerd314
da5cb099a6 pull: Move libsoup-related code from ostree-repo.c to ostree-repo-pull.c
This centralizes the ifdef's in one file, which will make it
easier to write new pull backends.

ostree-repo-pull.c is now built unconditionally

Closes: #327
Approved by: cgwalters
2016-06-09 18:15:49 +00:00
Krzesimir Nowak
51e4e3b364 libostree: Add g_autoptr() support for the public types
To avoid breaking the ostree users we hide the autoptr setup behind an
OSTREE_WITH_AUTOCLEANUPS variable. The variable is set to 0 by
default, so users that want to start using them, have to define the
variable themselves to 1.

Probably after some transition period, the variable will be set 1 by
default.

This is conditional on GLib >= 2.44.0 being available. It does not
bump ostree's dependency on GLib.

Closes: #295
Approved by: cgwalters
2016-05-17 21:13:19 +00:00
Colin Walters
817060e772 build: Don't run glib-mkenums if there's nothing to do
This fixes the build without libsoup.

Closes: #296
Approved by: gatispaeglis
2016-05-12 17:15:50 +00:00
Colin Walters
a5f703799f Make enum generation private by default, export stub symbol
When we added enum type generation, the generated symbols used
the `ostree_` prefix, and at the time that implied they were public.

So we started (if built with libsoup) exporting
`ostree_fetcher_config_flags_get_type`.

I think it's not worth confusing ABI checkers, so let's export the
dummy symbol forever, and switch enums to be private by default.

We should revisit this and also export some auto-generated enum
types for public enums, but that's a separate patch.

Closes: #296
Approved by: gatispaeglis
2016-05-12 17:15:50 +00:00
Colin Walters
8894bb3949 deploy: Handle a read-only /boot
I'd like to encourage people to make OSTree-managed systems more
strictly read-only in multiple places.  Ideally everywhere is
read-only normally besides `/var/`, `/tmp/`, and `/run`.

`/boot` is a good example of something to make readonly.  Particularly
now that there's work on the `admin unlock` verb, we need to protect
the system better against things like `rpm -Uvh kernel.rpm` because
the RPM-packaged kernel won't understand how to do OSTree right.

In order to make this work of course, we *do* need to remount `/boot`
as writable when we're doing an upgrade that changes the kernel
configuration.  So the strategy is to detect whether it's read-only,
and if so, temporarily mount read-write, then remount read-only when
the upgrade is done.

We can generalize this in the future to also do `/etc` (and possibly
`/sysroot/ostree/` although that gets tricky).

One detail: In order to detect "is this path a mountpoint" is
nontrivial - I looked at copying the systemd code, but the right place
is to use `libmount` anyways.
2016-03-21 12:49:05 -04:00
Colin Walters
16979cc5ed lib: Introduce versioned symbols
As rpm-ostree evolves, it keeps driving API additions to libostree.
This creates a relatively tight coupling.

However, if delivering via e.g. RPM, unless one manually remembers to
increment the `Requires:` in the spec file, it's possible for the two
to become desynchronized.

RPM handles versioned symbols and will ensure a dependency if the
application starts using a newer version.

To implement this, switch to `-fvisibility=hidden`, along with an
annotation in the header, and finally add a `.sym` file.

This matches what other projects like systemd and libvirt do.

Although rather than attempting to retroactively version symbols, glom
them all onto the current one.
2016-03-01 21:45:26 -05:00
Giuseppe Scrivano
b0163d6182 build: 'make clean' removes parse-datetime.c
and fix make dist while at it.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2016-02-07 14:56:21 +01:00
Colin Walters
403e05af24 build: Fix srcdir != builddir
Caught by GContinuous.  Also change the file writes to be atomic,
otherwise we're not Ctrl-c safe.
2015-12-14 14:19:35 -05:00
Matthew Barnes
af30fc764a fetcher: Add "config-flags" construct-only property
A lot of effort here just to avoid touching SoupSession directly in
ostree_fetcher_new().  The reason will become apparent in subsequent
commits.

Note this introduces generated enum/flags GTypes using glib-mkenums.
I could have just made the property type as plain integer, but doing
properties right will henceforth be easier now that the automake-fu
is established.
2015-12-14 09:41:29 -05:00
Matthew Barnes
4f33515316 Juggling libglnx.h includes 2015-05-06 21:50:06 -04:00
Matthew Barnes
ceacc57206 libotutil: Establish a place for GPG utilities
Add ot-gpg-utils.[ch] and move _ostree_gpg_error_to_gio_error() here.
2015-05-01 10:20:34 -04:00
Colin Walters
64ff3444f8 build: ostree-gpg-verify-result.h is a public header, install it 2015-03-20 10:56:23 -04:00
Matthew Barnes
4a2733f9e7 gpg: Add OstreeGpgVerifyResult
Wrappers a referenced gpgme_verify_result_t so detailed verify results
can be examined independently of executing a verify operation.

_ostree_gpg_verifier_check_signature() now returns this object instead
of a single valid/invalid boolean, but the idea is for OstreeRepo to also
return this object for commit signature verification so it can be utilized
at the CLI layer (and possibly by other programs).
2015-03-18 11:52:22 -04:00
Matthew Barnes
187e8d632e configure.ac: Make gpgme a hard dependency
In anticipation of API enhancements for GPG signature verification, which
would otherwise require a non-functional stub version were GPGME excluded.

GPGME is a pretty lightweight dependency, and the motivation to exclude
it is not clear.
2015-03-11 12:03:33 -04:00
Matthew Barnes
70cabcea0a gpg: Rewrite OstreeGpgVerifier to use GPGME
This sets the stage for more advanced signature management.

(Also, talking to GPG over pipes sucks.)

Previously we were spawning gpgv2 with a bunch of --keyring options
for /usr/share/ostree/trusted.gpg.d/ and whatever other keyring files
were explicitly added.  GPGME has no public API for multiple keyrings,
so we work around the issue by setting up a temp directory to serve as
a fake "home" directory for the crypto engine and then concatenate all
the keyring files into a single public keyring (pubring.gpg).

Unfortunately at present we do this on every signature verification.
There's a desire to cache this concatenation, but the problem is the
user may be unprivileged.  So it seems the cache would have to be per
user under $XDG_CACHE_HOME, which OSTree doesn't otherwise use.  I'm
open to suggestions.

We do at least clean up the temp directory when finished, and I have
further API changes planned to OstreeGpgVerifier to help mitigate the
performance impact.
2015-03-06 08:22:44 -05:00
Giuseppe Scrivano
a705d9cf29 Add bsdiff submodule
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 12:16:17 -05:00
Giuseppe Scrivano
0a7a70ed67 build: build libbupsplit separately
It avoids gobject-introspection warnings:

src/libostree/bupsplit.h:42: Warning: OSTree: symbol='bupsplit_find_ofs': Unknown namespace for symbol 'bupsplit_find_ofs'
src/libostree/bupsplit.h:43: Warning: OSTree: symbol='bupsplit_selftest': Unknown namespace for symbol 'bupsplit_selftest'
src/libostree/bupsplit.h:33: Warning: OSTree: symbol='BUP_BLOBBITS': Unknown namespace for symbol 'BUP_BLOBBITS'
src/libostree/bupsplit.h:34: Warning: OSTree: symbol='BUP_BLOBSIZE': Unknown namespace for symbol 'BUP_BLOBSIZE'
src/libostree/bupsplit.h:35: Warning: OSTree: symbol='BUP_WINDOWBITS': Unknown namespace for symbol 'BUP_WINDOWBITS'
src/libostree/bupsplit.h:36: Warning: OSTree: symbol='BUP_WINDOWSIZE': Unknown namespace for symbol 'BUP_WINDOWSIZE'

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-23 17:07:13 -05:00
Matthew Barnes
12c8ef1425 Add explicit zlib dependency
Needed for CRC32 computations.
2015-02-23 14:48:52 -05:00
Colin Walters
965a304a17 Use libglnx
Starting down the path of not using libgsystem.  The main win here
will be code sharing between ostree/rpm-ostree as well as going down
the path of not using GFile * for local files.
2015-02-22 21:02:27 -05:00
Colin Walters
c54df89771 deltas: Search for similar objects (possibly renamed across directories)
The previous diff algorithm was file tree based, and only looked
at modified files that lived at the same path.

However, components like the Linux kernel have versioned
subdirectories, e.g. /usr/lib/modules/$kver/.../ext4.ko.  We want to
be able to detect these "modified renames" so that we can compute
diffs (rollsum, bsdiff).
2015-02-16 10:10:35 -05:00
Colin Walters
9aa7e30b38 deltas: Implement rollsums
This does an rsync-style prepared delta basically.  On my test data,
it shaves ~6MB of uncompressed data.  Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
2015-02-16 10:10:35 -05:00
Colin Walters
3df8be0d92 deltas: Compute rollsum targets 2015-02-16 10:10:35 -05:00
Giuseppe Scrivano
7ebf7676cb Fix make distcheck
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-30 15:27:36 +01:00
Matthew Barnes
a4d904ee38 build: Include SELinux cflags with libostree build too
This is likely another issue with newer automake.

https://bugzilla.gnome.org/738875
2014-10-20 15:23:55 -04:00
Colin Walters
d546abfa2a libostree: Add initial GRUB2 support
In this approach, we drop a /etc/grub.d/15_ostree file which is a
hybrid of shell/C that picks up bits from the GRUB2 library (e.g. the
block device script generation), and then calls into libostree's
GRUB2 code which knows about the BLS entries.

This is admittedly ugly.  There exists another approach for GRUB2 to
learn the BLS specification.  However, the spec has a few issues:

https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html

This approach also gives a bit more control to the admin via the
naming of the 15_ostree symlink; they can easily disable it:

Or reorder the ostree entries ahead of 10_linux:

Also, this approach doesn't require patches for grub2, which is an
issue with the pressure to backport (rpm-)OSTree to EL7.
2014-10-16 14:15:00 -04:00
Colin Walters
1242704d68 build: Unify CPPFLAGS settings
The libostree core uses SYSCONFDIR now, so we should ensure it's used
consistently.  Someone else was seeing SYSCONFDIR not being defined
while compiling with a newer automake version, which may process
CPPFLAGS more precisely.
2014-09-08 11:47:58 -04:00
Colin Walters
f8f5da219e Add repository "summary" file and metalink support
For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.

In order to make metalink work though, it needs *one* file which can
be checksummed.  (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).

This commit adds a new command:

$ ostree summary -u

To regenerate the summary file.  Can only be run by one process at a
time.

After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.

https://bugzilla.gnome.org/show_bug.cgi?id=729585
2014-09-03 13:21:52 -04:00
Colin Walters
b4d21e9406 deploy: Set the immutable bit on the deployment root
This prevents people from creating new directories there and expecting
them to be persisted.  The OSTree model has all local state to be in
/etc and /var.

This introduces a compile-time dependency on libe2fsprogs.

We're only doing this for the root directory at the moment.

https://bugzilla.gnome.org/show_bug.cgi?id=728006
2014-06-03 13:41:48 -04:00