Commit Graph

188 Commits

Author SHA1 Message Date
Jonathan Lebon
75342035d5 Makefile-libostree.am: Uncomment BUILDOPT_IS_DEVEL_BUILD conditional
We shouldn't have to toggle the conditional itself during release
builds. It should only evaluate to true during devel builds.
2020-09-25 15:07:36 -04:00
Frédéric Danis
02a19b2c96 lib/deltas: Add signature check API for static-delta superblock
This retrieves the signatures and pass the static delta block as an array
of bytes to ostree_sign_data_verify().

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Colin Walters
0d91206a62 Post-release version bump 2020-07-22 14:09:26 +00:00
Colin Walters
1b8fed247c lib: Move gpg/signapi bits into ostree-repo-pull-verify.c
`ostree-repo-pull.c` is huge; separate some of the GPG/signing
verification functions into their own file so they're more easily seen.
2020-04-18 13:35:36 +00:00
Denis Pynkin
edbbe1c4f2 lib/sign: initial implementation
Added the initial version of signing interface allowing to allowing to
sign and verify commits.
Implemented initial signing modules:
 - dummy -- simple module allowing to sign/verify with ASCII string
 - ed25519 -- module allowing to sign/verify commit with ed25519
   (EdDSA) signature scheme provided by libsodium library.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-03-25 15:23:54 +03:00
Alex Kiernan
562b60022b build: fix systemd feature advertisement
17db0f15a7 ("configure: add option for libsystemd") exposed
--without-libsystemd to allow systemd to be disabled even if the systemd
pkgconfig script was present, introducing a new variable
with_libsystemd; there are now three, almost identical variables:

- with_libsystemd [yes, no, maybe] - controlled by --without-libsystemd,
  resolved into yes/no by the initial checks
- have_libsystemd [yes, no, <undefined>] - only set if with_libsystemd
  is yes/maybe, otherwise undefined
- with_systemd [yes, <undefined>] - yes if have_systemd is yes,
  otherwise undefined

with_systemd is the earliest variable and was previously set by a set of
checks for dracut and mkinitcpio. These checks were changed for a
systemd check in 9e2763106b ("lib: Use sd_journal directly
(optionally)"). This commit also introduced BUILDOPT_LIBSYSTEMD, which
will always match BUILDOPT_SYSTEMD.

Fix the confusion by removing with_systemd which will always be yes when
with_libsystemd=yes, or undefined if with_libsystemd=no. We can ignore
the with_libsystemd=maybe case because it will always be resolved into
yes/no before with_systemd is set.

And replace all uses of BUILDOPT_SYSTEMD with BUILDOPT_LIBSYSTEMD, since
they again always match.

This fixes both the advertised features and the summary output when
systemd is disabled by using with_libsystemd which is always defined.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Fixes: 5c62a7e4d0 ("build: Expose systemd in OSTREE_FEATURES")
Fixes: 17db0f15a7 ("configure: add option for libsystemd")
Supersedes: #1992
2020-01-25 13:12:41 +00:00
Colin Walters
c61234a428 bootloader: Add a zipl bootloader backend
zipl is a bit special in that it parses the BLS config files
directly *but* we need to run the command to update the "boot block".

Hence, we're not generating a separate config file like the other
backends.  Instead, extend the bootloader interface with a `post_bls_sync`
method that is run in the same place we swap the `boot/loader` symlink.

We write a "stamp file" in `/boot` that says we need to run this command.
The reason we use stamp file is to prevent the case where the system is
interrupted after BLS file is updated, but before zipl is triggered,
then zipl boot records are not updated.
This opens the door to making things eventually-consistent/reconcilable
by later adding a systemd unit to run `zipl` if we're interrupted via
a systemd unit - I think we should eventually take this approach
everywhere rather than requiring `/boot/loader` to be a symlink.

Author: Colin Walters <walters@verbum.org>
Tested-by: Tuan Hoang <tmhoang@linux.ibm.com>
Co-Authored-By: Tuan Hoang <tmhoang@linux.ibm.com>
2019-10-16 14:22:02 +00:00
Denis Pynkin
8322e77a88 gpg: add dummy public interface
Add dummy stubs for GPG public functions to be compiled instead of
original code in case if support of GPG is disabled.
Need that to keep API backward compatibility.

Based on original code from file `ostree-gpg-verify-result.c`.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Denis Pynkin
ca8752e01b build: Conditionally build GPGME-related sources
Do not build GPGME-related sources if flag USE_GPGME is not defined.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>

Closes: #1889
Approved by: cgwalters
2019-08-01 02:06:47 +00:00
Robert Fairley
fd8795f387 Makefile-libostree.am: Work around clang issues with g-ir-scanner
Work around an issue with clang-7 in Fedora 29 where the
`-fstack-clash-protection` flag is not recognized. Force
the compiler used by `g-ir-scanner` to `gcc`, so that the flag
is recognized.

Related: https://github.com/projectatomic/rpm-ostree/pull/1787#issuecomment-473971585

Closes: #1871
Approved by: jlebon
2019-06-21 13:17:10 +00:00
Allen Bai
be2572bf68 lib/kargs: Make API public and upstream new rpm-ostree APIs
This change makes public the current kargs API in src/libostree/ostree-kernel-args.c
and adds documentations.

Upstreams the new kargs API from rpm-ostree/src/libpriv/rpmostree-kargs-process.c

Merges libostree_kernel_args_la_SOURCES to libostree_1_la_SOURCES in Makefile-libostree.am

Upstreams tests/check/test-kargs.c from rpm-ostree.

Closes: #1833

Closes: #1869
Approved by: jlebon
2019-06-18 14:29:14 +00:00
Marcus Folkesson
8af389b758 build: add ostree-soup-* to build process when configured with avahi
Avoid getting these link errors:
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_path'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_new'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_free'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_scheme'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_to_string'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_host'
    ./.libs/libostree-1.so: undefined reference to `soup_uri_set_port'
    collect2: error: ld returned 1 exit status

Reproduce with:
    ./configure --with-avahi --without-soup

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1666
Approved by: cgwalters
2018-07-03 19:42:49 +00:00
Matthew Leeds
8fbf19c9f5 Make P2P API public (no longer experimental)
Currently the API that allows P2P operations (e.g. pulling an ostree ref
from a LAN or USB source) is hidden behind the configure flag
--enable-experimental-api. This commit makes the API public and makes
that flag essentially a no-op (leaving it in place in case we want to
use it again in the future). The P2P API has been tested over the last
several months and proven to work.

This means that since we're no longer using the "experimental" feature
flag, P2P builds of Flatpak will fail when using versions of OSTree from
this commit onwards, until Flatpak is patched in the near future. If you
want to build Flatpak < 0.11.8 with P2P enabled and link against OSTree
2018.6, you'll have to patch Flatpak.  However, since Flatpak won't yet
have a hard dependency on OSTree 2018.6, it needs a new way to determine
if the P2P API in OSTree is available, so this commit adds a "p2p"
feature flag. This way the feature set is more semantically correct than
if we had continued to use the "experimental" feature flag.

In addition to making the P2P API public, this commit makes the P2P unit
tests run by default, removes the f27-experimental CI instance that's no
longer needed, changes a few man pages to reflect the changes, and
updates the bash completion script to accept the new commands and
options.

Closes: #1596
Approved by: cgwalters
2018-06-04 19:20:10 +00:00
Marcus Folkesson
6bf4b3e1d8 Add SPDX-License-Identifier to source files
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.

Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
  text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
  environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
  license for their code is respected

See http://spdx.org for further reading.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1439
Approved by: cgwalters
2018-01-30 20:03:42 +00:00
Colin Walters
1f832597fc build-sys: Link with -ldl for rust build
I didn't dive into this too much, it looks like something in rust changed that
broke our build. Probably libstd gained a dependency on `-ldl` or so, and that's
handled by cargo? Anyways linking against it isn't going to hurt.

Closes: #1391
Approved by: smcv
2018-01-04 12:23:06 +00:00
Philip Withnall
1b7d83114e lib/pull: Split verify_bindings() out into a cmdprivate method
It will be used by the fsck utility in future. We could expose it
publicly in future too, if needed.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1347
Approved by: cgwalters
2017-12-14 14:18:44 +00:00
Philip Withnall
7d2ad351c4 build: Add a TODO comment about improving glib-mkenums usage in future
Signed-off-by: Philip Withnall <withnall@endlessm.com>

https://github.com/ostreedev/ostree/pull/1329

Closes: #1330
Approved by: cgwalters
2017-11-07 15:14:56 +00:00
Dan Nicholson
69c4737491 build: Define OSTREE_ENABLE_EXPERIMENTAL_API for g-ir-scanner
When compiling libostree, OSTREE_ENABLE_EXPERIMENTAL_API is managed via
config.h. However, g-ir-scanner can't use that since it gets confused
about the namespace of all the random macros. It won't include the
experimental APIs unless the macro is defined through another means.
Without this, none of the experimental APIs were being included in the
gir data.

Closes: #1322
Approved by: pwithnall
2017-11-03 22:05:47 +00:00
Dan Nicholson
6ec4cd3eb5 build: Ensure enumtypes.h is built before enumtypes.c
ostree-enumtypes.c includes ostree-enumtypes.h, so make needs to be told
about the dependency. Without it, parallel make could try to build
ostree-enumtypes.c before the header file exists.

I hit this when running `make -j OSTree-1.0.gir`.

Closes: #1322
Approved by: pwithnall
2017-11-03 22:05:47 +00:00
Philip Withnall
2531d8fe63 lib/repo-finder: Add OstreeRepoFinderOverride
This is another OstreeRepoFinder implementation; it returns results from
a given set of URIs. It’s designed to be used for implementing user
overrides to other repo-finders, or for implementing unit tests.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1281
Approved by: mwleeds
2017-10-19 19:11:58 +00:00
Jussi Laako
f91acf5226 Add --with-crypto=gnutls
Introduce support for GnuTLS for computing cryptograpic
hashes, similar to the OpenSSL backend.  A reason to do
this is some distributors want to avoid GPLv3, and GPG
pulls that in.

A possible extension of using GnuTLS would be replacing the GPG signing
with `PKCS#7` signatures and `X.509` keys.

We also support `--with-crypto=openssl`, which has the same effect
as `--with-openssl`, and continues to be supported.

Changes by Colin Walters <walters@verbum.org>:

 - Drop libgcrypt option for now
 - Unify buildsystem on --with-crypto

Link: https://mail.gnome.org/archives/ostree-list/2017-June/msg00002.html

Signed-off-by: Jussi Laako <jussi.laako@linux.intel.com>

Closes: #1189
Approved by: cgwalters
2017-09-25 12:58:54 +00:00
Emmanuele Bassi
47a54bf876 Move the include directive to the enum template
There is no actual written guarantee in glib-mkenums that the template
line specified using --fhead will be added after the templates specified
inside the template file. Since the template file is only used once, we
can simply move the `#include` directive inside the template, so that it
is guaranteed to be in the right place.

Closes: #1007
Approved by: cgwalters
2017-07-17 15:14:44 +00:00
Philip Withnall
f6ba20d9d6 build: Ensure all .sym files are distributed in tarballs
Since we’re using a custom variable for listing the .sym files,
automake’s magic support for automatically distributing all files in
conditionals doesn’t work, and the devel and experimental .sym files
were only being distributed if `make dist` was run on a source tree
which had been configured with --enable-experimental-api or not a
release flag.

Fix that by explicitly listing all the .sym files in EXTRA_DIST.

Specifically, this fixes the case of trying to compile with
--enable-experimental-api from a release tarball which was disted with
--disable-experimental-api.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #1001
Approved by: cgwalters
2017-07-12 13:04:32 +00:00
Colin Walters
54db9ecab3 build: Don't scan ostree-remote.h for introspection if !experimental-api
It's a bit unfortunate that the scanner doesn't error on this, but
anyways

Closes: https://github.com/ostreedev/ostree/issues/966

Closes: #967
Approved by: jlebon
2017-06-27 17:33:08 +00:00
Colin Walters
21cb4d1715 ci: Make introspection warnings fatal
Closes: #965
Approved by: pwithnall
2017-06-27 11:56:44 +00:00
Philip Withnall
e3d4eeacbc lib/repo-finder: Add Avahi based OstreeRepoFinder implementation
This is a more complex implementation of OstreeRepoFinder which resolves
ref names to remote URIs by looking for refs advertised by peers on the
local network using DNS-SD records and mDNS (Avahi). The idea is to
allow OS and app updates to be propagated over local networks, without
the internet.

It requires an OSTree server and code to generate the DNS-SD adverts in
order to be fully functional — support for this will be added
separately.

Unit tests are included.

Includes fixes by Krzesimir Nowak <krzesimir@kinvolk.io>.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
7ee4e1295a lib/bloom: Add an internal bloom filter implementation
This will be used in an upcoming commit. It adds a basic bloom filter
implementation, using the SipHash family of hash functions.

The implementation (including its parameter choices and hash functions)
will become a protocol detail in future, so must not be changed so that
its output is bitwise incompatible between OSTree versions.

Unit tests are included.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
ae335f24dc lib/repo-finder: Add mount based OstreeRepoFinder implementation
This is a basic implementation of OstreeRepoFinder which resolves ref
names to remote URIs by looking for them on any currently mounted
removable storage volumes. The idea is to support OS and app updates via
USB stick.

Unit tests are included.

This bumps libostree’s maximum GLib dependency from 2.44 to 2.50 for
g_drive_is_removable(). If GLib 2.50 is not available, the call which
needs it will be omitted and the OstreeRepoFinderMount implementation
will scan all volumes (not just removable ones); this is a performance
hit, but not a functionality hit.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
d15f83c922 lib/repo-finder: Add config-file based OstreeRepoFinder implementation
This is a basic implementation of OstreeRepoFinder which resolves ref
names to remote URIs by looking their collection IDs up in the local
configuration of remotes who have their collection-id key set.

Unit tests are included.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
292230301d lib/repo-finder: Add basic support for finding remote URIs by ref name
Add an initial OstreeRepoFinder interface (but no implementations),
which will find remote URIs by ref names and collection IDs, the
combination of which is globally unique.

The new API is used in a new ostree_repo_find_updates() function, which
resolves a list of ref names to update into a set of remote URIs to pull
them from, which can be treated as mirrors. It is an attempt to
generalise resolution of the URIs to pull from, and to generalise
determination of the order and parallelisation which they should be
downloaded from in.

Includes fixes by Krzesimir Nowak <krzesimir@kinvolk.io>.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
0a20e7d43c lib/ref: Add OstreeCollectionRef type for globally unique refs
This is a type representing the tuple (collection ID, ref name), which is
guaranteed to be globally unique. It will be used in upcoming commits.

It introduces the concept of a ‘collection’ which is a unique, curated
set of refs which lie in the same trust domain (i.e. all signed by the
same key and validated by the same developer). Flathub might be a
collection, for example; or the set of OS refs coming from a particular
OS vendor.

It includes a function for validating collection IDs.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Colin Walters
a45dc0fd0b build-sys: Add "release build" flag, use for symbol versioning
I was trying to do a release and move the symbols from `-devel.sym` into
`-release.sym`, but it turns out that at least GNU binutils `ld` treats an empty
version script as a syntax error.  Fix this by adding a "release build"
flag, and only include `-devel` in non-release builds.

This would also make it easier to inject that flag into our `.pc` and
`ostree-version.h` and `ostree --version` metadata, but I didn't do that yet.

EDIT: Turns out a simpler fix is just to add an empty section. However I kept
this commit since it's a useful sanity check for whether we should include
`-devel.sym` in builds, and we may want to inject the metadata later.

Closes: #942
Approved by: jlebon
2017-06-19 14:54:59 +00:00
Colin Walters
2bab43fb22 lib: Split symbol versioning into -released and -devel
So far a lot of submitted PR have added symbols into the first
section.  Split the file into `-released` and `-devel` to make
this more obvious.

To further enforce things, we hardcode a checksum of the `-released`
file in `test-symbols.sh`.  Only release commits should update that
checksum.

Did you notice I like checksums?

Closes: #931
Approved by: pwithnall
2017-06-16 09:31:16 +00:00
Colin Walters
30705889cb Switch to using a systemd generator for /var
If one wants to set up a mount for `/var` in `/etc/fstab`, it
won't be mounted since `ostree-prepare-root` set up a bind mount for
`/var` to `/sysroot/ostree/$stateroot/var`, and systemd will take
the already extant mount over what's in `/etc/fstab`.

There are a few options to fix this, but what I settled on is parsing
`/etc/fstab` in a generator (exactly like `systemd-fstab-generator` does),
except here we look for an explicit mount for `/var`, and if one *isn't* found,
synthesize the default ostree mount to the stateroot. Another nice property is
that if an admin creates a `var.mount` unit in `/etc` for example, that will
also override our mount.

Note that today ostree doesn't hard depend on systemd, so this behavior only
kicks in if we're built with systemd *and* libmount support (for parsing
`/etc/fstab`).  I didn't really test that case though.

Initially I started writing this as a "pure libc" program, but at one point
decided to use `libostree.so` to find the booted deployment. That didn't work
out because `/boot` wasn't necessarily mounted and hence we couldn't find the
bootloader config. A leftover artifact from this is that the generator code
calls into libostree via the "cmd private" infrastructure. But it's an easy way
to share code, and doesn't hurt.

Closes: #859
Approved by: jlebon
2017-05-16 16:13:05 +00:00
Philip Withnall
6eac575f21 libostree: Make OstreeRemote a public and internal API
Previously it was static to ostree-repo.c. Make it usable throughout
libostree so it can be used by an upcoming commit, but also expose the
typedef and reference counting functions so that opaque OstreeRemote
pointers can be used by user code, in anticipation of exposing more of
its API publicly in future.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Philip Withnall
c9244b1bb2 build: Add --enable-experimental-api configure option for unstable APIs
There are currently no unstable APIs, but some will be added in
following commits. They will be built and exposed in the libostree
global symbol list iff configured with --enable-experimental-api.

Distributions should not package OSTree with --enable-experimental-api.
This is designed for previewing new APIs on controlled platforms; any of
the APIs hidden behind this option may be changed or removed at any
point.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Colin Walters
ffc178311e build: Dist ostree-sepolicy-private.h
Should fix the Travis builds which actually generate a legacy tarball via
Automake.

Closes: #755
Approved by: jlebon
2017-03-23 21:58:55 +00:00
Colin Walters
94c47cde6a build: Various fixes for openssl build
This actually worked before because `pkg-config --cflags openssl`
is empty, and the linker was satisfying `-lssl -lcrypto` from our
indirect dependencies.

Also, in fact we *currently* just want `pkg-config libcrypto` i.e.
`-lcrypto`, since we aren't actually using openssl to speak TLS.

This doesn't actually matter on Fedora at least since they're both in the
`openssl-libs` package, but they are separate for a reason.

Closes: #749
Approved by: jlebon
2017-03-21 21:16:04 +00:00
Colin Walters
4ebe1ee94b build: Quiet automake warning for bupsplit
Use `_SRCS` and not `_SOURCES`, the latter of which makes automake assume
there's something for it to do.

Closes: #749
Approved by: jlebon
2017-03-21 21:16:04 +00:00
Colin Walters
741911d0fc lib: Exclude soup header from introspection
We should fix this more reliably by defining a separate variable for
introspection (and gtk-doc) sources at some point, but this does the trick for
now.

Avoids a spam of warnings, and it's definitely not public API.

Closes: #734
Approved by: jlebon
2017-03-13 15:20:45 +00:00
Georges Basile Stavracas Neto
fda4a47cae libostree: add versioning macros
OSTree currently provides no way to inspect the versioning
information at run time, being only available at compile
time through pkg-config.

This is a problem for e.g. Flatpak, that needs to check
whether the 'update-frequency' option is available. Checking
at compile time isn't great since it's not looking for new
symbols, but only if an optional feature is present.

This commit, then, adds a new header that is generated
at compile time, exposing OSTree's versioning information.

Closes: #728
Approved by: cgwalters
2017-03-11 15:38:56 +00:00
Colin Walters
ba350982e8 build: Remove .PHONY for Rust shared library
I have no idea why I made the lib `.PHONY` originally; it's clearly wrong, and I
noticed because when I was doing `sudo make install`, we were doing a rebuild,
which in turn triggered other things to be built, and they'd be owned by root.

Closes: #682
Approved by: jlebon
2017-02-14 14:27:39 +00:00
Colin Walters
361aa449fb libcurl backend
For rpm-ostree, we already link to libcurl indirectly via librepo, and
only having one HTTP library in process makes sense.

Further, libcurl is (I think) more popular in the embedded space.  It
also supports HTTP/2.0 today, which is a *very* nice to have for OSTree.

This seems to be working fairly well for me in my local testing, but it's
obviously brand new nontrivial code, so it's going to need some soak time.

The ugliest part of this is having to vendor in the soup-url code. With
Oxidation we could follow the path of Firefox and use the
[Servo URL parser](https://github.com/servo/rust-url).  Having to redo
cookie parsing also sucked, and that would also be a good oxidation target.

But that's for the future.

Closes: #641
Approved by: jlebon
2017-02-09 16:37:45 +00:00
Colin Walters
9169268c31 rust: Support make dist -> cargo vendor
What we do here basically is set things up in a `dist-hook` so that our Rust
sources are vendored at `dist` time. This gives us a single tarball still, and
ideally should be transparent to downstream builders, as long as they have the
`cargo/rust` toolchain.

Closes: #669
Approved by: jlebon
2017-02-07 16:50:37 +00:00
Jonathan Lebon
0400df75d1 trusted.gpg.d: keep in the same location
With the package rename from ostree to libostree, the trusted.gpg.d/ dir
changed install location from /usr/share/ostree to /usr/share/libostree.
Let's keep the same dir to remain compatible with existing installations
that may already have keys there.

Closes: #668
Approved by: cgwalters
2017-02-03 19:28:41 +00:00
Colin Walters
d894f609db oxidation: Add implementation of bupsplit in Rust
This is an initial drop of "oxidation", or adding implementation
of components in Rust.  The bupsplit code is a good target - no
dependencies, just computation.

Translation into Rust had a few twists -

 - The C code relies a lot on overflowing unsigned ints, and
   also on the C promotion rules for e.g. `uint8_t -> int32_t`
 - There were some odd loops that I introduced bugs in while
   translating...in particular, the function always returns `len`,
   but I mistakenly translated to `len+1`, resulting in an OOB
   read on the C side, which was hard to debug.

On the plus side, an off-by-one array indexing in the Rust code paniced nicely.

In practice, we'll need a lot more build infrastructure to make this work, such
as using `cargo vendor` when producing build artifacts for example. Also, Cargo
is yet another thing we need to cache.

Where do we go with this? Well, I think we should merge this, it's not a lot of
code. We can just have it be an alternative CI target. Should we do a lot more
right now? Probably not immediately, but I find the medium/long term prospects
pretty exciting!

Closes: #656
Approved by: jlebon
2017-02-03 14:29:00 +00:00
Simon McVittie
ce10fd9261 libostree: Don't distribute generated enumtypes in tarballs
They are built at "make" time and cleaned up by "make clean", so there
is no need to distribute them.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #665
Approved by: cgwalters
2017-01-31 22:30:37 +00:00
Colin Walters
9d0d0a26db fetcher: Move high level functions into "fetcher-util"
Conceptually these now lay on top of the core API, and don't reference libsoup.
This is preparation for libcurl porting, but it's also just generally better.

Closes: #636
Approved by: jlebon
2017-01-04 16:32:11 +00:00
Colin Walters
5155662b57 build: Always do enum scanning now
Since we stopped including the libsoup headers in `ostree-fetcher.h`,
we can now unconditionally do enum scanning, and drop a build time
conditional.

Prep for libcurl porting.

Closes: #620
Approved by: jlebon
2016-12-08 16:34:06 +00:00
Colin Walters
7bb0ff46a4 Define and use cleanup helpers for libarchive
This should fix some of the ASAN leaks around libarchive usage,
and is generally better.

Closes: #609
Approved by: jlebon
2016-12-05 15:20:56 +00:00
Giuseppe Scrivano
8867693240 libostree: mark ostree_repo_checkout_tree_at as deprecated
and move its definition to a separate file.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Colin Walters
56e652035b build-sys: Make libostree-1.so depend on the symbol file
Otherwise one changing it doesn't cause a symbol to be exported.

Closes: #365
Approved by: jlebon
2016-06-24 21:06:56 +00:00
Colin Walters
9e2763106b lib: Use sd_journal directly (optionally)
This was the last caller of libgsystem that isn't
`gs_file_get_path_cached()`.  I think the use case ostree has where
the same code can be called via command line and via a shared library
*and* via a daemon is rather unusual, so let's just copy the code for
logging from libgsystem into here.

For example rpm-ostree hard depends on a daemon mode, so it'll just
use `sd_journal` directly.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Simon McVittie
19cfe2d73a build: distribute libostree.sym in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:03 +00:00
Mathnerd314
da5cb099a6 pull: Move libsoup-related code from ostree-repo.c to ostree-repo-pull.c
This centralizes the ifdef's in one file, which will make it
easier to write new pull backends.

ostree-repo-pull.c is now built unconditionally

Closes: #327
Approved by: cgwalters
2016-06-09 18:15:49 +00:00
Krzesimir Nowak
51e4e3b364 libostree: Add g_autoptr() support for the public types
To avoid breaking the ostree users we hide the autoptr setup behind an
OSTREE_WITH_AUTOCLEANUPS variable. The variable is set to 0 by
default, so users that want to start using them, have to define the
variable themselves to 1.

Probably after some transition period, the variable will be set 1 by
default.

This is conditional on GLib >= 2.44.0 being available. It does not
bump ostree's dependency on GLib.

Closes: #295
Approved by: cgwalters
2016-05-17 21:13:19 +00:00
Colin Walters
817060e772 build: Don't run glib-mkenums if there's nothing to do
This fixes the build without libsoup.

Closes: #296
Approved by: gatispaeglis
2016-05-12 17:15:50 +00:00
Colin Walters
a5f703799f Make enum generation private by default, export stub symbol
When we added enum type generation, the generated symbols used
the `ostree_` prefix, and at the time that implied they were public.

So we started (if built with libsoup) exporting
`ostree_fetcher_config_flags_get_type`.

I think it's not worth confusing ABI checkers, so let's export the
dummy symbol forever, and switch enums to be private by default.

We should revisit this and also export some auto-generated enum
types for public enums, but that's a separate patch.

Closes: #296
Approved by: gatispaeglis
2016-05-12 17:15:50 +00:00
Colin Walters
8894bb3949 deploy: Handle a read-only /boot
I'd like to encourage people to make OSTree-managed systems more
strictly read-only in multiple places.  Ideally everywhere is
read-only normally besides `/var/`, `/tmp/`, and `/run`.

`/boot` is a good example of something to make readonly.  Particularly
now that there's work on the `admin unlock` verb, we need to protect
the system better against things like `rpm -Uvh kernel.rpm` because
the RPM-packaged kernel won't understand how to do OSTree right.

In order to make this work of course, we *do* need to remount `/boot`
as writable when we're doing an upgrade that changes the kernel
configuration.  So the strategy is to detect whether it's read-only,
and if so, temporarily mount read-write, then remount read-only when
the upgrade is done.

We can generalize this in the future to also do `/etc` (and possibly
`/sysroot/ostree/` although that gets tricky).

One detail: In order to detect "is this path a mountpoint" is
nontrivial - I looked at copying the systemd code, but the right place
is to use `libmount` anyways.
2016-03-21 12:49:05 -04:00
Colin Walters
16979cc5ed lib: Introduce versioned symbols
As rpm-ostree evolves, it keeps driving API additions to libostree.
This creates a relatively tight coupling.

However, if delivering via e.g. RPM, unless one manually remembers to
increment the `Requires:` in the spec file, it's possible for the two
to become desynchronized.

RPM handles versioned symbols and will ensure a dependency if the
application starts using a newer version.

To implement this, switch to `-fvisibility=hidden`, along with an
annotation in the header, and finally add a `.sym` file.

This matches what other projects like systemd and libvirt do.

Although rather than attempting to retroactively version symbols, glom
them all onto the current one.
2016-03-01 21:45:26 -05:00
Giuseppe Scrivano
b0163d6182 build: 'make clean' removes parse-datetime.c
and fix make dist while at it.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2016-02-07 14:56:21 +01:00
Colin Walters
403e05af24 build: Fix srcdir != builddir
Caught by GContinuous.  Also change the file writes to be atomic,
otherwise we're not Ctrl-c safe.
2015-12-14 14:19:35 -05:00
Matthew Barnes
af30fc764a fetcher: Add "config-flags" construct-only property
A lot of effort here just to avoid touching SoupSession directly in
ostree_fetcher_new().  The reason will become apparent in subsequent
commits.

Note this introduces generated enum/flags GTypes using glib-mkenums.
I could have just made the property type as plain integer, but doing
properties right will henceforth be easier now that the automake-fu
is established.
2015-12-14 09:41:29 -05:00
Matthew Barnes
4f33515316 Juggling libglnx.h includes 2015-05-06 21:50:06 -04:00
Matthew Barnes
ceacc57206 libotutil: Establish a place for GPG utilities
Add ot-gpg-utils.[ch] and move _ostree_gpg_error_to_gio_error() here.
2015-05-01 10:20:34 -04:00
Colin Walters
64ff3444f8 build: ostree-gpg-verify-result.h is a public header, install it 2015-03-20 10:56:23 -04:00
Matthew Barnes
4a2733f9e7 gpg: Add OstreeGpgVerifyResult
Wrappers a referenced gpgme_verify_result_t so detailed verify results
can be examined independently of executing a verify operation.

_ostree_gpg_verifier_check_signature() now returns this object instead
of a single valid/invalid boolean, but the idea is for OstreeRepo to also
return this object for commit signature verification so it can be utilized
at the CLI layer (and possibly by other programs).
2015-03-18 11:52:22 -04:00
Matthew Barnes
187e8d632e configure.ac: Make gpgme a hard dependency
In anticipation of API enhancements for GPG signature verification, which
would otherwise require a non-functional stub version were GPGME excluded.

GPGME is a pretty lightweight dependency, and the motivation to exclude
it is not clear.
2015-03-11 12:03:33 -04:00
Matthew Barnes
70cabcea0a gpg: Rewrite OstreeGpgVerifier to use GPGME
This sets the stage for more advanced signature management.

(Also, talking to GPG over pipes sucks.)

Previously we were spawning gpgv2 with a bunch of --keyring options
for /usr/share/ostree/trusted.gpg.d/ and whatever other keyring files
were explicitly added.  GPGME has no public API for multiple keyrings,
so we work around the issue by setting up a temp directory to serve as
a fake "home" directory for the crypto engine and then concatenate all
the keyring files into a single public keyring (pubring.gpg).

Unfortunately at present we do this on every signature verification.
There's a desire to cache this concatenation, but the problem is the
user may be unprivileged.  So it seems the cache would have to be per
user under $XDG_CACHE_HOME, which OSTree doesn't otherwise use.  I'm
open to suggestions.

We do at least clean up the temp directory when finished, and I have
further API changes planned to OstreeGpgVerifier to help mitigate the
performance impact.
2015-03-06 08:22:44 -05:00
Giuseppe Scrivano
a705d9cf29 Add bsdiff submodule
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-03 12:16:17 -05:00
Giuseppe Scrivano
0a7a70ed67 build: build libbupsplit separately
It avoids gobject-introspection warnings:

src/libostree/bupsplit.h:42: Warning: OSTree: symbol='bupsplit_find_ofs': Unknown namespace for symbol 'bupsplit_find_ofs'
src/libostree/bupsplit.h:43: Warning: OSTree: symbol='bupsplit_selftest': Unknown namespace for symbol 'bupsplit_selftest'
src/libostree/bupsplit.h:33: Warning: OSTree: symbol='BUP_BLOBBITS': Unknown namespace for symbol 'BUP_BLOBBITS'
src/libostree/bupsplit.h:34: Warning: OSTree: symbol='BUP_BLOBSIZE': Unknown namespace for symbol 'BUP_BLOBSIZE'
src/libostree/bupsplit.h:35: Warning: OSTree: symbol='BUP_WINDOWBITS': Unknown namespace for symbol 'BUP_WINDOWBITS'
src/libostree/bupsplit.h:36: Warning: OSTree: symbol='BUP_WINDOWSIZE': Unknown namespace for symbol 'BUP_WINDOWSIZE'

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-23 17:07:13 -05:00
Matthew Barnes
12c8ef1425 Add explicit zlib dependency
Needed for CRC32 computations.
2015-02-23 14:48:52 -05:00
Colin Walters
965a304a17 Use libglnx
Starting down the path of not using libgsystem.  The main win here
will be code sharing between ostree/rpm-ostree as well as going down
the path of not using GFile * for local files.
2015-02-22 21:02:27 -05:00
Colin Walters
c54df89771 deltas: Search for similar objects (possibly renamed across directories)
The previous diff algorithm was file tree based, and only looked
at modified files that lived at the same path.

However, components like the Linux kernel have versioned
subdirectories, e.g. /usr/lib/modules/$kver/.../ext4.ko.  We want to
be able to detect these "modified renames" so that we can compute
diffs (rollsum, bsdiff).
2015-02-16 10:10:35 -05:00
Colin Walters
9aa7e30b38 deltas: Implement rollsums
This does an rsync-style prepared delta basically.  On my test data,
it shaves ~6MB of uncompressed data.  Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
2015-02-16 10:10:35 -05:00
Colin Walters
3df8be0d92 deltas: Compute rollsum targets 2015-02-16 10:10:35 -05:00
Giuseppe Scrivano
7ebf7676cb Fix make distcheck
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-30 15:27:36 +01:00
Matthew Barnes
a4d904ee38 build: Include SELinux cflags with libostree build too
This is likely another issue with newer automake.

https://bugzilla.gnome.org/738875
2014-10-20 15:23:55 -04:00
Colin Walters
d546abfa2a libostree: Add initial GRUB2 support
In this approach, we drop a /etc/grub.d/15_ostree file which is a
hybrid of shell/C that picks up bits from the GRUB2 library (e.g. the
block device script generation), and then calls into libostree's
GRUB2 code which knows about the BLS entries.

This is admittedly ugly.  There exists another approach for GRUB2 to
learn the BLS specification.  However, the spec has a few issues:

https://www.redhat.com/archives/anaconda-devel-list/2014-July/msg00002.html

This approach also gives a bit more control to the admin via the
naming of the 15_ostree symlink; they can easily disable it:

Or reorder the ostree entries ahead of 10_linux:

Also, this approach doesn't require patches for grub2, which is an
issue with the pressure to backport (rpm-)OSTree to EL7.
2014-10-16 14:15:00 -04:00
Colin Walters
1242704d68 build: Unify CPPFLAGS settings
The libostree core uses SYSCONFDIR now, so we should ensure it's used
consistently.  Someone else was seeing SYSCONFDIR not being defined
while compiling with a newer automake version, which may process
CPPFLAGS more precisely.
2014-09-08 11:47:58 -04:00
Colin Walters
f8f5da219e Add repository "summary" file and metalink support
For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.

In order to make metalink work though, it needs *one* file which can
be checksummed.  (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).

This commit adds a new command:

$ ostree summary -u

To regenerate the summary file.  Can only be run by one process at a
time.

After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.

https://bugzilla.gnome.org/show_bug.cgi?id=729585
2014-09-03 13:21:52 -04:00
Colin Walters
b4d21e9406 deploy: Set the immutable bit on the deployment root
This prevents people from creating new directories there and expecting
them to be persisted.  The OSTree model has all local state to be in
/etc and /var.

This introduces a compile-time dependency on libe2fsprogs.

We're only doing this for the root directory at the moment.

https://bugzilla.gnome.org/show_bug.cgi?id=728006
2014-06-03 13:41:48 -04:00
Colin Walters
7ccfc93f71 build: Add missing DESTDIR
Hooray for read-only bind mounts and building as non-root.
2014-05-09 09:07:00 -04:00
Colin Walters
f47a20fb81 Support /etc/ostree/remotes.d
For many OS install scenarios, one runs through an installer which may
come with embedded data, and then the OS is configured post-install to
receive updates.

In this model, it'd be nice to avoid the post-install having to rewrite
the /ostree/repo/config file.

Additionally, it feels weird for admins to interact with "/ostree" -
let's make the system feel more like Unix and have our important
configuration in /etc.

https://bugzilla.gnome.org/show_bug.cgi?id=729343
2014-05-08 18:59:24 -04:00
Colin Walters
25ad4a9f65 pull: Add tls-client-cert-{path,key} (if we have new enough libsoup)
This is an actually working version of client-side certificates.
Depends on:
See: https://bugzilla.gnome.org/show_bug.cgi?id=334021

We detect whether libsoup is new enough for this.

https://bugzilla.gnome.org/show_bug.cgi?id=729356
2014-05-01 17:13:13 -04:00
Colin Walters
006e8f0073 deltas: Link to liblzma, add internal API to use it
For future static deltas work, we'll be linking to liblzma.  Since
it's fairly widespread, let's just make it a hard dependency.
2014-04-29 10:59:57 -04:00
Colin Walters
1bdabda5f3 Use external libgsystem 2014.2
It's been split off for a while, let's kill the code duplication.

Among other things, this fixes the systemd detection for the journal
logging.
2014-04-04 16:52:37 -04:00
Colin Walters
7baa600e23 Add an OstreeSysrootUpgrader API
This moves some utility code from the ostree tool into the shared
library, which will make it easier to consume by external tools.
2014-03-24 18:08:22 -04:00
Colin Walters
3337334be5 libostree: Split off SELinux OstreeSePolicy class
It's better if this is independent from the OstreeSysroot; for
example, a policy is active in a given deployment root at once, not
for a sysroot globally.

We can also collect SELinux-related API in one place.

Unfortunately at the moment there can be only one instance of this
class per process.
2014-02-19 08:43:45 -05:00
Colin Walters
2d6374822b Initial basic static delta code drop
This has a very basic level of functionality (deltas can be generated,
and applied offline).  There is only some stubbed out pull code to
fetch them via HTTP.

But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
2014-02-04 10:31:44 -05:00
Colin Walters
c7bcfc1c61 build: Look for /usr/bin/gpgv2 vs /usr/bin/gpgv
For some reason, RHEL has gpgv, but Fedora doesn't.  We need to detect
which to use, since presumably Debian only has gpgv.
2014-01-30 19:32:59 -05:00
Colin Walters
ebe6207847 build: Install README-gpg in /usr/share/ostree/trusted.gpg.d
Since this is what the current code actually reads.
2014-01-30 15:28:13 -05:00
Colin Walters
b2d0ba7ac1 deploy: Rework kernel arguments, add --karg-append to "admin deploy"
The "ordered hash" code was really just for kernel arguments.  And it
turns out it needs to be a multihash (for e.g. multiple console=
arguments).

So turn the OstreeOrderedHash into OstreeKernelArgs, and move the bits
to split key=value and such into there.

Now we're not making this public API yet - the public OstreeSysroot
just takes char **kargs.  To facilitate code reuse between ostree/ and
libostree/, make it a noinst libtool library.  It'll be duplicated in
the binary and library, but that's OK for now.  We can investigate
making OstreeKernelArgs public later.

https://bugzilla.gnome.org/show_bug.cgi?id=721136
2014-01-16 15:07:55 -05:00
Colin Walters
c65923e642 Add OstreeAsyncProgress, use it for ostree_repo_pull
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications.  This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.

From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.

PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.

So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.

Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
2013-10-24 14:27:13 -04:00
Colin Walters
6500026ba7 trivial: Add missing files to dist 2013-10-15 16:39:22 -04:00
Colin Walters
3b700ccb50 core: Add code to read/write "varints"
Adapted from Google protobufs.  For several cases, we want to support
e.g. file sizes up to guint64, but paying the cost of 8 bytes for each
number is too high.

This will be used for static deltas and sizes metadata.
2013-10-11 12:21:37 -04:00
Jeremy Whiting
7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Jeremy Whiting
7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters
ac0154713d libostree: Make OstreeBootloader private
It was only temporarily public while functionality was being merged
down; that's done now.
2013-09-15 20:26:13 -04:00
Colin Walters
c6292942ff libostree: Nearly complete move of API into OstreeSysroot
Move the deployment code too.
2013-09-15 20:16:20 -04:00