Commit Graph

142 Commits

Author SHA1 Message Date
Philip Withnall
fbf8df8829 lib/refs: Add methods for setting/listing collection–refs
These are tuples of (collection ID, ref name) which are a globally-unique
form of local ref. They use OstreeCollectionRef as an identifier, and hence
need to be accessed using new API, as the existing API uses string
identifiers and sometimes accepts refspecs. Remote names are not
supported as part an OstreeCollectionRef.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
4de736fdfa lib/repo: Add collection ID support to OstreeRepo
Add {get,set}_collection_id() methods to OstreeRepo and some documentation
about the concept of a collection ID which globally identifies an
upstream repository. See the documentation for more details.

This will be used in future commits. For now, the new API is marked as
experimental (--enable-experimental-api).

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
0a20e7d43c lib/ref: Add OstreeCollectionRef type for globally unique refs
This is a type representing the tuple (collection ID, ref name), which is
guaranteed to be globally unique. It will be used in upcoming commits.

It introduces the concept of a ‘collection’ which is a unique, curated
set of refs which lie in the same trust domain (i.e. all signed by the
same key and validated by the same developer). Flathub might be a
collection, for example; or the set of OS refs coming from a particular
OS vendor.

It includes a function for validating collection IDs.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #924
Approved by: cgwalters
2017-06-26 15:56:07 +00:00
Philip Withnall
20dc9454b3 lib/core: Add ostree_validate_remote_name() for remote names
There are a few places in the code where ad-hoc validation was being
performed. Might as well formalise it a bit more.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #948
Approved by: cgwalters
2017-06-20 21:52:22 +00:00
Anton Gerasimov
64ab8334b7 lib/sysroot: Add API to get pending/rollback for given stateroot
This imports a function that is used in rpm-ostree, and it's also intended for
use by https://github.com/advancedtelematic/aktualizr to display
what deployment we're going to boot next after the reboot.

Updated-by: Colin Walters <walters@verbum.org>

Closes: #897
Approved by: OYTIS
2017-06-14 09:56:01 +00:00
Brian C. Lane
c651982929 Remove the OSTREE_MAX_RECURSION limit on metadata depth
This was making it impossible to pull or mirror a large ostree repo, and
according to Colin is no longer necessary. It works fine with a test
against a repo with 2741 commit and 451468 objects in it.

Closes: #899

Closes: #904
Approved by: jlebon
2017-06-02 16:18:28 +00:00
Colin Walters
9bf8a8503a lib/sysroot: Add non-failable ostree_sysroot_repo()
Having a failable accessor is annoying, since it's really common
to reference both.  Instead, open the repo once when we load
the sysroot, and provide a non-failable accessor.

This is also prep for `ostree_repo_open_at()`, which collapses the separation
between `ostree_repo_new()` and `ostree_repo_open()`.

Closes: #886
Approved by: jlebon
2017-05-26 19:17:59 +00:00
Philip Withnall
d2eaded90d lib/remote: Add a getter for OstreeRemote.name
Now that we’ve got a public, sealed OstreeRemote structure, we can start
carefully exposing members of it as API.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #875
Approved by: cgwalters
2017-05-19 15:01:59 +00:00
Philip Withnall
6eac575f21 libostree: Make OstreeRemote a public and internal API
Previously it was static to ostree-repo.c. Make it usable throughout
libostree so it can be used by an upcoming commit, but also expose the
typedef and reference counting functions so that opaque OstreeRemote
pointers can be used by user code, in anticipation of exposing more of
its API publicly in future.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Philip Withnall
c9244b1bb2 build: Add --enable-experimental-api configure option for unstable APIs
There are currently no unstable APIs, but some will be added in
following commits. They will be built and exposed in the libostree
global symbol list iff configured with --enable-experimental-api.

Distributions should not package OSTree with --enable-experimental-api.
This is designed for previewing new APIs on controlled platforms; any of
the APIs hidden behind this option may be changed or removed at any
point.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #832
Approved by: cgwalters
2017-05-08 18:48:07 +00:00
Philip Withnall
c27b66de80 libostree: Add multiple getter/setter support to OstreeAsyncProgress
OstreeAsyncProgress is thread-safe: it can have keys changed by one
thread while another is getting the same keys (modulo some locking
contention). However, the thread safety is done at the function call
level: if some code calls an OstreeAsyncProgress getter several times,
the key fetches are not atomic with respect to each other.

In the case of contention on the lock, this can result in consumers of
OstreeAsyncProgress data seeing an inconsistent state between the
properties they query, which could result in progress reporting
inaccuracies.

In the uncontested case, this results in the OstreeAsyncProgress lock
being locked and unlocked many times more than necessary.

Try to improve this by adding new API, which supports getting and
setting multiple keys atomically:
 • ostree_async_progress_get()
 • ostree_async_progress_set()

The new API uses GVariants and varargs: keys are passed as a
GVariantType string followed by arguments as for g_variant_new() or
g_variant_get(), followed by the next key, etc.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #819
Approved by: cgwalters
2017-04-29 11:50:15 +00:00
Philip Withnall
f74e52a3a0 libostree: Rework OstreeAsyncProgress to use GVariants internally
OstreeAsyncProgress currently does some contortions to try and avoid
allocating space for guints and guint64s (on 64-bit platforms), but this
means it uses two GHashTables. A GHashTable allocates 8 buckets even
when empty. Given that the largest usage of OstreeAsyncProgress in
libostree puts 13 uints and 5 uint64s in it, this optimisation does not
save significant (if any) memory.

Instead, change OstreeAsyncProgress to store values internally as
GVariants, and expose this with some new API:
 • ostree_async_progress_get_variant()
 • ostree_async_progress_set_variant()
Each GVariant is allocated on the heap. As they are immutable, they are
thread-safe once returned by a getter.

The existing API continues to work as before, except in the case where a
key is set/got as both a uint and a uint64 — there will now be a
collision (and a GVariant type checking failure) whereas previously
there was no collision. Nothing in OSTree uses OstreeAsyncProgress this
way though.

The new API can be used to share more complex data via the progress API.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #819
Approved by: cgwalters
2017-04-29 11:50:15 +00:00
Krzesimir Nowak
1ceb17ff11 apidoc: Add missing enums to sections file
So they and their values show up in the documentation.

Closes: #812
Approved by: cgwalters
2017-04-26 11:41:38 +00:00
Colin Walters
6a7ee4860f Fix a few gtk-doc warnings
Just continuing to chip away at this.

Closes: #788
Approved by: jlebon
2017-04-12 15:36:46 +00:00
Colin Walters
a5d5333c83 sysroot: Add ostree_sysroot_write_deployments_with_options()
More sophisticated users of libostree like rpm-ostree need control over things
like the system repository. Previously we introduced a "no cleanup" flag to
`ostree_sysroot_simple_write_deployment()`, but that's a high level API that
does filtering on its own.

Since rpm-ostree needs more control, let's expose the bare essentials of the
"sysroot commit" operation with an extensible options structure, where one of
the options is whether or not to do post-transaction repository operations.

Closes: #745
Approved by: jlebon
2017-03-23 19:28:42 +00:00
Colin Walters
d7f4a326b9 sepolicy: Add ostree_sepolicy_new_at()
I'm porting other code away from `GFile`, and while we don't use this
internally, it will let us do so at a later date. I'm averse to changing the
code right now as we don't have good CI coverage of this.

Closes: #746
Approved by: jlebon
2017-03-22 16:24:06 +00:00
Erik Larsson
e665e51408 diff: Add ostree_diff_dirs_with_options(), expose via cmdline
The first options are owner_uid/owner_gid, which makes it possible to use diff
on local files where --owner-uid/gid have been passed to commit.

Closes: #740
Approved by: cgwalters
2017-03-21 13:38:04 +00:00
Colin Walters
46001f4a5b core: Add runtime ostree_check_version()
[Previously](https://github.com/ostreedev/ostree/pull/728) we added compile-time
checking for versions, but there are use cases for runtime checking as well,
because in a number of API calls we use `GVariant` as an API extension
mechanism.

Closes: #735
Approved by: jlebon
2017-03-13 14:29:02 +00:00
Georges Basile Stavracas Neto
fda4a47cae libostree: add versioning macros
OSTree currently provides no way to inspect the versioning
information at run time, being only available at compile
time through pkg-config.

This is a problem for e.g. Flatpak, that needs to check
whether the 'update-frequency' option is available. Checking
at compile time isn't great since it's not looking for new
symbols, but only if an optional feature is present.

This commit, then, adds a new header that is generated
at compile time, exposing OSTree's versioning information.

Closes: #728
Approved by: cgwalters
2017-03-11 15:38:56 +00:00
Philip Withnall
574c3ea6f9 libostree: Allow compression level to be set for archive-z2 stream
Add a ostree_raw_file_to_archive_z2_stream_with_options() variant of
ostree_raw_file_to_archive_z2_stream(), to allow a compression-level
option to be passed in and passed through to zlib.

This is useful when building archive-z2 files on the fly for
transmission over a non-bandwidth-limited channel, such as a local
network. In this case, CPU time is more valuable than bandwidth, so we
want a low compression level.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #721
Approved by: cgwalters
2017-03-06 18:19:45 +00:00
Colin Walters
9c0af41710 lib: Add ostree_repo_reload_config()
For a long time we've cached the remote configs in the repo, which
mostly makes sense for the `repo/config` file, but less sense
for `/etc/ostree/remotes.d`, because we want to support admins
interactively editing them.

One can delete the repo instance and create a new one, but that's a bit ugly.
Let's introduce an API for this so rpm-ostree can reload remotes after
admins/scripts edit them in `/etc`.  We also might as well reload
any other entries in the config.

Structurually now, `ostree_repo_open()` deals with file descriptors, and then
calls `ostree_repo_reload_config()`. Except for the uncompressed cache, which is
the only thing that deals with FDs that can be configured. But we want to delete
that anyways.

No tests, since...we don't have a daemon in this codebase, don't want to shave
that yak just today.

Closes: #662
Approved by: jlebon
2017-02-07 16:12:58 +00:00
Colin Walters
5c940987e7 Add support for more selective pruning
There are use cases for having a single repo with branches
with different lifecycles; a simple example of what I was
trying to do in CentOS Atomic Host work is have "stable"
and "devel" branches, were we want to prune devel, but
retain *all* of stable.

This patch is split into two parts - first we add a low level "delete all
objects not in this set" API, and change the current prune API
to use this.

Next, we move more logic into the "ostree prune" command. This paves the way for
demonstrating how more sophisticated algorithms/logic could be developed outside
of the ostree core.

Also, the --keep-younger-than logic already lived in the commandline, so it
makes sense to keep extending it there.

Closes: https://github.com/ostreedev/ostree/issues/604

Closes: #646
Approved by: jlebon
2017-01-19 16:28:00 +00:00
Colin Walters
24bf257ee9 lib: Add an API to GPG verify a commit given a remote
Conceptually we've been moving towards having our GPG verification
paths be per-remote.  The code internally supports this, but we
didn't expose an API to use it conveniently.

This came up when trying to add a new `gpgkeypath` option, since
right now rpm-ostree manually finds keyrings for the remote, and
hence it wasn't looking at the keypath, and said "Unknown key"
in status.

Adding an API fixes this nicely.

Closes: #576
Approved by: giuseppe
2016-11-17 11:33:41 +00:00
Giuseppe Scrivano
30963766c7 libostree: new function ostree_repo_checkout_at
Provide a gobject introspection safe version for
`ostree_repo_checkout_tree_at'.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Dan Nicholson
db974b0596 core: Add allocating b64 checksum functions
The checksum_b64_inplace variants can't be used in bindings. Provide
versions that allocate and return the output rather than working on a
passed in buffer. These can then be used in GI bindings to get the
ostree modified base64 encodings.

Closes: #398
Approved by: cgwalters
2016-07-15 02:18:27 +00:00
Yu Qi Zhang
02a2b689dd refs: resolve conflict between local/remote repos
Add the functionality to use the same name for refs in local and remote
repos. This helps users keep track of local refs of remote origin, much
like local and remote git branches.

Previously, when a local ref is specified, resolve_refspec would fall
back to searching through remote repos if the ref is not found locally.
This function now takes an extra flag to specify whether it should
search through remote repos. Additionally, ostree_repo_resove_rev_ext
was added to call resolve_refspec with fallback_remote being false, so
refs --create would no longer complain when trying to create a local
ref of the same name as a remote one.

Fix remote repo parsing not being handled correctly on refs --create.

Closes: #363
Approved by: jlebon
2016-06-23 19:52:26 +00:00
Mathnerd314
1b88dc7f90 docs: Get API docs working again
This changes around a few things that didn't work for me:
* Section names seem to be ostree-* instead of libostree-*
* Also XML files are ostree-* (they didn't show up at all)
- gtk-doc doesn't seem to parse const _OSTREE_PUBLIC correctly
* pull documentation is now on the actual functions rather than stubs
* Update gitignore with some more files

And there some changes to make gtk-doc give fewer warnings (not finished)

Closes: #327
Approved by: cgwalters
2016-06-09 18:15:49 +00:00
Krzesimir Nowak
4929ab4033 repo: Add functions for verifying any data in repository
This can be useful for validating the 3rd party data that is put in
the extensions directory and is signed with the same keys as commits
or the summary file.

Closes: #310
Approved by: cgwalters
2016-05-27 11:20:00 +00:00
Krzesimir Nowak
89bfb1d503 repo: Factor out the check of gpg result to a separate function
I plan to add a function for verifying any data which may return the
error about lack of trusted signatures, so let's avoid the redundancy
and put the check in the separate function.

Closes: #310
Approved by: cgwalters
2016-05-27 11:20:00 +00:00
Krzesimir Nowak
569e43c280 core: Add a function creating an archive-z2 content stream
It is quite similar to the already existing
ostree_raw_file_to_content_stream function, so I factored the common
part to a separate function. The difference is that we cannot report
the size of the resulting stream.

Can be useful for serving a "bare" repository as a faked "archive-z2"
repository.

Closes: #308
Approved by: cgwalters
2016-05-26 16:53:08 +00:00
Krzesimir Nowak
aa946cc136 repo: Allow using options for fetching summary
This adds a _with_options variant of the
ostree_repo_remote_fetch_summary function, so we can tell the fetcher
to use a specific URL instead taking it from the remote config.

Closes: #290
Approved by: cgwalters
2016-05-10 13:47:36 +00:00
Colin Walters
e9a1809c4d Rename test-abi to test-symbols, start verifying symbols are documented
We keep forgetting to update `apidoc/ostree-sections.txt`, so let's
start enforcing it.  Of course it turns out we had some bugs here
like symbols marked as public but never implemented, etc.  Those
are fixed in the prior commits.

Closes: #263
Approved by: giuseppe
2016-04-17 13:52:07 +00:00
Colin Walters
6923c088c5 lib: Remove ostree_repo_file_make_empty_tree
This was removed in 14d682305b - long
before we had a stable library.

I again claim removing unimplemented symbols from the header is not an
API/ABI break.

Closes: #263
Approved by: giuseppe
2016-04-17 13:52:07 +00:00
Colin Walters
1ac46828f2 core: Remove ostree_checksum_update_meta that was not implemented
If a symbol falls in a git merkle tree forest, but no one hears it,
did it ever exist?

I'm claiming this isn't an API/ABI break because nothing could
have actually used this.  `git log -S checksum_update_meta` leads
me to 38ef75e6e0 which was before
we really had a stable shared library.

Closes: #263
Approved by: giuseppe
2016-04-17 13:52:07 +00:00
Colin Walters
984bf91826 Use git.mk
It's a lot nicer than manually maintaining .gitignore in general.

Closes: #235
Approved by: giuseppe
2016-04-07 12:49:40 +00:00
Alexander Larsson
ed1e0c6d04 pull: Add OSTREE_REPO_PULL_FLAGS_UNTRUSTED flag
If this is set we verify all objects we pull, even for local remotes,
and we avoid hard-linking into local source repos.

https://bugzilla.gnome.org/show_bug.cgi?id=764125

Closes: #221
Approved by: cgwalters
2016-03-25 12:56:55 +00:00
Colin Walters
fea786cb2d lib: Add ostree_sysroot_load_if_changed() API
This will allow daemons like rpm-ostree to detect if there are any new
deployments efficiently, in combination with using inotify.  If there
are any changes, rpm-ostree wants publish them on DBus.

While we're here, add some changes to start doing unit C testing of
the sysroot API.
2016-03-03 21:56:23 -05:00
Colin Walters
fbd9409ebb lib: Add ostree_sysroot_init_osname() API, bump mtime
And change the command line to use it.  rpm-ostree had a copy
of this code, and thus there's a clear reason to have an API.

While we're moving this into API, ensure the mtime on deploy is bumped
after an osname is created, so that daemons like rpm-ostree can notice
changes.  (In reality, creating the directory should do this, but
let's be double sure)
2016-03-03 14:21:57 -05:00
Giuseppe Scrivano
898c7b6577 ostree-repo: new public function ostree_repo_list_refs_ext
It accepts a `flags` argument to control its behavior.  Differently
from `ostree_repo_list_refs`, the `refspec_prefix` is not removed from
the results.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2016-03-02 14:52:02 -05:00
Colin Walters
5bab946b80 apidoc: Remove unnecessary srcdir != builddir workaround
It seems to be fine with `gtk-doc-1.19-3.el7.noarch`, so let's drop
this workaround, as it causes `make` warnings.
2016-01-28 09:31:37 -05:00
Colin Walters
64ebe2b82a Rewrite manual in mkdocs
I don't much like Docbook (and am considering converting the man pages
too), but let's start with the manual.

I looked at various documentation generators (there are a lot), and
I had a few requirements:

 - Markdown
 - Packaged in Fedora
 - Suitable for upload to a static webserver

`mkdocs` seems to fit the bill.
2016-01-28 09:31:37 -05:00
Colin Walters
32c360b5a0 build: Rename doc/ -> apidoc/
This is preparation for introducing a `mkdocs` manual under `doc/`
which should be significantly more useful for the world at large than
the minimal manual that exists there now.
2016-01-28 09:31:34 -05:00