Commit Graph

3026 Commits

Author SHA1 Message Date
Colin Walters
4d873389f0 commit: Prefix error with target object name on failure to write
Helpful to debug things later.

Closes: #759
Approved by: jlebon
2017-03-24 22:16:43 +00:00
Colin Walters
ffc178311e build: Dist ostree-sepolicy-private.h
Should fix the Travis builds which actually generate a legacy tarball via
Automake.

Closes: #755
Approved by: jlebon
2017-03-23 21:58:55 +00:00
Colin Walters
193f5c16f2 core: Convert some functions to new code style
Not doing the whole file since it's big, these are some simple ones to keep
momentum up.

Closes: #753
Approved by: jlebon
2017-03-23 19:45:33 +00:00
Colin Walters
a5d5333c83 sysroot: Add ostree_sysroot_write_deployments_with_options()
More sophisticated users of libostree like rpm-ostree need control over things
like the system repository. Previously we introduced a "no cleanup" flag to
`ostree_sysroot_simple_write_deployment()`, but that's a high level API that
does filtering on its own.

Since rpm-ostree needs more control, let's expose the bare essentials of the
"sysroot commit" operation with an extensible options structure, where one of
the options is whether or not to do post-transaction repository operations.

Closes: #745
Approved by: jlebon
2017-03-23 19:28:42 +00:00
Colin Walters
ed60a854e4 repo/refs: Convert to new code style
I happened to be reading this one recently for a reason I forget,
and it's a relatively easy conversion.

Also one not conflicting with any outstanding patches.

Closes: #752
Approved by: jlebon
2017-03-23 19:09:52 +00:00
Colin Walters
f6255763fd cfg.mk: Add a syntax check for a redundant : in glnx_throw
This is going to be an easy bug to make when porting.

Example spotted by jlebon in 435c2dc35c (diff-dbc2359685171239c4708f2a8f6aa921)

Closes: #752
Approved by: jlebon
2017-03-23 19:09:52 +00:00
Colin Walters
7b2370dc86 sepolicy: Add better private API for setfscreatecon
Use `g_auto()` more sanely with a struct implmenting the "is initialized"
pattern.  This is way less ugly for callers, and fixes bugs like
us calling `setfscreatecon()` even if an error occurred beforehand.

Also fold in the logic for "NULL or not loaded" sepolicy into the setup rather
than requiring callers to inline it.

Prep for more users of this function.

Closes: #746
Approved by: jlebon
2017-03-22 16:24:06 +00:00
Colin Walters
d7f4a326b9 sepolicy: Add ostree_sepolicy_new_at()
I'm porting other code away from `GFile`, and while we don't use this
internally, it will let us do so at a later date. I'm averse to changing the
code right now as we don't have good CI coverage of this.

Closes: #746
Approved by: jlebon
2017-03-22 16:24:06 +00:00
Colin Walters
4cf210b128 Bump libglnx, port a few callers to new error API
Testing out the waters here. I think we should roll this into any future code
cleanup reworking we do.

Closes: #747
Approved by: jlebon
2017-03-22 16:04:58 +00:00
Colin Walters
94c47cde6a build: Various fixes for openssl build
This actually worked before because `pkg-config --cflags openssl`
is empty, and the linker was satisfying `-lssl -lcrypto` from our
indirect dependencies.

Also, in fact we *currently* just want `pkg-config libcrypto` i.e.
`-lcrypto`, since we aren't actually using openssl to speak TLS.

This doesn't actually matter on Fedora at least since they're both in the
`openssl-libs` package, but they are separate for a reason.

Closes: #749
Approved by: jlebon
2017-03-21 21:16:04 +00:00
Colin Walters
4ebe1ee94b build: Quiet automake warning for bupsplit
Use `_SRCS` and not `_SOURCES`, the latter of which makes automake assume
there's something for it to do.

Closes: #749
Approved by: jlebon
2017-03-21 21:16:04 +00:00
Erik Larsson
e665e51408 diff: Add ostree_diff_dirs_with_options(), expose via cmdline
The first options are owner_uid/owner_gid, which makes it possible to use diff
on local files where --owner-uid/gid have been passed to commit.

Closes: #740
Approved by: cgwalters
2017-03-21 13:38:04 +00:00
Colin Walters
5d413dff88 sysroot: Prep refactoring of cleanup logic
For future work I'm going to tweak how we handle cleanup, and
the private cleanup flags didn't really end up being used - we
only specify "prune repo or not".  So fold that into a boolean for now.

The sysroot deploy logic then has a single "do_postclean" boolean, which is all
I want to expose as public API.

Closes: #744
Approved by: jlebon
2017-03-20 20:16:12 +00:00
Colin Walters
ec2f52e625 sysroot/deploy: Some cleanup to decl-after-stmt/return FALSE style
And fd-relative. I also introduced some helpers here which I'll use later in
more invasive patches.

Closes: #742
Approved by: jlebon
2017-03-20 19:56:48 +00:00
Colin Walters
df5cbc9be9 core: Support building with OpenSSL for checksums
Add an OpenSSL backend to the checksum input stream, which is where we do a lot
of checksumming (object commit, static deltas).

The raw OpenSSL performance is
[approximately double](https://gist.github.com/cgwalters/169349fd1c06fd4fb4d3a7ce33303222) on
my laptop; not only does OpenSSL have e.g. hand-tuned x86_64 assembly, the
current implementation uses the
[Intel SHA extensions](https://en.wikipedia.org/wiki/Intel_SHA_extensions).

Another reason to do this is I was idly thinking about adding
[Curve25519](https://en.wikipedia.org/wiki/Curve25519) signatures (like e.g.
Alpine does) instead of/in addition to GPG.  The rationale for that is
that GPG is pretty heavyweight, both in code footprint and the simple
fact that EC keys are way smaller.

I didn't benchmark ostree with this; we have bigger performance problems
really like the fact we just malloc way too much.  But, it's a step
in the right direction I think in combination with the libcurl work
where we're linking to openssl anyways.

Closes: #738
Approved by: jlebon
2017-03-20 18:32:40 +00:00
Colin Walters
c2f5a999bf lib: Add a private copy of checksum-instream
The current `OstreeChecksumInputStream` is public due to a historical
mistake.  I'd like to add an OpenSSL checksum backend, but that's
harder without breaking this API.

Let's ignore it and create a new private version, so it's easier to do the
GLib/OpenSSL abstraction in one place.

Closes: #738
Approved by: jlebon
2017-03-20 18:32:40 +00:00
Colin Walters
0b214566a2 lib: Squash most of the gtk-doc warnings for missing parameters
This isn't all of them, just trying to make a dent.

Closes: #734
Approved by: jlebon
2017-03-13 15:20:45 +00:00
Colin Walters
741911d0fc lib: Exclude soup header from introspection
We should fix this more reliably by defining a separate variable for
introspection (and gtk-doc) sources at some point, but this does the trick for
now.

Avoids a spam of warnings, and it's definitely not public API.

Closes: #734
Approved by: jlebon
2017-03-13 15:20:45 +00:00
Colin Walters
340144265f pull: Squash a -Wmaybe-uninitialized warning
Tempting to make it fatal...but we'd have to basically init all
of our variables I think.

Closes: #734
Approved by: jlebon
2017-03-13 15:20:45 +00:00
Colin Walters
943c6c36c7 builtin/show: Convert to direct return/decl-after-stmt style
I just happened to be reading this code; still testing
the waters for the new code style.  If we decide to do more
of this it's likely worth trying out a [Coccinelle](http://coccinelle.lip6.fr/)
patch or something.

Closes: #733
Approved by: jlebon
2017-03-13 15:03:25 +00:00
Colin Walters
46001f4a5b core: Add runtime ostree_check_version()
[Previously](https://github.com/ostreedev/ostree/pull/728) we added compile-time
checking for versions, but there are use cases for runtime checking as well,
because in a number of API calls we use `GVariant` as an API extension
mechanism.

Closes: #735
Approved by: jlebon
2017-03-13 14:29:02 +00:00
Colin Walters
b825aac0ea Disable "ostree trivial-httpd" by default now
This goes farther along the path of deprecating it; consumers should
have been ported at this point.  Though probably a lot of people
may still use `rpm-ostree-toolbox`.

Closes: #723
Approved by: jlebon
2017-03-13 14:07:18 +00:00
Georges Basile Stavracas Neto
fda4a47cae libostree: add versioning macros
OSTree currently provides no way to inspect the versioning
information at run time, being only available at compile
time through pkg-config.

This is a problem for e.g. Flatpak, that needs to check
whether the 'update-frequency' option is available. Checking
at compile time isn't great since it's not looking for new
symbols, but only if an optional feature is present.

This commit, then, adds a new header that is generated
at compile time, exposing OSTree's versioning information.

Closes: #728
Approved by: cgwalters
2017-03-11 15:38:56 +00:00
André Klitzing
75907cb513 Fix includes if built against musl
LOCK_* is defined in sys/file.h

http://git.musl-libc.org/cgit/musl/tree/include/sys/file.h

Closes: #730
Approved by: cgwalters
2017-03-10 22:11:47 +00:00
André Klitzing
f72726c7d8 Avoid unnecessary includes
"return" will do the same here. Also that style is used at the
end of the function.

Closes: #732
Approved by: jlebon
2017-03-10 21:46:50 +00:00
Colin Walters
e02e900206 Release 2017.3
It's been almost a month, I think the current git is working well and
not too risky.  We have some new API additions which I think the
various consumers of them are going to want.

Closes: #726
Approved by: jlebon
2017-03-09 18:58:48 +00:00
Colin Walters
3e32d5c4b6 repo/checkout: Convert a few functions to new "stmt-decl/FALSE" style
Just testing the waters a bit more.  Yeah, definitely nicer.

Closes: #722
Approved by: jlebon
2017-03-09 14:26:17 +00:00
Georges Basile Stavracas Neto
bb3a0e3fa4 ostree: allow setting update frequency from command line
After commit 80b3edc64731a5f0 introducing the option to set
a custom timeout, adapt the ostree program to be able to update
that.

Closes: #725
Approved by: jlebon
2017-03-08 20:28:27 +00:00
Georges Basile Stavracas Neto
d8ac9f75cd repo-pull: add option to set the async update frequency
When using Flatpak with GNOME Software, it is important to
show the progress of the download and install as close as
possible to the real progress.

However, OSTree forces the frequency to call the async
progress callback to 1 second, which causes an unpleasant
effect on the UI, specially when the download size is so
small that everything happens in less than 1 second.

Fix that by adding making OSTree read a custom 'update-frequency'
option and set the timeout source timeout to that. If
no custom frequency is passed, we assume the default 1
second timeout, maintaining the current behavior.

Closes: #725
Approved by: jlebon
2017-03-08 20:28:27 +00:00
Christian Hergert
031d7898cc repo/checkout: fix 32-bit builds
__dev_t is 64-bit even on 32-bit Linux systems such as i386.

Closes: #724
Approved by: cgwalters
2017-03-08 14:01:10 +00:00
Colin Walters
94948e3522 checkout: Support a "pure addition" mode
I plan to use this for `rpm-ostree livefs`.
https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #714
Approved by: jlebon
2017-03-06 20:58:04 +00:00
Colin Walters
ff34810097 repo/checkout: Verify early if src/destination are on same device
At least in all Linux kernels up to today, one can never `link()` across
devices, so we might as well verify that up front. This will help for a future
patch to add a new type of union-add checkout, since Linux checks for `EEXIST`
before `EXDEV`.

Closes: #714
Approved by: jlebon
2017-03-06 20:58:04 +00:00
Colin Walters
3219a5d0ee Allow and start using C99 declaration-after-statement
I've seen code in a few places that I think on balance is definitely better this
way.  Some of our functions have huge variable declaration sections.
This change includes one small example where we could start using declarations
after statements.

A concern I had was - how does this interact with `__attribute__((cleanup))` and
early returns? I tested it, and AFAICS the behavior is what you'd expect - the
cleanup function isn't called if its variable isn't reachable.

Closes: #718
Approved by: jlebon
2017-03-06 18:33:50 +00:00
Philip Withnall
574c3ea6f9 libostree: Allow compression level to be set for archive-z2 stream
Add a ostree_raw_file_to_archive_z2_stream_with_options() variant of
ostree_raw_file_to_archive_z2_stream(), to allow a compression-level
option to be passed in and passed through to zlib.

This is useful when building archive-z2 files on the fly for
transmission over a non-bandwidth-limited channel, such as a local
network. In this case, CPU time is more valuable than bandwidth, so we
want a low compression level.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #721
Approved by: cgwalters
2017-03-06 18:19:45 +00:00
Colin Walters
72336f1c48 fetcher/curl: Fix leaks caught by ASAN
I had to rebuild `glib` with `-fsanitize=address` in order to get a stack trace
to finally get this one.  However, *installing* that glib "system wide"
in my container breaks everything (including `rpm-ostree`, `dnf`, `pkg-config` etc.)
that wasn't built with ASAN.

So my test scenario right now is to extract the libs and do e.g.:
```
make && env LD_LIBRARY_PATH=$HOME/src/distgit/fedora/glib2/asan-libs make check TESTS=tests/test-basic.sh
```

Closes: #719
Approved by: jlebon
2017-03-06 17:30:19 +00:00
Philip Withnall
f667a82fc1 build: Fix disabling --enable-man if xsltproc is not available
If --enable-man=maybe (the default), and xsltproc is not available, the
configuration code would check for it, correctly set enable_man=no, then
incorrectly overwrite that with enable_man=yes, which would result in
later trying to execute $(XSLTPROC) when it’s empty.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #720
Approved by: cgwalters
2017-03-06 13:48:17 +00:00
Colin Walters
a787e0c072 pull: Use all available commits for delta sources
The previous logic for static deltas was to use as a FROM
revision the current branch tip.  However, we want
to support deltas between branches in an automatic
fashion.

If a summary file is available, we already have an
enumerated list of deltas - so the logic introduced
here is to search it, and find the newest commit
we have locally that matches the TO revision target.

This builds on some thoughts from
https://github.com/ostreedev/ostree/pull/151#issuecomment-232390232

Closes: https://github.com/ostreedev/ostree/pull/151

Closes: #710
Approved by: giuseppe
2017-03-04 10:11:14 +00:00
Colin Walters
b41f150a72 grub2: Use "linux16" only on x86/x86_64
Got a report that a Fedora Atomic Host built for ppc64le didn't work with the
`linux16`, it needed `linux`. See the comments for more links.

Closes: #716
Approved by: vathpela
2017-03-03 15:36:36 +00:00
Jonathan Lebon
c4f6522829 libglnx: bump for -Wmaybe-uninitialized fix
https://github.com/GNOME/libglnx/pull/37

Closes: #715
Approved by: cgwalters
2017-03-02 20:35:24 +00:00
Jonathan Lebon
199646ccfa pull: don't use static deltas if archive repo
In https://github.com/ostreedev/ostree/pull/408, we disabled the use of
static deltas when mirroring. Later,
https://github.com/ostreedev/ostree/pull/506 loosened this up again so
that we could use static deltas when mirroring into bare{-user} repos.

However, the issue which originally spurrred #408 is even more generic
than that: we want to avoid static deltas for any archive repo, not just
when doing a mirror pull. This patch tightens this up, and also
relocates the decision code to make it easier to read.

Closes: #715
Approved by: cgwalters
2017-03-02 20:35:24 +00:00
Gatis Paeglis
64422a7d0b deltas: Expose the filename parameter
The C API (ostree_repo_static_delta_generate) knows what to do
with it, but this parameter was never exposed via command line
tool.

Closes: https://github.com/ostreedev/ostree/issues/695

Closes: #703
Approved by: jlebon
2017-02-27 22:10:26 +00:00
Colin Walters
9695c47684 grub2: Use g_spawn_sync() rather than GSubprocess to avoid SIGCHLD
Due to the async nature of `GSubprocess` it grabs `SIGCHLD` which
affects other software which might be using libostree, such as
QtOTA.

Closes: https://github.com/ostreedev/ostree/issues/696

Closes: #702
Approved by: jlebon
2017-02-27 21:55:17 +00:00
Colin Walters
36b28cb4d2 upgrade: Add support for --pull-only and --deploy-only
This makes it easier to script downloading updates in the background,
and only do deployments just before rebooting.

Partially addresses https://github.com/ostreedev/ostree/issues/640

Closes: #642
Approved by: jlebon
2017-02-27 20:57:23 +00:00
Colin Walters
2c326d705e fetcher: Log failures into journal
Particularly when HTTP requests fail, I really want a lot more information.
We could theoretically stuff it into the `GError` message field, but
that gets ugly *fast*.

Using the systemd journal allows us to log things in a structured fashion.
Right now e.g. rpm-ostree won't be aware of this additional information,
but I think we could teach it to be down the line.

In the short term, users can learn to find it from `systemctl status rpm-ostreed`
or `journalctl -b -r -u rpm-ostreed`, etc.

One thing I'd like to do next is log successful fetches of e.g. commit objects
as well with more information about the originating server (things like the
final URL if we were redirected, did we use TLS pinning, what was the negotiated
TLS version+cipher, etc).

Closes: #708
Approved by: jlebon
2017-02-24 22:30:24 +00:00
Colin Walters
cee57a0268 deploy/libmount: Fix build with old util-linux 2.23 (CentOS7)
https://github.com/ostreedev/ostree/pull/705 broke the build
on CentOS 7 which only has util-linux 2.23.

When I was thinking about this, I realized that there must really be a way to
make this safe even for older versions. Looking at that version of util-linux,
all we need to do is invert the order of frees so we `mnt_free_table()` *before*
`mnt_free_cache()`, like util-linux does:

https://github.com/karelzak/util-linux/blob/stable/v2.23/sys-utils/eject.c#L1131

We still use the `_unref()` versions if available.  I also fixed
the ordering there too for double plus redundant safety.

Closes: #712
Approved by: jlebon
2017-02-24 17:24:15 +00:00
Colin Walters
877a27da0f tree-wide: Squash noncritical compiler warnings
Should fix everything from
<https://kojipkgs.fedoraproject.org//packages/ostree/2017.2/3.fc25/data/logs/x86_64/build.log>

Anything that uses autocleanups should *always* be initialized directly I think,
even if a few lines down we directly assign, since this way it's more robust
against refactoring.

And the `freopen()` warnings are right - IMO we should *always* check return
values.

Closes: #711
Approved by: jlebon
2017-02-24 17:08:27 +00:00
Colin Walters
a71d550860 man/repo-config: Document mirrorlist
We should get more strict about docs. Add some text about `contenturl` and
mirrorlists.

Closes: #709
Approved by: jlebon
2017-02-24 14:39:35 +00:00
Colin Walters
0817be61a1 deploy: Correctly use libmount unref() calls rather than free()
We saw a random ostree SEGV start popping up in our CI environment:
https://github.com/projectatomic/rpm-ostree/pull/641#issuecomment-281870424

Looking at this code more and comparing it to what util-linux does, I noticed we
had a write-after-free, since `mnt_unref_table()` will invoke
`mnt_unref_cache()` on its cache, and that function does:

```
	if (cache) {
		cache->rfcount--;
```

unconditionally.

Fix this by using `unref()`.

Closes: #705
Approved by: jlebon
2017-02-23 15:31:25 +00:00
Colin Walters
09b392675a main: Make ostree --version output YAML (and add gitrev)
I learned today that `docker version` does this and I really like
the idea.  While we have the patient open, also add the gitrev
with code taken from https://github.com/projectatomic/rpm-ostree/pull/584

Closes: #691
Approved by: giuseppe
2017-02-22 18:57:18 +00:00
Colin Walters
3ec509c89b build: Add --with-smack, use it to reset contexts for writing objects
At some point we'll want to follow what systemd is doing and add
better support for smack, along the lines of `OstreeSePolicy`.  However,
short term this patch fixes AGL which uses Smack.

See: https://jira.automotivelinux.org/browse/SPEC-386
See: https://github.com/ostreedev/ostree/pull/698

Closes: #698
Approved by: OYTIS
2017-02-22 14:37:19 +00:00