Commit Graph

3006 Commits

Author SHA1 Message Date
Colin Walters
46001f4a5b core: Add runtime ostree_check_version()
[Previously](https://github.com/ostreedev/ostree/pull/728) we added compile-time
checking for versions, but there are use cases for runtime checking as well,
because in a number of API calls we use `GVariant` as an API extension
mechanism.

Closes: #735
Approved by: jlebon
2017-03-13 14:29:02 +00:00
Colin Walters
b825aac0ea Disable "ostree trivial-httpd" by default now
This goes farther along the path of deprecating it; consumers should
have been ported at this point.  Though probably a lot of people
may still use `rpm-ostree-toolbox`.

Closes: #723
Approved by: jlebon
2017-03-13 14:07:18 +00:00
Georges Basile Stavracas Neto
fda4a47cae libostree: add versioning macros
OSTree currently provides no way to inspect the versioning
information at run time, being only available at compile
time through pkg-config.

This is a problem for e.g. Flatpak, that needs to check
whether the 'update-frequency' option is available. Checking
at compile time isn't great since it's not looking for new
symbols, but only if an optional feature is present.

This commit, then, adds a new header that is generated
at compile time, exposing OSTree's versioning information.

Closes: #728
Approved by: cgwalters
2017-03-11 15:38:56 +00:00
André Klitzing
75907cb513 Fix includes if built against musl
LOCK_* is defined in sys/file.h

http://git.musl-libc.org/cgit/musl/tree/include/sys/file.h

Closes: #730
Approved by: cgwalters
2017-03-10 22:11:47 +00:00
André Klitzing
f72726c7d8 Avoid unnecessary includes
"return" will do the same here. Also that style is used at the
end of the function.

Closes: #732
Approved by: jlebon
2017-03-10 21:46:50 +00:00
Colin Walters
e02e900206 Release 2017.3
It's been almost a month, I think the current git is working well and
not too risky.  We have some new API additions which I think the
various consumers of them are going to want.

Closes: #726
Approved by: jlebon
2017-03-09 18:58:48 +00:00
Colin Walters
3e32d5c4b6 repo/checkout: Convert a few functions to new "stmt-decl/FALSE" style
Just testing the waters a bit more.  Yeah, definitely nicer.

Closes: #722
Approved by: jlebon
2017-03-09 14:26:17 +00:00
Georges Basile Stavracas Neto
bb3a0e3fa4 ostree: allow setting update frequency from command line
After commit 80b3edc64731a5f0 introducing the option to set
a custom timeout, adapt the ostree program to be able to update
that.

Closes: #725
Approved by: jlebon
2017-03-08 20:28:27 +00:00
Georges Basile Stavracas Neto
d8ac9f75cd repo-pull: add option to set the async update frequency
When using Flatpak with GNOME Software, it is important to
show the progress of the download and install as close as
possible to the real progress.

However, OSTree forces the frequency to call the async
progress callback to 1 second, which causes an unpleasant
effect on the UI, specially when the download size is so
small that everything happens in less than 1 second.

Fix that by adding making OSTree read a custom 'update-frequency'
option and set the timeout source timeout to that. If
no custom frequency is passed, we assume the default 1
second timeout, maintaining the current behavior.

Closes: #725
Approved by: jlebon
2017-03-08 20:28:27 +00:00
Christian Hergert
031d7898cc repo/checkout: fix 32-bit builds
__dev_t is 64-bit even on 32-bit Linux systems such as i386.

Closes: #724
Approved by: cgwalters
2017-03-08 14:01:10 +00:00
Colin Walters
94948e3522 checkout: Support a "pure addition" mode
I plan to use this for `rpm-ostree livefs`.
https://github.com/projectatomic/rpm-ostree/issues/639

Closes: #714
Approved by: jlebon
2017-03-06 20:58:04 +00:00
Colin Walters
ff34810097 repo/checkout: Verify early if src/destination are on same device
At least in all Linux kernels up to today, one can never `link()` across
devices, so we might as well verify that up front. This will help for a future
patch to add a new type of union-add checkout, since Linux checks for `EEXIST`
before `EXDEV`.

Closes: #714
Approved by: jlebon
2017-03-06 20:58:04 +00:00
Colin Walters
3219a5d0ee Allow and start using C99 declaration-after-statement
I've seen code in a few places that I think on balance is definitely better this
way.  Some of our functions have huge variable declaration sections.
This change includes one small example where we could start using declarations
after statements.

A concern I had was - how does this interact with `__attribute__((cleanup))` and
early returns? I tested it, and AFAICS the behavior is what you'd expect - the
cleanup function isn't called if its variable isn't reachable.

Closes: #718
Approved by: jlebon
2017-03-06 18:33:50 +00:00
Philip Withnall
574c3ea6f9 libostree: Allow compression level to be set for archive-z2 stream
Add a ostree_raw_file_to_archive_z2_stream_with_options() variant of
ostree_raw_file_to_archive_z2_stream(), to allow a compression-level
option to be passed in and passed through to zlib.

This is useful when building archive-z2 files on the fly for
transmission over a non-bandwidth-limited channel, such as a local
network. In this case, CPU time is more valuable than bandwidth, so we
want a low compression level.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #721
Approved by: cgwalters
2017-03-06 18:19:45 +00:00
Colin Walters
72336f1c48 fetcher/curl: Fix leaks caught by ASAN
I had to rebuild `glib` with `-fsanitize=address` in order to get a stack trace
to finally get this one.  However, *installing* that glib "system wide"
in my container breaks everything (including `rpm-ostree`, `dnf`, `pkg-config` etc.)
that wasn't built with ASAN.

So my test scenario right now is to extract the libs and do e.g.:
```
make && env LD_LIBRARY_PATH=$HOME/src/distgit/fedora/glib2/asan-libs make check TESTS=tests/test-basic.sh
```

Closes: #719
Approved by: jlebon
2017-03-06 17:30:19 +00:00
Philip Withnall
f667a82fc1 build: Fix disabling --enable-man if xsltproc is not available
If --enable-man=maybe (the default), and xsltproc is not available, the
configuration code would check for it, correctly set enable_man=no, then
incorrectly overwrite that with enable_man=yes, which would result in
later trying to execute $(XSLTPROC) when it’s empty.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #720
Approved by: cgwalters
2017-03-06 13:48:17 +00:00
Colin Walters
a787e0c072 pull: Use all available commits for delta sources
The previous logic for static deltas was to use as a FROM
revision the current branch tip.  However, we want
to support deltas between branches in an automatic
fashion.

If a summary file is available, we already have an
enumerated list of deltas - so the logic introduced
here is to search it, and find the newest commit
we have locally that matches the TO revision target.

This builds on some thoughts from
https://github.com/ostreedev/ostree/pull/151#issuecomment-232390232

Closes: https://github.com/ostreedev/ostree/pull/151

Closes: #710
Approved by: giuseppe
2017-03-04 10:11:14 +00:00
Colin Walters
b41f150a72 grub2: Use "linux16" only on x86/x86_64
Got a report that a Fedora Atomic Host built for ppc64le didn't work with the
`linux16`, it needed `linux`. See the comments for more links.

Closes: #716
Approved by: vathpela
2017-03-03 15:36:36 +00:00
Jonathan Lebon
c4f6522829 libglnx: bump for -Wmaybe-uninitialized fix
https://github.com/GNOME/libglnx/pull/37

Closes: #715
Approved by: cgwalters
2017-03-02 20:35:24 +00:00
Jonathan Lebon
199646ccfa pull: don't use static deltas if archive repo
In https://github.com/ostreedev/ostree/pull/408, we disabled the use of
static deltas when mirroring. Later,
https://github.com/ostreedev/ostree/pull/506 loosened this up again so
that we could use static deltas when mirroring into bare{-user} repos.

However, the issue which originally spurrred #408 is even more generic
than that: we want to avoid static deltas for any archive repo, not just
when doing a mirror pull. This patch tightens this up, and also
relocates the decision code to make it easier to read.

Closes: #715
Approved by: cgwalters
2017-03-02 20:35:24 +00:00
Gatis Paeglis
64422a7d0b deltas: Expose the filename parameter
The C API (ostree_repo_static_delta_generate) knows what to do
with it, but this parameter was never exposed via command line
tool.

Closes: https://github.com/ostreedev/ostree/issues/695

Closes: #703
Approved by: jlebon
2017-02-27 22:10:26 +00:00
Colin Walters
9695c47684 grub2: Use g_spawn_sync() rather than GSubprocess to avoid SIGCHLD
Due to the async nature of `GSubprocess` it grabs `SIGCHLD` which
affects other software which might be using libostree, such as
QtOTA.

Closes: https://github.com/ostreedev/ostree/issues/696

Closes: #702
Approved by: jlebon
2017-02-27 21:55:17 +00:00
Colin Walters
36b28cb4d2 upgrade: Add support for --pull-only and --deploy-only
This makes it easier to script downloading updates in the background,
and only do deployments just before rebooting.

Partially addresses https://github.com/ostreedev/ostree/issues/640

Closes: #642
Approved by: jlebon
2017-02-27 20:57:23 +00:00
Colin Walters
2c326d705e fetcher: Log failures into journal
Particularly when HTTP requests fail, I really want a lot more information.
We could theoretically stuff it into the `GError` message field, but
that gets ugly *fast*.

Using the systemd journal allows us to log things in a structured fashion.
Right now e.g. rpm-ostree won't be aware of this additional information,
but I think we could teach it to be down the line.

In the short term, users can learn to find it from `systemctl status rpm-ostreed`
or `journalctl -b -r -u rpm-ostreed`, etc.

One thing I'd like to do next is log successful fetches of e.g. commit objects
as well with more information about the originating server (things like the
final URL if we were redirected, did we use TLS pinning, what was the negotiated
TLS version+cipher, etc).

Closes: #708
Approved by: jlebon
2017-02-24 22:30:24 +00:00
Colin Walters
cee57a0268 deploy/libmount: Fix build with old util-linux 2.23 (CentOS7)
https://github.com/ostreedev/ostree/pull/705 broke the build
on CentOS 7 which only has util-linux 2.23.

When I was thinking about this, I realized that there must really be a way to
make this safe even for older versions. Looking at that version of util-linux,
all we need to do is invert the order of frees so we `mnt_free_table()` *before*
`mnt_free_cache()`, like util-linux does:

https://github.com/karelzak/util-linux/blob/stable/v2.23/sys-utils/eject.c#L1131

We still use the `_unref()` versions if available.  I also fixed
the ordering there too for double plus redundant safety.

Closes: #712
Approved by: jlebon
2017-02-24 17:24:15 +00:00
Colin Walters
877a27da0f tree-wide: Squash noncritical compiler warnings
Should fix everything from
<https://kojipkgs.fedoraproject.org//packages/ostree/2017.2/3.fc25/data/logs/x86_64/build.log>

Anything that uses autocleanups should *always* be initialized directly I think,
even if a few lines down we directly assign, since this way it's more robust
against refactoring.

And the `freopen()` warnings are right - IMO we should *always* check return
values.

Closes: #711
Approved by: jlebon
2017-02-24 17:08:27 +00:00
Colin Walters
a71d550860 man/repo-config: Document mirrorlist
We should get more strict about docs. Add some text about `contenturl` and
mirrorlists.

Closes: #709
Approved by: jlebon
2017-02-24 14:39:35 +00:00
Colin Walters
0817be61a1 deploy: Correctly use libmount unref() calls rather than free()
We saw a random ostree SEGV start popping up in our CI environment:
https://github.com/projectatomic/rpm-ostree/pull/641#issuecomment-281870424

Looking at this code more and comparing it to what util-linux does, I noticed we
had a write-after-free, since `mnt_unref_table()` will invoke
`mnt_unref_cache()` on its cache, and that function does:

```
	if (cache) {
		cache->rfcount--;
```

unconditionally.

Fix this by using `unref()`.

Closes: #705
Approved by: jlebon
2017-02-23 15:31:25 +00:00
Colin Walters
09b392675a main: Make ostree --version output YAML (and add gitrev)
I learned today that `docker version` does this and I really like
the idea.  While we have the patient open, also add the gitrev
with code taken from https://github.com/projectatomic/rpm-ostree/pull/584

Closes: #691
Approved by: giuseppe
2017-02-22 18:57:18 +00:00
Colin Walters
3ec509c89b build: Add --with-smack, use it to reset contexts for writing objects
At some point we'll want to follow what systemd is doing and add
better support for smack, along the lines of `OstreeSePolicy`.  However,
short term this patch fixes AGL which uses Smack.

See: https://jira.automotivelinux.org/browse/SPEC-386
See: https://github.com/ostreedev/ostree/pull/698

Closes: #698
Approved by: OYTIS
2017-02-22 14:37:19 +00:00
Colin Walters
515f832067 ci: Hard error on all -fsanitize=undefined warnings
I saw in a recent test log a ton of spam
```
libglnx/glnx-dirfd.c:253:3: runtime error: null pointer passed as argument 1, which is declared to never be null
```
which actually turned out to be libglnx getting reverted.  But
let's be sure now we actually bomb out quickly on UBSAN warnings
in general.

Closes: #693
Approved by: jlebon
2017-02-21 16:59:19 +00:00
Colin Walters
6d7e85bb38 libglnx: Re-bump to master due to accidental reversion
Commit a1805d6101 reverted
this unintentionally.

We should have some CI check that requires a commit message has
something like "libglnx bump" or something?

Closes: #693
Approved by: jlebon
2017-02-21 16:59:19 +00:00
Colin Walters
7d7ab92a05 README.md: Add more/clean up links to consuming projects
Add meta-updater and QtOTA, and delete OpenEmbedded since it's implied by the
first two. Merge rpm-ostree + Atomic Host since they're close. Clarify
gnome-continuous a bit.

Closes: #700
Approved by: jlebon
2017-02-21 16:34:14 +00:00
Colin Walters
f02dcc4997 libtest: Re-enable quiet mode for building fs tree
I think I commented this out while debugging something, and forgot to re-enable
it. Reading the log files should be a better again after this.

Closes: #699
Approved by: giuseppe
2017-02-21 15:35:23 +00:00
Colin Walters
c9356a50b8 lib: Ensure an error is set in ensure_unlinked() if errno != ENOENT
We hit this with:
```
27411 unlink("/boot/efi/EFI/fedora/grub.cfg.new") = -1 EROFS (Read-only file system)
```
from the grub2 code.

https://github.com/projectatomic/rpm-ostree/issues/633

Closes: #694
Approved by: giuseppe
2017-02-20 14:36:09 +00:00
Colin Walters
6b93cb3173 ci: Install PyYAML
We'll use it for https://github.com/ostreedev/ostree/pull/691

Closes: #692
Approved by: jlebon
2017-02-17 21:38:14 +00:00
Colin Walters
b5c5003ff6 pull: Fold together deltapart+fallback count for display
It's just simpler, and I'm not sure people are going to care
much about the difference by default.

We already folded in the fallback sizes into the download totals, so folding in
the count makes things consistent; previously you could see e.g.
`3/3 parts, 100MB/150MB` and be confused.

Closes: #678
Approved by: giuseppe
2017-02-17 14:58:25 +00:00
Colin Walters
693f7c5f60 pull: Explicitly error out if metadata objects are fallbacks
I don't know why I added support for this; it makes no sense really. If we have
large metadata objects something has gone badly wrong.

The delta compiler has always only processed fallbacks for regular
content files.

Dropping support in the fetcher for this will simplify later handling of
fallback progress accounting.

Closes: #678
Approved by: giuseppe
2017-02-17 14:58:25 +00:00
Colin Walters
e1118e320d repo: Fix static delta progress display
There were a few bugs here.

- We need to keep track of the size of the delta parts we've already processed,
  in order to make progress reliable at all in the face of interruptions.  Add
  a new `fetched-delta-part-size` async progress variable for this.
- The total before disregarded what we'd already downloaded, which was confusing.
  Now, a progress percentage is `fetched/total`.
- Correctly handle "unknown bytes/sec" in the progress display.

However, to be fully correct we need to show the fallback objects too. That
would require tracking in the pull code when we fetch an object as a fallback
versus "normally". This would be simpler really if we could assume in a run we
were *only* processing a delta, but currently we don't do that.

Related: https://github.com/ostreedev/ostree/issues/475

Closes: #678
Approved by: giuseppe
2017-02-17 14:58:25 +00:00
Colin Walters
0142e5ff39 delta-show: Don't dump whole superblock, do show fallback checksums
Doing `g_variant_print (superblock)` is unreadable and not very useful,
since we show the checksums as byte arrays.

However, do show the checksums for fallback objects. This makes it easier to see
which objects are fallbacks (and inspect why).

Closes: #678
Approved by: giuseppe
2017-02-17 14:58:25 +00:00
Colin Walters
98a4547510 deltas: Don't put unreadable *from* objects in fallback
In https://github.com/ostreedev/ostree/pull/634 we introduced
a subtle regression - the unreadable object was added to the *new*
reachable objects, when it shouldn't have been.  Because it
was a *from* object, clients already had it.

This became more obvious now that I'm working on fixing delta
progress - I noticed my deltas were always starting out with 40MB
fetched, which turned out to be a non-world-readable initramfs object.

This code should simply *skip* the unreadable object, and the delta processing
below properly iterates over "new objects", so we'll pick it up from there.

Closes: #678
Approved by: giuseppe
2017-02-17 14:58:25 +00:00
Giuseppe Scrivano
a1805d6101 contrib/golang: rm directory
rm -r the directory since we are keeping the Go bindings separately
under https://github.com/ostreedev/ostree-go

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #690
Approved by: cgwalters
2017-02-16 20:00:44 +00:00
Colin Walters
da21d7350e Release 2017.2
We should get a release out to try to keep with at least a once-a-month cadence.
This one has some exciting stuff like libcurl and Rust, and various bugfixes.
Also importantly I want to cut this *before* we land some other bigger stuff, so
rpm-ostree can start using the reload_config API etc.

Closes: #685
Approved by: jlebon
2017-02-14 16:21:57 +00:00
Alexander Larsson
4e908f867d rofiles-fuse: Support write/read_buf()
These allow us to avoid copying a lot of data around
in userspace. Instead we splice the data directly from
the fd to the destination fd.

Closes: #684
Approved by: cgwalters
2017-02-14 14:59:28 +00:00
Colin Walters
ba350982e8 build: Remove .PHONY for Rust shared library
I have no idea why I made the lib `.PHONY` originally; it's clearly wrong, and I
noticed because when I was doing `sudo make install`, we were doing a rebuild,
which in turn triggered other things to be built, and they'd be owned by root.

Closes: #682
Approved by: jlebon
2017-02-14 14:27:39 +00:00
Colin Walters
46544f5b4d commit: Support -F/--body-file, like git
This is more convenient to script for projects which haven't
yet made the leap to using the API.

Closes: https://github.com/ostreedev/ostree/issues/674

Closes: #681
Approved by: jlebon
2017-02-14 14:15:08 +00:00
Colin Walters
5a73a366b9 libglnx: Bump
Pulls in the xattr fixes and the tempname perf improvement.

Closes: #680
Approved by: jlebon
2017-02-14 14:02:59 +00:00
Anton Gerasimov
b8f4465b50 admin-switch: Don't segfault if there's no remote
Switching between local branches should be supported too.

Signed-off-by: Anton Gerasimov <anton@advancedtelematic.com>

Closes: #683
Approved by: cgwalters
2017-02-13 20:32:07 +00:00
Philip Withnall
e6a8979e05 ostree-repo: Clarify error behaviour of remote option getters
Clarify the documentation for functions like
ostree_repo_get_remote_boolean_option(), stating what out_value will be
set to on error.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Closes: #676
Approved by: cgwalters
2017-02-10 14:26:00 +00:00
Krisztian Litkey
b13ead1c5b libostree: added empty ot_cleanup_{read,write}_archive macros.
Added empty macros for ot_cleanup_{read,write}_archive to fix
errors when compiling without libarchive.

Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>

Closes: #677
Approved by: cgwalters
2017-02-10 14:06:37 +00:00