Commit Graph

2748 Commits

Author SHA1 Message Date
William Manley
598e3240ff switchroot: Fix test-switchroot now autotools can build static
This test previously depended on manually building ostree-prepare-root.
Since 42dab85 we've been able to build static binaries with the usual
autotools build-system.  This change reflects the fact that
`ostree-prepare-root` is built into $srcdir rather than `src/switchroot`
where I was building manually.

This test now passes with `./configure --with-static-compiler=gcc` (glibc)
but still fails with `./configure --with-static-compiler=musl-gcc` (musl).

Closes: #485
Approved by: cgwalters
2016-09-01 20:17:58 +00:00
William Manley
4d3b93e2ad switchroot: Fix build on Ubuntu
Was failing with error:

    src/switchroot/ostree-prepare-root.c:30:20: fatal error: config.h: No such file or directory
    compilation terminated.

Reported by and fix provided by @gatispaeglis.

Closes: #485
Approved by: cgwalters
2016-09-01 20:17:58 +00:00
Dan Nicholson
39918179d1 build: Actually distribute man page XML source
The make substitution pattern was wrong. The source files are
"ostree.xml", not "ostree.1.xml", for instance.

Closes: #488
Approved by: cgwalters
2016-09-01 19:23:49 +00:00
Dan Nicholson
7adc943596 build: Distribute man page XML source
Without this, the manual pages can't actually be regenerated from a dist
tarball, and running make clean will remove all traces of them.

Closes: #486
Approved by: cgwalters
2016-09-01 19:03:23 +00:00
Dan Nicholson
973ec467ce build: Set --enable-man during distcheck
If xsltproc is not installed, then ENABLE_MAN will be false and the
generated man pages won't be distributed. Pass --enable-man to enforce
that the man pages will be generated and distributed.

Closes: #486
Approved by: cgwalters
2016-09-01 19:03:23 +00:00
Jonathan Lebon
3b55db9661 pull code: clean up mirrorlist hack
While converting the mirrorlist code from using GSList to GPtrArray, I
completely missed the fact that there is now a much cleaner way to do
this.

Closes: #484
Approved by: cgwalters
2016-08-31 18:54:59 +00:00
Jonathan Lebon
661e4636f5 tests: add tests for contenturl and mirrorlist
Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Jonathan Lebon
c4c030cc79 libtest: add has_gpgme() helper function
Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Jonathan Lebon
157d878ce1 pull: add mirrorlist support
This commit adds mirrorlist support to the fetcher. Users can now
prepend url or/and contenturl by mirrorlist= to interpret the link as a
mirrorlist.

If an object is not found, the fetcher will automatically try the next
mirror in the order given in the list (assuming the order returned by
the server is significant).

Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Jonathan Lebon
9546b93382 pull: drop fetching_sync_uri
This made sense back when we used a main loop even when we needed to
fetch objects synchronously. Nowadays, we no longer actually update
progress before the FETCHING_OBJECTS phase, which is only for async
requests.

This allows us to get rid of fetch_uri_contents_membuf_sync() and to
generalize fetch_uri_contents_utf8_sync() so that it only requires a
fetcher. This will be needed later.

Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Jonathan Lebon
bfa8eaccd3 trivial-httpd: prepend timestamp in log file
I've found this useful when monitoring multiple logs at the same time to
test the upcoming content & meta URL splitting.

Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Jonathan Lebon
8d9d3a1d4a pull code: support contenturl setting
Allow users to pass a --contenturl during `remote add` and store it in
the remote config.

Fish out the contenturl setting from the remote config and use it when
downloading static deltas and objects (except for commit signatures).
The idea here is that items in the trust chain (summary & sigs) can be
fetched from a more secure e.g. TLS-pinned location, while objects
themselves are fetched from another location. Once mirrorlist support is
added, this use-case will become even more advantageous.

Closes: #469
Approved by: cgwalters
2016-08-31 16:52:12 +00:00
Colin Walters
dd71999dc9 Release 2016.9
Closes: #481
Approved by: jlebon
2016-08-31 16:36:19 +00:00
Georges Basile Stavracas Neto
cafc517c6b repo-pull: properly store the cancellable
OSTree function ostree_repo_pull_with_options starts a
series of operations that makes heavy use of the PullData's
cancellable.

This isn't effective, however, since nowhere in the code
the OtPullData.cancellable field is set. This is visible,
for example, when trying to cancel a Flatpak pull and nothing
happens, because the cancellable is not properly passed
to the pull data.

Fix that by setting the cancellable field of the pull data. It
owns a reference for safety reasons, and unreferences it at the
end of the operation.

ostreedev/ostree#482

Closes: #483
Approved by: cgwalters
2016-08-31 15:21:06 +00:00
William Manley
1dc69dc879 switchroot: Move path_is_on_readonly_fs to header file
This simplifies the build system by removing the need for
`libswitchroot-mountutil.la`.

Original idea by @cgwalters in #477.

Closes: #478
Approved by: cgwalters
2016-08-30 22:41:27 +00:00
William Manley
a128abd9bc switchroot: Replace custom error printing with err/warn functions from libc
The `warn()` libc extension has exactly the same behaviour as our own
`perrorv` function, but is available in (at least) glibc and musl.  As an
added bonus the similar function `err()` which will exit with an error
code afterwards.

This implementation is tidier and allows us to get rid of our own
`perrorv`.  It paves the way to removing `ostree-mount-util.c` to simplify
the build scripts.

Original idea by @cgwalters in #477.

Closes: #478
Approved by: cgwalters
2016-08-30 22:41:27 +00:00
William Manley
42dab85728 ostree-prepare-root: Allow building statically with musl
If the `--with-static-compiler=musl-gcc` configure flag is given.

ostree-prepare-root can be used as init in a system without a populated
/lib.  To support this use case we need to link statically as we will be
unable to locate libc.so at run time if it's not installed in /lib.

We support building ostree-prepare-root with a different compiler to the
rest of ostree so we can use musl rather than glibc.  This reduces the size
of the executable significantly: from ~700K -> ~30K.  We have to use
`_SCRIPTS` here to get autotools to install this as an executable but
without generating rules to make it itself which we have specified
manually.

See https://lists.gnu.org/archive/html/help-gnu-utils/2007-01/msg00007.html
for advice on using autotools in this manner.

Closes: #477
Approved by: cgwalters
2016-08-30 20:50:33 +00:00
William Manley
faee3df8ae switchroot: Fix building with musl libc
POSIX and GNU define conflicting versions of `strerror_r`.  The GNU
version returns the string but doesn't necessilary write into buf.
The POSIX version writes into buf and returns the length but doesn't
necessilary append a terminate the string with a NUL if it's too long
to fit in buf.

This commit fixes building ostree-prepare-root with musl libc.  The
stripped static build with musl on my machine is 30K vs. 724K with glibc
static and 11K with glibc shared.

Closes: #477
Approved by: cgwalters
2016-08-30 20:50:33 +00:00
Colin Walters
a269075724 commit: Don't delete tmp/cache dir
We hold a fd open on this, and it's basically now expected
to be immortal.  Confer that status.

This was showing up in flatpak crashers, because we'd get
an unexpected errno.

(I didn't test this fixes the crasher, but it's clearly right)

https://bugzilla.redhat.com/show_bug.cgi?id=1347293

Closes: #476
Approved by: alexlarsson
2016-08-29 17:30:15 +00:00
Colin Walters
8ece4d6d51 sysroot: Add a flag to suppress post-deploy cleanup
I noticed seeing the output of `prune` twice in rpm-ostree, and had
always wondered why.  When reading the rpm-ostree code to fix
something else, reasons, I noticed the reason - we were pruning once
here, and then once after rpm-ostree regenerates its "base" refs.

There's no reason to clean twice, so let's add a flag so rpm-ostree
can suppress doing it inside libostree.

Closes: #474
Approved by: giuseppe
2016-08-29 07:45:24 +00:00
Colin Walters
cfc3934e81 sysroot: Drop unnecessary dup() invocation
It's close-on-exec, not close-on-fork.  I was clearly confused when
writing this; it works just fine to reference the fd in the child and
`fchdir()` before exec.  So drop the unnecessary duplication.

Just noticed this while reading the code for a random other reason.

Closes: #473
Approved by: giuseppe
2016-08-28 13:56:03 +00:00
Gatis Paeglis
b6ec7526b5 u-boot: Merge ostree's and systems uEnv.txt
This is a proper fix for:
https://bugzilla.gnome.org/show_bug.cgi?id=755787

With this patch, an admin (system builder) can now:

1) Edit /usr/lib/ostree-boot/uEnv.txt
2) Deploy the new tree. OSTree will append system's uEnv.txt
   to the OSTree's managed uEnv.txt (loader/uEnv.txt).

It is common for u-boot systems to read in an extra env
from external /uEnv.txt. The same file OSTree uses to pass
in its env. With this patch /uEnv.txt now contains OSTree's
env + custom env added by system builders.

Closes: #466
Approved by: cgwalters
2016-08-27 13:11:22 +00:00
Giuseppe Scrivano
47b0b4120a pull: use same name for parameter and documentation comment
Fixes this warning:

src/libostree/ostree-repo-pull.c:2162: Warning: OSTree: ostree_repo_pull_with_options: unknown parameter 'remote_name_or_baseurl' in documentation comment, should be 'remote_name'

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #472
Approved by: jlebon
2016-08-27 12:40:46 +00:00
Colin Walters
3ef4cc2e5b lib: Add an API to list only "our" objects, fix prune to use it
When doing a prune, we should not try to delete objects in parent
repos, since it'll fail.  There is a bigger discussion about the
semantics of `parent=` to be had, but this will fix trying to use
`ostree prune --repo=/ostree/repo/extensions/rpmostree/pkgcache`.

Closes: https://github.com/ostreedev/ostree/issues/467

Closes: #471
Approved by: jlebon
2016-08-25 20:01:37 +00:00
Colin Walters
683e060099 repo: Add prefixes to errors for querying size/deleting
I hit these while causing errors in prune.  Let's add the specific
object we were looking for.

Closes: #471
Approved by: jlebon
2016-08-25 20:01:37 +00:00
Colin Walters
05ac14326c pull: Make .commitpartial files world readable
In CentOS, these happened to appear in a repo that is served
via rsync, and having them not be world-readable caused mirroring
tools to fail.

They aren't secret, so don't make them so.

Closes: #468
Approved by: giuseppe
2016-08-25 13:03:16 +00:00
Jonathan Lebon
76166cb52e pull_with_options: fix stray return FALSE
Closes: #465
Approved by: cgwalters
2016-08-18 14:45:01 +00:00
Jonathan Lebon
21d3063fcf pull_with_options: allow GPG verification override
We used to only look at the "gpg-verify" and "gpg-verify-summary"
options when we're passed a local URL. Make these options also have an
effect in the configured remote case.

Closes: #465
Approved by: cgwalters
2016-08-18 14:45:01 +00:00
Jonathan Lebon
e005da74ef pull_with_options: fix remote parameter name & desc
Closes: #465
Approved by: cgwalters
2016-08-18 14:45:01 +00:00
Colin Walters
77af6844d8 rofiles-fuse: Rework to be based on nlink
Programs like `useradd` try to `open(/etc/passwd, O_RDWR)` to append,
which didn't work with rofiles-fuse.  Thinking about this, I realized
that there's a simpler algorithm for "can we write to this file" which
is "does it have a hardlink count <= 1"?

Switching to this both drops complexity (we no longer need to keep a
hash table of files we created), and also lets useradd work.

Closes: #462
Approved by: jlebon
2016-08-16 21:22:28 +00:00
Colin Walters
3469161e41 test-rofiles-fuse: Actually check out via hardlinks
The test suite was actually doing something before, just
not quite what I intended.  Without `-U` for bare-user checkouts
we end up doing a copy.

Now, a future commit will change how rofiles work, which would cause
the test suite to permit inplace mutation for non-hardlinked files.
So let's ensure they *are* hardlinked.

Closes: #462
Approved by: jlebon
2016-08-16 21:22:28 +00:00
Colin Walters
3c6d2bc120 fixup! fix typo in docs/manual/atomic-upgrades.md
Closes: #461
Approved by: giuseppe
2016-08-12 21:20:08 +00:00
Colin Walters
f9933a2631 prune: Elaborate on what formats are accepted by dates
Inquiring minds want to know.

Closes: #461
Approved by: giuseppe
2016-08-12 21:20:08 +00:00
Adam Miller
e5cae81404 fix typo in docs/manual/atomic-upgrades.md
Signed-off-by: Adam Miller <maxamillion@fedoraproject.org>

Closes: #460
Approved by: cgwalters
2016-08-12 19:40:53 +00:00
Dan Nicholson
b1d13bb356 repo: Really ignore progress changed user data
The documentation says this is ignored, implying that you should pass
NULL to it. However, the function immediately returns in this case even
though the argument isn't used anywhere.

Closes: #458
Approved by: cgwalters
2016-08-12 14:56:07 +00:00
Jonathan Lebon
fa59f5adfb manual/repo.md: reword bits about the summary file
Closes: #459
Approved by: cgwalters
2016-08-11 20:58:07 +00:00
Jonathan Lebon
0728304a8a ostree_bootdir: properly preprend $(prefix)
Closes: #457
Approved by: cgwalters
2016-08-11 15:52:28 +00:00
Jonathan Lebon
182cb273c6 ostree_bootdir: prepend $(prefix) to path
Otherwise we break local installs.

Closes: #456
Approved by: cgwalters
2016-08-11 14:37:55 +00:00
Gatis Paeglis
fd1a044d6e Move ostree-* executables to /usr/lib/ostree
Why not to use libexecdir?

Because this directory does not exist on some distros or paths
between distros varies. There are several reasons why a well
known path is prefered, for example when generating a kernel
command line:

init=${ostree}/usr/lib/ostree-boot/ostree-prepare-root

In addition this saves us some typing in a console when wanting
to access the "ostree" cmd line.

Closes: #449
Approved by: cgwalters
2016-08-11 14:04:59 +00:00
Dan Nicholson
7ed2f1e5af tests: Test that local pulls do not use deltas
Pulls from local repos now default to disabling static deltas so that
objects are copied. To check this is the case, see if the object files
are hardlinked after pulling.

Closes: #447

Closes: #448
Approved by: cgwalters
2016-08-10 10:54:46 +00:00
Dan Nicholson
1caef17c7f pull: Disable static deltas by default for local pulls
For local pulls there's no benefit pulling the static delta over the
individual object files since there's no HTTP overhead. Furthermore,
processing deltas always generates the objects whereas a standard pull
ensures that the exact object files are copied. Using deltas also
prevents hardlinking the objects if the repos exist on the same
filesystem.

Closes: #447

Closes: #448
Approved by: cgwalters
2016-08-10 10:54:46 +00:00
Dan Nicholson
2a810fbef9 tests: Ensure deltas for pulling when needed
When testing pulling of deltas, use the new --require-static-deltas
option to pull-local to ensure that deltas are actually used. To support
the require-static-deltas mode, the summary in the remote repo must be
generated.

Closes: #447

Closes: #448
Approved by: cgwalters
2016-08-10 10:54:46 +00:00
Dan Nicholson
80c573478b pull-local: Support requiring static deltas
Add the --require-static-deltas option like pull to ensure static deltas
are processed for local pulls.

Closes: #447

Closes: #448
Approved by: cgwalters
2016-08-10 10:54:46 +00:00
Dan Nicholson
c3c07a2d49 deltas: Use F_DUPFD_CLOEXEC properly
You need to supply an argument to F_DUPFD_CLOEXEC or fcntl will return
EINVAL. Use 3 as the minimum fd number as is standard.

Closes: #454

Closes: #448
Approved by: cgwalters
2016-08-10 10:54:46 +00:00
Dan Nicholson
1e5ff71c49 deltas: Handle cleanup of fd array properly
If there's an early error, part_temp_fds will be NULL and dereferencing
the len member will segfault.

Closes: #454

Closes: #448
Approved by: cgwalters
2016-08-10 10:54:46 +00:00
Colin Walters
07ec35ef03 libostree.sym: Add 2016.9 section
We should remember to do this in the commit updating configure.ac.

Closes: #453
Approved by: jlebon
2016-08-09 20:00:24 +00:00
Colin Walters
73eabca5e8 Release 2016.8
It's time for a new release, there's enough queued and it's
been ~a month.

Closes: #452
Approved by: jlebon
2016-08-09 19:12:25 +00:00
Colin Walters
56a922280f tests: Port sysroot.js away from libgsystem
My case-sensitive `git grep` didn't find this one originally,
but the Travis+Debian setup was configured to run it.

Closes: #451
Approved by: jlebon
2016-08-09 19:01:35 +00:00
Colin Walters
519292114f travis: Disable email notifications
I'm getting spammed.  We'll see the results in the status.

Closes: #450
Approved by: jlebon
2016-08-09 17:36:39 +00:00
Colin Walters
a29bb0ab6b Final excision of libgsystem dependency
Lots and lots of preparation led to this moment - when nothing
apparent changes for users!  Woo!

But seriously, having the extra dependency is a minor annoyance, and
in the big picture I think the libgsystem idea was wrong - we need to
land things in GLib, and use git submodules for API-unstable or
Linux-specific sharing.  For a lot of OSTree, the libgsystem `GFile*`
orientation was also wrong, we really want fd-relative.

Closes: #444
Approved by: jlebon
2016-08-09 15:36:09 +00:00