Commit Graph

2603 Commits

Author SHA1 Message Date
Colin Walters
56e652035b build-sys: Make libostree-1.so depend on the symbol file
Otherwise one changing it doesn't cause a symbol to be exported.

Closes: #365
Approved by: jlebon
2016-06-24 21:06:56 +00:00
Alexander Larsson
3640725439 tests: Test partial commits for local remotes
This was broken before, fixed in the previous commit.

Closes: #324
Approved by: yuqi-zhang
2016-06-24 15:21:38 +00:00
Colin Walters
073c34ca08 pull: Write commitpartial files for local imports too
Just like HTTP fetches, these can be interrupted, so we need to write
the commitpartial files.

Closes: #324
Approved by: yuqi-zhang
2016-06-24 15:21:38 +00:00
Yu Qi Zhang
02a2b689dd refs: resolve conflict between local/remote repos
Add the functionality to use the same name for refs in local and remote
repos. This helps users keep track of local refs of remote origin, much
like local and remote git branches.

Previously, when a local ref is specified, resolve_refspec would fall
back to searching through remote repos if the ref is not found locally.
This function now takes an extra flag to specify whether it should
search through remote repos. Additionally, ostree_repo_resove_rev_ext
was added to call resolve_refspec with fallback_remote being false, so
refs --create would no longer complain when trying to create a local
ref of the same name as a remote one.

Fix remote repo parsing not being handled correctly on refs --create.

Closes: #363
Approved by: jlebon
2016-06-23 19:52:26 +00:00
Alexander Larsson
fc4a7ec35e pull: Correctly handle repo->parent_repo when applying static deltas
In flatpak i was using a parent repo, and it failed to update
with ENOENT when dispatching an set-read-source opcode, because the
object it referenced was in the parent repo.

This fixes that by making _ostree_repo_read_bare_fd look
at parent_repo.

Closes: #362
Approved by: cgwalters
2016-06-23 11:36:33 +00:00
Mathnerd314
9a779563bb refs: Fix a logic error
I encountered the Opening remotes/ dir error with some broken pull code, and this fixes it.

Closes: #358
Approved by: cgwalters
2016-06-22 20:15:42 +00:00
Mathnerd314
23049bbd01 core: Add OSTREE_OBJECT_TYPE_COMMIT_META
This is cleaner than the loose_path_with_suffix approach

Closes: #359
Approved by: cgwalters
2016-06-22 16:10:01 +00:00
Mathnerd314
55f5f73d80 configure: Turn on -Wempty-body
I spent half an hour debugging an extra semicolon,
and this C "feature" is not used at all in ostree

Closes: #359
Approved by: cgwalters
2016-06-22 16:10:01 +00:00
Mathnerd314
4cb77c51db core: Use OSTREE_SHA256_STRING_LEN instead of 64
Closes: #359
Approved by: cgwalters
2016-06-22 16:10:01 +00:00
Krzesimir Nowak
91ccaff197 core: Fix wrong return value docs
ostree_commit_get_parent() returns a string form of a checksum, not a
binary form.

Closes: #360
Approved by: cgwalters
2016-06-22 12:45:12 +00:00
Yu Qi Zhang
744543110e refs: allow overwrite of empty folders
We noticed that once a ref folder is created, there is no existing
command that can remove it. For example, once "foo/bar" is created,
even if the user deletes foo or all the refs under foo, the folder
will persist.

Now when the user attempts to create a ref "foo" either through commit
or refs --create, if a folder "foo" exists but is empty of refs, the
folder is removed and the new ref "foo" is created.

New unit tests in tests-ref.sh verify this functionality.

Closes: #354
Approved by: cgwalters
2016-06-21 20:20:15 +00:00
Colin Walters
f6ce04e480 libglnx porting: Drop uses of gs_file_openat_noatime
We're not really doing the "noatime" thing anymore.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Colin Walters
90b9a06277 lib: Use g_file_enumerator_iterate() if available, with fallback
Import `gs_file_enumerator_iterate()` for the next six months or
so...after RHEL 7.3 is released I'm strongly considering hard
requiring 2.46 or so.

Likely at some point we should figure out how to share more "glib
backport" code with NetworkManager at least.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Colin Walters
9e2763106b lib: Use sd_journal directly (optionally)
This was the last caller of libgsystem that isn't
`gs_file_get_path_cached()`.  I think the use case ostree has where
the same code can be called via command line and via a shared library
*and* via a daemon is rather unusual, so let's just copy the code for
logging from libgsystem into here.

For example rpm-ostree hard depends on a daemon mode, so it'll just
use `sd_journal` directly.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Colin Walters
4819b44189 libglnx porting: Use of GSDirFdIterator
This one was pretty simple.  One of the uses in `repo.c` was just a
leftover variable.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Colin Walters
0134c62157 libostree.sym: Fix test-symbols
The test isn't smart enough to ignore comments, so change the prefix.

Closes: #356
Approved by: jlebon
2016-06-20 22:59:27 +00:00
Colin Walters
6c7e191147 Release 2016.6 2016-06-20 11:51:32 -04:00
Colin Walters
535033a4f0 pull: Ensure we always process queue only from main thread
I was easily reproducing a hang on pulls with thousands of requests on
current git master.  The initial symptom seemed to be that there are
multiple code paths where we don't invoke
`session_thread_process_pending_queue()`.  We really need to do
that any time we remove something from the outstanding queue,
to ensure it gets filled again.

A further issue is that we were tying the lifecycle of the pending
object to the `GTask`, but the task could be unref'd from the main
thread (via a `GSource` on the main thread), and that introduced
threadsafety issues, because the hash table and other data suddenly
could be concurrently modified.

Both of these need to be fixed together.  First, we introduce
`Arc<Pending>`, and ensure that both the main and worker threads hold
references.

Second, we ensure that we re-process the queue *immediately* whenever
a task is done, inside the worker thread, rather than doing it
incidentally via an unref.  This architecture is quite similar to what
the outside pull code is doing.

Closes: #350
Approved by: jlebon
2016-06-17 18:19:23 +00:00
Colin Walters
d262fc2e0f tests: Support OT_SKIP_READDIR_RAND
For some reason I'm really not inclined to debug right now, `libtool`
is eating the `-fsanitize=address` argument when linking
`libreaddir-rand.so`, which causes failures since we're trying to
`LD_PRELOAD`.

Closes: #351
Approved by: jlebon
2016-06-17 14:43:44 +00:00
Colin Walters
71e768d230 build: Fix libreaddir-rand to honor global CFLAGS
It's unfortunate that in automake one has to explicitly include the globa
`$(AM_CFLAGS)` if one sets `CFLAGS`, and similarly for other variables.

I'm trying to use `-fsanitize=address`, and not including it was
causing linker failures.

We also weren't inheriting the global warnings etc., so I had to fix a
decl-after-statement.

Closes: #351
Approved by: jlebon
2016-06-17 14:43:44 +00:00
Colin Walters
d03ae448d0 manual: Discuss mirroring
This should likely be its own section, but it makes enough sense here
for now too.

Closes: #347
Approved by: yuqi-zhang
2016-06-17 14:14:08 +00:00
Colin Walters
651f4bc3b9 repo: Avoid a possible divide by zero in progress
The previous code was subject to a divide by zero if less than a
second had passed.  Rework it so we only do the divide if more than a
second has passed.

Closes: #349
Approved by: Mathnerd314
2016-06-17 02:30:13 +00:00
Yu Qi Zhang
30c34b2f55 libglnx porting: delete temp files on failure of file creation
We noticed the temp files being left over in ostree when (mistakenly)
trying to create refs with names in use by folders. This fix removes
temp files created by glnx_file_replace_contents_at on failure.

Closes: #348
Approved by: cgwalters
2016-06-16 19:18:47 +00:00
Colin Walters
ab47a8a030 Add "archive" as an alias for "archive-z2"
I find the "-z2" is really a long ago relic of the past when I changed
the format.  We no longer have anything to do with the original
`archive`, so let's start allowing people to type `--mode=archive`
which just looks saner.

At some point later I'll update the docs too, but it'll be an annoying
transition period as we'll have to say "On older OSTree, use -z2" etc.

Closes: #346
Approved by: giuseppe
2016-06-16 14:34:23 +00:00
Colin Walters
85f202d0b2 manual: Link to mender.io
Came out of a discussion on the list.

Closes: #344
Approved by: jlebon
2016-06-15 20:54:55 +00:00
Yu Qi Zhang
f8fcdba0a5 refs: add "ostree refs --create" and unit tests
Added the ability to create a ref (much like a git tag) for an
existing commit through "ostree refs EXISTING --create=NEWREF".
Previously the only way to create a new ref was by creating a new commit,
but refs --create allows multiple refs to point to the same commit.

The command will fail if:
 - None/more than one existing ref is specified
 - The specified EXISTING tag does not exist, or was not specified
 - The specified NEWREF already exists, or is the name of a folder

Add unit tests in tests-ref.sh to verify above functionality

Closes: #340
Approved by: jlebon
2016-06-15 20:42:30 +00:00
Colin Walters
7847bc7394 lib: Port some manual close() cleanups to be glnx_fd_close
Just noticed this while reading some code, we didn't have many manual
`out: close()` bits left, this pushes us over the edge to autocleanup
almost everywhere.

Closes: #332
Approved by: jlebon
2016-06-13 14:58:55 +00:00
Simon McVittie
14ff4f94fb build: re-create tests/ostree-symlink-stamp if Makefile changes
The filename of the real ostree executable could be either .libs/ostree
or .libs/lt-ostree.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
4801e9b30b libostreetest: include libtest.sh from srcdir, not builddir
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
8011d8c627 libtest: make gpghome in tmpdir writeable
Otherwise, during "make distcheck" it will be read-only (because it
is a copy of the read-only ${srcdir} with permissions preserved), and
deletion will fail during cleanup.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
70a1118955 build: distribute more test files
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
906575271c build: distribute TAP helper scripts in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:04 +00:00
Simon McVittie
c337d4dab5 build: distribute test scripts in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:03 +00:00
Simon McVittie
19cfe2d73a build: distribute libostree.sym in tarballs
Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:03 +00:00
Simon McVittie
c930eefeea autogen.sh: rely on VPATH for submodules' Makefile-*.am.inc
This avoids hard-coding the autogen-time ${srcdir} into our build
system, and might fix "make distcheck".

We don't need to specify ${srcdir} explicitly in the various variables,
because Automake always uses make's VPATH feature to look in both
${builddir} and ${srcdir} at build time.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #337
Approved by: cgwalters
2016-06-13 00:23:03 +00:00
Simon McVittie
b9e18b83fd Link libreaddir-rand to libdl
It uses dlsym(). There's no point in being extra-portable here
because OSTree only targets Linux anyway.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #336
Approved by: cgwalters
2016-06-13 00:12:55 +00:00
Simon McVittie
0cf0fa8154 test-sysroot.js: set "strict mode" when sourcing libtest.sh
As with the C tests in commit 08580118, this makes sure the test
fails as soon as something goes wrong.

Signed-off-by: Simon McVittie <smcv@debian.org>

Closes: #335
Approved by: giuseppe
2016-06-12 11:26:58 +00:00
Alexander Larsson
d258b124a2 tests: Exit valgrind tests if a leak is detected
This fails the test if OT_TESTS_VALGRIND is set and valgrind
detects a leak.

Closes: #334
Approved by: cgwalters
2016-06-10 12:14:40 +00:00
Alexander Larsson
f6b70a32e6 pull: Fix leak of refs_to_fetch
Closes: #333
Approved by: giuseppe
2016-06-10 11:12:59 +00:00
Alexander Larsson
c9c4ab851e pull: Fix leaked uri in request_static_delta_superblock_sync
Closes: #333
Approved by: giuseppe
2016-06-10 11:12:59 +00:00
Alexander Larsson
785f894de1 pull: Fix leak of FetchObjectData in some cases
Closes: #333
Approved by: giuseppe
2016-06-10 11:12:59 +00:00
Alexander Larsson
bd4e250e93 pull: Fix leak of debug temporary strings
Closes: #333
Approved by: giuseppe
2016-06-10 11:12:59 +00:00
Alexander Larsson
8e3d558f23 libotutil: Fix leak of VariantMapData
Closes: #333
Approved by: giuseppe
2016-06-10 11:12:59 +00:00
Colin Walters
70af1d26b1 tests: Modernize valgrind infrastructure
The recent memleak fixes motivated me to look at the bitrotted code to
run invocations of `ostree` in the test suite underneath valgrind.

There are a few things here.  First, update suppressions file from
libhif, since I recently worked on it.

When running *uninstalled* as we now support, we need
`libtool --mode=execute` in the mix so it expands out to
the uninstalled binary and we don't valgrind the intermediate shell.

However, it's harder than that because we chdir into a tmpdir,
which defeats the libtool logic.  AFAICS, the only fix for this
is to determine the realbin path before we chdir, and then unfortunately
we need to change every use of `ostree` to `${OSTREE}` =(

Then this immediately breaks for me on RHEL7 because my ancient
copy of `valgrind-3.10.0-16.el7.x86_64` is unaware of syscall 306, i.e.
`syncfs`.

But let's do this first before I dive into that.

Closes: #292
Approved by: krnowak
2016-06-09 21:10:35 +00:00
Colin Walters
c015fe13fb lib: Add OSTREE_SUPPRESS_SYNCFS environment variable
Just to work around valgrind not understanding the `syncfs()` syscall
in EL7 right now.

Closes: #292
Approved by: krnowak
2016-06-09 21:10:35 +00:00
Colin Walters
0f7bf7be95 lib: Fix a leak in remote parsing
As the docs say, `g_regex_match()` still allocates a match even if it
returns `FALSE`.  Using `g_autoptr` is just plain better.

Closes: #292
Approved by: krnowak
2016-06-09 21:10:35 +00:00
Mathnerd314
04ca15cb0e related-projects: Update with more information and projects
In particular, NixOS has changed somewhat, and Conda is worth
looking at. Also it seems reasonable to mention rpm-ostree /
Gnome Continuous.

Closes: #331
Approved by: cgwalters
2016-06-09 18:36:52 +00:00
Mathnerd314
1b88dc7f90 docs: Get API docs working again
This changes around a few things that didn't work for me:
* Section names seem to be ostree-* instead of libostree-*
* Also XML files are ostree-* (they didn't show up at all)
- gtk-doc doesn't seem to parse const _OSTREE_PUBLIC correctly
* pull documentation is now on the actual functions rather than stubs
* Update gitignore with some more files

And there some changes to make gtk-doc give fewer warnings (not finished)

Closes: #327
Approved by: cgwalters
2016-06-09 18:15:49 +00:00
Mathnerd314
693b4ec4be pull: Move pull and pull_onedir back into ostree-repo.c
They call pull_with_options, which will fail anyway.

Closes: #327
Approved by: cgwalters
2016-06-09 18:15:49 +00:00
Mathnerd314
da5cb099a6 pull: Move libsoup-related code from ostree-repo.c to ostree-repo-pull.c
This centralizes the ifdef's in one file, which will make it
easier to write new pull backends.

ostree-repo-pull.c is now built unconditionally

Closes: #327
Approved by: cgwalters
2016-06-09 18:15:49 +00:00