Commit Graph

1049 Commits

Author SHA1 Message Date
Colin Walters
ee73c0fa34 core: Add dynamic uncompressed object cache for archive-z
This gives us something closer to the advantages of archive and
archive-z when using the latter.  Concretely we get deduplication
among multiple checkouts, along with the "devino" hash table trick
during commits to avoid checksumming content again.

This is enabled by default.
2012-10-11 18:33:03 -04:00
Colin Walters
3c0fe01152 pull: Print elapsed time 2012-10-05 17:20:40 -04:00
Colin Walters
52a0b7bdb1 core: Only do devino scan on commit (speeds up pull)
When fetching data remotely, there's no point to the devino scan
because we're not going to be committing local files.

Only do it for "commit".
2012-10-05 17:12:39 -04:00
Colin Walters
efe27f1b05 pull: Ensure worker queue finishes if we unhold an empty queue too 2012-10-04 20:32:14 -04:00
Colin Walters
9618232f4d pull: Stage content asynchronously
For similar reasons as metadata, this avoids having the main thread
blocked in fdatasync(), and even better - we can achieve much higher
parallelism if we have multiple threads blocked on fdatasync().
2012-10-04 20:00:00 -04:00
Colin Walters
5b8e833351 pull: Stage metadata objects asynchronously
This avoids the main thread being blocked on fdatasync(); also as a
bonus we checksum metadata in a separate thread too.
2012-10-04 18:24:37 -04:00
Colin Walters
f1b4db15a2 pull: Ensure queued filemeta requests don't starve everything else
We need this hack for "archive mode" repositories; otherwise,
what ends up happening is that we get 10000+ requests pending
for .filemeta files, which we can't process until we also get
the .filecontent.

Note this hack is unneccessary when fetching from archive-z
repositories.
2012-10-04 18:23:18 -04:00
Colin Walters
4e51701bea pull: Merge metadata and content fetch phases
This is a notable speedup when the metadata scanner is working, and we
don't have a lot of traffic; we can pull down data at the same time.
2012-10-04 15:37:23 -04:00
Colin Walters
5c1dc390ae pull: Asynchronous metadata fetch
Create a worker thread for processing metadata, reserving the main
thread for HTTP requests.

This can create a very significant efficiency win for large pull
requests since we are much more likely to keep a full pipeline open.

The status display is also nicer now.
2012-10-03 22:35:33 -04:00
Colin Walters
5f25762122 admin: Install grub2 config file if we detect /etc/grub.d
Decouple this from the kernel postinst one, since it's possible to
have one but not the other.
2012-10-01 19:34:25 -04:00
Colin Walters
8153799121 admin: Prepare /var/log/dracut.log when creating initramfs
Otherwise dracut complains.
2012-10-01 10:25:36 -04:00
Colin Walters
758d6132e3 Release 2012.11 2012-09-25 22:25:06 -04:00
Colin Walters
3e9c358628 Update libgsystem
This ensures we're disting the README, which makes the autogen.sh do
the right thing.
2012-09-25 22:25:06 -04:00
Colin Walters
044881b60f build: Add one header file missing from dist 2012-09-25 19:13:05 -04:00
Colin Walters
8922871733 build: Add autogen.sh, the COPYING.{GPL,LGPL} files to EXTRA_DIST
We want them in tarballs too.
2012-09-25 18:44:08 -04:00
Colin Walters
503535da48 Release 2012.10 2012-09-25 18:08:18 -04:00
Colin Walters
7559759a91 packages: Two spec file fixes 2012-09-25 18:08:18 -04:00
Colin Walters
0aab024602 autogen: Only do submodule update if we don't have libgsystem
Otherwise we'll fail due to not being a git repository in the
embedded-dependency tarball case.
2012-09-24 22:20:36 -04:00
Colin Walters
a864e2cc19 autogen.sh: Use set -e
Otherwise we blindly continue if autoreconf fails, which is pretty
damn lame.
2012-09-24 08:30:54 -04:00
Colin Walters
26e487bacf core: use open(..., O_NOATIME) for reading loose objects
More efficient, and atime shouldn't matter for these.
2012-09-24 07:36:42 -04:00
Colin Walters
7aa620109e Add tests missing from previous commit 2012-09-23 19:25:09 -04:00
Colin Walters
40ce43036f Add an archive-z repository mode
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode.  This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
2012-09-23 19:23:45 -04:00
Colin Walters
35a1ff51d0 pull: Fix two minor memory leaks 2012-09-23 17:32:11 -04:00
Colin Walters
34c49f0411 core: Clean up staging API and internals
Cleanly separate metadata/content APIs, rather than defaulting to
raw streams.  This helps most use cases.

Also, drop support for staging content without knowing the total
length.  This complicated the code, and for things like streaming
HTTP, we should be able to figure this out from Content-Length.
2012-09-23 16:44:12 -04:00
Colin Walters
2a0601efc7 core: Drop packfiles as they are now
They're not a large efficiency win at the moment, because we don't
do any delta compression.

At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
2012-09-23 16:02:03 -04:00
Colin Walters
aa9aaa9a62 tests: Drop tests of removed --atomic-retarget option 2012-09-23 15:56:28 -04:00
Colin Walters
a8764bf996 diff: Fix case when used on native files 2012-09-23 15:55:02 -04:00
Colin Walters
08be599d50 packaging: Update spec file per Fedora review
See https://bugzilla.redhat.com/show_bug.cgi?id=819951
2012-09-20 20:12:01 -04:00
Colin Walters
1a274aecd6 packages: Update ostree.spec for file changes 2012-09-20 19:52:13 -04:00
Colin Walters
c746f3b37c packages: Tweak srpm command
This spec should be usable for both embedded dependencies and not.
2012-09-20 18:00:06 -04:00
Colin Walters
a6e9e54891 admin: Initialize /var more fully
This helps out gnome-ostree, because gdm needs /var/lib for example.
2012-09-18 08:58:07 -04:00
Giovanni Campagna
2881194186 Grub2: use /etc/system-release now that we have one
This matches the upstream code more, and avoids the GNOME OS trademark
in ostree, which is otherwise platform agnostic.

https://bugzilla.gnome.org/show_bug.cgi?id=684049
2012-09-17 00:25:30 +02:00
Giovanni Campagna
8c9406637c Add /ostree/current-etc symlink to current configuration
Configuration associated with a specific revision is stored in a folder
named <revision>-etc. In a similar spirit, add /ostree/current-etc, pointing
to the -etc folder for the revision named by current. This allows
easy editing of configuration from the host distribution, and allows
diffing current/etc and current-etc for configuration changes.

https://bugzilla.gnome.org/show_bug.cgi?id=684049
2012-09-17 00:25:29 +02:00
Giovanni Campagna
8a98e8d964 Update kernel files for command naming changes
ostadmin is now ostree admin.

https://bugzilla.gnome.org/show_bug.cgi?id=684049
2012-09-17 00:00:33 +02:00
Colin Walters
59ddbcc500 pull: Check the remote repo type
We can only pull from archive repositories right now.  This will also
be useful for adding compressed archives later.
2012-09-15 12:41:25 -04:00
Colin Walters
31153913ff Extract keyfile helpers into libotutil
Will be used by ostree-pull too soon.
2012-09-15 11:20:56 -04:00
Colin Walters
117b9c109e ostadmin: Fix up deploy config logic again
Clearly we need tests here...
2012-09-09 11:42:17 -04:00
Colin Walters
2f8b3517f4 admin: Ensure we pass options to subcommands
Otherwise ostree admin --ostree-dir=/foo deploy --no-kernel bar
doesn't work.
2012-09-09 11:31:22 -04:00
Colin Walters
7831ae81c2 update-kernel: Fix free -> g_object_unref 2012-09-09 10:42:13 -04:00
Colin Walters
fa5485c6c4 Move "ostadmin" => "ostree admin"
This helps us avoid polluting the global binary namespace.
2012-09-08 19:34:10 -04:00
Colin Walters
2ddbeb1f71 Make /etc a writable mount
First, move deployments to /ostree/deploy.  Having them in the
toplevel clutters the filesystem layout too much.

When we deploy a tree like /ostree/deploy/NAME, there is now also a
writable directory /ostree/deploy/NAME-etc.  This is mounted as
read-write inside the system.

On an initial install, that directory is copied from
/ostree/deploy/NAME/etc.  On subsequent deployments, we find any
changes made in the current deployment's /etc, and apply that set of
changes to the new deployment's /etc.

See https://live.gnome.org/OSTree/EverythingInEtcIsABug
2012-09-06 08:17:23 -04:00
Colin Walters
9ba968a5f4 otutil: Add a few more helper functions
To be used by wip/etc-writable branch.
2012-09-03 10:43:42 -07:00
Colin Walters
cf31942a69 core: Refactor diff to be an internal API
For future use by other code.
2012-08-31 17:39:04 -07:00
Colin Walters
38d62d79a2 ostadmin: Also ensure /ostree/modules exists
We depend on host kernel for now.
2012-08-30 17:42:36 -07:00
Colin Walters
c690416b72 build: Add dist-snapshot
Take the current git HEAD and make a tarball from it; should be useful
for putting snapshots into Poky.
2012-08-28 10:14:11 -04:00
Colin Walters
60368fae76 Use correct libgsystem
Accidentally pushed a local patch.
2012-08-28 09:58:07 -04:00
Colin Walters
2527f81abc Merge remote-tracking branch 'aperezdc/fixes' 2012-08-28 09:45:39 -04:00
Adrian Perez
b314b25e6a configure.ac: Fix checks for libarchive
When configuring without passing --enable-triggers-only, checks for
libarchive were not being done. Shuffling around the checks solves
the issue.
2012-08-28 16:36:07 +03:00
Colin Walters
a17a8e1f4e Update libgsystem 2012-08-27 18:48:14 -04:00
Colin Walters
4d2f770748 core: Use O_NOATIME to open metadata
We really don't need atime for metadata, it's just a speed hit.
2012-08-27 16:07:39 -04:00