1059 Commits

Author SHA1 Message Date
Colin Walters
9bd4d35c2b pull: Fix a race condition causing an early exit
This is a little less magical than what we had before.
2012-12-23 17:22:08 -05:00
Colin Walters
3c67c209db pull: Use a basic exponential moving average for bytes/sec
Still should look what other downloading programs do, but this
is better then what we had before.
2012-12-23 14:29:15 -05:00
Colin Walters
0ab1f78ec8 admin: Add new run-triggers command
In some cases we want the ability to run triggers independently of
checking out a tree.  For example, due to kernel limitations which
impact the gnome-ostree build system, we may need to run triggers on
first boot via systemd.

Secondarily, if the user installs a system extension which adds a new
shared library to /usr/lib for example, the system will need to run
the triggers again.

Also, I think I want to take triggers out of the core and put them in
ostree admin anyways.
2012-12-22 14:52:33 -05:00
Colin Walters
3832544ac4 admin: Rework /ostree/deploy to support multiple independent operating systems
The real vision of OSTree is to "multiple versions of multiple
operating systems".  Up until now, it's worked to install gnome-ostree
inside a host distribution, but several things don't work quite right
if you try to do completely different systems.

In the new model, there's the concept of an "osname" which encompasses
a few properties:

1) Its own /var
2) A set of trees deployed in /ostree/deploy/OSNAME/
3) Its own "current" and "previous" links.

Now it no longer really makes sense to boot with "ostree=current".
Instead, you specify e.g. "ostree=gnome/current".

This is an incompatible change to the deployment code - you will need
to run init-os gnome and redeploy.

All "ostree admin" subcommands now take an OSNAME argument.
2012-12-21 13:47:15 -05:00
Colin Walters
16d312e82f admin: Support installing a kernel from the tree, default to it
The "protocol" here is rather lame; we just look for
/boot/vmlinuz-RELEASE and /lib/modules/RELEASE.  But good enough for
now.
2012-12-21 12:20:55 -05:00
Colin Walters
19df2d372d core: Use new libgsystem GSSubprocess API
It's less screwing around with GPtrArray in most cases.
2012-12-21 11:52:28 -05:00
Colin Walters
b3d8c8c0e5 build: Set libsoup min/max defines
So we don't get warnings about soup_requester being deprecated.
2012-12-21 11:33:05 -05:00
Colin Walters
90ccc7b2d2 core: Pick better names for tmpfiles if no prefix given
Just makes it easier to track down where files came from.
2012-12-21 11:30:58 -05:00
Colin Walters
3d41d1d2db OtWorkerQueue: Don't exit until we've fulfilled initial requests
This counts as an implicit "hold" on the first items, which we must
complete.  Otherwise there are race conditions where the queue may
signal idle when in fact the main process is still working.
2012-12-21 09:36:38 -05:00
Colin Walters
fddc11ca74 Update to latest libgsystem
Has a bugfix change to gs_subprocess_new().
2012-12-20 21:14:31 -05:00
Colin Walters
ad956ff9e2 core: Allow creating tempfile without associated output stream
Just slightly more efficient, and we'll use this for spawning curl to
a tempfile.
2012-12-20 21:14:27 -05:00
Colin Walters
a97211e3d0 core: Clean up tmpfile creation code
I have been seeing random failures here...not entirely sure why yet,
but this code is cleaner.
2012-12-13 15:12:51 -05:00
Colin Walters
380b54d689 Update libgsystem
Just keeping up.
2012-12-13 15:12:33 -05:00
Colin Walters
b637ea7674 admin init-fs: New builtin
Should be used when initializing a new root filesystem for a "pure
OSTree" system; for example, what "ostbuild privhelper-deploy-qemu"
does when creating a filesystem image loopback.
2012-12-09 17:01:51 -05:00
Colin Walters
a2acaedfdc admin: Process newly added config files recursively
Previously we just failed out if there was a new directory in /etc
added by the user (or the system).
2012-12-07 18:29:03 -05:00
Colin Walters
227d22adca Update to the latest libgsystem 2012-12-03 12:05:41 -05:00
Colin Walters
a9a9baeb2a Update to latest libgsystem
Has a rather important bugfix.
2012-12-02 23:02:39 -05:00
Colin Walters
fc7f05940c core: Use mmap() with O_NOATIME if merely reading archive-z2 entries
Previously we'd open(path, O_NOATIME) and do a series of small read()
calls to just parse the header.  I think this will trigger kernel readahead
into the compressed portion, but we don't care about that.

This should be more efficient.
2012-12-02 15:20:17 -05:00
Colin Walters
7fcebed4e8 cleanup: Rebase spawn API on top of libgsystem GSSubprocess
We didn't need most the parameters anyways, so this is fewer lines of
code.  It also helps me test GSubprocess.
2012-12-01 16:53:06 -05:00
Colin Walters
b423ea2041 Update to the latest libgsystem 2012-11-29 18:16:45 -05:00
Colin Walters
6dda8da6c5 Switch to using libgsystem shutil
More code moved into libgsystem.
2012-11-29 17:11:24 -05:00
Colin Walters
1ac307e6b8 Switch to libgsystem file reading API
More draining common utilities into libgsystem.
2012-11-29 16:58:39 -05:00
Colin Walters
f3d765c903 Switch to libgsystem directory creation API
More code being dropped into a common place.
2012-11-29 16:40:17 -05:00
Colin Walters
6d81120455 otutil: Move checksum related bits into checksum-utils.[ch]
This paves the way for slimming ot-gio-utils.[ch] into libgsystem.
2012-11-29 16:28:24 -05:00
Colin Walters
5761733218 Drop some bits from libotutil that are now merged in libgsystem 2012-11-29 16:24:49 -05:00
Colin Walters
d1ddfc973c libotutil: Drop dead prototype
Was never implemented.
2012-11-29 15:56:17 -05:00
Colin Walters
f2b1be6a1e deploy: With --no-kernel, do copy host kernel modules
This is used by the qemu scripts, and there we *do* need the kernel
modules inside the target system.
2012-11-21 12:38:29 -05:00
Colin Walters
05e7b6d596 pull-local: Make multithreaded
We were blocking for easily 1/10 or 1/5 of a second in fdatasync(),
which drastically slows down the whole process.

This threading isn't quite as good as the ostree-pull command, but it
lets us avoid the dependency on libsoup everywhere, and it's simpler.
2012-11-19 12:08:41 -05:00
Colin Walters
de1ce843f1 core: Drop old GLIB_CHECK_VERSION(2.32) bits
We hard require 2.34 now; if it doesn't exist on the system, that's
what embedded dependencies are for.
2012-11-18 15:44:21 -05:00
Colin Walters
3f4d223361 admin prune: New builtin for cleaning up deployments and repo
After a while of pull-deploy cycles, you start to accumulate a lot of
them.  While the deployment read-only part is hardlinked, the -etc
space adds up.

Additionally, the repository itself just gets large.

The new command "ostree admin prune" deletes everything except the
"current" and "previous" deployments.
2012-11-16 17:41:46 -05:00
Colin Walters
05895780a3 deploy: Write refs into repo for current/previous deployments
We don't want this data to be gc'd by default by a prune.
2012-11-16 17:14:23 -05:00
Colin Walters
ffd774e2e6 prune: Report how much disk space we freed
Just nicer to see...
2012-11-16 15:45:05 -05:00
Colin Walters
7861b98673 prune: Add option to traverse refs only
The previous code (unintentionally) only traversed from refs; so data
only reachable from previous commits would be deleted.  That shouldn't
be the default, but we do want to offer it as an option.

So add a --refs-only option.
2012-11-16 12:50:13 -05:00
Colin Walters
51b1dd7cbd traverse: Skip already traversed commits
This is just more efficient, since e.g. "ostree prune" may invoke
ostree_traverse_commit() multiple times for the same commit.
2012-11-16 12:35:52 -05:00
Colin Walters
4169d628c5 prune: Default to deleting loose objects, code cleanup
We're getting closer to matching 'git gc'.
2012-11-16 10:58:35 -05:00
Colin Walters
ee533d6ab9 admin: code cleanup
Refactor internal functions so they can be used by a later patch.
2012-11-16 10:40:49 -05:00
Colin Walters
8f624ab51b ostree.spec.in: Update from Fedora review comments
See https://bugzilla.redhat.com/show_bug.cgi?id=819951
2012-11-16 10:11:39 -05:00
Colin Walters
2769ff36ff Release 2012.12 v2012.12 2012-10-31 22:02:35 -04:00
Jasper St. Pierre
cda5b28bab ot-main: Ignore lt- prefixes in binary names
libtool is the best

https://bugzilla.gnome.org/show_bug.cgi?id=682438
2012-10-24 12:20:15 -04:00
Colin Walters
cc4df4f3c2 admin: Add new pull-deploy command
Fetch the latest for the current tree, and deploy it in one go.
2012-10-22 20:41:49 -04:00
Colin Walters
f405737164 libgsystem: Update 2012-10-22 18:24:31 -04:00
Matthias Clasen
a0449ed246 Make ostree admin --help output more useful
At a minimum, it should list the available subcommands. This is
still not perfect, since there is no way to get at the help output
of the subcommands - getting that right needs more refactoring.

Signed-off-by: Colin Walters <walters@verbum.org>
2012-10-22 18:10:56 -04:00
Colin Walters
5a51bee024 core: quiet a compiler warning 2012-10-17 20:54:52 -04:00
Colin Walters
0c3e901dae core: Check out files before directories
Otherwise we pretty easily hit maximum FD limits =(
2012-10-15 20:03:13 -04:00
Colin Walters
bdbf76b6eb core: Ensure correct file data is synced in non-overwrite case 2012-10-15 18:44:55 -04:00
Colin Walters
1642310486 core: Use linkat() rather than link() in threaded checkouts
This seems to work around a likely Linux kernel VFS bug, where I
randomly see ENOENT on link() when we *definitely* called mkdir() at
an earlier point in time.
2012-10-15 10:49:48 -04:00
Colin Walters
d6956bfc19 core: And one more race in multithreaded mkdir -p
Sigh =(
2012-10-15 10:49:48 -04:00
Colin Walters
c0a6b5da71 core: Fix devino speedup for archive-z 2012-10-14 17:19:30 -04:00
Colin Walters
0d04738801 core: Rework archive-z mode to have header be uncompressed
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.

I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient.  Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
2012-10-14 17:10:57 -04:00
Colin Walters
2b90e987bf core: Call fdatasync() before we rename()
Just noticed this from strace, doesn't really matter, but looks nicer.
2012-10-14 15:34:33 -04:00