1336 Commits

Author SHA1 Message Date
Colin Walters
c9d7050d3d main: Support passing bare directory names to commit
It segfaulted before if you passed non-options, and a single directory
name is probably what people want.
2013-08-28 17:32:21 -04:00
Jeremy Whiting
499df2a90b pull: Add support for resuming downloads via range requests
Use a consistent temporary filename to download uri's.
Check for downloaded files before fetching from uri.
Download to hash.part file, then copy/move to hash.done when complete.
Add argument support to setup_fake_remote_repo1 function.
Add test for pull resume.
To implement this, pass --force-range-requests into the trivial-httpd,
which will only serve half of the objects to clients at a time.

https://bugzilla.gnome.org/show_bug.cgi?id=706344
2013-08-28 14:35:54 -04:00
Jeremy Whiting
902848cd71 test: Improve pull corruption test
To verify pull with a second repo works.
2013-08-27 20:35:45 -04:00
Colin Walters
8abad0b452 admin: Write out correct version fields in boot/loader/entries files
Before, we were writing the "bootversion", which is either 0 or 1, for
all entries.  This is completely wrong; the idea of the "version"
field is to compare between entries.

Fix this by writing out the inverted index - internally, index 0 is
the *first* boot entry, so we give it the highest version number, and
index N is the last, so give it version 0.

Then fix the deployment sorting code to correctly reverse the version
number comparison, so we read back the right order.

In practice before this bug didn't matter because "normally" you only
have at most two deployments.

https://bugzilla.gnome.org/show_bug.cgi?id=706546
2013-08-22 05:46:11 -04:00
Javier Martinez Canillas
2033edc658 test: fix a trivial typo in libtests.sh
When running the test-admin-deploy-1.sh unit test,
cat shows the following error:

cat: boot/vmlinuz-3-6.0: No such file or directory

due a trivial typo in the kernel image file name.

Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>

https://bugzilla.gnome.org/show_bug.cgi?id=706371
2013-08-20 06:46:52 -04:00
Colin Walters
1ec7c30408 core: Remove old "archive" mode
We'll always have "bare" mode for keeping files-as-hardlinks as root.
But "archive" was my second attempt at a format for non-root file
storage, used by the gnome-ostree buildsystem which runs as non-root.

It was really handy to have a "tar" like mode where I can create
tarballs as a user, that contain files owned by root for example.

The "archive" mode stored content files as two pieces in the
filesystem; ".file" contained metadata, and ".filecontent" was the
actual content, uncompressed.  The nice thing about this was that to
check out a tree as non-root, you could just hardlink into the repo.

However, archive was fairly bad for serving via HTTP; it required
*two* HTTP requests per content object, greatly magnifing the already
inefficient fetch process.  So "archive-z2" was introduced.

To allow gnome-ostree to still check out trees as a user, the
"uncompressed-object-cache" was introduced, and that's how things have
been working for a while.

So we should just be able to kill this code.  Specifically note just
how much better the stage_object() function became.

https://bugzilla.gnome.org/show_bug.cgi?id=706057
2013-08-16 22:56:42 -04:00
Stef Walter
f9b2c45fc0 Add 'ostree reset' command to undo a bad commit
Accepts the following arguments: ref checksum

Checks that the checksum is a parent of the ref before rewriting
the ref.

https://bugzilla.gnome.org/show_bug.cgi?id=705979
2013-08-16 17:28:43 +02:00
Stef Walter
3989e0d397 Add 'ostree log' command
Follows the parent of commits showing each in turn until it reaches
the top of the commit tree.

https://bugzilla.gnome.org/show_bug.cgi?id=705973
2013-08-15 07:01:30 +02:00
Stef Walter
790132a81a Intelligible display for 'ostree show'
Show something similar to git metadata display. Show raw variant
data when --raw is specified

https://bugzilla.gnome.org/show_bug.cgi?id=705973
2013-08-15 06:48:25 +02:00
Colin Walters
f68ac018c2 test-xattrs: Skip if current FS doesn't support user_xattr
tmpfs doesn't =(
2013-08-14 20:46:46 +02:00
Stef Walter
1904e3e0f4 Don't pass non-checksums to ostree_repo_has_object()
The 'ostree show' command passed non-checksum arguments
to the ostree_repo_has_object() function which led to
an assertion.

https://bugzilla.gnome.org/show_bug.cgi?id=705967
2013-08-14 16:08:18 +02:00
Colin Walters
8c31e6fda7 tests: Add xattr test case
Since it was broken up until now for multiple attributes =(
2013-08-14 15:07:14 +02:00
Colin Walters
760b866104 main: Assume / for ls if no path given
Since it just is kind of a sane default.
2013-07-27 20:12:07 -04:00
Colin Walters
187c75eae5 main: Support using repo from current directory
$ cd repo
$ ostree ls foo /
...

Can be a lot more convenient than typing --repo=repo a lot.
2013-07-24 14:05:12 -04:00
Colin Walters
c9b61cbfee Drop support for related objects and metadata in commit objects
While the actual commit object format is presently the same, for a
number of reasons we'd like to change it fairly radically.  Among
other things, we need to drop our a{sv} types in objects, to protect
against GVariant changing format.

Since now gnome-ostree now longer uses related objects, and nothing
ever used metadata, just drop them both.
2013-07-24 12:59:27 -04:00
Colin Walters
3b9da094d8 main: Drop log builtin
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.

More interesting things are diffs between commits, object statistics,
etc.
2013-07-23 18:19:14 -04:00
Colin Walters
3d7bff2d41 admin: Add an "undeploy" command
Otherwise it's really easy to keep accumulating deployments.  Also, we
may want to run this after rebooting, so we're back down to one
operating system.
2013-07-23 09:19:24 -04:00
Colin Walters
67823beb1f core: Don't strip setuid bits when creating files
This ugly regression occurred because I overlooked the fact that our
chown() invocation would strip off setuid.
2013-07-18 14:23:55 -04:00
Colin Walters
f0c5a5f334 main: Add --delete option to fsck
This is useful for clearing out all corrupted objects locally.
2013-07-18 12:09:44 -04:00
Colin Walters
a40b09310c tests: Add a test for link checkout speedup 2013-07-17 19:47:40 -04:00
Colin Walters
62a896350b Drop support for fifos and devices
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.

Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.

Less complexity and code here if we just support directories, regular
files, and symbolic links.
2013-07-16 09:35:44 -04:00
Colin Walters
fb93b95807 admin: Rename prune -> cleanup, avoid doing repo prune twice
Calling it "cleanup" is better since it does more than repo pruning.

We were also doing a prune twice; ot_admin_cleanup() already does one,
so drop the bits to do it in cleanup.c.
2013-07-15 16:13:12 -04:00
Colin Walters
420763e90d tests: Add a test of corruption during a pull process
These corruption tests could be a lot better...like randomly try
single bit flips, range flips.  Better, content-aware fuzzing.  But
this is useful for now.
2013-07-09 11:28:49 -04:00
Colin Walters
43d69ac8a3 tests: Drop numeric prefix
Theoretically it's useful to have layers of tests, but in practice
it's just annoying to assign numbers.
2013-07-09 10:41:07 -04:00
Colin Walters
bea4a7538f Fix test reference to obsolete ostree-pull
It was leftover in my buildroot =/
2013-07-09 09:28:02 -04:00
Colin Walters
c66148160c admin: Also delete unsed boot directories
My VM filled up /boot.  Oops.
2013-07-07 13:45:18 -04:00
Colin Walters
bb6eedfb25 [INCOMPATIBLE CHANGE] Implement new deployment model
See https://wiki.gnome.org/OSTree/DeploymentModel2

This is a major rework of the on-disk filesystem layout, and the boot
process.  OSTree now explicitly supports upgrading kernels, and these
upgrades are also atomic.

The core concept of the new model is the "deployment list", which is
an ordered list of bootable operating system trees.  The deployment
list is reflected in the bootloader configuration; which has a kernel
argument that tells the initramfs (dracut) which operating system root
to use.

Invidiual notable changes that come along with this:

1) Operating systems should now come with their etc in usr/etc; OSTree
   will perform a 3-way merge at deployment time, and place etc in
   the actual root.  This avoids the need for a bind mount, and is
   just a lot cleaner.
2) OSTree no longer bind mounts /root, /home, and /tmp.  It is expected
   that the the OS/ has these as symbolic links into /var.

At the moment, OSTree only supports managing syslinux; other
bootloader backends will follow.
2013-07-07 11:31:26 -04:00
Colin Walters
ec21dc4242 Add "trivial-httpd" builtin, use it in tests
A simple HTTP server implementation is so few lines of code when one
is linking to libsoup anyways, so let's just have one here in ostree
that will be used for the test suite.

This allows us to run the archive tests that previously required
apache even in gnome-ostree.
2013-07-05 16:28:40 -04:00
Colin Walters
2ed49a3749 tests: Pull from file:/// only works with archive-z right now 2013-07-02 11:24:07 -04:00
Colin Walters
a93f2b8d16 pull: Make fetcher work for file:/// URIs too
Even if very suboptimally, for now; we copy the files, then copy them
again.

The obvious long term plan is to merge pull-local and pull together,
but truly optimizing that requires the pull code to know how to use
the OstreeRepo APIs when operating on local repositories (as
pull-local does), rather than assuming the remote is an archive-z
fetched over HTTP.
2013-07-01 22:43:17 -04:00
Colin Walters
dc0f3c3dcb Add support for deleting refs
The internal API will be used by admin, and "ostree refs --delete"
is handy for interactive management.
2013-07-01 15:41:27 -04:00
Colin Walters
5b3fca8426 Add "refs" builtin
This is just useful to look at before pruning, etc.
2013-06-29 14:51:08 -04:00
Colin Walters
aa1eb19eea Add a bit more testing for prune 2013-06-29 14:02:17 -04:00
Colin Walters
a167bb7342 pull-local: Support --remote argument
For offline upgrades, pull-local can now write the refs into a
specific remote, rather than using the local heads.
2013-06-29 13:00:42 -04:00
Colin Walters
82f444020e Add a --version argument that also outputs features
And use it in the tests to skip libarchive tests when the build
doesn't support it.
2013-05-01 19:15:06 -04:00
Colin Walters
b1ace57ef8 archive-test: Handle umask of 022 or 002
Red Hat appears to default to 002, whereas Debian systems default
to 022.  Fun.
2013-05-01 18:43:01 -04:00
Colin Walters
e9a181c67d Install tests
Matching https://live.gnome.org/GnomeGoals/InstalledTests
2013-05-01 15:26:21 -04:00
Colin Walters
c60c70e9a9 checkout: Add --allow-noent option
This is useful for the gnome-ostree build system where each build is
one commit, but it's split up into /runtime /devel /debug etc. trees.
Ideally we wouldn't have a /debug subdirectory for "noarch"
components for example.

So add an option to not error out if the given path doesn't exist in
the commit.
2013-05-01 12:15:02 -04:00
Colin Walters
a268b53dc9 run-apache: Use -DFOREGROUND, not -X
-X implies debug mode which limits the workers to 1, which totally
kills parallelism.
2013-04-01 18:55:35 -04:00
Colin Walters
bde7e5c719 pull: Drop support for original archive repositories
For future refactoring here, it's just too complex to maintain support
for both of them, and the advantages of archive-z2 outweigh archive
too much.
2013-03-29 17:22:56 -04:00
Colin Walters
8a2628fc2c tests: Update expected diff output
For previous diff commit.
2013-03-26 08:29:30 -04:00
Colin Walters
0d04738801 core: Rework archive-z mode to have header be uncompressed
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.

I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient.  Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
2012-10-14 17:10:57 -04:00
Colin Walters
7aa620109e Add tests missing from previous commit 2012-09-23 19:25:09 -04:00
Colin Walters
40ce43036f Add an archive-z repository mode
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode.  This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
2012-09-23 19:23:45 -04:00
Colin Walters
2a0601efc7 core: Drop packfiles as they are now
They're not a large efficiency win at the moment, because we don't
do any delta compression.

At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
2012-09-23 16:02:03 -04:00
Colin Walters
aa9aaa9a62 tests: Drop tests of removed --atomic-retarget option 2012-09-23 15:56:28 -04:00
Colin Walters
a7b917c856 core: Check out asynchronously
This can be a large performance win in certain circumstances:

 * Cold buffer cache (we don't block the whole process)
 * Requiring a copy instead of hardlink
2012-06-21 18:05:13 -04:00
Colin Walters
eba0ff75cc tests: Just use OT_TESTS_DEBUG to say "save test data" 2012-06-21 11:42:45 -04:00
Colin Walters
c18817eca1 tests: Tweak valgrind suppression 2012-05-18 16:30:44 -04:00
Colin Walters
0fc291b954 core: Drop --link-cache option, we do it by default now for archives 2012-05-04 16:10:30 -04:00