Commit Graph

624 Commits

Author SHA1 Message Date
Colin Walters
760b866104 main: Assume / for ls if no path given
Since it just is kind of a sane default.
2013-07-27 20:12:07 -04:00
Colin Walters
187c75eae5 main: Support using repo from current directory
$ cd repo
$ ostree ls foo /
...

Can be a lot more convenient than typing --repo=repo a lot.
2013-07-24 14:05:12 -04:00
Colin Walters
c9b61cbfee Drop support for related objects and metadata in commit objects
While the actual commit object format is presently the same, for a
number of reasons we'd like to change it fairly radically.  Among
other things, we need to drop our a{sv} types in objects, to protect
against GVariant changing format.

Since now gnome-ostree now longer uses related objects, and nothing
ever used metadata, just drop them both.
2013-07-24 12:59:27 -04:00
Colin Walters
3b9da094d8 main: Drop log builtin
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.

More interesting things are diffs between commits, object statistics,
etc.
2013-07-23 18:19:14 -04:00
Colin Walters
3d7bff2d41 admin: Add an "undeploy" command
Otherwise it's really easy to keep accumulating deployments.  Also, we
may want to run this after rebooting, so we're back down to one
operating system.
2013-07-23 09:19:24 -04:00
Colin Walters
67823beb1f core: Don't strip setuid bits when creating files
This ugly regression occurred because I overlooked the fact that our
chown() invocation would strip off setuid.
2013-07-18 14:23:55 -04:00
Colin Walters
f0c5a5f334 main: Add --delete option to fsck
This is useful for clearing out all corrupted objects locally.
2013-07-18 12:09:44 -04:00
Colin Walters
a40b09310c tests: Add a test for link checkout speedup 2013-07-17 19:47:40 -04:00
Colin Walters
62a896350b Drop support for fifos and devices
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.

Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.

Less complexity and code here if we just support directories, regular
files, and symbolic links.
2013-07-16 09:35:44 -04:00
Colin Walters
fb93b95807 admin: Rename prune -> cleanup, avoid doing repo prune twice
Calling it "cleanup" is better since it does more than repo pruning.

We were also doing a prune twice; ot_admin_cleanup() already does one,
so drop the bits to do it in cleanup.c.
2013-07-15 16:13:12 -04:00
Colin Walters
420763e90d tests: Add a test of corruption during a pull process
These corruption tests could be a lot better...like randomly try
single bit flips, range flips.  Better, content-aware fuzzing.  But
this is useful for now.
2013-07-09 11:28:49 -04:00
Colin Walters
43d69ac8a3 tests: Drop numeric prefix
Theoretically it's useful to have layers of tests, but in practice
it's just annoying to assign numbers.
2013-07-09 10:41:07 -04:00
Colin Walters
bea4a7538f Fix test reference to obsolete ostree-pull
It was leftover in my buildroot =/
2013-07-09 09:28:02 -04:00
Colin Walters
c66148160c admin: Also delete unsed boot directories
My VM filled up /boot.  Oops.
2013-07-07 13:45:18 -04:00
Colin Walters
bb6eedfb25 [INCOMPATIBLE CHANGE] Implement new deployment model
See https://wiki.gnome.org/OSTree/DeploymentModel2

This is a major rework of the on-disk filesystem layout, and the boot
process.  OSTree now explicitly supports upgrading kernels, and these
upgrades are also atomic.

The core concept of the new model is the "deployment list", which is
an ordered list of bootable operating system trees.  The deployment
list is reflected in the bootloader configuration; which has a kernel
argument that tells the initramfs (dracut) which operating system root
to use.

Invidiual notable changes that come along with this:

1) Operating systems should now come with their etc in usr/etc; OSTree
   will perform a 3-way merge at deployment time, and place etc in
   the actual root.  This avoids the need for a bind mount, and is
   just a lot cleaner.
2) OSTree no longer bind mounts /root, /home, and /tmp.  It is expected
   that the the OS/ has these as symbolic links into /var.

At the moment, OSTree only supports managing syslinux; other
bootloader backends will follow.
2013-07-07 11:31:26 -04:00
Colin Walters
ec21dc4242 Add "trivial-httpd" builtin, use it in tests
A simple HTTP server implementation is so few lines of code when one
is linking to libsoup anyways, so let's just have one here in ostree
that will be used for the test suite.

This allows us to run the archive tests that previously required
apache even in gnome-ostree.
2013-07-05 16:28:40 -04:00
Colin Walters
2ed49a3749 tests: Pull from file:/// only works with archive-z right now 2013-07-02 11:24:07 -04:00
Colin Walters
a93f2b8d16 pull: Make fetcher work for file:/// URIs too
Even if very suboptimally, for now; we copy the files, then copy them
again.

The obvious long term plan is to merge pull-local and pull together,
but truly optimizing that requires the pull code to know how to use
the OstreeRepo APIs when operating on local repositories (as
pull-local does), rather than assuming the remote is an archive-z
fetched over HTTP.
2013-07-01 22:43:17 -04:00
Colin Walters
dc0f3c3dcb Add support for deleting refs
The internal API will be used by admin, and "ostree refs --delete"
is handy for interactive management.
2013-07-01 15:41:27 -04:00
Colin Walters
5b3fca8426 Add "refs" builtin
This is just useful to look at before pruning, etc.
2013-06-29 14:51:08 -04:00
Colin Walters
aa1eb19eea Add a bit more testing for prune 2013-06-29 14:02:17 -04:00
Colin Walters
a167bb7342 pull-local: Support --remote argument
For offline upgrades, pull-local can now write the refs into a
specific remote, rather than using the local heads.
2013-06-29 13:00:42 -04:00
Colin Walters
82f444020e Add a --version argument that also outputs features
And use it in the tests to skip libarchive tests when the build
doesn't support it.
2013-05-01 19:15:06 -04:00
Colin Walters
b1ace57ef8 archive-test: Handle umask of 022 or 002
Red Hat appears to default to 002, whereas Debian systems default
to 022.  Fun.
2013-05-01 18:43:01 -04:00
Colin Walters
e9a181c67d Install tests
Matching https://live.gnome.org/GnomeGoals/InstalledTests
2013-05-01 15:26:21 -04:00
Colin Walters
c60c70e9a9 checkout: Add --allow-noent option
This is useful for the gnome-ostree build system where each build is
one commit, but it's split up into /runtime /devel /debug etc. trees.
Ideally we wouldn't have a /debug subdirectory for "noarch"
components for example.

So add an option to not error out if the given path doesn't exist in
the commit.
2013-05-01 12:15:02 -04:00
Colin Walters
a268b53dc9 run-apache: Use -DFOREGROUND, not -X
-X implies debug mode which limits the workers to 1, which totally
kills parallelism.
2013-04-01 18:55:35 -04:00
Colin Walters
bde7e5c719 pull: Drop support for original archive repositories
For future refactoring here, it's just too complex to maintain support
for both of them, and the advantages of archive-z2 outweigh archive
too much.
2013-03-29 17:22:56 -04:00
Colin Walters
8a2628fc2c tests: Update expected diff output
For previous diff commit.
2013-03-26 08:29:30 -04:00
Colin Walters
0d04738801 core: Rework archive-z mode to have header be uncompressed
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.

I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient.  Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
2012-10-14 17:10:57 -04:00
Colin Walters
7aa620109e Add tests missing from previous commit 2012-09-23 19:25:09 -04:00
Colin Walters
40ce43036f Add an archive-z repository mode
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode.  This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
2012-09-23 19:23:45 -04:00
Colin Walters
2a0601efc7 core: Drop packfiles as they are now
They're not a large efficiency win at the moment, because we don't
do any delta compression.

At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
2012-09-23 16:02:03 -04:00
Colin Walters
aa9aaa9a62 tests: Drop tests of removed --atomic-retarget option 2012-09-23 15:56:28 -04:00
Colin Walters
a7b917c856 core: Check out asynchronously
This can be a large performance win in certain circumstances:

 * Cold buffer cache (we don't block the whole process)
 * Requiring a copy instead of hardlink
2012-06-21 18:05:13 -04:00
Colin Walters
eba0ff75cc tests: Just use OT_TESTS_DEBUG to say "save test data" 2012-06-21 11:42:45 -04:00
Colin Walters
c18817eca1 tests: Tweak valgrind suppression 2012-05-18 16:30:44 -04:00
Colin Walters
0fc291b954 core: Drop --link-cache option, we do it by default now for archives 2012-05-04 16:10:30 -04:00
Colin Walters
8d926c3e36 core: Add valgrind framework, plug various memory leaks 2012-05-04 10:04:32 -04:00
Colin Walters
d4321629f4 core: Rename local-clone to pull-local
Also change it to copy selective refs, rather than enumerating
all objects.
2012-05-01 17:44:29 -04:00
Colin Walters
9cb87ffc07 core: pack: Add --metadata-only argument
Useful for bare repositories.
2012-04-29 12:15:12 -04:00
Colin Walters
5947b5b145 core: Add ability for repositories to have a "parent"
This will be useful for ostbuild; a user can create their own archive
mode repository which transparently inherits objects from the
root-owned one in /ostree.
2012-04-18 23:12:34 -04:00
Colin Walters
262f426997 core: Add --link-cache option to checkout
This is a convenient way to have a lookaside directory of hard links,
which can greatly speed up checkouts.  In the future we probably want
to push this down into the repository.
2012-04-13 14:33:38 -04:00
Colin Walters
38ef75e6e0 core: Re-unify on one OSTREE_OBJECT_TYPE_FILE
Having the archived vs not distinction in the object system wasn't
useful in light of pack files.  In fact, we should probably move
towards generating a pack file per commit by default.
2012-04-11 22:27:59 -04:00
Colin Walters
188621f0b2 core: Drop 'compose' builtin
We'll just add this functionality to commit.
2012-04-11 19:54:55 -04:00
Colin Walters
3a847a571b tests: Actually test pack functionality
With --keep-loose we were just using the loose objects.
2012-04-11 19:54:28 -04:00
Colin Walters
bae584c64a core: Associate branches with remotes, move trigger runs into checkout
Also add --atomic-retarget option to checkout. This does the magical
symlink dance to do atomic swaps between trees.
2012-04-03 23:46:34 -04:00
Colin Walters
80bdfd7f42 core: Add pack files
This concept is also directly inspired by git.  At present, our
implementation is quite similar, except we don't have delta
compression.
2012-03-31 11:51:47 -04:00
Colin Walters
76bc35186e core: Add --union mode to checkout
This is another step towards ostbuild using this instead of the
"compose" builtin.
2012-03-06 11:59:06 -05:00
Colin Walters
4db485dd5f core: Add "subpath" option to checkout
Will be used by ostbuild to extract e.g. /runtime or /devel from
artifacts.
2012-03-06 11:59:06 -05:00
Colin Walters
eb7c3d01a3 core: Add 'cat' builtin 2012-03-06 11:59:06 -05:00
Colin Walters
d3a73225b0 core: Add API to enumerate all refs 2012-02-27 05:58:42 -05:00
Colin Walters
e8dbaa3c07 core: Add generic "commit filter" API, use it to implement statoverride
The builder wants the ability to mark a given file as e.g. setuid.  To
implement this, the repo now has a callback-based API when importing a
directory to modify or remove items.

The commit tool accepts a "statoverride" file as input which looks like:

+mode /path/to/file
2012-01-22 13:27:38 -05:00
Colin Walters
a322ecacba core: Fix test now that ls prints actual size 2012-01-22 13:10:01 -05:00
Colin Walters
e36c48f923 core: Fix compose tests 2012-01-11 14:33:13 -05:00
Colin Walters
0044ff133a core: Add from-file argument to compose
We don't want to have to pass a million arguments.
2012-01-10 18:55:00 -05:00
Colin Walters
95d0e94370 core: Make --recompose skip passed arguments if they were already composed
This allows us to do "--recompose -b foo bar baz" and add bar and baz
iff they weren't already there.
2012-01-06 14:04:39 -05:00
Colin Walters
5119cbdb7c core: Add --no-xattrs option for commit
Will be used by ostbuild.
2012-01-05 19:10:26 -05:00
Colin Walters
af62271eed tests: Fix metadata test, we removed the spurious = 2011-12-27 10:31:23 -05:00
Colin Walters
1687f3efa8 core: Add --add-metadata-string option for commit
A convenience function for the common case of slapping a string into
the metadata.
2011-12-23 07:17:46 -05:00
Colin Walters
2e48474263 core: Fix libarchive test count 2011-12-23 07:12:03 -05:00
Colin Walters
3e59cc1305 core: Add --tar-autocreate-parents option for commit
The tar files we're making of artifacts don't include parent
directories.  Now we could change the builder to make them, but we can
also just autocreate them on import.  Mode 0755 with no xattrs seems
OK here.
2011-12-23 06:49:04 -05:00
Colin Walters
dab4611263 core: Add --skip-if-unchanged option for commit
There's not much point for OS builds to have "empty" commits.
2011-12-22 18:47:30 -05:00
Colin Walters
3292dcb6d6 core: Allow / in ref names like git does
Using / is nice basically.
2011-12-22 15:59:47 -05:00
Colin Walters
8a499c4a2a core: Add --recompose option for compose
This will allow us to easily regenerate a compose without passing the
full list of components each time.
2011-12-22 10:29:16 -05:00
Colin Walters
e9fd921afe core: Change compose to operate purely in-memory
This is *significantly* faster than checking out each branch into the
real filesystem, then importing it again.
2011-12-21 21:42:13 -05:00
Colin Walters
add55849ab core: Allow specifying a previous commit tree as a basis for a new commit
It's pretty trivial to map a previously existing commit tree into a
mutable tree too.  While we're here change the command line arguments
for commit so that we can now properly overlay any combination of
directory, commit, or tarfile.
2011-12-21 20:10:10 -05:00
Colin Walters
bdfde03b61 core: Support committing multiple tarballs in the same transaction
ostbuild will generate two artifacts: foo-runtime.tar.gz and
foo-devel.tar.gz in the general case.  When committing to the devel
tree, it'd be lame (i.e. slower and not atomic) to have to commit
twice.
2011-12-20 17:21:15 -05:00
Colin Walters
11d57d63e3 core: Support --owner-uid and --owner-gid options for commit
This allows us to more easily import user-built tarballs into
a root-owned OSTree repo.
2011-12-18 17:36:46 -05:00
Colin Walters
a287274935 tests: Ensure we quote grep pattern
Otherwise we fail if the pattern starts with -
2011-12-18 17:35:30 -05:00
Colin Walters
134283afbf core: In user mode checkouts of archives, use hard links
Now that we've done all the gyrations to separate content from
metadata in archives, we can just hard link when doing user checkouts.
2011-12-15 16:12:31 -05:00
Colin Walters
78f435d245 core: Fix handling of hardlinks for tar commits
We were creating files with the wrong name.  Add a test.
2011-12-04 14:38:30 -05:00
Colin Walters
17cc772cf3 core: Support for committing from any libarchive-supported format
We really want the ability to take a .tar.gz and directly import
it into a repository, without creating a temporary filesystem tree.

First, doing it this way is significantly faster.  Also, this allows
us to handle importing tar files with e.g. uid 0 files into packed
repositories as non-root, which is very useful for tests and builds.
2011-12-04 14:08:55 -05:00
Colin Walters
e8865af09e core: Split pull functionality into separate ostree-pull binary
This is to avoid everything depending on libsoup.
2011-11-30 09:21:14 -05:00
Colin Walters
7e32bc6cd7 core: Make commit always operate on directory contents, not file list
This simplifies things significantly, at some cost in flexibility.
We'll later add the ability to e.g. filter out files by regular
expression.
2011-11-28 12:57:25 -05:00
Colin Walters
a042731673 core: Add local-clone builtin
This is useful for converting between e.g. archive and non-archive
repositories.
2011-11-18 18:50:53 -05:00
Colin Walters
12f2d89174 core: Add checksum builtin
This necessitated reworking things so that builtins can specify no
--repo is required.
2011-11-18 07:29:13 -05:00
Colin Walters
b8cef545d1 core: CHANGE CHECKSUM ALGORITHM, port checksum API to GFile
This commit originally was to port ostree_stat_and_checksum_file() to
GFile*, but I noticed that the checksum code was reading data in host
endianness.  Fix that while we're here.

This invalidates all existing repositories.
2011-11-17 19:32:01 -05:00
Colin Walters
8e9b9c59d6 core: Fix pull to actually download files too 2011-11-17 10:40:55 -05:00
Colin Walters
4a26be1bef core: diff: Correctly handle files changing type 2011-11-16 23:09:32 -05:00
Colin Walters
e759c1cce5 tests: Fix number of tests 2011-11-16 22:51:56 -05:00
Colin Walters
9fb390664a core: Implement diff command 2011-11-16 18:16:41 -05:00
Colin Walters
659c99417c Switch to LGPLv2+ for most code
Since we're making a shared library, it should be usable by non-GPL
apps.

To allow more code sharing between the core and the tests, move them
to the LGPLv2+ too.

A few bits of test and other code are still GPL.  See the new COPYING
file for more information.
2011-11-10 13:17:04 -05:00
Colin Walters
a103218fd1 core: Support named pipes
This is mainly useful for sysvinit's "/dev/initctl".
2011-11-09 09:08:58 -05:00
Colin Walters
1d23e4b8e8 core: Actually return an error from fsck
And add some tests that fsck reports an error.
2011-11-09 08:27:47 -05:00
Colin Walters
8bda1c6a84 tests: Add OT_TEST_DEBUG environment variable 2011-11-09 05:29:53 -05:00
Colin Walters
2be1407622 commit: Drop the ability to --add --remove
This forces us to have two code paths for doing a commit; let's just
drop this ability for now since it's not very useful.
2011-11-08 18:58:01 -05:00
Colin Walters
ea6a761265 Reorder compose and pull tests 2011-11-04 16:14:12 -04:00
Colin Walters
9877f7eea3 tests: A few more compose tests 2011-11-04 16:13:21 -04:00
Colin Walters
ed90bf77cc core: Make compose commits contain metadata about their composition
This allows tracking of their history better.
2011-11-04 15:59:08 -04:00
Colin Walters
d2cb5fb49a core: Support actually adding metadata to commits 2011-11-04 15:59:08 -04:00
Colin Walters
889f454cb1 tests: Make GLib warnings fatal 2011-11-04 15:59:08 -04:00
Colin Walters
dd37e46172 tests: Output errors to stderr, so they show up under "prove" 2011-11-04 12:31:40 -04:00
Colin Walters
b15eee447a commit: Support reading current directory rather than requiring 'find'
Implement 'find' internally.
2011-11-04 10:53:47 -04:00
Colin Walters
d6b3fb5118 core: Take --repo as the first argument
I kept doing this over and over...it feels more natural.  The "prefix"
thing was (almost) unused anyways, and it was easy enough to replace.
2011-11-03 23:08:28 -04:00
Colin Walters
3b61a21c0f tests: Add some more tests of compose content 2011-11-03 22:35:42 -04:00
Colin Walters
69f104c558 core: Make compose its own builtin
This will allow more flexibility down the line.
2011-11-03 22:32:06 -04:00
Colin Walters
d23f203108 core: Add ostree checkout --compose
This allows merging e.g. multiple artifact trees into one big tree.
2011-11-03 21:50:00 -04:00
Colin Walters
2b00cf3810 core: add ^ for rev-parse 2011-11-02 13:22:13 -04:00
Colin Walters
796dc7fc57 core: remove stub http backend for now
The plain http pull is probably good enough.
2011-11-02 12:43:25 -04:00
Colin Walters
3ceec68a54 pull: Properly clean up after test 2011-11-01 12:35:50 -04:00
Colin Walters
e125dffbfd misc test fixes 2011-11-01 08:49:01 -04:00
Colin Walters
c910f29c00 Consolidate tests
Rather than having a ton of separate tests, be like git and have
a "basic" test that does a lot of the, well, basics.
2011-10-31 22:42:14 -04:00
Colin Walters
a160a2a5fa Add support for pulling from remote archives
This necessitated a large set of changes.

We now support an "archive" mode for repositories.  In this mode,
files are stored "packed" rather than hard linked.  This allows one to
e.g. store an OSTree repository with root-owned files as non-root.  It
is also used as the basis for serving repositories via HTTP.

While doing this I realized that GVariant is endianness-dependent; I
decided to just store all data in big endian.
2011-10-31 20:28:47 -04:00
Colin Walters
79cff61ae8 ostree-http-server: Exit when our directory goes away 2011-10-28 16:56:50 -04:00
Colin Walters
dc6a355f76 ostree-http-server: New simple HTTP server for tests 2011-10-28 16:26:37 -04:00
Colin Walters
d2938ad6cc core: Add "ostree remote add" 2011-10-28 10:25:36 -04:00
Colin Walters
ee4edf114b Don't default to "master", require a branch
It doesn't really make sense to have a default branch, since we expect
people to have multiple roots.  Thus, require a branch
specification always.
2011-10-27 21:42:44 -04:00
Colin Walters
dd13d172ef core: Add rev-parse builtin 2011-10-27 15:44:08 -04:00
Colin Walters
7ca1c3d2e7 core: remove HEAD file, use branches instead
HEAD in git describes a working copy, and we don't have those.
Instead, default to a "master" branch.  This also lets us support
multiple branches.
2011-10-27 14:32:05 -04:00
Colin Walters
d0b3a07cf1 core: Fix checksum for symlinks walking off into uninitialized memory 2011-10-26 18:33:33 -04:00
Colin Walters
b27df6fd72 "Hacktree" is now known as "OSTree"
It just sounds better.
2011-10-18 14:44:48 -04:00
Colin Walters
97003b48da Add log builtin
The "less" spawning is buggy for some reason =(
2011-10-15 15:43:43 -04:00
Colin Walters
c768c2a746 Support for committing files from stdin or a file list file 2011-10-15 13:07:26 -04:00
Colin Walters
2bd973f645 Make file deletion work, add test 2011-10-15 09:56:31 -04:00
Colin Walters
05c35f2cf7 Actually checksum xattr values, not just names
We need to call lgetxattr() and not just llistxattr().
2011-10-15 03:03:51 -04:00
Colin Walters
3eb77d4ce8 Importing and checking out nested trees works! 2011-10-15 01:23:01 -04:00
Colin Walters
401ab27c11 checkout: New command 2011-10-15 00:45:07 -04:00
Colin Walters
f16aad3710 We can commit multiple times now 2011-10-14 23:17:50 -04:00
Colin Walters
98a043c671 Add probably working commits 2011-10-14 21:09:11 -04:00
Colin Walters
5e0d58dd72 tests: Support saving temporary directory 2011-10-14 21:09:11 -04:00
Colin Walters
42a3a68d17 Add test for --repo 2011-10-14 21:09:10 -04:00
Colin Walters
f90994320b Add a COPYING, and put GPL headers in more places 2011-10-12 16:50:55 -04:00
Colin Walters
03425c0dcd Initial test suite 2011-10-12 16:46:23 -04:00