Operating system and container binary deployment and upgrades
Go to file
Colin Walters 5c7d2dd8be Deduplicate and fix up our use of mmap()
Buried in this large patch is a logical fix:

```
-  if (!map)
-    return glnx_throw_errno_prefix (error, "mmap");
+  if (map == (void*)-1)
+    return glnx_null_throw_errno_prefix (error, "mmap");
```

Which would have helped me debug another patch I was working
on.  But it turns out that actually correctly checking for
errors from `mmap()` triggers lots of other bugs - basically
because we sometimes handle zero-length variants (in detached
metadata).  When we start actually returning errors due to
this, things break.  (It wasn't a problem in practice before
because most things looked at the zero size, not the data).

Anyways there's a bigger picture issue here - a while ago
we made a fix to only use `mmap()` for reading metadata from disk
only if it was large enough (i.e. `>16k`).  But that didn't
help various other paths in the pull code and others that were
directly doing the `mmap()`.

Fix this by having a proper low level fs helper that does "read all data from
fd+offset into GBytes", which handles the size check. Then the `GVariant` bits
are just a clean layer on top of this. (At the small cost of an additional
allocation)

Side note: I had to remind myself, but the reason we can't just use
`GMappedFile` here is it doesn't support passing an offset into `mmap()`.

Closes: #1251
Approved by: jlebon
2017-10-04 20:42:39 +00:00
apidoc lib/repo-refs: Add first version of ostree_repo_resolve_collection_ref() 2017-09-27 14:44:00 +00:00
bash lib/checkout: Rename disjoint union, change to merge identical files 2017-09-13 19:19:33 +00:00
bsdiff@1edf9f6568 bsdiff: change submodule location 2015-03-26 23:33:07 +01:00
build-aux Add infrastructure for "make syntax-check" 2015-01-30 15:27:36 +01:00
buildutil tests: allow specifying tmpdir 2017-09-21 21:50:40 +00:00
ci ci: really turn on warnings 2017-10-04 12:54:53 +00:00
coccinelle tree-wide: Add+run spatch to use glnx_throw() 2017-05-26 19:27:11 +00:00
docs lib/deploy: Add .img to end of initramfs in /usr/lib/modules 2017-08-21 16:41:50 +00:00
libglnx@e30154431d tree-wide: Bump libglnx, port to new lockfile init 2017-09-27 20:08:34 +00:00
man ostree/summary: Generate an ostree-metadata ref when updating summary 2017-10-02 13:39:41 +00:00
manual-tests repo+tests: Add [core]disable-xattrs=true, use it on overlayfs 2017-03-24 22:16:43 +00:00
rust rust: Support make dist -> cargo vendor 2017-02-07 16:50:37 +00:00
src Deduplicate and fix up our use of mmap() 2017-10-04 20:42:39 +00:00
tests lib/pull: Fix regression with pull-local for nonexistent refs 2017-10-02 16:02:28 +00:00
.dir-locals.el .dir-locals.el: Standard Emacs indentation config 2017-01-12 16:09:34 +00:00
.editorconfig Add a .vimrc and .editorconfig 2017-09-21 22:03:11 +00:00
.gitmodules bsdiff: change submodule location 2015-03-26 23:33:07 +01:00
.papr.yml ci: Record primary context results using g-d-t-r --log-directory 2017-09-27 18:08:06 +00:00
.travis.yml ci: Move travis scripts from tests/ → ci/ 2017-05-09 18:25:13 +00:00
.vimrc Add a .vimrc and .editorconfig 2017-09-21 22:03:11 +00:00
autogen.sh autogen.sh: Fix running out of tree 2017-08-08 12:44:17 +00:00
cfg.mk tests: add a syntax-check rule for glnx_prefix_error() 2017-06-26 15:09:12 +00:00
configure.ac build-sys: Post-release version bump 2017-10-02 15:11:42 +00:00
CONTRIBUTING.md Rewrite manual in mkdocs 2016-01-28 09:31:37 -05:00
COPYING COPYING: Update to latest FSF with current address 2014-01-16 10:22:30 -05:00
git.mk Use git.mk 2016-04-07 12:49:40 +00:00
GNUmakefile Add infrastructure for "make syntax-check" 2015-01-30 15:27:36 +01:00
maint.mk build/maint.mk: Comment out setting of LC_ALL 2017-08-23 17:41:06 +00:00
Makefile-bash.am build: Add distcheck configure flags to fix systemd and bash-completion 2017-08-22 17:37:48 +00:00
Makefile-boot.am build: Ensure ostree-tmpfiles.conf is distributed 2017-08-22 17:37:48 +00:00
Makefile-decls.am build-sys: Minor makefile tweaks 2017-01-04 16:32:11 +00:00
Makefile-libostree-defines.am lib/repo-finder: Add Avahi based OstreeRepoFinder implementation 2017-06-26 15:56:07 +00:00
Makefile-libostree.am Add --with-crypto=gnutls 2017-09-25 12:58:54 +00:00
Makefile-man.am build: Don't distribute generated man pages 2017-07-18 21:51:43 +00:00
Makefile-ostree.am create-usb: Add a create-usb command to complement OstreeRepoFinderMount 2017-09-27 14:44:00 +00:00
Makefile-otutil.am lib/sysroot: Add journal-msg signal 2017-08-10 14:20:00 +00:00
Makefile-switchroot.am build: Add distcheck configure flags to fix systemd and bash-completion 2017-08-22 17:37:48 +00:00
Makefile-tests.am tests: Add a manual integration test for OstreeRepoFinderMount 2017-09-28 14:08:40 +00:00
Makefile.am bash: Add bash completion 2017-08-15 15:51:34 +00:00
mkdocs.yml docs: Add a section on repository management 2016-03-29 14:10:24 +00:00
ostree.doap doap category infrastructure 2014-07-31 11:26:32 +02:00
README-historical.md README: Just link to wiki, move most of it to README-historical.md 2014-01-20 18:00:09 -05:00
README.md docs/build: s/libOSTree/libostree/ 2017-08-18 17:51:09 +00:00
TODO Fix repeated words. 2015-01-30 15:27:36 +01:00

libostree

New! See the docs online at Read The Docs (OSTree)


This project is now known as "libostree", though it is still appropriate to use the previous name: "OSTree" (or "ostree"). The focus is on projects which use libostree's shared library, rather than users directly invoking the command line tools (except for build systems). However, in most of the rest of the documentation, we will use the term "OSTree", since it's slightly shorter, and changing all documentation at once is impractical. We expect to transition to the new name over time.

As implied above, libostree is both a shared library and suite of command line tools that combines a "git-like" model for committing and downloading bootable filesystem trees, along with a layer for deploying them and managing the bootloader configuration.

The core OSTree model is like git in that it checksums individual files and has a content-addressed-object store. It's unlike git in that it "checks out" the files via hardlinks, and they should thus be immutable. Therefore, another way to think of OSTree is that it's just a more polished version of Linux VServer hardlinks.

Features:

  • Transactional upgrades and rollback for the system
  • Replicating content incrementally over HTTP via GPG signatures and "pinned TLS" support
  • Support for parallel installing more than just 2 bootable roots
  • Binary history on the server side (and client)
  • Introspectable shared library API for build and deployment systems
  • Flexible support for multiple branches and repositories, supporting projects like flatpak which use libostree for applications, rather than hosts.

Projects using OSTree

meta-updater is a layer available for OpenEmbedded systems.

QtOTA is Qt's over-the-air update framework which uses libostree.

rpm-ostree is a next-generation hybrid package/image system for Fedora and CentOS, used by the Atomic Host project. By default it uses libostree to atomically replicate a base OS (all dependency resolution is done on the server), but it supports "package layering", where additional RPMs can be layered on top of the base. This brings a "best of both worlds"" model for image and package systems.

flatpak uses libostree for desktop application containers. Unlike most of the other systems here, flatpak does not use the "libostree host system" aspects (e.g. bootloader management), just the "git-like hardlink dedup". For example, flatpak supports a per-user OSTree repository.

Endless OS uses libostree for their host system as well as flatpak. See their eos-updater and deb-ostree-builder projects.

GNOME Continuous is where OSTree was born - as a high performance continuous delivery/testing system for GNOME.

Building

Releases are available as GPG signed git tags, and most recent versions support extended validation using git-evtag.

However, in order to build from a git clone, you must update the submodules. If you're packaging OSTree and want a tarball, I recommend using a "recursive git archive" script. There are several available online; this code in OSTree is an example.

Once you have a git clone or recursive archive, building is the same as almost every autotools project:

git submodule update --init
env NOCONFIGURE=1 ./autogen.sh
./configure --prefix=...
make
make install DESTDIR=/path/to/dest

More documentation

New! See the docs online at Read The Docs (OSTree)

Contributing

See Contributing.