rpm-ostree/docs/HACKING.md
Jonathan Lebon 271954a41c app: Add rpm-ostree compose extensions
This adds support for a new `rpm-ostree compose extensions` command`
which takes a treefile, a new extensions YAML file, and an OSTree repo
and ref. It performs a depsolve and downloads the extensions to a
provided output directory.

This is intended to replace cosa's `download-extensions`:
https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions

The input YAML schema matches the one accepted by that script.

Some differences from the script:
- We have a guaranteed depsolve match and thus can avoid silly issues
  we've hit in RHCOS (like downloading the wrong `libprotobuf` for
  `usbguard` -- rhbz#1889694).
- We seamlessly re-use the same repos defined in the treefile, whereas
  the cosa script uses `reposdir=$dir` which doesn't have the same
  semantics (repo enablement is in that case purely based on the
  `enabled` flag in those repos, which may be different than what the
  rpm-ostree compose ran with).
- We perform more sanity-checks against the requested extensions, such
  as whether the extension is already in the base.
- We support no-change detection via a state SHA512 file for better
  integration in cosa and pipelines.
- We support a `match-base-evr` key, which forces the extension to have
  the same EVR as the one from a base package: this is helpful in the
  case of extensions which complement a base package, esp. those which
  may not have strong enough reldeps to enforce matching EVRs by
  depsolve alone (`kernel-headers` is an example of this).
- We don't try to organize the RPMs into separate directories by
  extension because IMO it's not at the right level. Instead, we should
  work towards higher-level metadata to represent extensions (see
  https://github.com/openshift/os/issues/409 which is related to this).

Closes: #2055
2021-01-23 17:12:09 +01:00

5.1 KiB

nav_order
7

Hacking on rpm-ostree

{: .no_toc }

  1. TOC {:toc}

Raw build instructions

First, releases are available as GPG signed git tags, and most recent versions support extended validation using git-evtag.

You'll need to get the submodules too: git submodule update --init

rpm-ostree has a hard requirement on a bleeding edge version of libhif - we now consume this as a git submodule automatically.

We also require a few other libraries like librepo.

On Fedora, you can install those with the command dnf builddep rpm-ostree.

So the build process now looks like any other autotools program:

env NOCONFIGURE=1 ./autogen.sh
./configure --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc
make

At this point you can run some of the unit tests with make check. For more information on this, see CONTRIBUTING.md.

Doing builds in a container

First, we recommend building in a container (for example docker); you can use other container tools obviously. See ci/build.sh for build and test dependencies.

Testing

You can use make check in a container to run the unit tests. However, if you want to test the daemon in a useful way, you'll need virtualization.

rpm-ostree has some tests that use the coreos-assembler/kola framework.

You will want to build a custom image, then run make install in the ${topsrcdir}/tests/kolainst/ directory, and finally kola run --qemu-image path/to/custom-rpm-ostree-qemu.qcow2 'ext.rpm-ostree.*'. See the kola external tests documentation for more information and also how to filter tests.

There's also a vmcheck test suite. This model always operates on an immutable base image. It takes that image and dynamically launches a separate VM for each test using kola spawn. For example, using the CoreOS Assembler, you can build a FCOS image that contains the version of rpm-ostree that you would like to test. To run the vmcheck test suite on it, symlink the built image to ${topsrcdir}/tests/vmcheck/image.qcow2 and execute tests/vmcheck.sh.

To filter tests, use the TESTS= environment variable. For example, to run only tests/vmcheck/test-misc-2.sh, you can do:

TESTS='misc-2' ./tests/vmcheck.sh

For development, there is also a make vmsync which copies the built rpm-ostree into an unlocked VM. To use this, you must have an ssh-config file with a host defined in it called vmcheck. You can provision the VM however you want; libvirt directly, vagrant, a remote OpenStack/EC2 instance, etc. If you choose vagrant for example, do something like this:

vagrant ssh-config > /path/to/src/rpm-ostree/ssh-config

Note that by default, these commands will retrieve the latest version of ostree from the build environment and include those binaries when syncing to the VM. So make sure to have the latest ostree installed or built. This allows you to not have to worry about using libostree APIs that are not yet released.

For more details on how tests are structured, see tests/README.md.

Testing with a custom libdnf

rpm-ostree bundles libdnf since commit 125c482b1d the rationale is:

  • libdnf broke ABI several times silently in the past
  • Today, dnf does not actually use libdnf much, which means for the most part any libdnf breakage is first taken by us
  • libdnf is trying to rewrite more in C++, which is unlikely to help API/ABI stability
  • dnf and rpm-ostree release on separate cycles (e.g. today rpm-ostree is used by OpenShift)

In general, until libdnf is defined 100% API/ABI stable, we will continue to bundle it.

However, because it's a git submodule, it's easy to test updates to it, and it also means we're not forking it.

So just do e.g.:

cd libdnf
git fetch origin
git reset --hard origin/master
cd ..

The various make targets will pick up the changes and recompile.

Testing with a custom ostree

It is sometimes necessary to develop against a version of ostree which is not even yet in git master. In such situations, one can simply do:

$ # from the rpm-ostree build dir
$ INSTTREE=$PWD/insttree
$ rm -rf $INSTTREE
$ # from the ostree build dir
$ make
$ make install DESTDIR=$INSTTREE
$ # from the rpm-ostree build dir
$ make
$ make install DESTDIR=$INSTTREE

At this point, simply set SKIP_INSTALL=1 when running vmsync and vmoverlay to reuse the installation tree and sync the installed binaries there:

$ make vmsync SKIP_INSTALL=1
$ make vmoverlay SKIP_INSTALL=1

Of course, you can use this pattern for not just ostree but whatever else you'd like to install into the VM (e.g. bubblewrap, libsolv, etc...).