rpm-ostree/docs/HACKING.md

146 lines
5.2 KiB
Markdown
Raw Normal View History

2020-10-01 12:57:05 +03:00
---
app: Add `rpm-ostree compose extensions` This adds support for a new `rpm-ostree compose extensions` command` which takes a treefile, a new extensions YAML file, and an OSTree repo and ref. It performs a depsolve and downloads the extensions to a provided output directory. This is intended to replace cosa's `download-extensions`: https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions The input YAML schema matches the one accepted by that script. Some differences from the script: - We have a guaranteed depsolve match and thus can avoid silly issues we've hit in RHCOS (like downloading the wrong `libprotobuf` for `usbguard` -- rhbz#1889694). - We seamlessly re-use the same repos defined in the treefile, whereas the cosa script uses `reposdir=$dir` which doesn't have the same semantics (repo enablement is in that case purely based on the `enabled` flag in those repos, which may be different than what the rpm-ostree compose ran with). - We perform more sanity-checks against the requested extensions, such as whether the extension is already in the base. - We support no-change detection via a state SHA512 file for better integration in cosa and pipelines. - We support a `match-base-evr` key, which forces the extension to have the same EVR as the one from a base package: this is helpful in the case of extensions which complement a base package, esp. those which may not have strong enough reldeps to enforce matching EVRs by depsolve alone (`kernel-headers` is an example of this). - We don't try to organize the RPMs into separate directories by extension because IMO it's not at the right level. Instead, we should work towards higher-level metadata to represent extensions (see https://github.com/openshift/os/issues/409 which is related to this). Closes: #2055
2021-01-12 00:53:05 +03:00
nav_order: 7
2020-10-01 12:57:05 +03:00
---
# Hacking on rpm-ostree
{: .no_toc }
1. TOC
{:toc}
## Building and testing in a container
2020-10-01 12:57:05 +03:00
The majority of developers on rpm-ostree build and test it
from a [toolbox container](https://github.com/containers/toolbox), separate from the host system.
The instructions below may *also* work when run in a traditional
login on a virtual machine, but are less frequently tested.
2020-10-01 12:57:05 +03:00
### Installing dependencies (cxx)
2020-10-01 12:57:05 +03:00
Today rpm-ostree uses [cxx.rs](https://cxx.rs/) - the CLI tools for
that aren't packaged in e.g. Fedora; we ship the pre-generated
source in the releases. But to build from git you need to install the
tools.
2020-10-01 12:57:05 +03:00
```
$ ./ci/installdeps.sh
```
2020-10-01 12:57:05 +03:00
You will also need to rerun this after the dependency changes in our
`Cargo.lock`. Eventually we will fix this.
2020-10-01 12:57:05 +03:00
### Installing dependencies: packages
2020-10-01 12:57:05 +03:00
Otherwise, you can use e.g. `sudo dnf builddep rpm-ostree` to get
the rest of the build dependencies.
2020-10-01 12:57:05 +03:00
### Baseline build
2020-10-01 12:57:05 +03:00
rpm-ostree uses autotools to build both our C/C++ side as well
as to invoke `cargo` to build the Rust code. After you've
[cloned the repository](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository):
2020-10-01 12:57:05 +03:00
```
$ git submodule update --init
$ ./autogen.sh --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc
$ make
```
### Unit tests
```
$ make check
```
2020-10-01 12:57:05 +03:00
### Virtualized integration testing
2020-10-01 12:57:05 +03:00
The unit tests today don't cover much; rpm-ostree is very oriented to run as a privileged systemd unit managing a host system.
2020-10-01 12:57:05 +03:00
rpm-ostree has some tests that use the [coreos-assembler/kola framework](https://coreos.github.io/coreos-assembler/kola/external-tests/).
2020-10-01 12:57:05 +03:00
You will want to [build a custom image](https://coreos.github.io/coreos-assembler/working/#using-overrides), then run `make install` in the `${topsrcdir}/tests/kolainst/` directory, and finally `kola run --qemu-image path/to/custom-rpm-ostree-qemu.qcow2 'ext.rpm-ostree.*'`. See the [kola external tests documentation](https://coreos.github.io/coreos-assembler/kola/external-tests/#using-kola-run-with-externally-defined-tests) for more information and also how to filter tests.
2020-10-01 12:57:05 +03:00
There's also a `vmcheck` test suite. This model always operates on an immutable base image. It takes that image and dynamically launches a separate VM for each test using `kola spawn`. For example, using the [CoreOS Assembler](https://coreos.github.io/coreos-assembler/building-fcos/), you can build a FCOS image that contains the version of rpm-ostree that you would like to test.
One approach for (somewhat) fast iteration is `cosa build-fast`, then run e.g. `./tests/vmcheck.sh`.
To filter tests, use the `TESTS=` environment variable. For example, to run only `tests/vmcheck/test-misc-2.sh`, you can do:
```sh
TESTS='misc-2' ./tests/vmcheck.sh
```
For development, there is also a `make vmsync` which copies the built rpm-ostree
into an unlocked VM. To use this, you must have an `ssh-config` file with a host
defined in it called `vmcheck`. You can provision the VM however you want;
libvirt directly, vagrant, a remote OpenStack/EC2 instance, etc. If you choose
2020-10-01 12:57:05 +03:00
vagrant for example, do something like this:
```sh
vagrant ssh-config > /path/to/src/rpm-ostree/ssh-config
```
Note that by default, these commands will retrieve the latest version of ostree
from the build environment and include those binaries when syncing to the VM.
So make sure to have the latest ostree installed or built. This allows you to
not have to worry about using libostree APIs that are not yet released.
2020-10-01 12:57:05 +03:00
For more details on how tests are structured, see [tests/README.md](tests/README.md).
## Testing with a custom libdnf
rpm-ostree bundles libdnf since commit https://github.com/coreos/rpm-ostree/commit/125c482b1d16ce8376378f220fc2f93a5b157bc1
the rationale is:
- libdnf broke ABI several times silently in the past
- Today, dnf does not actually *use* libdnf much, which means
for the most part any libdnf breakage is first taken by us
- libdnf is trying to rewrite more in C++, which is unlikely to help
API/ABI stability
- dnf and rpm-ostree release on separate cycles (e.g. today rpm-ostree
is used by OpenShift)
In general, until libdnf is defined 100% API/ABI stable, we will
continue to bundle it.
However, because it's a git submodule, it's easy to test updates
to it, and it also means we're not *forking* it.
So just do e.g.:
```
cd libdnf
git fetch origin
2021-05-07 17:12:45 +03:00
git reset --hard origin/main
2020-10-01 12:57:05 +03:00
cd ..
```
The various `make` targets will pick up the changes and recompile.
## Testing with a custom ostree
It is sometimes necessary to develop against a version of ostree which is not
2021-05-07 17:12:45 +03:00
even yet in git main. In such situations, one can simply do:
2020-10-01 12:57:05 +03:00
```sh
$ # from the rpm-ostree build dir
$ INSTTREE=$PWD/insttree
$ rm -rf $INSTTREE
$ # from the ostree build dir
$ make
$ make install DESTDIR=$INSTTREE
$ # from the rpm-ostree build dir
$ make
$ make install DESTDIR=$INSTTREE
```
At this point, simply set `SKIP_INSTALL=1` when running `vmsync` and `vmoverlay`
to reuse the installation tree and sync the installed binaries there:
```sh
$ make vmsync SKIP_INSTALL=1
$ make vmoverlay SKIP_INSTALL=1
```
Of course, you can use this pattern for not just ostree but whatever else you'd
like to install into the VM (e.g. bubblewrap, libsolv, etc...).