keremet/rpm-ostree
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
058178680f
rpm-ostree/tests/compose/test-basic-unified.sh

128 lines
4.4 KiB
Bash
Raw Normal View History

compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
#!/bin/bash
set -xeuo pipefail
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
dn=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=libcomposetest.sh
. "${dn}/libcomposetest.sh"
# Add a local rpm-md repo so we can mutate local test packages
treefile_append "repos" '["test-repo"]'
# test `recommends: false` (test-misc-tweaks tests the true path)
build_rpm foobar recommends foobar-rec
build_rpm foobar-rec
echo gpgcheck=0 >> yumrepo.repo
ln "$PWD/yumrepo.repo" config/yumrepo.repo
treefile_append "packages" '["foobar"]'
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
# Test --print-only. We also
Move varsubst code into Rust, use it in treefile parsing External tools often want to parse the ref; for example coreos-assembler currently does so. Let's ensure `${basearch}` is expanded with `--print-only` so they can parse that JSON to get the expanded version reliably. Implementation note: this is the first Rust code which exposes a "GLib-like" C API, notably with GHashTable, so we're making more use of the glib-rs bindings. Closes: #1653 Closes: #1655 Approved by: jlebon
2018-11-01 22:13:30 +03:00
# just in this test (for now) use ${basearch} to test substitution.
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
# shellcheck disable=SC2016
treefile_set_ref '"fedora/stable/${basearch}/basic-unified"'
rpm-ostree compose tree --print-only "${treefile}" > treefile.json
compose: Make --print-only happen earlier and be quiet Don't print the version or other warnings; ensure we output clean JSON. Prep for fixing https://github.com/coreos/fedora-coreos-config/pull/24 (This helps, but we still need to expand `${basearch}`) Closes: #1648 Approved by: jlebon
2018-10-30 17:19:49 +03:00
# Verify it's valid JSON
jq -r .ref < treefile.json > ref.txt
Move varsubst code into Rust, use it in treefile parsing External tools often want to parse the ref; for example coreos-assembler currently does so. Let's ensure `${basearch}` is expanded with `--print-only` so they can parse that JSON to get the expanded version reliably. Implementation note: this is the first Rust code which exposes a "GLib-like" C API, notably with GHashTable, so we're making more use of the glib-rs bindings. Closes: #1653 Closes: #1655 Approved by: jlebon
2018-11-01 22:13:30 +03:00
# Test substitution of ${basearch}
compose: Make --print-only happen earlier and be quiet Don't print the version or other warnings; ensure we output clean JSON. Prep for fixing https://github.com/coreos/fedora-coreos-config/pull/24 (This helps, but we still need to expand `${basearch}`) Closes: #1648 Approved by: jlebon
2018-10-30 17:19:49 +03:00
assert_file_has_content_literal ref.txt "${treeref}"
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
treefile_pyedit "tf['add-commit-metadata']['foobar'] = 'bazboo'"
treefile_pyedit "tf['add-commit-metadata']['overrideme'] = 'old var'"
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
# Test metadata json with objects, arrays, numbers
cat > metadata.json <<EOF
{
"exampleos.gitrepo": {
"rev": "97ec21c614689e533d294cdae464df607b526ab9",
"src": "https://gitlab.com/exampleos/custom-atomic-host"
},
treefile: Add new `add-commit-metadata` key Add support for a new `add-commit-metadata` key in the treefile so that we can directly specify commit metadata we want to inject from there. This will be useful in Fedora CoreOS, where we'll have separate treefiles for each streams, each with stream-specific metadata values required. Closes: #1865 Approved by: cgwalters
2019-07-08 17:34:59 +03:00
"exampleos.tests": ["smoketested", "e2e"],
"overrideme": "new val"
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
}
EOF
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
compose: Add --parent option This may seem like a backflip on #1829, but there's a common theme here: in a promotion workflow, the parent (or lack of parent) of a commit is an important parameter, so we need full flexibility in configuring it. But again, like #1829, we still want e.g. change detection, versioning, and various optimizations to happen on whatever the latest commit on that ref is in the build repo. Closes: #1871 Approved by: cgwalters
2019-07-18 18:25:24 +03:00
# Test --parent at the same time (hash is `echo | sha256sum`)
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
runcompose --add-metadata-from-json $(pwd)/metadata.json \
compose: Add --parent option This may seem like a backflip on #1829, but there's a common theme here: in a promotion workflow, the parent (or lack of parent) of a commit is an important parameter, so we need full flexibility in configuring it. But again, like #1829, we still want e.g. change detection, versioning, and various optimizations to happen on whatever the latest commit on that ref is in the build repo. Closes: #1871 Approved by: cgwalters
2019-07-18 18:25:24 +03:00
--parent 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
compose: Fix EBADF in unified core mode without cachedir If no cache dir is given in the workdir, we would alias the cache dir fd to the workdir fd. But of course, this meant that we'd try to close the same fd twice when freeing the compose context. Instead, let's just copy the fd as is also done in the non-unified path. Closes: #1697 Closes: #1698 Approved by: lucab
2018-12-06 00:51:40 +03:00
# Run it again, but without RPMOSTREE_PRESERVE_TMPDIR. Should be a no-op. This
# exercises fd handling in the tree context.
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
(unset RPMOSTREE_PRESERVE_TMPDIR && runcompose)
compose: Fix EBADF in unified core mode without cachedir If no cache dir is given in the workdir, we would alias the cache dir fd to the workdir fd. But of course, this meant that we'd try to close the same fd twice when freeing the compose context. Instead, let's just copy the fd as is also done in the non-unified path. Closes: #1697 Closes: #1698 Approved by: lucab
2018-12-06 00:51:40 +03:00
echo "ok no cachedir"
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
# shellcheck source=libbasic-test.sh
. "${dn}/libbasic-test.sh"
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
basic_test
# This one is done by postprocessing /var
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
ostree --repo="${repo}" cat "${treeref}" /usr/lib/tmpfiles.d/pkg-filesystem.conf > autovar.txt
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
# Picked this one at random as an example of something that won't likely be
# converted to tmpfiles.d upstream. But if it is, we can change this test.
assert_file_has_content_literal autovar.txt 'd /var/cache 0755 root root - -'
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
ostree --repo="${repo}" cat "${treeref}" /usr/lib/tmpfiles.d/pkg-chrony.conf > autovar.txt
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
# And this one has a non-root uid
assert_file_has_content_literal autovar.txt 'd /var/log/chrony 0755 chrony chrony - -'
importer: Don't generate var-tmpfiles.d for rpm itself This caused `/var/lib/rpm` to be a directory in my Fedora CoreOS builds. Closes: #1532 Approved by: sinnykumari
2018-09-07 17:48:28 +03:00
# see rpmostree-importer.c
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
if ostree --repo="${repo}" cat "${treeref}" /usr/lib/tmpfiles.d/pkg-rpm.conf > rpm.txt 2>/dev/null; then
importer: Don't generate var-tmpfiles.d for rpm itself This caused `/var/lib/rpm` to be a directory in my Fedora CoreOS builds. Closes: #1532 Approved by: sinnykumari
2018-09-07 17:48:28 +03:00
assert_not_file_has_content rpm.txt 'd /var/lib/rpm'
fi
importer: Use /run instead of /var/run Translate RPM paths under `/var/run` to `/run` automatically; this quiets down systemd. Since we end up running `systemd-tmpfiles` a few times in FCOS reducing spew here is particularly valuable. The bug is really in the packages here but...we don't have an agile process for fixing them. Note that for this fix to take effect, if you have a `cache/pkgcache-repo` you'll need to remove it.
2020-02-02 18:15:51 +03:00
ostree --repo="${repo}" cat "${treeref}" /usr/lib/tmpfiles.d/pkg-pam.conf > autovar.txt
# Verify translating /var/run -> /run
assert_file_has_content_literal autovar.txt 'd /run/console'
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
echo "ok autovar"
compose: Do relabel before downloading in --ex-unified-core We removed this in review, but I rediscovered why I added it. We fail the `g_assert (sepolicy_matches)` if we already had packages done with the final label when we go to reuse the cache. (Basically, if we use the cache multiple times it's hard to avoid relabeling all the time which is unfortunate...gets back a bit to a way to annotate pkgcache commits as supporting multiple policies) Closes: #1109 Approved by: jlebon
2017-11-17 23:06:50 +03:00
compose: Add --no-parent option There are cases where we do want all the things that specifying a ref provides (e.g. change detection, version incrementing, SELinux labeling optimizations, and of course writing the ref) but we *don't* want the new commit to have a parent. Add a new `--no-parent` option to accommodate this. This will be used by coreos-assembler. See discussions at https://github.com/coreos/coreos-assembler/issues/159. Closes: #1829 Approved by: cgwalters
2019-05-08 07:13:43 +03:00
# And redo it to trigger relabeling. Also test --no-parent at the same time.
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
origrev=$(ostree --repo="${repo}" rev-parse "${treeref}")
runcompose --force-nocache --no-parent
newrev=$(ostree --repo="${repo}" rev-parse "${treeref}")
compose: Do relabel before downloading in --ex-unified-core We removed this in review, but I rediscovered why I added it. We fail the `g_assert (sepolicy_matches)` if we already had packages done with the final label when we go to reuse the cache. (Basically, if we use the cache multiple times it's hard to avoid relabeling all the time which is unfortunate...gets back a bit to a way to annotate pkgcache commits as supporting multiple policies) Closes: #1109 Approved by: jlebon
2017-11-17 23:06:50 +03:00
assert_not_streq "${origrev}" "${newrev}"
echo "ok rerun"
compose: Add --no-parent option There are cases where we do want all the things that specifying a ref provides (e.g. change detection, version incrementing, SELinux labeling optimizations, and of course writing the ref) but we *don't* want the new commit to have a parent. Add a new `--no-parent` option to accommodate this. This will be used by coreos-assembler. See discussions at https://github.com/coreos/coreos-assembler/issues/159. Closes: #1829 Approved by: cgwalters
2019-05-08 07:13:43 +03:00
# And check that --no-parent worked.
tests/compose: Target FCOS 31, move off of PAPR Again, a lot going on here, but essentially, we adapt the compose tests to run either privileged or fully unprivileged via supermin, just like cosa. I actually got more than halfway through this initially using `cosa build` directly for testing. But in the end, we simply need more flexibility than that. We want to be able to manipulate exactly how rpm-ostree is called, and cosa is very opinionated about this (and may also change from under us in the future). (Another big difference for example is that cosa doesn't care about non-unified mode, whereas we *need* to have coverage for this until we fully kill it.) Really, the most important bit we want from there is the unprivileged-via-supermin bits. So we copy and adapt that here. One obvious improvement then is sharing this code more easily (e.g. a `cosa runasroot` or something?) However, we still use the FCOS manifest (frozen at a specific tag). It's a realistic example, and because of the lockfiles and pool, we get good reproducibility.
2019-12-22 01:42:09 +03:00
if ostree rev-parse --repo "${repo}" "${newrev}"^ 2>error.txt; then
compose: Add --no-parent option There are cases where we do want all the things that specifying a ref provides (e.g. change detection, version incrementing, SELinux labeling optimizations, and of course writing the ref) but we *don't* want the new commit to have a parent. Add a new `--no-parent` option to accommodate this. This will be used by coreos-assembler. See discussions at https://github.com/coreos/coreos-assembler/issues/159. Closes: #1829 Approved by: cgwalters
2019-05-08 07:13:43 +03:00
assert_not_reached "New revision has a parent even with --no-parent?"
fi
tests/compose: Fix unified rev-parse test We were hitting the classic "negative test passes for the wrong reason". It was failing not because it didn't have a parent, but because we didn't pass `--repo`. Fix this and also explicitly check for the error message we expect. Closes: #1865 Approved by: cgwalters
2019-07-08 19:22:29 +03:00
assert_file_has_content_literal error.txt 'has no parent'
compose: Add --no-parent option There are cases where we do want all the things that specifying a ref provides (e.g. change detection, version incrementing, SELinux labeling optimizations, and of course writing the ref) but we *don't* want the new commit to have a parent. Add a new `--no-parent` option to accommodate this. This will be used by coreos-assembler. See discussions at https://github.com/coreos/coreos-assembler/issues/159. Closes: #1829 Approved by: cgwalters
2019-05-08 07:13:43 +03:00
echo "ok --no-parent"
app: Add `rpm-ostree compose extensions` This adds support for a new `rpm-ostree compose extensions` command` which takes a treefile, a new extensions YAML file, and an OSTree repo and ref. It performs a depsolve and downloads the extensions to a provided output directory. This is intended to replace cosa's `download-extensions`: https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions The input YAML schema matches the one accepted by that script. Some differences from the script: - We have a guaranteed depsolve match and thus can avoid silly issues we've hit in RHCOS (like downloading the wrong `libprotobuf` for `usbguard` -- rhbz#1889694). - We seamlessly re-use the same repos defined in the treefile, whereas the cosa script uses `reposdir=$dir` which doesn't have the same semantics (repo enablement is in that case purely based on the `enabled` flag in those repos, which may be different than what the rpm-ostree compose ran with). - We perform more sanity-checks against the requested extensions, such as whether the extension is already in the base. - We support no-change detection via a state SHA512 file for better integration in cosa and pipelines. - We support a `match-base-evr` key, which forces the extension to have the same EVR as the one from a base package: this is helpful in the case of extensions which complement a base package, esp. those which may not have strong enough reldeps to enforce matching EVRs by depsolve alone (`kernel-headers` is an example of this). - We don't try to organize the RPMs into separate directories by extension because IMO it's not at the right level. Instead, we should work towards higher-level metadata to represent extensions (see https://github.com/openshift/os/issues/409 which is related to this). Closes: #2055
2021-01-12 00:53:05 +03:00
build_rpm dodo-base
build_rpm dodo requires dodo-base
build_rpm solitaire
cat > extensions.yaml << EOF
extensions:
extinct-birds:
packages:
- dodo
- solitaire
extensions: Write JSON to output dir Let's include the final extensions file in JSON format as part of the output directory. A key difference from the input file (apart from YAML vs JSON) is that this is post-filtering, so any extensions which were removed because the architecture does not match are not present. This JSON file will be used by cosa and the MCO. See discussions in: https://github.com/openshift/os/issues/409
2021-01-27 01:46:02 +03:00
another-arch:
packages:
- nonexistent
architectures:
- badarch
app: Add `rpm-ostree compose extensions` This adds support for a new `rpm-ostree compose extensions` command` which takes a treefile, a new extensions YAML file, and an OSTree repo and ref. It performs a depsolve and downloads the extensions to a provided output directory. This is intended to replace cosa's `download-extensions`: https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions The input YAML schema matches the one accepted by that script. Some differences from the script: - We have a guaranteed depsolve match and thus can avoid silly issues we've hit in RHCOS (like downloading the wrong `libprotobuf` for `usbguard` -- rhbz#1889694). - We seamlessly re-use the same repos defined in the treefile, whereas the cosa script uses `reposdir=$dir` which doesn't have the same semantics (repo enablement is in that case purely based on the `enabled` flag in those repos, which may be different than what the rpm-ostree compose ran with). - We perform more sanity-checks against the requested extensions, such as whether the extension is already in the base. - We support no-change detection via a state SHA512 file for better integration in cosa and pipelines. - We support a `match-base-evr` key, which forces the extension to have the same EVR as the one from a base package: this is helpful in the case of extensions which complement a base package, esp. those which may not have strong enough reldeps to enforce matching EVRs by depsolve alone (`kernel-headers` is an example of this). - We don't try to organize the RPMs into separate directories by extension because IMO it's not at the right level. Instead, we should work towards higher-level metadata to represent extensions (see https://github.com/openshift/os/issues/409 which is related to this). Closes: #2055
2021-01-12 00:53:05 +03:00
EOF
# we don't actually need root here, but in CI the cache may be in a qcow2 and
# the supermin code is gated behind `runasroot`
runasroot rpm-ostree compose extensions --repo=${repo} \
--cachedir=${test_tmpdir}/cache --base-rev ${treeref} \
--output-dir extensions ${treefile} extensions.yaml \
--touch-if-changed extensions-changed
ls extensions/{dodo-1.0,dodo-base-1.0,solitaire-1.0}-*.rpm
test -f extensions-changed
extensions: Write JSON to output dir Let's include the final extensions file in JSON format as part of the output directory. A key difference from the input file (apart from YAML vs JSON) is that this is post-filtering, so any extensions which were removed because the architecture does not match are not present. This JSON file will be used by cosa and the MCO. See discussions in: https://github.com/openshift/os/issues/409
2021-01-27 01:46:02 +03:00
assert_jq extensions/extensions.json \
'.extensions|length == 1' \
'.extensions["extinct-birds"]'
app: Add `rpm-ostree compose extensions` This adds support for a new `rpm-ostree compose extensions` command` which takes a treefile, a new extensions YAML file, and an OSTree repo and ref. It performs a depsolve and downloads the extensions to a provided output directory. This is intended to replace cosa's `download-extensions`: https://github.com/coreos/coreos-assembler/blob/master/src/download-extensions The input YAML schema matches the one accepted by that script. Some differences from the script: - We have a guaranteed depsolve match and thus can avoid silly issues we've hit in RHCOS (like downloading the wrong `libprotobuf` for `usbguard` -- rhbz#1889694). - We seamlessly re-use the same repos defined in the treefile, whereas the cosa script uses `reposdir=$dir` which doesn't have the same semantics (repo enablement is in that case purely based on the `enabled` flag in those repos, which may be different than what the rpm-ostree compose ran with). - We perform more sanity-checks against the requested extensions, such as whether the extension is already in the base. - We support no-change detection via a state SHA512 file for better integration in cosa and pipelines. - We support a `match-base-evr` key, which forces the extension to have the same EVR as the one from a base package: this is helpful in the case of extensions which complement a base package, esp. those which may not have strong enough reldeps to enforce matching EVRs by depsolve alone (`kernel-headers` is an example of this). - We don't try to organize the RPMs into separate directories by extension because IMO it's not at the right level. Instead, we should work towards higher-level metadata to represent extensions (see https://github.com/openshift/os/issues/409 which is related to this). Closes: #2055
2021-01-12 00:53:05 +03:00
echo "ok extensions"
rm extensions-changed
runasroot rpm-ostree compose extensions --repo=${repo} \
--cachedir=${test_tmpdir}/cache --base-rev ${treeref} \
--output-dir extensions ${treefile} extensions.yaml \
--touch-if-changed extensions-changed
if test -f extensions-changed; then
fatal "found extensions-changed"
fi
echo "ok extensions no change"
Reference in New Issue Copy Permalink