2016-12-03 02:07:08 +03:00
#!/bin/bash
2016-12-09 01:31:20 +03:00
set -euo pipefail
2016-12-03 02:07:08 +03:00
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
# freeze on a specific commit for tests for reproducibility and since it should
# always work to target older treefiles
2020-10-13 17:58:15 +03:00
FEDORA_COREOS_CONFIG_COMMIT = 501c4edc0c3d2bfea7d8fcf857eeb0add765d11d
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
dn = $( cd " $( dirname " $0 " ) " && pwd )
topsrcdir = $( cd " $dn /.. " && pwd )
commondir = $( cd " $dn /common " && pwd )
export topsrcdir commondir
# shellcheck source=common/libtest-core.sh
. " ${ commondir } /libtest.sh "
read -r -a tests <<< " $( filter_tests " ${ topsrcdir } /tests/compose " ) "
if [ ${# tests [*] } -eq 0 ] ; then
echo "No tests selected; mistyped filter?"
exit 0
2018-08-09 19:14:01 +03:00
fi
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
JOBS = ${ JOBS :- $( ncpus) }
outputdir = " ${ topsrcdir } /compose-logs "
fixtures = " $( pwd ) /compose-cache "
# re-use the same FCOS config and RPMs if it already exists
if [ ! -d compose-cache ] ; then
mkdir -p compose-cache
# first, download all the RPMs into a directory
echo "Caching test fixtures in compose-cache/"
# Really want to use cosa fetch for this and just share the pkgcache repo.
# Though for now we still need to support non-unified mode. Once we don't, we
# can clean this up.
pushd compose-cache
git clone https://github.com/coreos/fedora-coreos-config config
pushd config
2020-04-14 23:44:05 +03:00
git checkout " ${ FEDORA_COREOS_CONFIG_COMMIT } "
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
# we flatten the treefile to make it easier to manipulate in tests (we have
# lots of tests that check for include logic already)
rpm-ostree compose tree --print-only manifest.yaml > manifest.json
rm manifest.yaml
mv manifests/{ passwd,group} .
rm -rf manifests/
2020-04-14 23:44:05 +03:00
popd # config
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
if ! has_compose_privileges; then
# Unlike cosa, we don't need as much flexibility since we don't e.g. build
# images. So just create the supermin appliance and root now so each test
# doesn't have to build it.
mkdir -p supermin.{ prepare,build}
# we just import the strict minimum here that rpm-ostree needs
rpms = "rpm-ostree bash rpm-build coreutils selinux-policy-targeted dhcp-client util-linux"
# shellcheck disable=SC2086
supermin --prepare --use-installed -o supermin.prepare $rpms
# the reason we do a heredoc here is so that the var substition takes
# place immediately instead of having to proxy them through to the VM
cat > init <<EOF
#!/bin/bash
set -xeuo pipefail
export PATH = /usr/sbin:$PATH
mount -t proc /proc /proc
mount -t sysfs /sys /sys
mount -t devtmpfs devtmpfs /dev
LANG = C /sbin/load_policy -i
# load kernel module for 9pnet_virtio for 9pfs mount
/sbin/modprobe 9pnet_virtio
# need fuse module for rofiles-fuse/bwrap during post scripts run
/sbin/modprobe fuse
# set up networking
/usr/sbin/dhclient eth0
# set the umask so that anyone in the group can rwx
umask 002
# mount once somewhere predictable to source env vars
mount -t 9p -o rw,trans= virtio,version= 9p2000.L testdir /mnt
source /mnt/tmp/env
umount /mnt
# we only need two dirs
mkdir -p " ${ fixtures } " "\${test_tmpdir}"
mount -t 9p -o ro,trans= virtio,version= 9p2000.L cache " ${ fixtures } "
mount -t 9p -o rw,trans= virtio,version= 9p2000.L testdir "\${test_tmpdir}"
mount /dev/sdb1 "\${test_tmpdir}/cache"
cd "\${test_tmpdir}"
# hack for non-unified mode
rm -rf cache/workdir && mkdir cache/workdir
rc = 0
sh -x tmp/cmd.sh || rc = \$ ?
echo \$ rc > tmp/cmd.sh.rc
if [ -b /dev/sdb1 ] ; then
/sbin/fstrim -v cache
2018-01-08 22:54:10 +03:00
fi
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
/sbin/reboot -f
EOF
chmod a+x init
tar -czf supermin.prepare/init.tar.gz --remove-files init
supermin --build " ${ fixtures } /supermin.prepare " --size 5G -f ext2 -o supermin.build
fi
mkdir cachedir
# we just need a repo so we can download stuff (but see note above about
# sharing pkgcache repo in the future)
ostree init --repo= repo --mode= archive
rpm-ostree compose tree --unified-core --download-only-rpms --repo= repo \
config/manifest.json --cachedir cachedir \
--ex-lockfile config/manifest-lock.x86_64.json \
--ex-lockfile config/manifest-lock.overrides.x86_64.yaml
rm -rf repo
( cd cachedir && createrepo_c .)
echo -e " [cache]\nbaseurl= $( pwd ) /cachedir\ngpgcheck=0 " > config/cache.repo
2016-12-09 01:31:20 +03:00
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
pushd config
python3 -c '
import sys, json
y = json.load( sys.stdin)
y[ "repos" ] = [ "cache" ]
2020-10-14 00:00:32 +03:00
y[ "postprocess" ] = [ ]
2020-04-14 23:44:08 +03:00
y.pop( "lockfile-repos" , None)
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
json.dump( y, sys.stdout) ' < manifest.json > manifest.json.new
mv manifest.json{ .new,}
git add .
git -c user.email= "composetest@localhost.com" -c user.name= "composetest" \
commit -am 'modifications for tests'
2020-04-14 23:44:05 +03:00
popd # config
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
2020-04-14 23:44:05 +03:00
popd # compose-cache
2018-01-08 22:54:10 +03:00
fi
2016-12-03 02:07:08 +03:00
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
echo " Running ${# tests [*] } tests ${ JOBS } at a time "
echo " Test results outputting to ${ outputdir } / "
echo -n " ${ tests [*] } " | parallel -d' ' -j " ${ JOBS } " --line-buffer \
" ${ topsrcdir } /tests/compose/runtest.sh " " ${ outputdir } " " ${ fixtures } "
