2014-07-16 17:02:45 +04:00
# Source library for shell script tests
#
# Copyright (C) 2011 Colin Walters <walters@verbum.org>
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the
# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
# Boston, MA 02111-1307, USA.
2016-02-26 20:52:42 +03:00
# Have we already been sourced?
if test -n " ${ LIBTEST_SH :- } " ; then
# would be good to know when it happens
echo "INFO: Skipping subsequent sourcing of libtest.sh"
return
fi
LIBTEST_SH = 1
2016-06-06 19:08:29 +03:00
self = " $( realpath $0 ) "
2020-05-06 18:29:08 +03:00
if test -z " ${ SRCDIR :- } " && test -n " ${ topsrcdir :- } " ; then
2017-05-19 20:24:42 +03:00
SRCDIR = ${ topsrcdir } /tests
2020-05-06 18:29:08 +03:00
commondir = ${ SRCDIR } /common
2017-05-19 20:24:42 +03:00
fi
2020-05-06 18:29:08 +03:00
commondir = ${ commondir :- ${ KOLA_EXT_DATA } }
. ${ commondir } /libtest-core.sh
2018-02-14 17:27:06 +03:00
2017-01-17 17:12:26 +03:00
for bin in jq; do
if ! command -v $bin >/dev/null; then
( echo ${ bin } is required to execute tests 1>& 2; exit 1)
fi
done
2016-02-10 13:42:57 +03:00
_cleanup_tmpdir ( ) {
2016-12-03 02:07:08 +03:00
if test -z " ${ TEST_SKIP_CLEANUP :- } " ; then
if test -f ${ test_tmpdir } /.test; then
rm ${ test_tmpdir } -rf
fi
else
echo " Skipping cleanup of ${ test_tmpdir } "
2016-02-10 13:42:57 +03:00
fi
}
2016-02-26 19:12:11 +03:00
2016-06-21 19:37:02 +03:00
# Create a tmpdir if we're running as a local test (i.e. through `make check`)
# or as a `vmcheck` test, which also needs some scratch space on the host.
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
if { test -n " ${ UNINSTALLEDTESTS :- } " || \
test -n " ${ VMTESTS :- } " || \
test -n " ${ COMPOSETESTS :- } " ; } && ! test -f " $PWD /.test " ; then
Rework vmcheck to use `kola spawn`, move off of PAPR
There's a lot going on here, but essentially:
1. We change the `vmcheck` model so that it always operates on an
immutable base image. It takes that image and dynamically launches a
separate VM for each test using `kola spawn`. This means we can drop
a lot of hacks around re-using the same VMs.
2. Following from 1., `vmoverlay` now takes as input a base image,
overlays the built rpm-ostree bits, then creates a new base image. Of
course, we don't have to do this in CI, because we build FCOS with
the freshly built RPMs (so it uses `SKIP_VMOVERLAY=1`). `vmoverlay`
then will be more for the developer case where one doesn't want to
iterate via `cosa build` to test rpm-ostree changes. I say "will"
because the functionality doesn't exist yet; I'd like to enhance
`cosa dev-overlay` to do this. (Note `vmsync` should still works just
as before too.)
3. `vmcheck` can be run without building the tree first, as
`tests/vmcheck.sh`. The `make vmcheck` target still exists though for
finger compatibility and better meshing with `vmoverlay` in the
developer case.
What's really nice about using kola spawn is that it takes care of a lot
of things for us, such as the qemu command, journal and console
gathering, and SSH.
Similarly to the compose testsuites, we're using parallel here to run
multiple vmcheck tests at once. (On developer laptops, we cap
parallelism at `$(nproc) - 1`).
2019-12-13 20:23:41 +03:00
# Use --tmpdir to keep it in /tmp. This also keeps paths short; this is
# important if we want to create UNIX sockets under there.
test_tmpdir = $( mktemp -d test.XXXXXX --tmpdir)
2016-02-10 13:42:57 +03:00
touch ${ test_tmpdir } /.test
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
trap _cleanup_tmpdir EXIT SIGINT
2016-02-10 13:42:57 +03:00
cd ${ test_tmpdir }
2016-12-03 02:07:08 +03:00
fi
if test -n " ${ UNINSTALLEDTESTS :- } " ; then
export PATH = ${ builddir } :${ PATH }
2016-02-26 19:12:11 +03:00
fi
2014-07-16 17:02:45 +04:00
test_tmpdir = $( pwd )
2017-01-11 23:24:30 +03:00
echo " Using tmpdir ${ test_tmpdir } "
2014-07-16 17:02:45 +04:00
export G_DEBUG = fatal-warnings
2016-02-19 20:34:36 +03:00
# Don't flag deployments as immutable so that test harnesses can
# easily clean up.
export OSTREE_SYSROOT_DEBUG = mutable-deployments
2017-03-25 19:01:03 +03:00
# See comment in ot-builtin-commit.c and https://github.com/ostreedev/ostree/issues/758
# Also keep this in sync with the bits in libostreetest.c
case $( stat -f --printf '%T' /) in
overlayfs)
echo "overlayfs found; enabling OSTREE_NO_XATTRS"
ostree --version
export OSTREE_SYSROOT_DEBUG = " ${ OSTREE_SYSROOT_DEBUG } ,no-xattrs "
export OSTREE_NO_XATTRS = 1 ; ;
*) echo "Not using overlayfs" ; ;
esac
2016-02-10 13:42:57 +03:00
if test -n " ${ OT_TESTS_DEBUG :- } " ; then
2014-07-16 17:02:45 +04:00
set -x
fi
2015-04-01 14:25:36 +03:00
check_root_test ( )
{
if test " $( id -u) " != "0" ; then
2016-10-28 23:43:16 +03:00
skip "This test requires uid 0"
fi
if ! capsh --print | grep -q 'Bounding set.*[^a-z]cap_sys_admin' ; then
skip "No CAP_SYS_ADMIN in bounding set"
2016-06-06 19:08:29 +03:00
fi
}
ensure_dbus ( )
{
if test -z " $RPMOSTREE_USE_SESSION_BUS " ; then
2016-06-06 19:46:48 +03:00
exec " $topsrcdir /tests/utils/setup-session.sh " " $self "
2015-04-01 14:25:36 +03:00
fi
}
2017-01-27 07:31:53 +03:00
2017-09-21 23:59:28 +03:00
# https://github.com/ostreedev/ostree/commit/47b4dd1b38e422254afa67756873957c25aeab6d
# Unfortunately, introspection uses dlopen(), which doesn't quite
# work when the DSO is compiled with ASAN but the outer executable
# isn't.
skip_one_with_asan ( ) {
if test -n " ${ BUILDOPT_ASAN :- } " ; then
echo "ok # SKIP - built with ASAN"
return 0
else
return 1
fi
}
2017-12-21 01:46:04 +03:00
get_obj_path( ) {
repo = $1 ; shift
csum = $1 ; shift
objtype = $1 ; shift
echo " ${ repo } /objects/ ${ csum : 0 : 2 } / ${ csum : 2 } . ${ objtype } "
}
2018-02-14 17:27:06 +03:00
uinfo_cmd( ) {
2020-05-06 18:29:08 +03:00
${ SRCDIR } /utils/updateinfo --repo " ${ test_tmpdir } /yumrepo " " $@ "
2018-02-14 17:27:06 +03:00
}
2017-06-29 17:06:36 +03:00
# builds a new RPM and adds it to the testdir's repo
# $1 - name
# $2+ - optional, treated as directive/value pairs
build_rpm( ) {
local name = $1 ; shift
2018-01-11 22:02:44 +03:00
# Unset, not zero https://github.com/projectatomic/rpm-ostree/issues/349
local epoch = ""
2017-06-29 17:06:36 +03:00
local version = 1.0
local release = 1
local arch = x86_64
mkdir -p $test_tmpdir /yumrepo/{ specs,packages}
local spec = $test_tmpdir /yumrepo/specs/$name .spec
# write out the header
cat > $spec << EOF
Name: $name
Summary: %{ name}
License: GPLv2+
EOF
2017-07-13 03:50:08 +03:00
local build = install = files = pretrans = pre = post = posttrans = post_args =
2018-02-14 17:27:06 +03:00
local verifyscript = uinfo =
2017-07-07 17:45:10 +03:00
local transfiletriggerin = transfiletriggerin_patterns =
local transfiletriggerin2 = transfiletriggerin2_patterns =
local transfiletriggerun = transfiletriggerun_patterns =
2017-06-29 17:06:36 +03:00
while [ $# -ne 0 ] ; do
local section = $1 ; shift
local arg = $1 ; shift
case $section in
requires)
echo " Requires: $arg " >> $spec ; ;
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
recommends)
echo " Recommends: $arg " >> $spec ; ;
2017-06-29 17:06:36 +03:00
provides)
echo " Provides: $arg " >> $spec ; ;
conflicts)
echo " Conflicts: $arg " >> $spec ; ;
2017-07-13 03:50:08 +03:00
post_args)
post_args = " $arg " ; ;
2018-02-14 17:27:06 +03:00
version| release| epoch| arch| build| install| files| pretrans| pre| post| posttrans| verifyscript| uinfo)
2017-06-29 17:06:36 +03:00
declare $section = " $arg " ; ;
2017-07-07 17:45:10 +03:00
transfiletriggerin)
transfiletriggerin_patterns = " $arg " ;
declare $section = " $1 " ; shift; ;
transfiletriggerin2)
transfiletriggerin2_patterns = " $arg " ;
declare $section = " $1 " ; shift; ;
transfiletriggerun)
transfiletriggerun_patterns = " $arg " ;
declare $section = " $1 " ; shift; ;
2017-06-29 17:06:36 +03:00
*)
assert_not_reached " unhandled section $section " ; ;
esac
done
cat >> $spec << EOF
Version: $version
Release: $release
2018-01-11 22:02:44 +03:00
${ epoch : +Epoch : $epoch }
2017-06-29 17:06:36 +03:00
BuildArch: $arch
%description
%{ summary}
# by default, we create a /usr/bin/$name script which just outputs $name
%build
2017-07-04 20:04:07 +03:00
echo -e " #!/bin/sh\necho $name - $version - $release . $arch " > $name
2017-06-29 17:06:36 +03:00
chmod a+x $name
$build
${ pretrans : +%pretrans }
$pretrans
${ pre : +%pre }
$pre
2017-07-13 03:50:08 +03:00
${ post : +%post } ${ post_args }
2017-06-29 17:06:36 +03:00
$post
${ posttrans : +%posttrans }
$posttrans
2017-07-07 17:45:10 +03:00
${ transfiletriggerin : +%transfiletriggerin -- ${ transfiletriggerin_patterns } }
$transfiletriggerin
${ transfiletriggerin2 : +%transfiletriggerin -- ${ transfiletriggerin2_patterns } }
$transfiletriggerin2
${ transfiletriggerun : +%transfiletriggerun -- ${ transfiletriggerun_patterns } }
$transfiletriggerun
2018-01-19 17:24:09 +03:00
${ verifyscript : +%verifyscript }
$verifyscript
2017-06-29 17:06:36 +03:00
%install
mkdir -p %{ buildroot} /usr/bin
install $name %{ buildroot} /usr/bin
$install
%clean
rm -rf %{ buildroot}
%files
/usr/bin/$name
$files
EOF
2018-02-05 19:52:02 +03:00
# because it'd be overkill to set up mock for this, let's just fool
# rpmbuild using setarch
local buildarch = $arch
2018-02-14 17:27:06 +03:00
if [ " $arch " = = "noarch" ] ; then
2018-02-05 19:52:02 +03:00
buildarch = $( uname -m)
fi
2017-06-29 17:06:36 +03:00
( cd $test_tmpdir /yumrepo/specs &&
2018-02-05 19:52:02 +03:00
setarch $buildarch rpmbuild --target $arch -ba $name .spec \
2017-12-15 20:01:46 +03:00
--define " _topdir $PWD " \
2017-06-29 17:06:36 +03:00
--define " _sourcedir $PWD " \
--define " _specdir $PWD " \
--define " _builddir $PWD /.build " \
--define " _srcrpmdir $PWD " \
--define " _rpmdir $test_tmpdir /yumrepo/packages " \
--define " _buildrootdir $PWD " )
2018-02-14 17:27:06 +03:00
# use --keep-all-metadata to retain previous updateinfo
( cd $test_tmpdir /yumrepo &&
createrepo_c --no-database --update --keep-all-metadata .)
# convenience function to avoid follow-up add-pkg
if [ -n " $uinfo " ] ; then
uinfo_cmd add-pkg $uinfo $name 0 $version $release $arch
fi
2017-12-15 20:01:46 +03:00
if test '!' -f $test_tmpdir /yumrepo.repo; then
cat > $test_tmpdir /yumrepo.repo.tmp << EOF
2017-06-29 17:06:36 +03:00
[ test-repo]
name = test-repo
baseurl = file:///$PWD /yumrepo
EOF
2017-12-15 20:01:46 +03:00
mv $test_tmpdir /yumrepo.repo{ .tmp,}
2017-06-29 17:06:36 +03:00
fi
}
2017-09-18 22:11:26 +03:00
# build an SELinux package ready to be installed -- really, we just support file
# context entries for now, though it's enough to test policy changes
# $1 - package name
# $2+ - pairs of file path regex and context types
build_selinux_rpm( ) {
local name = $1 ; shift
local module_dir = $test_tmpdir /policies/$name
mkdir -p $module_dir
local module_te = $module_dir /$name .te
local module_fc = $module_dir /$name .fc
# also declare a type associated with the app; any non-trivial SELinux
# package will have some type enforcement rules that will require policy
# recompilation
cat > $module_te <<EOF
policy_module( ${ name } , 1.0.0)
type ${ name } _t;
EOF
echo -n "" > $module_fc
while [ $# -ne 0 ] ; do
local fc_regex = $1 ; shift
local fc_type = $1 ; shift
local fc_label = " gen_context(system_u:object_r: $fc_type ,s0) "
echo " $fc_regex -- $fc_label " >> $module_fc
done
make -C $module_dir -f /usr/share/selinux/devel/Makefile $name .pp
# We point the spec file directly at our pp. This is a bit underhanded, but
# it's cleaner than copying it in and using e.g. Source0 or something.
local pp = $( realpath $module_dir /$name .pp)
local install_dir = /usr/share/selinux/packages
build_rpm $name install " mkdir -p %{buildroot} ${ install_dir }
install ${ pp } %{ buildroot} ${ install_dir } " \
post " semodule -n -i ${ install_dir } / ${ name } .pp " \
files " ${ install_dir } / ${ name } .pp "
}
2018-04-09 21:10:33 +03:00
files_are_hardlinked( ) {
inode1 = $( stat -c %i $1 )
inode2 = $( stat -c %i $2 )
test -n " ${ inode1 } " && test -n " ${ inode2 } "
[ " ${ inode1 } " = = " ${ inode2 } " ]
}
assert_files_hardlinked( ) {
if ! files_are_hardlinked " $1 " " $2 " ; then
fatal " Files ' $1 ' and ' $2 ' are not hardlinked "
fi
}
2019-02-26 19:04:48 +03:00
# $1 - json file
# $2+ - assertions
assert_jq( ) {
f = $1 ; shift
for expression in " $@ " ; do
if ! jq -e " ${ expression } " >/dev/null < $f ; then
jq . < $f | sed -e 's/^/# /' >& 2
echo 1>& 2 " ${ expression } failed to match $f "
exit 1
fi
done
}
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
# This function below was taken and adapted from coreos-assembler. We
# should look into sharing this code more easily.
# Determine if current user has enough privileges for composes
_privileged =
has_compose_privileges( ) {
if [ -z " ${ _privileged :- } " ] ; then
if [ -n " ${ FORCE_UNPRIVILEGED :- } " ] ; then
echo "Detected FORCE_UNPRIVILEGED; using virt"
_privileged = 0
elif ! capsh --print | grep -q 'Bounding.*cap_sys_admin' ; then
echo "Missing CAP_SYS_ADMIN; using virt"
_privileged = 0
elif [ " $( id -u) " != "0" ] ; then
echo "Not running as root; using virt"
_privileged = 0
else
_privileged = 1
fi
fi
[ ${ _privileged } = = 1 ]
}