2016-12-03 02:07:08 +03:00
#!/bin/bash
set -xeuo pipefail
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
# XXX: nuke this test once we fully drop non-unified core mode
dn = $( cd " $( dirname " $0 " ) " && pwd )
# shellcheck source=libcomposetest.sh
. " ${ dn } /libcomposetest.sh "
# Add a local rpm-md repo so we can mutate local test packages
treefile_append "repos" '["test-repo"]'
# test `recommends: false` (test-misc-tweaks tests the true path)
build_rpm foobar recommends foobar-rec
build_rpm foobar-rec
echo gpgcheck = 0 >> yumrepo.repo
ln " $PWD /yumrepo.repo " config/yumrepo.repo
2021-04-27 23:38:46 +03:00
treefile_pyedit "
tf[ 'repo-packages' ] = [ {
'repo' : 'test-repo' ,
'packages' : [ 'foobar' ] ,
} ]
"
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
treefile_pyedit "tf['add-commit-metadata']['foobar'] = 'bazboo'"
treefile_pyedit "tf['add-commit-metadata']['overrideme'] = 'old var'"
2016-12-03 02:07:08 +03:00
compose: Add --add-metadata-from-json
I'd like to embed structured metadata about the originating git
repository. See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master). However, I think what we really
want here is a *single* value which has subkeys.
One thing in the back of my mind too is...we could use this to
enhance our "change detection". Right now we checksum the sack,
treefile, and treecompose-post. But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.
In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).
Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.
Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.
I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.
Closes: #676
Approved by: jlebon
2017-03-06 19:20:25 +03:00
# Test metadata json with objects, arrays, numbers
cat > metadata.json <<EOF
{
"exampleos.gitrepo" : {
"rev" : "97ec21c614689e533d294cdae464df607b526ab9" ,
"src" : "https://gitlab.com/exampleos/custom-atomic-host"
} ,
2019-07-08 17:34:59 +03:00
"exampleos.tests" : [ "smoketested" , "e2e" ] ,
"overrideme" : "new val"
compose: Add --add-metadata-from-json
I'd like to embed structured metadata about the originating git
repository. See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master). However, I think what we really
want here is a *single* value which has subkeys.
One thing in the back of my mind too is...we could use this to
enhance our "change detection". Right now we checksum the sack,
treefile, and treecompose-post. But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.
In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).
Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.
Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.
I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.
Closes: #676
Approved by: jlebon
2017-03-06 19:20:25 +03:00
}
EOF
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
# drop the --unified-core and add --workdir
mkdir -p cache/workdir
export compose_base_argv = " ${ compose_base_argv /--unified-core/--workdir= $PWD /cache/workdir } "
2019-07-18 18:25:24 +03:00
# Test --parent at the same time (hash is `echo | sha256sum`)
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
rm -rf cache/workdir && mkdir cache/workdir
2019-07-18 18:25:24 +03:00
runcompose --add-metadata-from-json metadata.json \
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
--parent= 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
2016-12-09 01:31:20 +03:00
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
# shellcheck source=libbasic-test.sh
. " ${ dn } /libbasic-test.sh "
2017-11-16 05:28:03 +03:00
basic_test
2017-09-25 19:58:44 +03:00
2021-04-19 10:19:09 +03:00
# Check tmpfiles.d entries created by postprocessing /var.
# Testcases picked at random to cover translation logic. They are unlikely to change upstream,
# but if that happens we will need to adapt the entries.
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
ostree --repo= " ${ repo } " cat " ${ treeref } " /usr/lib/tmpfiles.d/rpm-ostree-1-autovar.conf > autovar.txt
2017-10-27 19:56:05 +03:00
assert_file_has_content_literal autovar.txt 'd /var/cache 0755 root root - -'
2021-04-28 05:10:36 +03:00
assert_file_has_content_literal autovar.txt 'd /var/lib/chrony 0750 chrony chrony - -'
2021-04-19 10:19:09 +03:00
assert_file_has_content_literal autovar.txt 'L /var/mail - - - - spool/mail'
assert_file_has_content_literal autovar.txt 'd /var/tmp 1777 root root - -'
assert_file_has_content_literal autovar.txt 'd /var/lib/polkit-1 0750 root polkitd - -'
2017-09-25 19:58:44 +03:00
echo "ok autovar"
2018-05-31 23:31:22 +03:00
2020-12-02 20:57:34 +03:00
# Validate this exists
ostree --repo= " ${ repo } " ls " ${ treeref } " /usr/lib/systemd/system/multi-user.target.wants/ostree-remount.service
2018-07-31 23:00:55 +03:00
2019-05-08 17:15:48 +03:00
python3 <<EOF
2018-05-31 23:31:22 +03:00
import json, yaml
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
tf = yaml.safe_load( open( " $treefile " ) )
with open( " $treefile .json " , "w" ) as f:
json.dump( tf, f)
2018-05-31 23:31:22 +03:00
EOF
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
export treefile = $treefile .json
rm -rf cache/workdir && mkdir cache/workdir
2018-09-10 15:50:33 +03:00
runcompose
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
echo "ok json"
2019-05-08 07:13:43 +03:00
# also check that --no-parent doesn't invalidate change detection
tests/compose: Target FCOS 31, move off of PAPR
Again, a lot going on here, but essentially, we adapt the compose tests
to run either privileged or fully unprivileged via supermin, just like
cosa.
I actually got more than halfway through this initially using `cosa
build` directly for testing. But in the end, we simply need more
flexibility than that. We want to be able to manipulate exactly how
rpm-ostree is called, and cosa is very opinionated about this (and may
also change from under us in the future).
(Another big difference for example is that cosa doesn't care about
non-unified mode, whereas we *need* to have coverage for this until we
fully kill it.)
Really, the most important bit we want from there is the
unprivileged-via-supermin bits. So we copy and adapt that here. One
obvious improvement then is sharing this code more easily (e.g. a
`cosa runasroot` or something?)
However, we still use the FCOS manifest (frozen at a specific tag). It's
a realistic example, and because of the lockfiles and pool, we get good
reproducibility.
2019-12-22 01:42:09 +03:00
rm -rf cache/workdir && mkdir cache/workdir
2019-05-08 07:13:43 +03:00
runcompose --no-parent | & tee out.txt
assert_file_has_content_literal out.txt "No apparent changes since previous commit"
echo "ok --no-parent"