2016-12-03 02:07:08 +03:00
#!/bin/bash
set -xeuo pipefail
dn = $( cd $( dirname $0 ) && pwd )
. ${ dn } /libcomposetest.sh
compose: Add --add-metadata-from-json
I'd like to embed structured metadata about the originating git
repository. See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master). However, I think what we really
want here is a *single* value which has subkeys.
One thing in the back of my mind too is...we could use this to
enhance our "change detection". Right now we checksum the sack,
treefile, and treecompose-post. But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.
In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).
Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.
Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.
I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.
Closes: #676
Approved by: jlebon
2017-03-06 19:20:25 +03:00
prepare_compose_test "basic"
# Test metadata json with objects, arrays, numbers
cat > metadata.json <<EOF
{
"exampleos.gitrepo" : {
"rev" : "97ec21c614689e533d294cdae464df607b526ab9" ,
"src" : "https://gitlab.com/exampleos/custom-atomic-host"
} ,
"exampleos.tests" : [ "smoketested" , "e2e" ]
}
EOF
runcompose --add-metadata-from-json metadata.json
ostree --repo= ${ repobuild } ls -R ${ treeref } /usr/lib/ostree-boot > bootls.txt
2017-03-17 23:45:35 +03:00
if ostree --repo= ${ repobuild } ls -R ${ treeref } /usr/etc/passwd-; then
assert_not_reached "Found /usr/etc/passwd- backup file in tree"
fi
2016-12-09 01:31:20 +03:00
echo "ok compose"
compose: Add --add-metadata-from-json
I'd like to embed structured metadata about the originating git
repository. See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master). However, I think what we really
want here is a *single* value which has subkeys.
One thing in the back of my mind too is...we could use this to
enhance our "change detection". Right now we checksum the sack,
treefile, and treecompose-post. But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.
In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).
Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.
Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.
I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.
Closes: #676
Approved by: jlebon
2017-03-06 19:20:25 +03:00
ostree --repo= ${ repobuild } show --print-metadata-key exampleos.gitrepo ${ treeref } > meta.txt
assert_file_has_content meta.txt 'rev.*97ec21c614689e533d294cdae464df607b526ab9'
assert_file_has_content meta.txt 'src.*https://gitlab.com/exampleos/custom-atomic-host'
ostree --repo= ${ repobuild } show --print-metadata-key exampleos.tests ${ treeref } > meta.txt
assert_file_has_content meta.txt 'smoketested.*e2e'
compose: Add rpmostree.rpmmd-repo metadata to commits by default
This is a revisit of a PR for client-side layering: https://github.com/projectatomic/rpm-ostree/pull/1072
Here though we're doing this by default for server-side composes.
There are a few reasons to do this; first, I'm seeing an issue
in some of our Jenkins jobs for Fedora that hit "mirror roulette"
and end up creating commits that "revert" to older versions temporarily.
While I've [certainly pitched](https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/IMPE6KCRBHCEJH5VBE6ZFIRLPAD743JT/) this as a feature, I think
we really want something like `--force-older-timestamp` - basically
error out if the timestamps on one or more input repos were older.
Not doing that in this patch, but it paves the way to do so.
Second, I'd like to use this data in the `ostree.source-title`
metadata key down the line. Something like:
`└ rpmmd: fedora-26 (20170310), fedora-26-updates (20171101)`
(This could be a lot nicer if we drive versioning in to the rpm-md repo info,
and e.g. there's some friendly "week number" style versioning for the updates
repo now that it's batched...for now we have timestamps)
For CentOS/RHELAH this gets interesting and potentially more verbose,
to the point where we may want to render it more explicitly.
But anyways, let's do this now, as it will be useful even without
an explicit rendering, since users can do e.g. `ostree show` on
a base commit hash to dump the data.
I had a concern that some users may not want to emit this metadata;
they can currently do `--add-metadata-string rpmostree.rpmmd-repos ''`
and that will "win".
Closes: #1079
Approved by: jlebon
2017-11-02 00:12:08 +03:00
ostree --repo= ${ repobuild } show --print-metadata-key rpmostree.rpmmd-repos ${ treeref } > meta.txt
assert_file_has_content meta.txt 'id.*fedora.*timestamp'
compose: Add --add-metadata-from-json
I'd like to embed structured metadata about the originating git
repository. See [this example](https://pagure.io/fedora-atomic-host-continuous/c/142b12020d7efe18b56d039304efea102a210790?branch=master). However, I think what we really
want here is a *single* value which has subkeys.
One thing in the back of my mind too is...we could use this to
enhance our "change detection". Right now we checksum the sack,
treefile, and treecompose-post. But down the line, I'd
like to support more sophisticated postprocessing, where the
script might reference external files or the like.
In that case, we could stop checksumming the post script, and rely on whether or
not the git repo changed. (This would conversely mean we would do a build even
if e.g. the repo's `README.md` changed, but we can address that with a
post-assemble content check).
Anyways though, for now, this gets us the ability to more easily drop more
structured metadata in the commit, whether it's input git repos, tests that
passed, etc.
Note a trap that bit me here: since the metadata we write here is *host* endian,
but `ostree show --raw` byteswaps (it needs to since the core ostree variant
is always big endian), we get inverted numbers if the host is little.
I think we should probably canonicalize our metadata to big endian; this should
be pretty backwards compatible since I doubt anyone has been adding raw numbers
so far.
Closes: #676
Approved by: jlebon
2017-03-06 19:20:25 +03:00
echo "ok metadata"
2017-08-26 04:06:09 +03:00
for path in /boot /usr/lib/ostree-boot; do
ostree --repo= ${ repobuild } ls -R ${ treeref } ${ path } > bootls.txt
assert_file_has_content bootls.txt vmlinuz-
assert_file_has_content bootls.txt initramfs-
echo "ok boot files"
done
kver = $( grep /vmlinuz bootls.txt | sed -e 's,.*/vmlinuz-\(.*\)-[0-9a-e].*$,\1,' )
ostree --repo= ${ repobuild } ls ${ treeref } /usr/lib/modules/${ kver } /{ vmlinuz,initramfs.img} >/dev/null
2016-12-03 02:07:08 +03:00
ostree --repo= ${ repobuild } ls -R ${ treeref } /usr/share/man > manpages.txt
assert_file_has_content manpages.txt man5/ostree.repo.5
2016-12-09 01:31:20 +03:00
echo "ok manpages"
compose: Add `tmp-is-dir` option to make `/tmp` a directory
There are a few reasons to do this. First, systemd changed to refuse mounts on
symlinks, and hence if one *wants* "/tmp-on-tmpfs", one would need to write a
different `sysroot-tmp.mount` unit.
Second, the original rationale for having this symlink was that if you had
multiple ostree stateroots ("osnames"), it's nicer if they had the same `/tmp`
to avoid duplication. But in practice today that's already an issue due to
`/var/tmp`, and further the multiple-stateroot case is pretty unusual. And that
case is *further* broken by SELinux (if one wanted to have e.g. an Ubuntu and
Fedora) stateroots. So let's fully decouple this and make `/tmp` a plain
old directory by default, so systemd's `tmp.mount` can become useful.
Now, things get interesting for the case where someone wants a physical `/tmp`
that *does* persist across reboots. Right now, if one just did a `systemctl mask
tmp.mount` as we do in Fedora Atomic Host's cloud images, you'd get a semantic
where `/tmp` stays per-deployment, which is weird. Our recommendation for
that should likely be to set up a bind mount for `/tmp` → `/var/tmp`.
For now, this stays an option to ensure compatibility; if FAH Cloud images
want to stay with "physical /tmp", then we'd have to change the kickstart.
Closes: https://github.com/projectatomic/rpm-ostree/issues/669
Closes: #778
Approved by: jlebon
2017-05-17 21:48:48 +03:00
# https://github.com/projectatomic/rpm-ostree/issues/669
ostree --repo= ${ repobuild } ls ${ treeref } /tmp > ls.txt
assert_file_has_content ls.txt 'l00777 0 0 0 /tmp -> sysroot/tmp'
echo "ok /tmp"
2017-09-19 22:20:09 +03:00
ostree --repo= ${ repobuild } ls ${ treeref } /usr/share/rpm > ls.txt
assert_not_file_has_content ls.txt '__db' 'lock'
ostree --repo= ${ repobuild } ls -R ${ treeref } /usr/etc/selinux > ls.txt
assert_not_file_has_content ls.txt 'LOCK'
echo "ok no leftover files"
2017-09-25 19:58:44 +03:00
ostree --repo= ${ repobuild } cat ${ treeref } /usr/lib/tmpfiles.d/rpm-ostree-1-autovar.conf > autovar.txt
# Picked this one at random as an example of something that won't likely be
# converted to tmpfiles.d upstream. But if it is, we can change this test.
2017-10-27 19:56:05 +03:00
assert_file_has_content_literal autovar.txt 'd /var/cache 0755 root root - -'
# And this one has a non-root uid
assert_file_has_content_literal autovar.txt 'd /var/log/chrony 0755 chrony chrony - -'
2017-09-25 19:58:44 +03:00
echo "ok autovar"
