keremet/rpm-ostree
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7c99809a71
rpm-ostree/ci/build.sh

43 lines
1.6 KiB
Bash
Raw Normal View History

ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
#!/usr/bin/bash
# Install build dependencies, run unit tests and installed tests.
set -xeuo pipefail
dn=$(dirname $0)
. ${dn}/libbuild.sh
ci: Use FAHC for build container Conceptually: we're going to move rpm-ostree and ostree at the same cadence most of the time; for both releases *and* for git master. The problem so far has been the latter part. Reusing FAHC for the build gets us half of the problem. The other trick I realized we can do - just pull ostree out from the build container. This avoids fetching it from the internet, and makes my workflow for hacking on both nicer - I just `sudo make install` in my build container for ostree. It's tempting to make the whole thing symmetric and require `sudo make install` for rpm-ostree and not do the insttree thing but...perhaps after. Closes: #758 Approved by: jlebon
2017-04-28 18:40:22 +03:00
# Use the latest ostree by default
ci: unite testsuites and run vmcheck on centos We can be a bit less wasteful here by merging the check and vmcheck suites into a single suite. The check suite today takes a negligible amount of time to run, so we're not gaining much by parallelizing them. It's more of a sanity check at this point before we start vmcheck. Also start running vmcheck on CentOS 7. We adapt the ci scripts to accomodate both Fedora and CentOS target machines. This commit also switches to Fedora 26 as the primary test base. Closes: #871 Approved by: cgwalters
2017-07-17 20:40:25 +03:00
id=$(. /etc/os-release && echo $ID)
version_id=$(. /etc/os-release && echo $VERSION_ID)
ci/build.sh: s/26/27/ for FAHC check Missed this in https://github.com/projectatomic/rpm-ostree/pull/1125 Closes: #1123 Approved by: jlebon
2017-12-04 22:42:55 +03:00
if [ "$id" == fedora ] && [ "$version_id" == 27 ]; then
ci: Reduce metadata expiration for rdgo At some point I'll fix rdgo to retain multiple, but I think this may help CI reliability in the short term. Closes: #1015 Approved by: jlebon
2017-09-26 18:41:29 +03:00
echo -e '[fahc]\nmetadata_expire=1m\nbaseurl=https://ci.centos.org/artifacts/sig-atomic/fahc/rdgo/build/\ngpgcheck=0\n' > /etc/yum.repos.d/fahc.repo
ci: unite testsuites and run vmcheck on centos We can be a bit less wasteful here by merging the check and vmcheck suites into a single suite. The check suite today takes a negligible amount of time to run, so we're not gaining much by parallelizing them. It's more of a sanity check at this point before we start vmcheck. Also start running vmcheck on CentOS 7. We adapt the ci scripts to accomodate both Fedora and CentOS target machines. This commit also switches to Fedora 26 as the primary test base. Closes: #871 Approved by: cgwalters
2017-07-17 20:40:25 +03:00
# Until we fix https://github.com/rpm-software-management/libdnf/pull/149
sed -i -e 's,metadata_expire=6h,exclude=ostree ostree-devel ostree-libs ostree-grub2\nmetadata_expire=6h,' /etc/yum.repos.d/fedora-updates.repo
elif [ "$id" == centos ]; then
ci: Reduce metadata expiration for rdgo At some point I'll fix rdgo to retain multiple, but I think this may help CI reliability in the short term. Closes: #1015 Approved by: jlebon
2017-09-26 18:41:29 +03:00
echo -e '[cahc]\nmetdata_expire=1m\nbaseurl=https://ci.centos.org/artifacts/sig-atomic/rdgo/centos-continuous/build\ngpgcheck=0\n' > /etc/yum.repos.d/cahc.repo
ci: unite testsuites and run vmcheck on centos We can be a bit less wasteful here by merging the check and vmcheck suites into a single suite. The check suite today takes a negligible amount of time to run, so we're not gaining much by parallelizing them. It's more of a sanity check at this point before we start vmcheck. Also start running vmcheck on CentOS 7. We adapt the ci scripts to accomodate both Fedora and CentOS target machines. This commit also switches to Fedora 26 as the primary test base. Closes: #871 Approved by: cgwalters
2017-07-17 20:40:25 +03:00
fi
ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
ci: workaround broken fedora:26 image This is essentially the same workaround as https://github.com/ostreedev/ostree/pull/1143. See https://bugzilla.redhat.com/show_bug.cgi?id=1483553. Closes: #975 Approved by: cgwalters
2017-09-06 20:53:46 +03:00
pkg_upgrade
ci: Sync with ostree We have some drift; keep this more in sync so we maintain the rpm/yum hackarounds for example in one place. I backed out the ASAN bits though pending at least a newer gpgme: <https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758> Closes: #1000 Approved by: jlebon
2017-09-19 17:40:35 +03:00
pkg_install_builddeps rpm-ostree
lib: Expose new API around basearch For https://pagure.io/atomic-wg/issue/299 we need to make it more convenient to substitute the architecture in an installation context. I plan to use this API inside `rpmostreepayload` in Anaconda, so we can substitute the same value of `${basearch}` we use in treefiles since https://github.com/projectatomic/rpm-ostree/pull/305 Now, you might wonder - why do we need an API wrapping libdnf? It's because libdnf is not API stable yet. We're just exposing a tiny subset. In theory we could use the Python dnf bindings in Anaconda, but things get slightly weird if rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for now this a small API surface to maintain (forever). This change reworks the internal `varsubst` bits to take a pure `DnfContext`, since we don't want to spin up a whole `RpmOstreeContext` just to do some string substitutions. Closes: #877 Approved by: jlebon
2017-07-13 23:39:01 +03:00
# Mostly dependencies for tests
ci: run clang build with -Werror Start running `clang` with `-Werror` like ostree. We can only run this on Fedora right now because the CentOS `clang` doesn't support `-Wno-error=macro-redefined`, though that's fine. Closes: #1036 Approved by: cgwalters
2017-10-04 22:48:26 +03:00
pkg_install ostree{,-devel,-grub2} createrepo_c /usr/bin/jq PyYAML \
vmcheck: add SELinux labeling tests There's a lot of paths in the core related to SELinux policy changes and relabeling packages. We currently have no test coverage for them. We add support in the test libraries here to build such packages. We also add a test that checks both that we correctly relabel RPMs when the policy changes and that we handle layered packages that install SELinux packages properly. Closes: #999 Approved by: cgwalters
2017-09-18 22:11:26 +03:00
libubsan libasan libtsan elfutils fuse sudo python-gobject-base \
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
selinux-policy-devel selinux-policy-targeted
ci: run clang build with -Werror Start running `clang` with `-Werror` like ostree. We can only run this on Fedora right now because the CentOS `clang` doesn't support `-Wno-error=macro-redefined`, though that's fine. Closes: #1036 Approved by: cgwalters
2017-10-04 22:48:26 +03:00
# For ex-container tests and clang build
pkg_install_if_os fedora parallel clang
ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
ci: workaround broken fedora:26 image This is essentially the same workaround as https://github.com/ostreedev/ostree/pull/1143. See https://bugzilla.redhat.com/show_bug.cgi?id=1483553. Closes: #975 Approved by: cgwalters
2017-09-06 20:53:46 +03:00
if [ -n "${CI_PKGS:-}" ]; then
pkg_install ${CI_PKGS}
fi
ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
# create an unprivileged user for testing
adduser testuser
ci: Sync with ostree We have some drift; keep this more in sync so we maintain the rpm/yum hackarounds for example in one place. I backed out the ASAN bits though pending at least a newer gpgme: <https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758> Closes: #1000 Approved by: jlebon
2017-09-19 17:40:35 +03:00
export LSAN_OPTIONS=verbosity=1:log_threads=1
ci: don't use CentOS Alpha anymore It's no longer being built and is now older than the latest CentOS AH release. This should help us no longer see messages like: (rpm-ostree pkg-add:5662): GLib-CRITICAL **: g_variant_dict_lookup: assertion 'is_valid_dict (dict)' failed which happen because in #1034, we started using `G_VARIANT_DICT_INIT`, whose special magic values only make sense in glib2 >= 2.50. (The alpha image stopped at 2.46). Saw this while debugging #1035. Closes: #1040 Approved by: cgwalters
2017-10-06 00:14:11 +03:00
BWRAP=/usr/bin/bwrap
scripts: Drop most capabilities Note this PR requires [bubblewrap 0.2.0](https://github.com/projectatomic/bubblewrap/releases/tag/v0.2.0). Change our bwrap invocations drop truly dangerous capabilities like `cap_sys_admin` and `cap_sys_module` just like Docker does today. Because of the popularity of Docker, we can be pretty sure that most RPM scripts should have adapted to this (although a problematic area here is that traditional librpm doesn't actually error out if scripts fail). There are two reasons to do this: - We want "offline" updates by default; updates shouldn't affect the running system. If we prepare the new root in the background, a %post shouldn't restart a service for example. We already "handle" this by making `systemctl` a symlink to `/bin/true`, but this approach also shuts off `%post`s that do e.g. `insmod`. - Protection against accidental system damage Closes: #1099 Approved by: jlebon
2017-11-09 22:24:15 +03:00
# we use smoketested now, which uses a git master-ish version of bwrap
#if [ "$id" == centos ]; then
# BWRAP=/usr/lib64/rpm-ostree/bwrap
#fi
ci: don't use CentOS Alpha anymore It's no longer being built and is now older than the latest CentOS AH release. This should help us no longer see messages like: (rpm-ostree pkg-add:5662): GLib-CRITICAL **: g_variant_dict_lookup: assertion 'is_valid_dict (dict)' failed which happen because in #1034, we started using `G_VARIANT_DICT_INIT`, whose special magic values only make sense in glib2 >= 2.50. (The alpha image stopped at 2.46). Saw this while debugging #1035. Closes: #1040 Approved by: cgwalters
2017-10-06 00:14:11 +03:00
build --enable-installed-tests --enable-gtk-doc --with-bubblewrap=$BWRAP
Reference in New Issue Copy Permalink