keremet/rpm-ostree
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
3e9c6cf230
rpm-ostree/ci/build.sh

39 lines
1.5 KiB
Bash
Raw Normal View History

ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
#!/usr/bin/bash
# Install build dependencies, run unit tests and installed tests.
set -xeuo pipefail
dn=$(dirname $0)
. ${dn}/libbuild.sh
ci: Use FAHC for build container Conceptually: we're going to move rpm-ostree and ostree at the same cadence most of the time; for both releases *and* for git master. The problem so far has been the latter part. Reusing FAHC for the build gets us half of the problem. The other trick I realized we can do - just pull ostree out from the build container. This avoids fetching it from the internet, and makes my workflow for hacking on both nicer - I just `sudo make install` in my build container for ostree. It's tempting to make the whole thing symmetric and require `sudo make install` for rpm-ostree and not do the insttree thing but...perhaps after. Closes: #758 Approved by: jlebon
2017-04-28 18:40:22 +03:00
# Use the latest ostree by default
ci: unite testsuites and run vmcheck on centos We can be a bit less wasteful here by merging the check and vmcheck suites into a single suite. The check suite today takes a negligible amount of time to run, so we're not gaining much by parallelizing them. It's more of a sanity check at this point before we start vmcheck. Also start running vmcheck on CentOS 7. We adapt the ci scripts to accomodate both Fedora and CentOS target machines. This commit also switches to Fedora 26 as the primary test base. Closes: #871 Approved by: cgwalters
2017-07-17 20:40:25 +03:00
id=$(. /etc/os-release && echo $ID)
version_id=$(. /etc/os-release && echo $VERSION_ID)
ci/build.sh: s/26/27/ for FAHC check Missed this in https://github.com/projectatomic/rpm-ostree/pull/1125 Closes: #1123 Approved by: jlebon
2017-12-04 22:42:55 +03:00
if [ "$id" == fedora ] && [ "$version_id" == 27 ]; then
ci: Reduce metadata expiration for rdgo At some point I'll fix rdgo to retain multiple, but I think this may help CI reliability in the short term. Closes: #1015 Approved by: jlebon
2017-09-26 18:41:29 +03:00
echo -e '[fahc]\nmetadata_expire=1m\nbaseurl=https://ci.centos.org/artifacts/sig-atomic/fahc/rdgo/build/\ngpgcheck=0\n' > /etc/yum.repos.d/fahc.repo
ci: unite testsuites and run vmcheck on centos We can be a bit less wasteful here by merging the check and vmcheck suites into a single suite. The check suite today takes a negligible amount of time to run, so we're not gaining much by parallelizing them. It's more of a sanity check at this point before we start vmcheck. Also start running vmcheck on CentOS 7. We adapt the ci scripts to accomodate both Fedora and CentOS target machines. This commit also switches to Fedora 26 as the primary test base. Closes: #871 Approved by: cgwalters
2017-07-17 20:40:25 +03:00
# Until we fix https://github.com/rpm-software-management/libdnf/pull/149
sed -i -e 's,metadata_expire=6h,exclude=ostree ostree-devel ostree-libs ostree-grub2\nmetadata_expire=6h,' /etc/yum.repos.d/fedora-updates.repo
elif [ "$id" == centos ]; then
ci: Reduce metadata expiration for rdgo At some point I'll fix rdgo to retain multiple, but I think this may help CI reliability in the short term. Closes: #1015 Approved by: jlebon
2017-09-26 18:41:29 +03:00
echo -e '[cahc]\nmetdata_expire=1m\nbaseurl=https://ci.centos.org/artifacts/sig-atomic/rdgo/centos-continuous/build\ngpgcheck=0\n' > /etc/yum.repos.d/cahc.repo
ci: unite testsuites and run vmcheck on centos We can be a bit less wasteful here by merging the check and vmcheck suites into a single suite. The check suite today takes a negligible amount of time to run, so we're not gaining much by parallelizing them. It's more of a sanity check at this point before we start vmcheck. Also start running vmcheck on CentOS 7. We adapt the ci scripts to accomodate both Fedora and CentOS target machines. This commit also switches to Fedora 26 as the primary test base. Closes: #871 Approved by: cgwalters
2017-07-17 20:40:25 +03:00
fi
ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
ci: workaround broken fedora:26 image This is essentially the same workaround as https://github.com/ostreedev/ostree/pull/1143. See https://bugzilla.redhat.com/show_bug.cgi?id=1483553. Closes: #975 Approved by: cgwalters
2017-09-06 20:53:46 +03:00
pkg_upgrade
ci: Sync with ostree We have some drift; keep this more in sync so we maintain the rpm/yum hackarounds for example in one place. I backed out the ASAN bits though pending at least a newer gpgme: <https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758> Closes: #1000 Approved by: jlebon
2017-09-19 17:40:35 +03:00
pkg_install_builddeps rpm-ostree
lib: Expose new API around basearch For https://pagure.io/atomic-wg/issue/299 we need to make it more convenient to substitute the architecture in an installation context. I plan to use this API inside `rpmostreepayload` in Anaconda, so we can substitute the same value of `${basearch}` we use in treefiles since https://github.com/projectatomic/rpm-ostree/pull/305 Now, you might wonder - why do we need an API wrapping libdnf? It's because libdnf is not API stable yet. We're just exposing a tiny subset. In theory we could use the Python dnf bindings in Anaconda, but things get slightly weird if rpmostreepayload depends on dnf. Perhaps we'll do that down the road, but for now this a small API surface to maintain (forever). This change reworks the internal `varsubst` bits to take a pure `DnfContext`, since we don't want to spin up a whole `RpmOstreeContext` just to do some string substitutions. Closes: #877 Approved by: jlebon
2017-07-13 23:39:01 +03:00
# Mostly dependencies for tests
ci: run clang build with -Werror Start running `clang` with `-Werror` like ostree. We can only run this on Fedora right now because the CentOS `clang` doesn't support `-Wno-error=macro-redefined`, though that's fine. Closes: #1036 Approved by: cgwalters
2017-10-04 22:48:26 +03:00
pkg_install ostree{,-devel,-grub2} createrepo_c /usr/bin/jq PyYAML \
vmcheck: add SELinux labeling tests There's a lot of paths in the core related to SELinux policy changes and relabeling packages. We currently have no test coverage for them. We add support in the test libraries here to build such packages. We also add a test that checks both that we correctly relabel RPMs when the policy changes and that we handle layered packages that install SELinux packages properly. Closes: #999 Approved by: cgwalters
2017-09-18 22:11:26 +03:00
libubsan libasan libtsan elfutils fuse sudo python-gobject-base \
compose: Add --ex-unified-core The "--ex" prefix here means it's an experimental option. A tremendous change here is that start to support non-uid 0, but there are various things to fix there; the unpacker for example needs to learn to set imported objects fully based on the rpmfi information (i.e. default to uid 0, since libarchive gives the current uid by default). And even when run as uid 0, there are some bugs, though I'm not sure of any showstoppers yet. For example, dracut's `dracut-install` calls `cp --preserve=xattrs` which fails to copy the `user.ostreemeta` xattrs from a checkout (it shouldn't be copying that anyways...) Nevertheless, the infrastructure behind this really helps (is almost a hard requirement for) the [jigdo effort](https://github.com/projectatomic/rpm-ostree/issues/1081). Which is really only true due to SELinux - we need to import the packages, then generate the final tree to get the final policy, then use that policy to relabel all of the packages. Closes: #940 Approved by: jlebon
2017-11-16 05:28:03 +03:00
selinux-policy-devel selinux-policy-targeted
ci: run clang build with -Werror Start running `clang` with `-Werror` like ostree. We can only run this on Fedora right now because the CentOS `clang` doesn't support `-Wno-error=macro-redefined`, though that's fine. Closes: #1036 Approved by: cgwalters
2017-10-04 22:48:26 +03:00
# For ex-container tests and clang build
pkg_install_if_os fedora parallel clang
ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
ci: workaround broken fedora:26 image This is essentially the same workaround as https://github.com/ostreedev/ostree/pull/1143. See https://bugzilla.redhat.com/show_bug.cgi?id=1483553. Closes: #975 Approved by: cgwalters
2017-09-06 20:53:46 +03:00
if [ -n "${CI_PKGS:-}" ]; then
pkg_install ${CI_PKGS}
fi
ci: Delete dockerfile, move to common scripts, consolidate gcc/clang The Dockerfile is problematic since we can't update it atomically. I also really dislike reliance on the Hub. Further, I think rather than caching our build deps as built containers, we should be caching RPMs in CI. And we should be using rpm-ostree at some point to assemble filesystem trees faster. Also, consolidate the clang to be serial with gcc, since while we lose a tiny bit of parallelism, it's not really worth its own context right now. Closes: #759 Approved by: jlebon
2017-04-28 19:29:25 +03:00
# create an unprivileged user for testing
adduser testuser
ci: Sync with ostree We have some drift; keep this more in sync so we maintain the rpm/yum hackarounds for example in one place. I backed out the ASAN bits though pending at least a newer gpgme: <https://github.com/projectatomic/rpm-ostree/pull/1000#issuecomment-331278758> Closes: #1000 Approved by: jlebon
2017-09-19 17:40:35 +03:00
export LSAN_OPTIONS=verbosity=1:log_threads=1
jigdo: V4: Use archful provides for jigdoRPM Requires When I tried to use my WIP client patches to do: `rpm-ostree rebase rojig://fahc:fedora-atomic-host`, I got a missing file object which turned out to be the client importing the i686 RPMs. This was passing in the test suite because we don't mirror i686 of course, but on the client side right now we end up using all enabled repos, and since Fedora is multiarch, the behavior is going to be...not predictable. Thinking a bit about on this problem I actually happened to recall the RPM `%{_isa}` macro which is used in Fedora in various places; for example to "arch bind" `-devel` packages to their base. See for example [this case](https://src.fedoraproject.org/rpms/ostree/blob/33c7dc02bc4f14a6acd7679cc4d84e9629294ae4/f/ostree.spec#_79) in libostree. As I noted at first, the core problem here is that the "final" RPM architecture field is not symmetric in any way with the definition of that `%{_isa}` macro. See: https://github.com/rpm-software-management/rpm/blob/d9d47e01146a5d4411691a71916b1030ac7da193/installplatform#L25 The *third* solution I ended up on here is to iterate over the `Provides` on the server side and we take the first thing that matches `Provides: %{name}(whatever)`. I briefly thought about trying to somehow drive into libsolv the logic to prefer the jigdoRPM's native architecture...IIRC yum did something like that in the past but it was never done in libsolv? Anyways the dependencies here are now more correct, so other tools will also handle it. Closes: #1213 Approved by: jlebon
2018-01-18 00:25:39 +03:00
# And now the build
build --enable-installed-tests --enable-gtk-doc ${CONFIGOPTS:-}
Reference in New Issue Copy Permalink