2020-03-24 14:01:12 +03:00
variables :
2020-03-24 15:04:08 +03:00
GIT_DEPTH : 100
2020-03-24 14:01:12 +03:00
2020-03-06 18:38:36 +03:00
stages :
2020-06-02 18:28:57 +03:00
- containers
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
- builds
2020-07-28 12:05:16 +03:00
- sanity_checks
2020-03-06 18:38:36 +03:00
2020-03-27 18:40:05 +03:00
.script_variables : &script_variables |
gitlab: Enable improved ccache usage
Setting CC="ccache cc" works in most cases, but sometimes it will
break the build: in particular, we have experienced issues in the
past with that approach when using cgo to build our Go bindings.
A more robust approach is to have a directory containing symlinks
from the compiler name to the ccache binary: in that case, ccache
itself will invoke the compiler, and the build system will be none
the wiser.
Since libvirt-ci commit 2563aebb6c5c, container images contain a
suitable symlink directory, so all that's needed to enable the new
approach is to add this directory to $PATH.
Since we're touching this anyway, we make a few more changes:
$CCACHE_DIR is no longer created manually, because ccache will
take care of creating it for us if it doesn't already exist; the
ccache setup is moved out of the job template and into
script_variables, removing unnecessary duplication; a limit is
set on the size of the cache (500 MB, which is twice the amount
used by a fresh build on my Fedora 31 machine).
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-30 19:29:06 +03:00
export CCACHE_BASEDIR="$(pwd)"
export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
export CCACHE_MAXSIZE="500M"
export PATH="$CCACHE_WRAPPERSDIR:$PATH"
2020-03-06 18:38:36 +03:00
2020-03-06 19:29:03 +03:00
# Common templates
2021-01-13 19:45:06 +03:00
.container_job :
2020-06-02 18:28:57 +03:00
image : docker:stable
stage : containers
2020-07-31 15:51:00 +03:00
needs : [ ]
2020-06-02 18:28:57 +03:00
services :
- docker:dind
2021-01-14 13:20:36 +03:00
rules :
2021-01-14 13:36:54 +03:00
- if : "$TEMPORARILY_DISABLED"
allow_failure : true
2021-01-14 13:20:36 +03:00
- when : on_success
2020-06-02 18:28:57 +03:00
before_script :
- export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest"
- export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest"
- docker info
- docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
script :
- docker pull "$TAG" || docker pull "$COMMON_TAG" || true
2020-12-01 18:22:09 +03:00
- docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/ci-$NAME.Dockerfile" ci/containers
2020-06-02 18:28:57 +03:00
- docker push "$TAG"
after_script :
- docker logout
# We build many containers which can be useful to debug problems but are not
# needed for the pipeline itself to complete: those sometimes fail, and when
# that happens it's mostly because of temporary issues with Debian sid. We
# don't want those failures to affect the overall pipeline status
2021-01-13 19:45:06 +03:00
.container_optional_job :
extends : .container_job
2020-06-02 18:28:57 +03:00
allow_failure : true
2021-01-13 19:45:06 +03:00
.native_build_job :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : builds
2020-06-02 18:28:57 +03:00
image : $CI_REGISTRY_IMAGE/ci-$NAME:latest
2021-01-14 13:20:36 +03:00
rules :
2021-01-14 13:36:54 +03:00
- if : "$TEMPORARILY_DISABLED"
allow_failure : true
2021-01-14 13:20:36 +03:00
- when : on_success
2020-03-25 18:01:43 +03:00
cache :
paths :
- ccache/
key : "$CI_JOB_NAME"
before_script :
2020-03-27 18:40:05 +03:00
- *script_variables
2020-03-06 19:29:03 +03:00
script :
2020-08-03 09:53:00 +03:00
- meson build --werror || (cat build/meson-logs/meson-log.txt && exit 1)
2020-10-08 16:30:50 +03:00
- if test "$DIST" != "skip";
then
ninja -C build dist;
else
ninja -C build;
ninja -C build test;
fi
2020-10-08 16:36:07 +03:00
- if test -x /usr/bin/rpmbuild && test "$RPM" != "skip";
then
rpmbuild --nodeps -ta build/meson-dist/libvirt-*.tar.xz;
fi
2020-03-06 19:29:03 +03:00
2020-06-05 21:37:55 +03:00
# Jobs that we delegate to Cirrus CI because they require an operating
# system other than Linux. These jobs will only run if the required
# setup has been performed on the GitLab account (see ci/README.rst).
2020-06-29 20:00:36 +03:00
#
# The Cirrus CI configuration is generated by replacing target-specific
# variables in a generic template: some of these variables are provided
# when the GitLab CI job is defined, others are taken from a shell
# snippet generated using lcitool.
#
# Note that the $PATH environment variable has to be treated with
# special care, because we can't just override it at the GitLab CI job
# definition level or we risk breaking it completely.
2021-01-13 19:45:06 +03:00
.cirrus_build_job :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : builds
2020-06-05 21:37:55 +03:00
image : registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
2020-07-29 02:06:30 +03:00
needs : [ ]
2020-06-05 21:37:55 +03:00
script :
2021-01-11 17:01:51 +03:00
- source ci/cirrus/$NAME.vars
2020-06-29 20:00:36 +03:00
- sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
-e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
-e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g"
-e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g"
-e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g"
-e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g"
-e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g"
-e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g"
-e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g"
-e "s|[@]PKGS@|$PKGS|g"
-e "s|[@]MAKE@|$MAKE|g"
-e "s|[@]PYTHON@|$PYTHON|g"
2021-01-11 17:12:02 +03:00
-e "s|[@]PIP3@|$PIP3|g"
2020-08-03 09:53:00 +03:00
-e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g"
2020-06-29 20:00:36 +03:00
<ci/cirrus/build.yml >ci/cirrus/$NAME.yml
- cat ci/cirrus/$NAME.yml
2020-08-04 13:00:41 +03:00
- cirrus-run -v --show-build-log always ci/cirrus/$NAME.yml
2021-01-14 13:20:36 +03:00
rules :
2021-01-14 13:36:54 +03:00
- if : "$TEMPORARILY_DISABLED"
allow_failure : true
2021-01-14 13:20:36 +03:00
- if : "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
2020-06-05 21:37:55 +03:00
2021-01-13 19:45:06 +03:00
.cross_build_job :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : builds
2020-06-02 18:28:57 +03:00
image : $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest
2020-03-25 18:01:43 +03:00
cache :
paths :
- ccache/
key : "$CI_JOB_NAME"
2021-01-14 13:20:36 +03:00
rules :
2021-01-14 13:36:54 +03:00
- if : "$TEMPORARILY_DISABLED"
allow_failure : true
2021-01-14 13:20:36 +03:00
- when : on_success
2020-03-25 18:01:43 +03:00
before_script :
2020-03-27 18:40:05 +03:00
- *script_variables
2019-01-25 20:38:53 +03:00
script :
2020-08-03 09:53:00 +03:00
- meson build --werror $MESON_OPTS || (cat build/meson-logs/meson-log.txt && exit 1)
- ninja -C build
2020-10-29 20:32:28 +03:00
- if test "$CROSS" = "i686" ; then ninja -C build test ; fi
2019-01-25 20:38:53 +03:00
2020-06-02 18:28:57 +03:00
# Native container build jobs
x64-centos-7-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : centos-7
x64-centos-8-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : centos-8
x64-centos-stream-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : centos-stream
x64-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10
x64-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid
2020-11-23 21:03:36 +03:00
x64-fedora-32-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
2020-11-23 21:03:36 +03:00
NAME : fedora-32
2020-06-02 18:28:57 +03:00
2020-11-23 21:03:36 +03:00
x64-fedora-33-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
2020-11-23 21:03:36 +03:00
NAME : fedora-33
2020-06-02 18:28:57 +03:00
x64-fedora-rawhide-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : fedora-rawhide
2021-01-19 10:28:42 +03:00
x64-opensuse-152-container :
extends : .container_job
variables :
NAME : opensuse-152
2020-06-02 18:28:57 +03:00
x64-ubuntu-1804-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : ubuntu-1804
x64-ubuntu-2004-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : ubuntu-2004
# Cross-build containers build jobs
aarch64-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-aarch64
armv6l-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-armv6l
2020-07-29 11:53:51 +03:00
armv7l-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-armv7l
i686-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-i686
mips-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-mips
mips64el-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-mips64el
mipsel-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-mipsel
ppc64le-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-ppc64le
s390x-debian-10-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-10-cross-s390x
aarch64-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-aarch64
armv6l-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-armv6l
2020-07-29 11:53:51 +03:00
armv7l-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-armv7l
i686-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-i686
mips64el-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-mips64el
mipsel-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-mipsel
ppc64le-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-ppc64le
s390x-debian-sid-container :
2021-01-13 19:45:06 +03:00
extends : .container_optional_job
2020-06-02 18:28:57 +03:00
variables :
NAME : debian-sid-cross-s390x
mingw32-fedora-rawhide-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : fedora-rawhide-cross-mingw32
mingw64-fedora-rawhide-container :
2021-01-13 19:45:06 +03:00
extends : .container_job
2020-06-02 18:28:57 +03:00
variables :
NAME : fedora-rawhide-cross-mingw64
2020-03-06 19:29:03 +03:00
# Native architecture build + test jobs
x64-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-debian-10-container
2020-06-02 18:28:55 +03:00
variables :
NAME : debian-10
2020-03-06 19:29:03 +03:00
2020-07-28 15:32:40 +03:00
x64-debian-10-clang :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-debian-10-container
2020-07-28 15:32:40 +03:00
variables :
NAME : debian-10
CC : clang
2020-03-06 19:29:03 +03:00
x64-debian-sid :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-debian-sid-container
2020-06-02 18:28:55 +03:00
variables :
NAME : debian-sid
2020-03-06 19:29:03 +03:00
x64-centos-7 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-centos-7-container
2020-06-02 18:28:55 +03:00
variables :
NAME : centos-7
2020-10-08 16:30:50 +03:00
# meson dist fails on CentOS 7 because of old git that fails to clone
# from shallow git repository which is done when running meson dist
DIST : skip
2020-10-08 16:36:07 +03:00
RPM : skip
2020-03-06 19:29:03 +03:00
x64-centos-8 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-centos-8-container
2020-06-02 18:28:55 +03:00
variables :
NAME : centos-8
2020-10-08 16:36:07 +03:00
RPM : skip
2020-03-06 19:29:03 +03:00
2020-07-28 15:32:40 +03:00
x64-centos-8-clang :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-centos-8-container
2020-07-28 15:32:40 +03:00
variables :
NAME : centos-8
CC : clang
2020-10-08 16:36:07 +03:00
RPM : skip
2020-07-28 15:32:40 +03:00
2020-06-02 18:28:57 +03:00
x64-centos-stream :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-centos-stream-container
2020-06-02 18:28:57 +03:00
variables :
NAME : centos-stream
2020-10-08 16:36:07 +03:00
RPM : skip
2020-06-02 18:28:57 +03:00
2020-11-23 21:03:36 +03:00
x64-fedora-32 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
2020-11-23 21:03:36 +03:00
- x64-fedora-32-container
2020-06-02 18:28:55 +03:00
variables :
2020-11-23 21:03:36 +03:00
NAME : fedora-32
2020-10-08 16:36:07 +03:00
RPM : skip
2020-03-06 19:29:03 +03:00
2020-11-23 21:03:36 +03:00
x64-fedora-33 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
2020-11-23 21:03:36 +03:00
- x64-fedora-33-container
2020-06-02 18:28:55 +03:00
variables :
2020-11-23 21:03:36 +03:00
NAME : fedora-33
2020-05-04 15:06:56 +03:00
2020-03-06 19:29:03 +03:00
x64-fedora-rawhide :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-fedora-rawhide-container
2020-06-02 18:28:55 +03:00
variables :
NAME : fedora-rawhide
2021-01-25 21:13:57 +03:00
# Temp workaround until we see what resolution is for
# https://bugzilla.redhat.com/show_bug.cgi?id=1919965
# Hopefully remove before F34 is released.
CFLAGS : -gdwarf-4
2020-03-06 19:29:03 +03:00
2020-07-28 15:32:40 +03:00
x64-fedora-rawhide-clang :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-fedora-rawhide-container
2020-07-28 15:32:40 +03:00
variables :
NAME : fedora-rawhide
CC : clang
2020-10-08 16:36:07 +03:00
RPM : skip
2020-07-28 15:32:40 +03:00
2021-01-19 10:28:42 +03:00
x64-opensuse-152 :
extends : .native_build_job
needs :
- x64-opensuse-152-container
variables :
NAME : opensuse-152
RPM : skip
2020-03-06 19:29:03 +03:00
x64-ubuntu-1804 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-ubuntu-1804-container
2020-06-02 18:28:55 +03:00
variables :
NAME : ubuntu-1804
2020-03-06 19:29:03 +03:00
2020-05-04 15:06:56 +03:00
x64-ubuntu-2004 :
2021-01-13 19:45:06 +03:00
extends : .native_build_job
2020-07-28 12:15:13 +03:00
needs :
- x64-ubuntu-2004-container
2020-06-02 18:28:55 +03:00
variables :
NAME : ubuntu-2004
2020-05-04 15:06:56 +03:00
2020-09-04 13:22:16 +03:00
x64-freebsd-11-build :
2021-01-13 19:45:06 +03:00
extends : .cirrus_build_job
2020-09-04 13:22:16 +03:00
variables :
NAME : freebsd-11
CIRRUS_VM_INSTANCE_TYPE : freebsd_instance
CIRRUS_VM_IMAGE_SELECTOR : image_family
CIRRUS_VM_IMAGE_NAME : freebsd-11-4
INSTALL_COMMAND : pkg install -y
2020-06-05 21:37:55 +03:00
x64-freebsd-12-build :
2021-01-13 19:45:06 +03:00
extends : .cirrus_build_job
2020-06-05 21:37:55 +03:00
variables :
NAME : freebsd-12
2020-06-29 20:00:36 +03:00
CIRRUS_VM_INSTANCE_TYPE : freebsd_instance
CIRRUS_VM_IMAGE_SELECTOR : image_family
CIRRUS_VM_IMAGE_NAME : freebsd-12-1
INSTALL_COMMAND : pkg install -y
2020-06-05 21:37:55 +03:00
x64-macos-1015-build :
2021-01-13 19:45:06 +03:00
extends : .cirrus_build_job
2020-06-05 21:37:55 +03:00
variables :
NAME : macos-1015
2020-06-29 20:00:36 +03:00
CIRRUS_VM_INSTANCE_TYPE : osx_instance
CIRRUS_VM_IMAGE_SELECTOR : image
CIRRUS_VM_IMAGE_NAME : catalina-base
INSTALL_COMMAND : brew install
PATH_EXTRA : /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin:/usr/local/opt/libpcap/bin:/usr/local/opt/libxslt/bin:/usr/local/opt/rpcgen/bin
PKG_CONFIG_PATH : /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/libpcap/lib/pkgconfig:/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
2020-06-05 21:37:55 +03:00
2020-03-06 19:29:03 +03:00
# Cross compiled build jobs
2020-07-14 19:49:41 +03:00
armv6l-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- armv6l-debian-10-container
2020-06-02 18:28:55 +03:00
variables :
2020-07-14 19:49:41 +03:00
NAME : debian-10
2020-06-02 18:28:55 +03:00
CROSS : armv6l
2019-01-25 20:38:53 +03:00
2020-07-16 11:26:02 +03:00
armv7l-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- armv7l-debian-10-container
2020-07-16 11:26:02 +03:00
variables :
NAME : debian-10
CROSS : armv7l
2020-07-14 19:49:41 +03:00
mips64el-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- mips64el-debian-10-container
2020-06-02 18:28:55 +03:00
variables :
2020-07-14 19:49:41 +03:00
NAME : debian-10
2020-06-02 18:28:55 +03:00
CROSS : mips64el
2019-07-11 13:34:50 +03:00
2020-07-14 19:49:41 +03:00
mips-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- mips-debian-10-container
2020-06-02 18:28:55 +03:00
variables :
2020-07-14 19:49:41 +03:00
NAME : debian-10
2020-06-02 18:28:55 +03:00
CROSS : mips
2019-01-25 20:38:53 +03:00
2020-03-06 21:34:45 +03:00
aarch64-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- aarch64-debian-10-container
2020-06-02 18:28:55 +03:00
variables :
NAME : debian-10
CROSS : aarch64
2019-01-25 20:38:53 +03:00
2020-06-11 21:06:02 +03:00
mipsel-debian-10 :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- mipsel-debian-10-container
2020-06-02 18:28:55 +03:00
variables :
NAME : debian-10
2020-06-11 21:06:02 +03:00
CROSS : mipsel
2019-01-25 20:38:53 +03:00
2020-07-14 19:49:41 +03:00
s390x-debian-sid :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
2021-02-09 12:26:55 +03:00
- s390x-debian-sid-container
2020-06-02 18:28:55 +03:00
variables :
2020-07-14 19:49:41 +03:00
NAME : debian-sid
2020-06-02 18:28:55 +03:00
CROSS : s390x
2019-01-25 20:38:53 +03:00
2020-03-06 21:34:45 +03:00
i686-debian-sid :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- i686-debian-sid-container
2020-06-02 18:28:55 +03:00
variables :
NAME : debian-sid
CROSS : i686
2019-01-25 20:38:53 +03:00
2020-06-11 21:06:02 +03:00
ppc64le-debian-sid :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
2021-02-09 12:26:55 +03:00
- ppc64le-debian-sid-container
2020-06-02 18:28:55 +03:00
variables :
NAME : debian-sid
2020-06-11 21:06:02 +03:00
CROSS : ppc64le
2020-03-06 17:56:42 +03:00
2020-05-04 15:06:56 +03:00
mingw32-fedora-rawhide :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- mingw32-fedora-rawhide-container
2020-06-02 18:28:55 +03:00
variables :
NAME : fedora-rawhide
CROSS : mingw32
2020-03-24 17:23:27 +03:00
2020-05-04 15:06:56 +03:00
mingw64-fedora-rawhide :
2021-01-13 19:45:06 +03:00
extends : .cross_build_job
2020-07-28 12:15:13 +03:00
needs :
- mingw64-fedora-rawhide-container
2020-06-02 18:28:55 +03:00
variables :
NAME : fedora-rawhide
CROSS : mingw64
2020-03-24 17:23:27 +03:00
2020-03-06 17:56:42 +03:00
# This artifact published by this job is downloaded by libvirt.org to
# be deployed to the web root:
# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website
website :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : builds
2020-06-02 18:28:57 +03:00
image : $CI_REGISTRY_IMAGE/ci-centos-8:latest
2020-07-28 12:15:13 +03:00
needs :
- x64-centos-8-container
2020-03-27 18:40:05 +03:00
before_script :
- *script_variables
2020-03-06 17:56:42 +03:00
script :
2020-08-03 09:53:00 +03:00
- meson build --prefix=$(pwd)/vroot || (cat build/meson-logs/meson-log.txt && exit 1)
- ninja -C build install-web
2020-03-06 17:56:42 +03:00
- mv vroot/share/doc/libvirt/html/ website
artifacts :
expose_as : 'Website'
name : 'website'
when : on_success
expire_in : 30 days
paths :
- website
2020-03-24 17:42:14 +03:00
2020-03-26 15:07:34 +03:00
codestyle :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : builds
2021-01-20 10:06:35 +03:00
image : $CI_REGISTRY_IMAGE/ci-opensuse-152:latest
2020-07-28 12:15:13 +03:00
needs :
2021-01-20 10:06:35 +03:00
- x64-opensuse-152-container
2020-03-27 18:40:05 +03:00
before_script :
- *script_variables
2020-03-26 15:07:34 +03:00
script :
2020-08-03 09:53:00 +03:00
- meson build || (cat build/meson-logs/meson-log.txt && exit 1)
- ninja -C build libvirt-pot-dep
- meson test -C build --suite syntax-check --no-rebuild || (cat build/meson-logs/testlog.txt && exit 1)
2020-03-26 15:07:34 +03:00
2020-03-24 17:42:14 +03:00
# This artifact published by this job is downloaded to push to Weblate
# for translation usage:
# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potfile
potfile :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : builds
2020-06-02 18:28:57 +03:00
image : $CI_REGISTRY_IMAGE/ci-centos-8:latest
2020-07-28 12:15:13 +03:00
needs :
- x64-centos-8-container
2021-01-14 13:20:36 +03:00
rules :
- if : "$CI_COMMIT_BRANCH == 'master'"
2020-03-27 18:40:05 +03:00
before_script :
- *script_variables
2020-03-24 17:42:14 +03:00
script :
2020-08-03 09:53:00 +03:00
- meson build || (cat build/meson-logs/meson-log.txt && exit 1)
- ninja -C build libvirt-pot-dep
- ninja -C build libvirt-pot
2020-06-04 16:26:39 +03:00
- cp po/libvirt.pot libvirt.pot
2020-03-24 17:42:14 +03:00
artifacts :
expose_as : 'Potfile'
name : 'potfile'
when : on_success
expire_in : 30 days
paths :
- libvirt.pot
2020-03-26 13:46:47 +03:00
2020-05-01 13:57:33 +03:00
# Check that all commits are signed-off for the DCO.
# Skip on "libvirt" namespace, since we only need to run
# this test on developer's personal forks from which
# merge requests are submitted
check-dco :
ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.
This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.
Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.
Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].
[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage : sanity_checks
2020-07-28 12:05:16 +03:00
needs : [ ]
2020-05-01 13:57:33 +03:00
image : registry.gitlab.com/libvirt/libvirt-ci/check-dco:master
2020-03-26 13:46:47 +03:00
script :
2020-05-01 13:57:33 +03:00
- /check-dco
2021-01-14 13:20:36 +03:00
rules :
- if : "$CI_PROJECT_NAMESPACE != 'libvirt'"
2020-07-14 17:29:26 +03:00
variables :
GIT_DEPTH : 1000
2020-11-12 16:56:25 +03:00
# Coverity job that is run only by schedules
coverity :
image : $CI_REGISTRY_IMAGE/ci-centos-8:latest
needs :
- x64-centos-8-container
stage : builds
script :
- curl https://scan.coverity.com/download/linux64 --form project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN -o /tmp/cov-analysis-linux64.tgz
- tar xfz /tmp/cov-analysis-linux64.tgz
- meson build
- cov-analysis-linux64-*/bin/cov-build --dir cov-int ninja -C build
- tar cfz cov-int.tar.gz cov-int
- curl https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN --form email=$GITLAB_USER_EMAIL --form file=@cov-int.tar.gz --form version="$(git describe --tags)" --form description="$(git describe --tags) / $CI_COMMIT_TITLE / $CI_COMMIT_REF_NAME:$CI_PIPELINE_ID"
2021-01-14 13:20:36 +03:00
rules :
- if : "$CI_PIPELINE_SOURCE == 'schedule' && $COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN"