shaba/libvirt
1
0
Fork 0
mirror of https://gitlab.com/libvirt/libvirt.git synced 2025-01-10 05:17:59 +03:00
Code Releases Activity
60a7c78348
libvirt/.gitlab-ci.yml

543 lines
12 KiB
YAML
Raw Normal View History

gitlab: add variable for make command name To facilitate future jobs that will use FreeBSD Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 14:01:12 +03:00
variables:
gitlab: restrict git history to 100 commits We don't need the full git history when running CI jobs. From a code POV we only need the most recent commit, but we want to be able to run checks on the commits too. In particular to validate the DCO signoff for each commit. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 15:04:08 +03:00
GIT_DEPTH: 100
gitlab: add variable for make command name To facilitate future jobs that will use FreeBSD Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 14:01:12 +03:00
gitlab: create an explicit stage for cross build jobs As we introduce more build jobs, it will be useful to have a grouping of jobs to more easily visualize the results and potentially control build ordering. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 18:38:36 +03:00
stages:
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
- sanity_checks
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
- containers
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
- builds
gitlab: create an explicit stage for cross build jobs As we introduce more build jobs, it will be useful to have a grouping of jobs to more easily visualize the results and potentially control build ordering. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 18:38:36 +03:00
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
.script_variables: &script_variables |
export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
gitlab: Enable improved ccache usage Setting CC="ccache cc" works in most cases, but sometimes it will break the build: in particular, we have experienced issues in the past with that approach when using cgo to build our Go bindings. A more robust approach is to have a directory containing symlinks from the compiler name to the ccache binary: in that case, ccache itself will invoke the compiler, and the build system will be none the wiser. Since libvirt-ci commit 2563aebb6c5c, container images contain a suitable symlink directory, so all that's needed to enable the new approach is to add this directory to $PATH. Since we're touching this anyway, we make a few more changes: $CCACHE_DIR is no longer created manually, because ccache will take care of creating it for us if it doesn't already exist; the ccache setup is moved out of the job template and into script_variables, removing unnecessary duplication; a limit is set on the size of the cache (500 MB, which is twice the amount used by a fresh build on my Fedora 31 machine). Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-30 19:29:06 +03:00
export CCACHE_BASEDIR="$(pwd)"
export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
export CCACHE_MAXSIZE="500M"
export PATH="$CCACHE_WRAPPERSDIR:$PATH"
gitlab: create an explicit stage for cross build jobs As we introduce more build jobs, it will be useful to have a grouping of jobs to more easily visualize the results and potentially control build ordering. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 18:38:36 +03:00
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
# Common templates
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
# Containers that are necessary for the default pipeline
.container_default_job_template: &container_default_job_definition
image: docker:stable
stage: containers
services:
- docker:dind
before_script:
- export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest"
- export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest"
- docker info
- docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
script:
- docker pull "$TAG" || docker pull "$COMMON_TAG" || true
- docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/libvirt-$NAME.Dockerfile" ci/containers
- docker push "$TAG"
after_script:
- docker logout
# Containers that are only needed for the full pipeline
.container_extra_job_template: &container_extra_job_definition
<<: *container_default_job_definition
only:
- master
- /^ci-full-.*$/
# We build many containers which can be useful to debug problems but are not
# needed for the pipeline itself to complete: those sometimes fail, and when
# that happens it's mostly because of temporary issues with Debian sid. We
# don't want those failures to affect the overall pipeline status
.container_optional_job_template: &container_optional_job_definition
<<: *container_extra_job_definition
allow_failure: true
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
# Default native build jobs that are always run
.native_build_default_job_template: &native_build_default_job_definition
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: builds
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
gitlab: introduce use of ccache for speeding up rebuilds For any given job there is a high likelihood that ccache will be able to reuse previously built object files. This will result in faster build pipelines in later updates. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 18:01:43 +03:00
cache:
paths:
- ccache/
key: "$CI_JOB_NAME"
before_script:
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- *script_variables
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
script:
- mkdir build
- cd build
- ../autogen.sh || (cat config.log && exit 1)
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- $MAKE distcheck
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
# Extra native build jobs that are only run post-merge, or
# when code is pushed to a branch with "ci-full-" name prefix
.native_build_extra_job_template: &native_build_extra_job_definition
<<: *native_build_default_job_definition
only:
- master
- /^ci-full-.*$/
ci: Enable Cirrus CI integration We use cirrus-run to trigger Cirrus CI jobs from GitLab CI jobs, making it possible to extend our platform coverage to include FreeBSD without having to maintain our own runners; additionally, we'll be able to ditch Travis CI and, since results for Cirrus CI jobs are reflected back to the GitLab CI jobs that triggered them, we will be able to get all information from a single dashboard. The FreeBSD and macOS job definitions can be improved further: for example, we will want to enable caching to speed up builds, and ultimately we should figure out a way to generate at least part of them, notably the list of packages to be installed, using lcitool. All of that will happen in later patches: for now, this is good enough to start using Cirrus CI. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-05 21:37:55 +03:00
# Jobs that we delegate to Cirrus CI because they require an operating
# system other than Linux. These jobs will only run if the required
# setup has been performed on the GitLab account (see ci/README.rst).
.cirrus_build_default_job_template: &cirrus_build_default_job_definition
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: builds
ci: Enable Cirrus CI integration We use cirrus-run to trigger Cirrus CI jobs from GitLab CI jobs, making it possible to extend our platform coverage to include FreeBSD without having to maintain our own runners; additionally, we'll be able to ditch Travis CI and, since results for Cirrus CI jobs are reflected back to the GitLab CI jobs that triggered them, we will be able to get all information from a single dashboard. The FreeBSD and macOS job definitions can be improved further: for example, we will want to enable caching to speed up builds, and ultimately we should figure out a way to generate at least part of them, notably the list of packages to be installed, using lcitool. All of that will happen in later patches: for now, this is good enough to start using Cirrus CI. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-05 21:37:55 +03:00
image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
script:
- cirrus-run ci/cirrus/$NAME.yml.j2
only:
variables:
- $CIRRUS_GITHUB_REPO
- $CIRRUS_API_TOKEN
.cirrus_build_extra_job_template: &cirrus_build_extra_job_definition
<<: *cirrus_build_default_job_definition
only:
variables:
- $CIRRUS_GITHUB_REPO
- $CIRRUS_API_TOKEN
refs:
- master
- /^ci-full-.*$/
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
# Default cross build jobs that are always run
gitlab: create an explicit stage for cross build jobs As we introduce more build jobs, it will be useful to have a grouping of jobs to more easily visualize the results and potentially control build ordering. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 18:38:36 +03:00
.cross_build_default_job_template: &cross_build_default_job_definition
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: builds
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest
gitlab: introduce use of ccache for speeding up rebuilds For any given job there is a high likelihood that ccache will be able to reuse previously built object files. This will result in faster build pipelines in later updates. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 18:01:43 +03:00
cache:
paths:
- ccache/
key: "$CI_JOB_NAME"
before_script:
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- *script_variables
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
script:
- mkdir build
- cd build
- ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1)
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- $MAKE
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
# Extra cross build jobs that are only run post-merge, or
# when code is pushed to a branch with "ci-full-" name prefix
.cross_build_extra_job_template: &cross_build_extra_job_definition
<<: *cross_build_default_job_definition
only:
- master
- /^ci-full-.*$/
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
# Native container build jobs
x64-centos-7-container:
<<: *container_default_job_definition
variables:
NAME: centos-7
x64-centos-8-container:
<<: *container_default_job_definition
variables:
NAME: centos-8
x64-centos-stream-container:
<<: *container_extra_job_definition
variables:
NAME: centos-stream
x64-debian-9-container:
<<: *container_extra_job_definition
variables:
NAME: debian-9
x64-debian-10-container:
<<: *container_default_job_definition
variables:
NAME: debian-10
x64-debian-sid-container:
<<: *container_extra_job_definition
variables:
NAME: debian-sid
x64-fedora-31-container:
<<: *container_extra_job_definition
variables:
NAME: fedora-31
x64-fedora-32-container:
<<: *container_default_job_definition
variables:
NAME: fedora-32
x64-fedora-rawhide-container:
<<: *container_default_job_definition
variables:
NAME: fedora-rawhide
x64-opensuse-151-container:
<<: *container_default_job_definition
variables:
NAME: opensuse-151
x64-ubuntu-1804-container:
<<: *container_extra_job_definition
variables:
NAME: ubuntu-1804
x64-ubuntu-2004-container:
<<: *container_default_job_definition
variables:
NAME: ubuntu-2004
# Cross-build containers build jobs
aarch64-debian-9-container:
<<: *container_optional_job_definition
variables:
NAME: debian-9-cross-aarch64
armv6l-debian-9-container:
<<: *container_extra_job_definition
variables:
NAME: debian-9-cross-armv6l
armv7l-debian-9-container:
<<: *container_optional_job_definition
variables:
NAME: debian-9-cross-armv7l
mips-debian-9-container:
<<: *container_extra_job_definition
variables:
NAME: debian-9-cross-mips
mips64el-debian-9-container:
<<: *container_extra_job_definition
variables:
NAME: debian-9-cross-mips64el
mipsel-debian-9-container:
<<: *container_optional_job_definition
variables:
NAME: debian-9-cross-mipsel
ppc64le-debian-9-container:
<<: *container_optional_job_definition
variables:
NAME: debian-9-cross-ppc64le
s390x-debian-9-container:
<<: *container_optional_job_definition
variables:
NAME: debian-9-cross-s390x
aarch64-debian-10-container:
<<: *container_extra_job_definition
variables:
NAME: debian-10-cross-aarch64
armv6l-debian-10-container:
<<: *container_optional_job_definition
variables:
NAME: debian-10-cross-armv6l
armv7-ldebian-10-container:
<<: *container_optional_job_definition
variables:
NAME: debian-10-cross-armv7l
i686-debian-10-container:
<<: *container_optional_job_definition
variables:
NAME: debian-10-cross-i686
mips-debian-10-container:
<<: *container_optional_job_definition
variables:
NAME: debian-10-cross-mips
mips64el-debian-10-container:
<<: *container_optional_job_definition
variables:
NAME: debian-10-cross-mips64el
mipsel-debian-10-container:
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
<<: *container_extra_job_definition
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
variables:
NAME: debian-10-cross-mipsel
ppc64le-debian-10-container:
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
<<: *container_optional_job_definition
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
variables:
NAME: debian-10-cross-ppc64le
s390x-debian-10-container:
<<: *container_default_job_definition
variables:
NAME: debian-10-cross-s390x
aarch64-debian-sid-container:
<<: *container_optional_job_definition
variables:
NAME: debian-sid-cross-aarch64
armv6l-debian-sid-container:
<<: *container_optional_job_definition
variables:
NAME: debian-sid-cross-armv6l
armv7-ldebian-sid-container:
<<: *container_default_job_definition
variables:
NAME: debian-sid-cross-armv7l
i686-debian-sid-container:
<<: *container_extra_job_definition
variables:
NAME: debian-sid-cross-i686
mips64el-debian-sid-container:
<<: *container_optional_job_definition
variables:
NAME: debian-sid-cross-mips64el
mipsel-debian-sid-container:
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
<<: *container_optional_job_definition
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
variables:
NAME: debian-sid-cross-mipsel
ppc64le-debian-sid-container:
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
<<: *container_extra_job_definition
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
variables:
NAME: debian-sid-cross-ppc64le
s390x-debian-sid-container:
<<: *container_optional_job_definition
variables:
NAME: debian-sid-cross-s390x
mingw32-fedora-rawhide-container:
<<: *container_default_job_definition
variables:
NAME: fedora-rawhide-cross-mingw32
mingw64-fedora-rawhide-container:
<<: *container_default_job_definition
variables:
NAME: fedora-rawhide-cross-mingw64
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
# Native architecture build + test jobs
x64-debian-9:
<<: *native_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-9
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-debian-10:
<<: *native_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-10
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-debian-sid:
<<: *native_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-sid
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-centos-7:
<<: *native_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: centos-7
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-centos-8:
<<: *native_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: centos-8
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
x64-centos-stream:
<<: *native_build_extra_job_definition
variables:
NAME: centos-stream
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-fedora-31:
<<: *native_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: fedora-31
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
gitlab: update list of distros to use latest versions Replace Fedora 30 with Fedora 32 Replace Ubuntu 16.04 with Ubuntu 20.04 Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 15:06:56 +03:00
x64-fedora-32:
<<: *native_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: fedora-32
gitlab: update list of distros to use latest versions Replace Fedora 30 with Fedora 32 Replace Ubuntu 16.04 with Ubuntu 20.04 Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 15:06:56 +03:00
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-fedora-rawhide:
<<: *native_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: fedora-rawhide
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-opensuse-151:
<<: *native_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: opensuse-151
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
x64-ubuntu-1804:
<<: *native_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: ubuntu-1804
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
gitlab: update list of distros to use latest versions Replace Fedora 30 with Fedora 32 Replace Ubuntu 16.04 with Ubuntu 20.04 Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 15:06:56 +03:00
x64-ubuntu-2004:
<<: *native_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: ubuntu-2004
gitlab: update list of distros to use latest versions Replace Fedora 30 with Fedora 32 Replace Ubuntu 16.04 with Ubuntu 20.04 Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 15:06:56 +03:00
ci: Enable Cirrus CI integration We use cirrus-run to trigger Cirrus CI jobs from GitLab CI jobs, making it possible to extend our platform coverage to include FreeBSD without having to maintain our own runners; additionally, we'll be able to ditch Travis CI and, since results for Cirrus CI jobs are reflected back to the GitLab CI jobs that triggered them, we will be able to get all information from a single dashboard. The FreeBSD and macOS job definitions can be improved further: for example, we will want to enable caching to speed up builds, and ultimately we should figure out a way to generate at least part of them, notably the list of packages to be installed, using lcitool. All of that will happen in later patches: for now, this is good enough to start using Cirrus CI. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-05 21:37:55 +03:00
x64-freebsd-12-build:
<<: *cirrus_build_default_job_definition
variables:
NAME: freebsd-12
x64-macos-1015-build:
<<: *cirrus_build_default_job_definition
variables:
NAME: macos-1015
gitlab: add x86_64 native CI jobs This patch adds x86_64 native CI jobs for all distros that we currently build container images for. This is a superset of the Linux jobs run on current Jenkins and Travis platforms. The remaining missing platforms are FreeBSD and macOS, neither of which can use the shared runner container based infrastructure. We may add further native jobs in the future which are not x86_64 based, if we get access to suitable hardware, thus the jobs all have an arch prefix in their name, just like the cross-built jobs do. As with the cross-arch builds, the native jobs are split into two groups. One group is run in all situations, while the other group is only run on the master branch, or branches with a name prefix 'ci-full-'. This avoids the build time getting too long when developers are testing their code prior to submission, while keeping full coverage of code that is merged. Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 19:29:03 +03:00
# Cross compiled build jobs
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
armv6l-debian-9:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-9
CROSS: armv6l
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
mips64el-debian-9:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-9
CROSS: mips64el
gitlab: Perform some builds on Debian 10 The split is mostly arbitrary, but we purposefully moved the mips64el builds off sid since currently that container is failing to build and is thus unavailable. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-11 13:34:50 +03:00
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
mips-debian-9:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-9
CROSS: mips
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
aarch64-debian-10:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-10
CROSS: aarch64
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
mipsel-debian-10:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-10
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
CROSS: mipsel
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
s390x-debian-10:
gitlab: create an explicit stage for cross build jobs As we introduce more build jobs, it will be useful to have a grouping of jobs to more easily visualize the results and potentially control build ordering. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 18:38:36 +03:00
<<: *cross_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-10
CROSS: s390x
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
armv7l-debian-sid:
gitlab: create an explicit stage for cross build jobs As we introduce more build jobs, it will be useful to have a grouping of jobs to more easily visualize the results and potentially control build ordering. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 18:38:36 +03:00
<<: *cross_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-sid
CROSS: armv7l
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
gitlab: rename the cross build jobs The pipeline UI will truncate the names of jobs after about 15 characters. As a result with the cross-builds, we truncate the most important part of the job name. Putting the most important part first is robust against truncation, and we can drop the redundant "-cross" stub. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <skultety.erik@gmail.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 21:34:45 +03:00
i686-debian-sid:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-sid
CROSS: i686
tests: perform cross compiler builds on GitLab CI GitLab CI provides some shared build runners that use Docker containers. This resource can usefully run cross-compiled builds since all other CI build testing is currently x86 only, and Travis CI is already very busy testing native builds. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-25 20:38:53 +03:00
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
ppc64le-debian-sid:
gitlab: reduce number of cross build jobs run by default Currently we have nine different cross build jobs, but as we introduce more native jobs this is going to result in a very long CI execution time. For developers testing their personal branches under development it is generally sufficient to just look at a couple of interesting scenarios, namely 32-bit and big endian. This splits the cross build jobs so that by default only the armv7 and s390x archs are built. The remainining archs are setup so that they are only built for code on the master branch, which will have the effect of doing post-merge testing. Developers can opt-in to full testing of their pre-merge code by pushing it to a branch with a name prefix of "ci-full-". Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-25 16:14:57 +03:00
<<: *cross_build_extra_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: debian-sid
ci: Swap mipsel and ppc64le builds Debian sid is currently broken on mipsel, so use Debian 10 for that architecture; at the same time, move the ppc64le build from Debian 10 to Debian sid to keep things balanced. Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 21:06:02 +03:00
CROSS: ppc64le
gitlab: use CI for building website contents Run the bare minimum build that is possible to create the docs, avoiding compiling code which other jobs will deal with. The generated website is published as an artifact and thus is browsable by developers on build completion and can be downloaded as a zip file. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 17:56:42 +03:00
gitlab: update list of distros to use latest versions Replace Fedora 30 with Fedora 32 Replace Ubuntu 16.04 with Ubuntu 20.04 Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 15:06:56 +03:00
mingw32-fedora-rawhide:
gitlab: add mingw cross build CI jobs This pulls in the mingw cross build jobs using Fedora 30 as a base, matching what is done on Jenkins and Travis. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:23:27 +03:00
<<: *cross_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: fedora-rawhide
CROSS: mingw32
gitlab: add mingw cross build CI jobs This pulls in the mingw cross build jobs using Fedora 30 as a base, matching what is done on Jenkins and Travis. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:23:27 +03:00
gitlab: update list of distros to use latest versions Replace Fedora 30 with Fedora 32 Replace Ubuntu 16.04 with Ubuntu 20.04 Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 15:06:56 +03:00
mingw64-fedora-rawhide:
gitlab: add mingw cross build CI jobs This pulls in the mingw cross build jobs using Fedora 30 as a base, matching what is done on Jenkins and Travis. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:23:27 +03:00
<<: *cross_build_default_job_definition
ci: Use variables to build image names This removes a lot of repetition and makes the configuration much easier to read. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:55 +03:00
variables:
NAME: fedora-rawhide
CROSS: mingw64
gitlab: add mingw cross build CI jobs This pulls in the mingw cross build jobs using Fedora 30 as a base, matching what is done on Jenkins and Travis. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:23:27 +03:00
gitlab: use CI for building website contents Run the bare minimum build that is possible to create the docs, avoiding compiling code which other jobs will deal with. The generated website is published as an artifact and thus is browsable by developers on build completion and can be downloaded as a zip file. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 17:56:42 +03:00
# This artifact published by this job is downloaded by libvirt.org to
# be deployed to the web root:
# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website
website:
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: builds
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
before_script:
- *script_variables
gitlab: use CI for building website contents Run the bare minimum build that is possible to create the docs, avoiding compiling code which other jobs will deal with. The generated website is published as an artifact and thus is browsable by developers on build completion and can be downloaded as a zip file. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 17:56:42 +03:00
script:
- mkdir build
- cd build
- ../autogen.sh --prefix=$(pwd)/../vroot || (cat config.log && exit 1)
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- $MAKE -C docs
- $MAKE -C docs install
gitlab: use CI for building website contents Run the bare minimum build that is possible to create the docs, avoiding compiling code which other jobs will deal with. The generated website is published as an artifact and thus is browsable by developers on build completion and can be downloaded as a zip file. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-06 17:56:42 +03:00
- cd ..
- mv vroot/share/doc/libvirt/html/ website
artifacts:
expose_as: 'Website'
name: 'website'
when: on_success
expire_in: 30 days
paths:
- website
gitlab: add job for building latest potfile Whenever there is a change to the translatable strings we need to push a new libvirt.pot to weblate. This only needs to be done when code merges into git master, so the job is restricted to that branch. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:42:14 +03:00
gitlab: add explicit early job for syntax-check Running the code style syntax-check as part of the build jobs leads to all jobs failing in the same way. Have a prebuild job for validating syntax-check to catch code style problems upfront and thus avoid needing to run all the build jobs. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>a Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-26 15:07:34 +03:00
codestyle:
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: builds
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
before_script:
- *script_variables
gitlab: add explicit early job for syntax-check Running the code style syntax-check as part of the build jobs leads to all jobs failing in the same way. Have a prebuild job for validating syntax-check to catch code style problems upfront and thus avoid needing to run all the build jobs. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>a Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-26 15:07:34 +03:00
script:
- mkdir build
- cd build
- ../autogen.sh || (cat config.log && exit 1)
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- $MAKE syntax-check
gitlab: add explicit early job for syntax-check Running the code style syntax-check as part of the build jobs leads to all jobs failing in the same way. Have a prebuild job for validating syntax-check to catch code style problems upfront and thus avoid needing to run all the build jobs. Reviewed-by: Erik Skultety <eskultet@redhat.com> Reviewed-by: Andrea Bolognani <abologna@redhat.com>a Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-26 15:07:34 +03:00
gitlab: add job for building latest potfile Whenever there is a change to the translatable strings we need to push a new libvirt.pot to weblate. This only needs to be done when code merges into git master, so the job is restricted to that branch. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:42:14 +03:00
# This artifact published by this job is downloaded to push to Weblate
# for translation usage:
# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potfile
potfile:
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: builds
ci: Use GitLab container registry Instead of using pre-built containers hosted on Quay, build containers as part of the GitLab CI pipeline and upload them to the GitLab container registry for later use. This will not significantly slow down builds, because containers are only rebuilt when the corresponding Dockerfile has been modified. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-02 18:28:57 +03:00
image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
gitlab: add job for building latest potfile Whenever there is a change to the translatable strings we need to push a new libvirt.pot to weblate. This only needs to be done when code merges into git master, so the job is restricted to that branch. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:42:14 +03:00
only:
- master
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
before_script:
- *script_variables
gitlab: add job for building latest potfile Whenever there is a change to the translatable strings we need to push a new libvirt.pot to weblate. This only needs to be done when code merges into git master, so the job is restricted to that branch. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:42:14 +03:00
script:
- mkdir build
- cd build
- ../autogen.sh || (cat config.log && exit 1)
gitlab: Add script_variables and define $MAKEFLAGS there This new template provides a standardized place where environment variables which are not static, but rather depend on the output of some shell command, can be defined for later use. This pattern is already used in libosinfo's GitLab CI integration. Defining $MAKEFLAGS there means we don't need to call getconf over and over, and the actual build steps don't end up drowned in the noise. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-27 18:40:05 +03:00
- $MAKE -C src generated-sources
- $MAKE -C po libvirt.pot
gitlab: add job for building latest potfile Whenever there is a change to the translatable strings we need to push a new libvirt.pot to weblate. This only needs to be done when code merges into git master, so the job is restricted to that branch. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:42:14 +03:00
- cd ..
gitlab: fix potfile job for new location Since we're storing the libvirt.pot file in git now, we should pull from the source dir, not the build dir. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-04 16:26:39 +03:00
- cp po/libvirt.pot libvirt.pot
gitlab: add job for building latest potfile Whenever there is a change to the translatable strings we need to push a new libvirt.pot to weblate. This only needs to be done when code merges into git master, so the job is restricted to that branch. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Erik Skultety <eskultet@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-24 17:42:14 +03:00
artifacts:
expose_as: 'Potfile'
name: 'potfile'
when: on_success
expire_in: 30 days
paths:
- libvirt.pot
gitlab: introduce a check for validate DCO sign-off This introduces a CI job for validating DCO sign-off in every commit message. The CI jobs are not provided any information on what the baseline commit for the branch was. We can't compare against the forked repo's master branch, as there's no guarantee the user is keeping master up2date in their fork. Thus we add the master upstream repo as a git remote and identify the common ancestor. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-26 13:46:47 +03:00
gitlab: convert to shared container for DCO check This removes the locally maintained DCO checking script in favour of the shared container image provided by libvirt-ci.git. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-01 13:57:33 +03:00
# Check that all commits are signed-off for the DCO.
# Skip on "libvirt" namespace, since we only need to run
# this test on developer's personal forks from which
# merge requests are submitted
check-dco:
ci: Reduce number of stages Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 13:11:30 +03:00
stage: sanity_checks
gitlab: convert to shared container for DCO check This removes the locally maintained DCO checking script in favour of the shared container image provided by libvirt-ci.git. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-01 13:57:33 +03:00
image: registry.gitlab.com/libvirt/libvirt-ci/check-dco:master
gitlab: introduce a check for validate DCO sign-off This introduces a CI job for validating DCO sign-off in every commit message. The CI jobs are not provided any information on what the baseline commit for the branch was. We can't compare against the forked repo's master branch, as there's no guarantee the user is keeping master up2date in their fork. Thus we add the master upstream repo as a git remote and identify the common ancestor. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-26 13:46:47 +03:00
script:
gitlab: convert to shared container for DCO check This removes the locally maintained DCO checking script in favour of the shared container image provided by libvirt-ci.git. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-01 13:57:33 +03:00
- /check-dco
gitlab: introduce a check for validate DCO sign-off This introduces a CI job for validating DCO sign-off in every commit message. The CI jobs are not provided any information on what the baseline commit for the branch was. We can't compare against the forked repo's master branch, as there's no guarantee the user is keeping master up2date in their fork. Thus we add the master upstream repo as a git remote and identify the common ancestor. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-26 13:46:47 +03:00
except:
gitlab: convert to shared container for DCO check This removes the locally maintained DCO checking script in favour of the shared container image provided by libvirt-ci.git. Reviewed-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-01 13:57:33 +03:00
variables:
- $CI_PROJECT_NAMESPACE == 'libvirt'
Copy Permalink