Tests/packaging: Remove CentOS/RHEL 7 specific content

Signed-off-by: Till Maas <opensource@till.name>
This commit is contained in:
Till Maas 2019-11-12 11:53:54 +01:00
parent 3af79606c2
commit a85839f3f5
18 changed files with 18 additions and 415 deletions

View File

@ -4,8 +4,6 @@ env:
matrix:
- DOCKER_IMAGE=nmstate/fedora-nmstate-dev
testflags="--test-type integ --pytest-args='-x'"
- DOCKER_IMAGE=nmstate/centos7-nmstate-dev
testflags="--test-type integ --pytest-args='-x'"
- DOCKER_IMAGE=nmstate/fedora-nmstate-dev
testflags="--test-type integ --pytest-args='-x'
--copr networkmanager/NetworkManager-master"
@ -25,8 +23,6 @@ env:
matrix:
allow_failures:
- env: DOCKER_IMAGE=nmstate/centos7-nmstate-dev
testflags="--test-type integ --pytest-args='-x'"
- env: DOCKER_IMAGE=nmstate/fedora-nmstate-dev
testflags="--test-type integ --pytest-args='-x'
--copr networkmanager/NetworkManager-master"

View File

@ -93,53 +93,11 @@ Alternatively, install Nmstate system-wide:
pip uninstall -y nmstate; pip install .
```
### Container Image
Nmstate also provides a container image based on CentOS 7 to try it:
```shell
CONTAINER_ID=$(sudo docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro nmstate/centos7-nmstate)
sudo docker exec -ti "${CONTAINER_ID}" /bin/bash
# now play with nmstatectl in the container
nmstatectl show
# remove the container at the end
sudo docker stop "${CONTAINER_ID}"
sudo docker rm "${CONTAINER_ID}"
```
## Development Environment
Nmstate uses `tox` to run unit tests and linters. Since Nmstate uses the binary
module PyGObject it also requires the build dependencies for it.
### RHEL 7.6
Recommended minimum installation:
```shell
yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm # install EPEL for python-pip
subscription-manager repos --enable "rhel-*-optional-rpms" --enable "rhel-*-extras-rpms" # recommended for EPEL
yum install git python-pip
yum-builddep python-gobject # install build dependencies for PyGObject
pip install tox # python-tox in EPEL seems to be too old
```
### CentOS 7.6
Recommended minimum installation:
```shell
yum -y install epel-release
yum -y install \
NetworkManager \
NetworkManager-libnm \
git \
python-pip
yum-builddep python-gobject # install build dependencies for PyGObject
pip install tox # python-tox in EPEL seems to be too old
```
Note: This will not run the unit tests for Python 3.6 because this Python version is not available there.
### Unit tests
Run Unit Tests:
```shell

View File

@ -9,7 +9,6 @@ A declarative network manager API for hosts.
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/nmstate/nmstate.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/nmstate/nmstate/context:python)
Copr build status:
* EPEL 7 GIT master: [![EPEL7 GIT master Copr build status](https://copr.fedorainfracloud.org/coprs/nmstate/nmstate-git-el7/package/nmstate/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/nmstate/nmstate-git-el7/package/nmstate/)
* Fedora GIT master: [![Fedora GIT master Copr build status](https://copr.fedorainfracloud.org/coprs/nmstate/nmstate-git-fedora/package/nmstate/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/nmstate/nmstate-git-fedora/package/nmstate/)
## What is it?

View File

@ -3,17 +3,9 @@ The automation env is serving the tests of Nmstate.
It may be used both locally and through CI.
## Components
- Dockerfile: Defines a Centos 7 based container image which includes systemd,
NetworkManager and other basic build tools (e.g. tox).
The image can be found at:
https://hub.docker.com/r/nmstate/centos7-nmstate-dev/
- Dockerfile.fedora: Defines a Fedora based container image for Nmstate test
purpose.
The image can be found at:
https://hub.docker.com/r/nmstate/fedora-nmstate-dev/
- Container specifications to be used for the tests are in the `packaging`
directory. The images are published on docker hub:
https://hub.docker.com/r/nmstate/
- run-tests.sh: Execute the tests in a container using
'nmstate/fedora-nmstate-dev' docker image.
@ -55,8 +47,8 @@ You may change the test type by using:
* `./automation/run-tests.sh --test-type unit_py37`:
Unit tests in Python 3.7 using 'nmstate/fedora-nmstate-dev' docker image.
* `./automation/run-tests.sh --test-type integ --el7`:
Integration tests using 'nmstate/centos7-nmstate-dev' docker image.
* `./automation/run-tests.sh --test-type integ --el8`:
Integration tests using 'nmstate/centos8-nmstate-dev' docker image.
* `./automation/run-tests.sh --test-type integ`:
Integration tests using 'nmstate/fedora-nmstate-dev' docker image.
@ -77,7 +69,7 @@ possible to provide the `--debug-shell` command-line option.
To specify a different container image for the tests, specify it with the
`DOCKER_IMAGE` variable:
`DOCKER_IMAGE=local/centos7-nmstate-dev debug_exit_shell=1 ./automation/run-tests.sh`
`DOCKER_IMAGE=local/centos8-nmstate-dev debug_exit_shell=1 ./automation/run-tests.sh`
It is also possible to pass extra arguments to PDB using the
`nmstate_pytest_extra_args` variable or via `--pytest-args` command-line
@ -112,14 +104,14 @@ tox -e check-integ-py27
### Build a new container image
```
sudo ../packaging/build-container.sh local/centos7-nmstate-dev
sudo ../packaging/build-container.sh local/centos8-nmstate-dev
sudo ../packaging/build-container.sh local/fedora-nmstate-dev
```
To test the image, either specify it manually as described above or tag it locally:
```
sudo docker tag local/centos7-nmstate-dev nmstate/centos7-nmstate-dev:latest
sudo docker tag local/centos8-nmstate-dev nmstate/centos8-nmstate-dev:latest
sudo docker tag local/fedora-nmstate-dev nmstate/fedora-nmstate-dev:latest
```
@ -131,8 +123,8 @@ persistent. If this is not feasible, a new build could be pushed as follow to
the Docker Hub:
```shell
sudo docker tag local/centos7-nmstate-dev nmstate/centos7-nmstate-dev:latest
sudo docker push nmstate/centos7-nmstate-dev:latest
sudo docker tag local/centos8-nmstate-dev nmstate/centos8-nmstate-dev:latest
sudo docker push nmstate/centos8-nmstate-dev:latest
sudo docker tag local/fedora-nmstate-dev nmstate/fedora-nmstate-dev:latest
sudo docker push nmstate/fedora-nmstate-dev:latest

View File

@ -1,20 +0,0 @@
#!/bin/bash
PROJECT=${PROJECT:-${PWD##*/}}
TOXWORKDIR="$HOME/.cache/.tox-$PROJECT"
CONTAINER_IMAGE="localhost/$PROJECT/centos7-$PROJECT-utest"
mkdir -p "$TOXWORKDIR"
podman run \
--rm \
-ti \
-v $PWD:/workspace/$PROJECT:Z \
-v $TOXWORKDIR:$TOXWORKDIR:Z \
$CONTAINER_IMAGE \
/bin/bash -c " \
cp -rf /workspace/$PROJECT /tmp/ \
&& \
cd /tmp/$PROJECT \
&& \
tox -e black,flake8,pylint,check-py27 --workdir $TOXWORKDIR"

View File

@ -19,7 +19,7 @@ TEST_TYPE_UNIT_PY37="unit_py37"
TEST_TYPE_INTEG="integ"
FEDORA_IMAGE_DEV="nmstate/fedora-nmstate-dev"
CENTOS_IMAGE_DEV="nmstate/centos7-nmstate-dev"
CENTOS_IMAGE_DEV="nmstate/centos8-nmstate-dev"
test -t 1 && USE_TTY="-t"
@ -213,7 +213,7 @@ function modprobe_ovs {
}
options=$(getopt --options "" \
--long customize:,pytest-args:,help,debug-shell,test-type:,el7,copr:,artifacts-dir:\
--long customize:,pytest-args:,help,debug-shell,test-type:,el8,copr:,artifacts-dir:\
-- "${@}")
eval set -- "$options"
while true; do
@ -237,7 +237,7 @@ while true; do
shift
TEST_TYPE="$1"
;;
--el7)
--el8)
DOCKER_IMAGE=$CENTOS_IMAGE_DEV
;;
--artifacts-dir)
@ -246,7 +246,7 @@ while true; do
;;
--help)
set +x
echo -n "$0 [--copr=...] [--customize=...] [--debug-shell] [--el7] "
echo -n "$0 [--copr=...] [--customize=...] [--debug-shell] [--el8] "
echo -n "[--help] [--pytest-args=...] "
echo "[--test-type=<TEST_TYPE>]"
echo " Valid TEST_TYPE are:"

View File

@ -1,20 +0,0 @@
FROM nmstate/centos7-nmstate-base
ARG SOURCE_COMMIT
# dbus-python is special and does not provide egg files, therefore it is not
# used by pip and needs to be compiled from PyPi. For this the build
# dependencies are needed.
# https://bugzilla.redhat.com/show_bug.cgi?id=1654774
RUN yum -y install \
python2-pip \
git \
&& \
yum-builddep -y dbus-python && \
yum clean all
RUN pip install git+https://github.com/nmstate/nmstate@${SOURCE_COMMIT:-master}
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]

View File

@ -1,31 +0,0 @@
# This Dockerfile is based on the recommendations provided in the
# Centos official repository (https://hub.docker.com/_/centos/).
# It enables systemd to be operational.
FROM centos:7.5.1804
ENV container docker
COPY docker_enable_systemd.sh docker_sys_config.sh ./
RUN bash ./docker_enable_systemd.sh && rm ./docker_enable_systemd.sh -f
RUN yum -y install yum-plugin-copr && \
yum -y copr enable nmstate/ovs-el7 && \
yum -y upgrade && \
yum -y install \
NetworkManager \
NetworkManager-libnm \
NetworkManager-ovs \
openvswitch \
&& \
yum -y install epel-release && \
yum -y install \
dbus-python \
python2-pyyaml \
python2-six \
python-gobject-base \
python-jsonschema \
python-setuptools \
&& \
yum clean all && \
bash ./docker_sys_config.sh && rm ./docker_sys_config.sh -f

View File

@ -1,22 +0,0 @@
FROM nmstate/centos7-nmstate-base
RUN yum -y install \
dnsmasq \
git \
iproute \
python2-devel \
python2-pip \
python36-pip \
rpm-build \
dhclient \
&& yum clean all
RUN pip install --upgrade pip pytest==4.6.6 pytest-cov==2.8.1
RUN pip3 install --user python-coveralls
RUN echo 'PATH="${HOME}/.local/bin:${PATH}"' >> /root/.bashrc
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]

View File

@ -1,27 +0,0 @@
# This image definition is defining
# a container image on which unit and lint tests
# can be run locally through `tox`.
FROM centos:7
RUN yum -y upgrade \
&& \
yum -y install \
NetworkManager \
NetworkManager-libnm \
&& \
yum -y install epel-release \
&& \
yum -y install \
python36 \
python2-pip \
python36-pip \
make \
&& \
yum-builddep -y \
python-gobject \
dbus-python \
&& \
yum clean all \
&& \
pip install tox

View File

@ -2,15 +2,12 @@
The images are automatically rebuilt on new GIT tags or pushes to the master branch:
`Dockerfile.centos7-nmstate` by https://cloud.docker.com/u/nmstate/repository/docker/nmstate/centos7-nmstate
`Dockerfile.centos7-nmstate-base` by https://cloud.docker.com/u/nmstate/repository/docker/nmstate/centos7-nmstate-base
`Dockerfile.centos7-nmstate-dev` by https://cloud.docker.com/u/nmstate/repository/docker/nmstate/centos7-nmstate-dev
`Dockerfile.fedora-nmstate-dev` by https://cloud.docker.com/u/nmstate/repository/docker/nmstate/fedora-nmstate-dev
The base image contains a common base that is used both for the development
image and for the distributed image.
Configuration (here for `centos7-nmstate-base`, the other build just specifies
Configuration (here for `fedora-nmstate-dev`, the other build just specifies
a different container spec (Dockerfile location):
Source repo: nmstate/nmstate
@ -20,7 +17,7 @@ Build rules:
Branch:
Source:master
Docker Tag:latest
Dockerfile location:Dockerfile.centos7-nmstate-base
Dockerfile location:Dockerfile.fedora-nmstate-dev
Build Context:/packaging
Autobuild:on
Build Caching:off
@ -28,7 +25,7 @@ Build Caching:off
Tag:
Source: /^v[0-9.]+$/
Docker Tag:{sourceref}
Dockerfile location:Dockerfile.centos7-nmstate-base
Dockerfile location:Dockerfile.fedora-nmstate-dev
Build Context:/packaging
Autobuild:on
Build Caching:off
@ -39,6 +36,6 @@ The Nmstate user image builds the master master branch by default. To specify a
different commit or tag, specify the `SOURCE_COMMIT` build argument:
```shell
./build-container.sh --extra-args "--build-arg SOURCE_COMMIT=v0.0.6" nmstate/centos7-nmstate
./build-container.sh --extra-args "--build-arg SOURCE_COMMIT=v0.0.6" nmstate/fedora-nmstate-dev
```

View File

@ -75,9 +75,6 @@ do
if [[ "${container_name}" == "all" ]]
then
for container_name in \
centos7-nmstate-base \
centos7-nmstate-dev \
centos7-nmstate \
fedora-nmstate-dev \
centos8-nmstate-dev
do

View File

@ -1,60 +0,0 @@
%define srcname nmstate
%define libname libnmstate
Name: nmstate
Version: @VERSION@
Release: @RELEASE@%{?dist}
Summary: Declarative network manager API
License: LGPLv2.1+
URL: https://github.com/%{srcname}/%{srcname}
Source0: https://github.com/%{srcname}/%{srcname}/archive/v%{version}/%{srcname}-%{version}.tar
BuildArch: noarch
BuildRequires: python2-devel
BuildRequires: python2-setuptools
Requires: python2-setuptools
Requires: python2-%{libname} = %{?epoch:%{epoch}:}%{version}-%{release}
%description
Nmstate is a library with an accompanying command line tool that manages host
networking settings in a declarative manner and aimed to satisfy enterprise
needs to manage host networking through a northbound declarative API and multi
provider support on the southbound.
%package -n python2-%{libname}
Summary: nmstate Python 2 API library
Requires: NetworkManager-libnm >= 1:1.12
Requires: python-gobject-base
Requires: python2-six
Requires: python-jsonschema
Requires: python2-pyyaml
Requires: dbus-python
%description -n python2-%{libname}
This package contains the Python 2 library for Nmstate.
%prep
%setup -q
sed -i -e '/^dbus-python$/d' requirements.txt
%build
%py2_build
%install
%py2_install
%files
%doc README.md
%doc examples/
%{_mandir}/man8/nmstatectl.8*
%{python2_sitelib}/nmstatectl
%{_bindir}/nmstatectl
%files -n python2-%{libname}
%license LICENSE
%{python2_sitelib}/%{libname}
%{python2_sitelib}/%{srcname}-*.egg-info/
%changelog
@CHANGELOG@
- snapshot build

View File

@ -1,22 +0,0 @@
# Integration Tests #
The integration tests are written as Ansible playbooks targetted for all hosts.
They follow the [Fedora's Standard Test
Interface](https://fedoraproject.org/wiki/CI/Standard_Test_Interface) and can
be run by the [Linux Sytem Roles Test
Harness](https://github.com/linux-system-roles/test-harness). You can also run
them directly, for example to run them against the [CentOS Cloud
Images](https://cloud.centos.org/centos/7/images/) download one of them,
install `standard-test-roles` from Fedora and execute `ansible-playbook`:
```
dnf install standard-test-roles
TEST_SUBJECTS=CentOS-7-x86_64-GenericCloud.qcow2c TEST_ARTIFACTS=$PWD ansible-playbook -i /usr/share/ansible/inventory/standard-inventory-qcow2 tests_*.yml
```
The tests are defined in Ansible playbooks named `tests_*.yml` that must run
sucessfully when the tests passes or skips and fail otherwise. The `playbooks/`
directory contains helper playbooks for the tests and `ci_*.yml` playbooks are
used to workaround a deficit in Ansible to skip tests when they do not work on
a certain hosts (e.g. hosts with a distribution that is missing Network
Manager).

View File

@ -1,88 +0,0 @@
---
- name: empty play to gather facts
hosts: all
- import_playbook: playbooks/rhel7_deps.yml
when:
- ansible_distribution == "RedHat"
- ansible_distribution_major_version == '7'
- import_playbook: playbooks/centos7_deps.yml
when:
- ansible_distribution == "CentOS"
- ansible_distribution_major_version == '7'
- name: Run tox tests and nmstatectl show
hosts: all
vars:
- nmstatedir: /tmp/nmstate-git-checkout
tasks:
- service:
state: started
enabled: yes
name: NetworkManager
- file:
state: directory
path: "{{ nmstatedir }}"
# TODO: using tar and copying the file is a workaround for the synchronize
# module being broken in Fedora 28 at the moment
#
- shell: 'tar -cvf ../nmstate.tar --exclude "*.pyc" --exclude "__pycache__" --exclude nmstate.tar ../'
delegate_to: localhost
- copy:
src: ../nmstate.tar
dest: "{{ nmstatedir }}"
- command: tar xf nmstate.tar
args:
chdir: "{{ nmstatedir }}"
# END workaround
- synchronize:
src: ../../
dest: "{{ nmstatedir }}"
recursive: yes
delete: yes
# tox reqires it to be a .git directory, so cannot exclude .git
rsync_opts:
- "--exclude=.pyc"
- "--exclude=__pycache__"
# TODO: Enable this when the module works again and remove the tar workaround
when: false
- command: git clean -fX
args:
chdir: "{{ nmstatedir }}"
- pip:
name: tox
extra_args: --user
- command: ~/.local/bin/tox
args:
chdir: "{{ nmstatedir }}"
# TODO: this needs https://github.com/ansible/ansible/pull/40793 to be
# merged, therefore we run pip via the command module
- pip:
name: .
extra_args: --user --upgrade
chdir: "{{ nmstatedir }}"
when: false
# TODO: This is a workaround for the broken pip module
- command: pip install --user --upgrade .
args:
chdir: "{{ nmstatedir }}"
- command: ~/.local/bin/nmstatectl show
register: nmstatectl_show
- debug:
var: nmstatectl_show.stdout_lines
- assert:
that: '''"name": "lo"'' in nmstatectl_show.stdout'

View File

@ -1,18 +0,0 @@
---
- name: Install CentOS 7 deps for nmstate tests
hosts: all
vars:
- rpmdependencies:
- git
- python2-pip
- NetworkManager
tasks:
- name: install EPEL
package:
name: epel-release
state: present
- name: install rpm dependencies
package:
state: present
name: "{{ rpmdependencies }}"

View File

@ -1,16 +0,0 @@
---
- name: Install RHEL 7 deps for nmstate tests
hosts: all
vars:
- rpmdependencies:
- git
- python2-pip
tasks:
- name: enable extras and optional
command: subscription-manager repos --enable "rhel-*-optional-rpms" --enable "rhel-*-extras-rpms"
- name: install EPEL
shell: rpm -q epel-release || yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
- name: install rpm dependencies
package:
state: present
name: "{{ rpmdependencies }}"

View File

@ -1,12 +0,0 @@
---
# empty playbook to gather facts for import_playbook when clause
- hosts: all
# workaround for: https://github.com/ansible/ansible/issues/27973
# There is no way in Ansible to abort a playbook for host that do not run RHEL
# 7 or CentOS 7 without failing the playbook. Therefore we include the playbook
# with the tests only if the hosts would support it. There will need to be a
# way to skip RHEL 6 hosts from running the tests since there is no Network
# Manager
- import_playbook: ci_tests_basic.yml
when: ansible_distribution_major_version == '7'