2019-04-11 17:49:00 +03:00
The requirements.txt and requirements_ansible.txt files are generated from requirements.in and requirements_ansible.in, respectively, using `pip-tools` `pip-compile` . The following commands should do this if ran inside the tools_awx container.
2016-12-14 17:53:24 +03:00
2019-04-11 17:49:00 +03:00
Run these commands from the root of the awx repo. This will produce python 3 requirements files.
Dependency Updates
* Dynamic Inventory Source
Template against ansible 2.3 dynamic inventory sources.
The major change is removal of `rax.py`. Most upstream scripts except
`foreman.py` has quite trivial coding style changes, or minor functional
extensions that does not affect Tower inventory update runs.
`foreman.py`, on the other hand, went through quite a major refactoring,
but functionalities stay the same.
Major python dependency updates include apache-libcloud (1.3.0 -->
2.0.0), boto (2.45.0 --> 2.46.1) and shade (1.19.0 --> 1.20.0). Minor
python dependency updates include indirect updates via `pip-compile`,
which are determined by base dependencies.
Some minor `task.py` extensions:
- `.ini` file for ec2 has one more field `stack_filter=False`, which
reveals changes in `ec2.py`.
- `.ini` file for cloudforms will catch these four options from
`source_vars_dict` of inventory update: `'version', 'purge_actions',
'clean_group_keys', 'nest_tags'`. These four options have always been
available in `cloudforms.py` but `cloudforms.ini.example` has not
mentioned them until the latest version. For consistency with upstream
docs, we should make these fields available for tower user to customize.
- YAML file of openstack will catch ansible options `use_hostnames`,
`expand_hostvars` and `fail_on_errors` from `source_vars_dict` of
inventory update as a response to issue #6075.
* Remove Rackspace support
Supports of Rackspace as both a dynamic inventory source and a cloud
credential are fully removed. Data migrations have been added to support
arbitrary credential types feature and delete rackspace inventory
sources.
Note also requirement `jsonschema` has been moved from
`requirements.txt` to `requirements.in` as a primary dependency to
reflect it's usage in `/main/fields.py`.
Connected issue: #6080.
* `pexpect` major update
`pexpect` stands at the very core of our task system and underwent a
major update from 3.1 to 4.2.1. Although verified during devel, please
still be mindful of any suspicious issues on celery side even after this
PR gets merged.
* Miscellaneous
- requests now explicitly declared in `requirements.in` at version 2.11.1
in response to upstream issue
- celery: 3.1.17 -> 3.1.25
- django-extensions: 1.7.4 -> 1.7.8
- django-polymorphic: 0.7.2 -> 1.2
- django-split-settings: 0.2.2 -> 0.2.5
- django-taggit: 0.21.3 -> 0.22.1
- irc: 15.0.4 -> 15.1.1
- pygerduty: 0.35.1 -> 0.35.2
- pyOpenSSL: 16.2.0 -> 17.0.0
- python-saml: 2.2.0 -> 2.2.1
- redbaron: 0.6.2 -> 0.6.3
- slackclient: 1.0.2 -> 1.0.5
- tacacs_plus: 0.1 -> 0.2
- xmltodict: 0.10.2 -> 0.11.0
- pip: 8.1.2 -> 9.0.1
- setuptools: 23.0.0 -> 35.0.2
- (requirements_ansible.in only)kombu: 3.0.35 -> 3.0.37
2017-04-20 23:47:53 +03:00
2016-12-14 17:53:24 +03:00
```
2019-04-11 17:49:00 +03:00
python3 -m venv /buildit
2016-12-14 17:53:24 +03:00
source /buildit/bin/activate
pip install pip-tools
pip install pip --upgrade
2019-04-11 17:49:00 +03:00
pip-compile -U -r --allow-unsafe --output-file requirements/requirements.txt requirements/requirements.in
pip-compile -U -r --allow-unsafe --output-file requirements/requirements_ansible_py3.txt requirements/requirements_ansible.in
2016-12-14 17:53:24 +03:00
```
2019-05-09 23:34:37 +03:00
Remove the `docutils` line from `requirements/requirements.txt` .
2019-04-11 17:49:00 +03:00
The Ansible venv requirements file needs to start with the python 2 version
as a base. Then we can run the tool again to get the python 3 version.
Consult the output of the `diff` command and add a conditional switch in those cases.
```
virtualenv -p python2 /buildit_py2
source /buildit_py2/bin/activate
pip install pip-tools
pip install pip --upgrade
pip-compile -U -r --allow-unsafe --output-file requirements/requirements_ansible.txt requirements/requirements_ansible.in
diff requirements/requirements_ansible_py3.txt requirements/requirements_ansible.txt
rm requirements/requirements_ansible_py3.txt
```
2016-12-14 17:53:24 +03:00
2019-04-11 17:49:00 +03:00
Python 3 exceptions should be added to relevant `requirements_ansible.txt` lines
after version numbers with the syntax of `; python_version < '3'` .
## Licenses and Source Files
If any library has a change to its license with the upgrade, then the license for that library
inside of `docs/licenses` needs to be updated.
For libraries that have source distribution requirements (LGPL as an example),
a tarball of the library is kept along with the license.
To download the PyPI tarball, you can run this command:
```
pip download < pypi library name > -d docs/licenses/ --no-binary :all: --no-deps
```
2016-12-14 17:53:24 +03:00
2019-04-11 17:49:00 +03:00
Make sure to delete the old tarball if it is an upgrade.