awx/requirements/README.md

The requirements.txt and requirements_ansible.txt files are generated from requirements.in and requirements_ansible.in, respectively, using `pip-tools` `pip-compile`. The following commands should do this if ran inside the tower_tools container.

NOTE: before running `pip-compile`, please copy-paste contents in `requirements/requirements_git.txt` to the top of `requirements/requirements.in` and prepend each copied line with `-e `. Later after `requirements.txt` is generated, don't forget to remove all `git+https://github.com...`-like lines from both `requirements.txt` and `requirements.in` (repeat for `requirements_ansible_git.txt` and `requirements_ansible.in`)

At the end of `requirements/requirements.in`, pip and setuptools need to have their versions pinned.

```
virtualenv /buildit
source /buildit/bin/activate
pip install pip-tools
pip install pip --upgrade

pip-compile requirements/requirements.in > requirements/requirements.txt
pip-compile requirements/requirements_ansible.in > requirements/requirements_ansible.txt
```

## Known Issues

* Remove the `-e` from packages of the form `-e git+https://github.com...` in the generated `.txt`. Failure to do so will result in a "bad" RPM and DEB due to the `pip install` laying down a symbolic link with an absolute path from the virtualenv to the git repository that will differ from when the RPM and DEB are build to when the RPM and DEB are installed on a machine. By removing the `-e` the symbolic egg link will not be created and all is well.

* As of `pip-tools` `1.8.1` `pip-compile` does not resolve packages specified using a git url. Thus, dependencies for things like `dm.xmlsec.binding` do not get resolved and output to `requirements.txt`. This means that:
  * can't use `pip install --no-deps` because other deps WILL be sucked in
  * all dependencies are NOT captured in our `.txt` files. This means you can't rely on the `.txt` when gathering licenses.
general update of requirements 2016-12-16 21:13:52 +03:00			The requirements.txt and requirements_ansible.txt files are generated from requirements.in and requirements_ansible.in, respectively, using `pip-tools` `pip-compile`. The following commands should do this if ran inside the tower_tools container.
uninstall certifi if installed in venv's 2016-12-14 17:53:24 +03:00
Dependency Updates Upgrades of minor dependency upgrades Inventory scripts were upgraded in separate commit Major exclusions from this update - celery was already downgraded for other reasons - Django / DRF major update already done, minor bumps here - asgi-amqp has fixes coming independently, not touched - TACACS plus added features not needed Removals of note - remove shade from AWX requirements - remove kombu from Ansible requirements Other notes Add note about pinning setuptools and pip, done but not mentioned previously Stop pinning gevent-websocket and twisted upgrade Azure to Ansible core requirements more detailed notes https://gist.github.com/AlanCoding/9442a512ab6977940bc7b5b346d4f70b upgrade version of Django for Exception 2018-03-01 19:17:46 +03:00			NOTE: before running `pip-compile`, please copy-paste contents in `requirements/requirements_git.txt` to the top of `requirements/requirements.in` and prepend each copied line with `-e `. Later after `requirements.txt` is generated, don't forget to remove all `git+https://github.com...`-like lines from both `requirements.txt` and `requirements.in` (repeat for `requirements_ansible_git.txt` and `requirements_ansible.in`)

			At the end of `requirements/requirements.in`, pip and setuptools need to have their versions pinned.
Dependency Updates * Dynamic Inventory Source Template against ansible 2.3 dynamic inventory sources. The major change is removal of `rax.py`. Most upstream scripts except `foreman.py` has quite trivial coding style changes, or minor functional extensions that does not affect Tower inventory update runs. `foreman.py`, on the other hand, went through quite a major refactoring, but functionalities stay the same. Major python dependency updates include apache-libcloud (1.3.0 --> 2.0.0), boto (2.45.0 --> 2.46.1) and shade (1.19.0 --> 1.20.0). Minor python dependency updates include indirect updates via `pip-compile`, which are determined by base dependencies. Some minor `task.py` extensions: - `.ini` file for ec2 has one more field `stack_filter=False`, which reveals changes in `ec2.py`. - `.ini` file for cloudforms will catch these four options from `source_vars_dict` of inventory update: `'version', 'purge_actions', 'clean_group_keys', 'nest_tags'`. These four options have always been available in `cloudforms.py` but `cloudforms.ini.example` has not mentioned them until the latest version. For consistency with upstream docs, we should make these fields available for tower user to customize. - YAML file of openstack will catch ansible options `use_hostnames`, `expand_hostvars` and `fail_on_errors` from `source_vars_dict` of inventory update as a response to issue #6075. * Remove Rackspace support Supports of Rackspace as both a dynamic inventory source and a cloud credential are fully removed. Data migrations have been added to support arbitrary credential types feature and delete rackspace inventory sources. Note also requirement `jsonschema` has been moved from `requirements.txt` to `requirements.in` as a primary dependency to reflect it's usage in `/main/fields.py`. Connected issue: #6080. * `pexpect` major update `pexpect` stands at the very core of our task system and underwent a major update from 3.1 to 4.2.1. Although verified during devel, please still be mindful of any suspicious issues on celery side even after this PR gets merged. * Miscellaneous - requests now explicitly declared in `requirements.in` at version 2.11.1 in response to upstream issue - celery: 3.1.17 -> 3.1.25 - django-extensions: 1.7.4 -> 1.7.8 - django-polymorphic: 0.7.2 -> 1.2 - django-split-settings: 0.2.2 -> 0.2.5 - django-taggit: 0.21.3 -> 0.22.1 - irc: 15.0.4 -> 15.1.1 - pygerduty: 0.35.1 -> 0.35.2 - pyOpenSSL: 16.2.0 -> 17.0.0 - python-saml: 2.2.0 -> 2.2.1 - redbaron: 0.6.2 -> 0.6.3 - slackclient: 1.0.2 -> 1.0.5 - tacacs_plus: 0.1 -> 0.2 - xmltodict: 0.10.2 -> 0.11.0 - pip: 8.1.2 -> 9.0.1 - setuptools: 23.0.0 -> 35.0.2 - (requirements_ansible.in only)kombu: 3.0.35 -> 3.0.37 2017-04-20 23:47:53 +03:00
uninstall certifi if installed in venv's 2016-12-14 17:53:24 +03:00			```
			`virtualenv /buildit`
			`source /buildit/bin/activate`
			`pip install pip-tools`
			`pip install pip --upgrade`

			`pip-compile requirements/requirements.in > requirements/requirements.txt`
			`pip-compile requirements/requirements_ansible.in > requirements/requirements_ansible.txt`
			```

			`## Known Issues`

			* Remove the `-e` from packages of the form `-e git+https://github.com...` in the generated `.txt`. Failure to do so will result in a "bad" RPM and DEB due to the `pip install` laying down a symbolic link with an absolute path from the virtualenv to the git repository that will differ from when the RPM and DEB are build to when the RPM and DEB are installed on a machine. By removing the `-e` the symbolic egg link will not be created and all is well.

			* As of `pip-tools` `1.8.1` `pip-compile` does not resolve packages specified using a git url. Thus, dependencies for things like `dm.xmlsec.binding` do not get resolved and output to `requirements.txt`. This means that:
			* can't use `pip install --no-deps` because other deps WILL be sucked in
			* all dependencies are NOT captured in our `.txt` files. This means you can't rely on the `.txt` when gathering licenses.