IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Adjust requirements to keep docutils in awx requirements; these
are downstream packaging adjustments. Override azure wheel
dependency to a version that can install bcrypt properly.
Remove build isolation flag
Do not use --ignore-installed for any pip install commands
Add Makefile comments
do not use system site packages for awx venv
Consolidate bootstrap pins
Do another upgrade, properly document wheel
this is in addition to pip and setuptools installs
add in --ignore-installed and --no-build-isolation flags
reasoning is that we are not installing needed packages
for building
Versions selected to be pre-19 pip
due to unresolved issues with the build systems
Upgrade everything, party on
document new process
rotate license files
fix Swagger schema generation target
Remove --ignore-installed flag
This commit includes all the changes involved in
converting the old Ansible Tower modules from commits
in Ansible core into the AWX collection that replaces it.
Also includes work needed to integrate it into the
AWX processes like tests, docs, and the Makefile.
Apply changes from content_collector tool
Add integrated module tests
operate via run_module fixture
add makefile target for them
Add flake8 target and fix flake8 errors
Update README
Make consolidated target for testing modules
- use awx-python in shebang in dev env
- scl enable where needed for rhel7 & container installs
- use scram-sha-256 pg user hashing by default
- ensure psycopg2 is using the correct PG_CONFIG at build time for the right libpq version
pip 19 added support for something called `pyproject.toml`. Several packages have been setting the option `build-backend = "setuptools.build_meta”` (bcrypt, attrs, jaraco) which seems to be the root of the problem when building from source. Until the community sorts this out I’m inclined to avoid pip 19.
When you do `make docker-compose-build` on your branch without any changes, this will use the devel tag as a cache source. This will speed things up if you havent made any local changes to the Makefile or requirements files.
There was quite a bit of drift between the main Dockerfile and this one since we last touched it. It’s heavier than it needs to be, but by using the main development image as the base here, we have less duplicated and outdated code overall.
This was intended to save us CI time if / when we switch away from static nodes for Zuul. After merging this in I discovered that this option does not work with `docker build --cache-from`, which will be more benefical in terms of speed improvements anyway.
This commit does a few things:
- Add the `--squash` option to the `awx-devel-build` make target. This reduces the resulting image size from 2.12 GB to 1.37 GB. I think we can get this down even more by inspecting the image contents.
- Reorganize commands so that the cache expires less often. Before this commit, any changes to the Makefile would essentially cause the entire image to rebuild.
- Break yum dependencies up into multiple lines. This makes it easier to see what changes in a diff.
- Use `n` to install our required version of node (rather `curl node | bash`). I’ve found this to be easier to maintain / more portable when working with other Dockerfiles.
- General organizational changes to make things easier to parse visually.
I wanted to pass `—user` to `docker-compose` up, but that option doesnt exist. To get around this, I had to record the uid on the host (CURRENT_UID), interpolate the variable in tools/docker-compose.yml, and detect that inside the container. I then piggy-backed on the /etc/passwd hack we use for scenarios with unpredictable uids.
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:
- a new decorator, `awx.main.dispatch.task`, which can be used to
decorate functions or classes so that they can be designated as
"Tasks"
- support for fanout/broadcast tasks (at this point in time, only
`conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
`handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
on demand
- minimal support for RPC, such as status checks and pool recycle/reload
the main goal of this change is to make `make docker-isolated` work out
of the box
- specify the proper version for awx-expect --version
- update some deprecated playbook bits
- change the isolated container to privileged so bwrap will work
- fix awx-manage test_isolated_connection
- expedite the first isolated heartbeat so you don't have to wait 10m;
this is accomplished by _not_ setting Instance.last_isolated_check to
now() at insertion time (which causes the next check not to happen for
10 minutes)
- fix a bug that caused isolated node execution to fail when bwrap was
enabled
see: https://github.com/ansible/tower/issues/2150
This reverts commit 9863fe71dc.
refactor existing handlers to be the related
"real" handler classes, which are swapped
out dynamically by external logger "proxy" handler class
real handler swapout only done on setting change
remove restart_local_services method
get rid of uWSGI fifo file
change TCP/UDP return type contract so that it mirrors
the request futures object
add details to socket error messages
* Before, we had a special group, tower, that ran any async work that
tower needed done. This allowed users fine grain control over which
nodes did background work. However, this granularity was too complicated
for users. So now, all tower system work goes to a special non-user
exposed celery queue. Tower remains the fallback instance group to
execute jobs on. The tower group will be created upon install and
protected from deletion.
* Switch policy router queue to not be "tower" so that we don't
fall into a chicken/egg scenario
* Show fixed policy list in serializer so a user can determine if
an instance is manually managed
* Change IG membership mixin to not directly handle applying topology
changes. Instead it just makes sure the policy instance list is
accurate
* Add create/delete hooks for instances and groups to trigger policy
re-evaluation
* Update policy algorithm for fairer distribution
* Fix an issue where CELERY_ROUTES wasn't renamed after celery/django
upgrade
* Update unit tests to be more explicit
* Update count calculations used by algorithm to only consider
non-manual instances
* Adding unit tests and fixture
* Don't propagate logging messages from awx.main.tasks and
awx.main.scheduler
* Use advisory lock to prevent policy eval conflicts
* Allow updating instance groups from view
* use embedded beat rather than standalone
* dynamically set celeryd hostname at runtime
* add embeded beat flag to celery startup
* Embedded beat mode routes will piggyback off of celery worker setup
signal
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process
add support for separate beat process
this is for the development environment only; when uwsgi notices a code
change, it automatically reloads the uwsgi workers; this patch includes
a hook that sends `SIGHUP` to the celery process, causing it to spawn
a new set of workers as well
We were having issues where an older tag was being outputed from `git describe`.
From the man page:
Follow only the first parent commit upon seeing a merge commit. This is useful when you wish to not match tags on branches merged in the history of the target commit.
* Jupyter starts alongside the other awx services and is available on
0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
When installing an sdist, setup.py is invoked on the machine you're installing on. We extract the version from a git tag, but the repo is not included in the sdist. The git describe --long command will silently fail and cause the installed package to report version 0.0.0.0.
* containerization:
Updating make targets for fixes to bringing cluster up and svcs
Adding Makefile targets for local builds/deployments
Moving Openshift work to installer directory to reduce conflicts
Support deploying Tower into OpenShift
Implementing containerized tower packaging
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
This moves the container-based code location and venvs.
The goal here is that the paths of Tower source for isolated
vs normal nodes matches (both in prod and local development) so that we
don't have to add a bunch of additional bwrap argument logic for
<location-of-isolated-tower-venv>.
* Increase total number of celery jobs to match what we are currently
supporting in production with very large instances
* Make sure celery in the development environment also uses fair
scheduling without prefetching
I was upgrading virtualenv for good measure in 4545fc1da1488bb1623de82ad19f78702bd6df62, but didnt foresee it causing problems. I was wrong. Worked for everything except Trusty builds.
* rampart_groups_setup_playbook:
Updating changelog for Instance Groups
Fix an incorrect reference on instance group jobs list
Purge remaining references to rampart groups
Simplify can_access for instance groups on job templates
Adding Instance Group permissions and tests
Increase test coverage for task scheduler inventory updates
Exit logic fixes for instance group tools
View Fixes for instance groups
new view to allow associations but no creations
Updating acceptance documentation and system docs
Updating unit tests for task manager refactoring
Update views and serializers to support instance group (ramparts)
Implementing models for instance groups, updating task manager
Updating the setup playbook to support instance group installation
Add nginx to server start and switch back to first tmux win
Fix an issue where the local queue wouldn't use the rabbitmq name
* includes top level views for instances and instance groups and
extending those views to be able to view running jobs
* Associative endpoints on Organizations, Inventories, and Job
Templates
* Related and summary field entries where appropriate
* Adding job model references to executing instance group
* Fix up default queue properties for clustering from the settings file
* Update production and default settings for instance queues in settings
* New InstanceGroup model and associative relationship with Instances
* Associative instances between Organizations, Inventory, and Job
Templates and InstanceGroups
* Migrations for adding fields and tables for Instance Groups
* Adding activity stream reference for instance groups
* Task Manager Refactoring:
** Simplifying task manager relationships and move away from the
interstitial hash tables
** Simplify dependency determination logic
** Reduce task manager runtime complexity by removing the partial
references and moving the logic into the task manager directly or
relying on Job model logic for determinism
* Dynamic Inventory Source
Template against ansible 2.3 dynamic inventory sources.
The major change is removal of `rax.py`. Most upstream scripts except
`foreman.py` has quite trivial coding style changes, or minor functional
extensions that does not affect Tower inventory update runs.
`foreman.py`, on the other hand, went through quite a major refactoring,
but functionalities stay the same.
Major python dependency updates include apache-libcloud (1.3.0 -->
2.0.0), boto (2.45.0 --> 2.46.1) and shade (1.19.0 --> 1.20.0). Minor
python dependency updates include indirect updates via `pip-compile`,
which are determined by base dependencies.
Some minor `task.py` extensions:
- `.ini` file for ec2 has one more field `stack_filter=False`, which
reveals changes in `ec2.py`.
- `.ini` file for cloudforms will catch these four options from
`source_vars_dict` of inventory update: `'version', 'purge_actions',
'clean_group_keys', 'nest_tags'`. These four options have always been
available in `cloudforms.py` but `cloudforms.ini.example` has not
mentioned them until the latest version. For consistency with upstream
docs, we should make these fields available for tower user to customize.
- YAML file of openstack will catch ansible options `use_hostnames`,
`expand_hostvars` and `fail_on_errors` from `source_vars_dict` of
inventory update as a response to issue #6075.
* Remove Rackspace support
Supports of Rackspace as both a dynamic inventory source and a cloud
credential are fully removed. Data migrations have been added to support
arbitrary credential types feature and delete rackspace inventory
sources.
Note also requirement `jsonschema` has been moved from
`requirements.txt` to `requirements.in` as a primary dependency to
reflect it's usage in `/main/fields.py`.
Connected issue: #6080.
* `pexpect` major update
`pexpect` stands at the very core of our task system and underwent a
major update from 3.1 to 4.2.1. Although verified during devel, please
still be mindful of any suspicious issues on celery side even after this
PR gets merged.
* Miscellaneous
- requests now explicitly declared in `requirements.in` at version 2.11.1
in response to upstream issue
- celery: 3.1.17 -> 3.1.25
- django-extensions: 1.7.4 -> 1.7.8
- django-polymorphic: 0.7.2 -> 1.2
- django-split-settings: 0.2.2 -> 0.2.5
- django-taggit: 0.21.3 -> 0.22.1
- irc: 15.0.4 -> 15.1.1
- pygerduty: 0.35.1 -> 0.35.2
- pyOpenSSL: 16.2.0 -> 17.0.0
- python-saml: 2.2.0 -> 2.2.1
- redbaron: 0.6.2 -> 0.6.3
- slackclient: 1.0.2 -> 1.0.5
- tacacs_plus: 0.1 -> 0.2
- xmltodict: 0.10.2 -> 0.11.0
- pip: 8.1.2 -> 9.0.1
- setuptools: 23.0.0 -> 35.0.2
- (requirements_ansible.in only)kombu: 3.0.35 -> 3.0.37
* release_3.1.3: (52 commits)
ack fact scan messages
making ldap user/group search fields into codemirror instances
removing UI parsing for LDAP User and Group Search fields
Allow exception view to accept all valid HTTP methods.
Restore ability of parsing extra_vars string for provisioning callback.
Fix up backup/restore role broken in f7a8e45809758322d9ee41c5305850dd70ed5faf
Stop / start ansible-tower-service during restores
value_to_python should encode lookup fields as ascii
fix brace interpolation on standard out pane
Adjust some hardcoded usages of 'awx' to use 'aw_user' and 'aw_group'.
Pull Spanish updates from Zanata
Temporarily grant awx user createdb role
Stop giving ownership of backups to postgres
don't display chunked lines'
Add dropdown li truncation with ellipsis
CTiT -> adhoc modules should allow the user to add new modules
Remove task that was replacing the supervisor systemd tmp file
Fix failing supervisorctl commands on RH-based distros
Give ownership of the supervisor socket to awx
Setting for external log emissions cert verification
...
These macros were intended to pull the version and release from a python file if they werent passed in with --define, but /usr/bin/python is not available at this point in the build process. I'm not sure when or if this ever worked. Perhaps before we were using mock.
This approach also works for our Brew builds, as we cannot specify a macro value at build time.
This looks pretty hairy, but essentially it's just re-invoking `pip download` with `--no-deps` so that I can record the name of the local archives that are created.
I had to pull the git urls out of the main requirements files because in order to install offline (--no-index), we need pip to install from local package archives rather than cloning repo.
The weird `cat` thing going on in the Makefile is because we need to install everything as part of a single `pip install` transaction. Without this, installing only requirements_git.txt will result in dependencies getting unintentionally updated.
For some reason -I wasn't cutting it. Even though we were trying to install pip 8.1.2, this was happening:
Collecting pip==8.1.2
Using cached pip-8.1.2-py2.py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-9.0.1
¯\_(ツ)_/¯
- Renamed npm scripts to reflect what they are actually doing. Nothing is being built here!
- Not sure why the default django_host was 0.0.0.0…
- Removed unnecessary logic from build-docker-cid
