IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
* Before, we had a special group, tower, that ran any async work that
tower needed done. This allowed users fine grain control over which
nodes did background work. However, this granularity was too complicated
for users. So now, all tower system work goes to a special non-user
exposed celery queue. Tower remains the fallback instance group to
execute jobs on. The tower group will be created upon install and
protected from deletion.
* Switch policy router queue to not be "tower" so that we don't
fall into a chicken/egg scenario
* Show fixed policy list in serializer so a user can determine if
an instance is manually managed
* Change IG membership mixin to not directly handle applying topology
changes. Instead it just makes sure the policy instance list is
accurate
* Add create/delete hooks for instances and groups to trigger policy
re-evaluation
* Update policy algorithm for fairer distribution
* Fix an issue where CELERY_ROUTES wasn't renamed after celery/django
upgrade
* Update unit tests to be more explicit
* Update count calculations used by algorithm to only consider
non-manual instances
* Adding unit tests and fixture
* Don't propagate logging messages from awx.main.tasks and
awx.main.scheduler
* Use advisory lock to prevent policy eval conflicts
* Allow updating instance groups from view
* use embedded beat rather than standalone
* dynamically set celeryd hostname at runtime
* add embeded beat flag to celery startup
* Embedded beat mode routes will piggyback off of celery worker setup
signal
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process
add support for separate beat process
this is for the development environment only; when uwsgi notices a code
change, it automatically reloads the uwsgi workers; this patch includes
a hook that sends `SIGHUP` to the celery process, causing it to spawn
a new set of workers as well
We were having issues where an older tag was being outputed from `git describe`.
From the man page:
Follow only the first parent commit upon seeing a merge commit. This is useful when you wish to not match tags on branches merged in the history of the target commit.
* Jupyter starts alongside the other awx services and is available on
0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
When installing an sdist, setup.py is invoked on the machine you're installing on. We extract the version from a git tag, but the repo is not included in the sdist. The git describe --long command will silently fail and cause the installed package to report version 0.0.0.0.
* containerization:
Updating make targets for fixes to bringing cluster up and svcs
Adding Makefile targets for local builds/deployments
Moving Openshift work to installer directory to reduce conflicts
Support deploying Tower into OpenShift
Implementing containerized tower packaging
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
This moves the container-based code location and venvs.
The goal here is that the paths of Tower source for isolated
vs normal nodes matches (both in prod and local development) so that we
don't have to add a bunch of additional bwrap argument logic for
<location-of-isolated-tower-venv>.
* Increase total number of celery jobs to match what we are currently
supporting in production with very large instances
* Make sure celery in the development environment also uses fair
scheduling without prefetching
I was upgrading virtualenv for good measure in 4545fc1da1488bb1623de82ad19f78702bd6df62, but didnt foresee it causing problems. I was wrong. Worked for everything except Trusty builds.
* rampart_groups_setup_playbook:
Updating changelog for Instance Groups
Fix an incorrect reference on instance group jobs list
Purge remaining references to rampart groups
Simplify can_access for instance groups on job templates
Adding Instance Group permissions and tests
Increase test coverage for task scheduler inventory updates
Exit logic fixes for instance group tools
View Fixes for instance groups
new view to allow associations but no creations
Updating acceptance documentation and system docs
Updating unit tests for task manager refactoring
Update views and serializers to support instance group (ramparts)
Implementing models for instance groups, updating task manager
Updating the setup playbook to support instance group installation
Add nginx to server start and switch back to first tmux win
Fix an issue where the local queue wouldn't use the rabbitmq name
* includes top level views for instances and instance groups and
extending those views to be able to view running jobs
* Associative endpoints on Organizations, Inventories, and Job
Templates
* Related and summary field entries where appropriate
* Adding job model references to executing instance group
* Fix up default queue properties for clustering from the settings file
* Update production and default settings for instance queues in settings
* New InstanceGroup model and associative relationship with Instances
* Associative instances between Organizations, Inventory, and Job
Templates and InstanceGroups
* Migrations for adding fields and tables for Instance Groups
* Adding activity stream reference for instance groups
* Task Manager Refactoring:
** Simplifying task manager relationships and move away from the
interstitial hash tables
** Simplify dependency determination logic
** Reduce task manager runtime complexity by removing the partial
references and moving the logic into the task manager directly or
relying on Job model logic for determinism
* Dynamic Inventory Source
Template against ansible 2.3 dynamic inventory sources.
The major change is removal of `rax.py`. Most upstream scripts except
`foreman.py` has quite trivial coding style changes, or minor functional
extensions that does not affect Tower inventory update runs.
`foreman.py`, on the other hand, went through quite a major refactoring,
but functionalities stay the same.
Major python dependency updates include apache-libcloud (1.3.0 -->
2.0.0), boto (2.45.0 --> 2.46.1) and shade (1.19.0 --> 1.20.0). Minor
python dependency updates include indirect updates via `pip-compile`,
which are determined by base dependencies.
Some minor `task.py` extensions:
- `.ini` file for ec2 has one more field `stack_filter=False`, which
reveals changes in `ec2.py`.
- `.ini` file for cloudforms will catch these four options from
`source_vars_dict` of inventory update: `'version', 'purge_actions',
'clean_group_keys', 'nest_tags'`. These four options have always been
available in `cloudforms.py` but `cloudforms.ini.example` has not
mentioned them until the latest version. For consistency with upstream
docs, we should make these fields available for tower user to customize.
- YAML file of openstack will catch ansible options `use_hostnames`,
`expand_hostvars` and `fail_on_errors` from `source_vars_dict` of
inventory update as a response to issue #6075.
* Remove Rackspace support
Supports of Rackspace as both a dynamic inventory source and a cloud
credential are fully removed. Data migrations have been added to support
arbitrary credential types feature and delete rackspace inventory
sources.
Note also requirement `jsonschema` has been moved from
`requirements.txt` to `requirements.in` as a primary dependency to
reflect it's usage in `/main/fields.py`.
Connected issue: #6080.
* `pexpect` major update
`pexpect` stands at the very core of our task system and underwent a
major update from 3.1 to 4.2.1. Although verified during devel, please
still be mindful of any suspicious issues on celery side even after this
PR gets merged.
* Miscellaneous
- requests now explicitly declared in `requirements.in` at version 2.11.1
in response to upstream issue
- celery: 3.1.17 -> 3.1.25
- django-extensions: 1.7.4 -> 1.7.8
- django-polymorphic: 0.7.2 -> 1.2
- django-split-settings: 0.2.2 -> 0.2.5
- django-taggit: 0.21.3 -> 0.22.1
- irc: 15.0.4 -> 15.1.1
- pygerduty: 0.35.1 -> 0.35.2
- pyOpenSSL: 16.2.0 -> 17.0.0
- python-saml: 2.2.0 -> 2.2.1
- redbaron: 0.6.2 -> 0.6.3
- slackclient: 1.0.2 -> 1.0.5
- tacacs_plus: 0.1 -> 0.2
- xmltodict: 0.10.2 -> 0.11.0
- pip: 8.1.2 -> 9.0.1
- setuptools: 23.0.0 -> 35.0.2
- (requirements_ansible.in only)kombu: 3.0.35 -> 3.0.37
* release_3.1.3: (52 commits)
ack fact scan messages
making ldap user/group search fields into codemirror instances
removing UI parsing for LDAP User and Group Search fields
Allow exception view to accept all valid HTTP methods.
Restore ability of parsing extra_vars string for provisioning callback.
Fix up backup/restore role broken in f7a8e45809758322d9ee41c5305850dd70ed5faf
Stop / start ansible-tower-service during restores
value_to_python should encode lookup fields as ascii
fix brace interpolation on standard out pane
Adjust some hardcoded usages of 'awx' to use 'aw_user' and 'aw_group'.
Pull Spanish updates from Zanata
Temporarily grant awx user createdb role
Stop giving ownership of backups to postgres
don't display chunked lines'
Add dropdown li truncation with ellipsis
CTiT -> adhoc modules should allow the user to add new modules
Remove task that was replacing the supervisor systemd tmp file
Fix failing supervisorctl commands on RH-based distros
Give ownership of the supervisor socket to awx
Setting for external log emissions cert verification
...
These macros were intended to pull the version and release from a python file if they werent passed in with --define, but /usr/bin/python is not available at this point in the build process. I'm not sure when or if this ever worked. Perhaps before we were using mock.
This approach also works for our Brew builds, as we cannot specify a macro value at build time.
This looks pretty hairy, but essentially it's just re-invoking `pip download` with `--no-deps` so that I can record the name of the local archives that are created.
I had to pull the git urls out of the main requirements files because in order to install offline (--no-index), we need pip to install from local package archives rather than cloning repo.
The weird `cat` thing going on in the Makefile is because we need to install everything as part of a single `pip install` transaction. Without this, installing only requirements_git.txt will result in dependencies getting unintentionally updated.
For some reason -I wasn't cutting it. Even though we were trying to install pip 8.1.2, this was happening:
Collecting pip==8.1.2
Using cached pip-8.1.2-py2.py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-9.0.1
¯\_(ツ)_/¯
- Renamed npm scripts to reflect what they are actually doing. Nothing is being built here!
- Not sure why the default django_host was 0.0.0.0…
- Removed unnecessary logic from build-docker-cid
A couple reasons for this:
- The command for generating the API l10n files is currently broken, and I want to get the translation folks looking at the UI ASAP.
- These things require different environments:
- Generating the UI files requires Grunt, which is only available *before* packaging. We generate the static files before invoking mock or pbuilder.
- Generating the API files requires Django, which is only available inside the virtual environment. This will likely need to be invoked somewhere inside of the install playbooks.
https://patchwork.kernel.org/patch/8784071/
While renaming the policy would have been a “cleaner” solution, this would create a scenario where our setup playbooks would need to check for the policy by it’s old name before installing the new one. Updating the filename to reflect the policy name is the least invasive and risky change.
* Fixes BrowserSync window.name namespace collision #3872
* Allow command-line arguments to be passed to development UI builds.
Add support for --ng-debug and --router-debug flags to turn on Angular's
$log service and ui-router's trace utility.
Cleans up development debug settings & defunct websocket configuration.
Resolves lingering issues raised by trying read asynchronously-loaded config file during AngularJS config phase. #3872
* shrinkwrap updated minimist
* update CONTRIBUTING.md to include UI build options
* feedback/review
* Set --master flag so we have a preforking master process to manage
subprocs
* Set max-requests to 1000 so uwsgi will recycle processes after 1000
requests
* Set --no-orphans so uwsgi will track child processes and clean them up
in order to respawn children
* Turn stats on in the container and install uwsgitop in dev
requirements so we can track uwsgi stats
Use `make ui-devel` to build a development/debug version of the UI, with source maps,
without raising any additional browser-sync or filesystem polling.
* Add a local node queue to execute targeted jobs
* Add a setting for active cluster node id (per-node)
* Base the heartbeat time on the `modified` time on the Instance table
* Add periodic task that calls save() on the instance to update the
heartbeat time if services are up
* Purge/update any ha/instance management commands
* Fix up CELERY_ROUTES settings data structure
- Added a script that allows for piping signing keys into the container’s stdin stream.
- Got sign.exp working
- I ended up getting mock to run as root, which means both RPM and DEB jobs both use —unsafe-perm now, so I added that back to the Makefile.
Don't `sudo` for me, I'll `sudo` when I need it, yo. This appears to
only affect the non-container based development workflow. For folks
that still need to `make develop` outside of a container, please use
`sudo make develop`
* ramparts_and_ha: (21 commits)
Rename database migrations for devel integration
Integrate callback receiver refactoring
Fix an issue running jobs in the cluster
Implement a more dynamic celery queue system
Purge old munin monitors and tools
Refactor Tower HA Instance logic and models
Docker compose improvements
Initial Docker Compose workflow for Tower cluster
Add memcached role for setup playbook
Removing qpid from deb packaging
Refactor rabbitmq role
Integrate memcached into setup playbook
Remove mongodb shutdown task
Remove dependency on erlang_sd_notify
Add initial rabbitmq role
Initial rabbitmq setup playbook integration
Update development environment for rabbit
Replace qpid with rabbitmq
Remove redis role from setup playbook
Update qpid packaging, remove migrations
...
* Meant to be a starting point to more efficiently manage work routing
and to balance work across all tower nodes
* Integrate flower as a dev tool that starts alongside other nodes.
Helpful for observing and monitoring the queues/exchanges
* For the moment, force the task manager to only run on one node (not
sure if this is needed)
* Define queues and routes for all task work
* Bump celery version to 3.1.23
* Expose flower through haproxy
* Gut the HA middleware
* Purge concept of primary and secondary.
* UUID is not the primary host identifier, now it's based mostly on the
username. Some work probably still left to do to make sure this is
legit. Also removed unique constraint from the uuid field. This
might become the cluster ident now... or it may just deprecate
* No more secondary -> primary redirection
* Initial revision of /api/v1/ping
* Revise and gut tower-manage register_instance
* Rename awx/main/socket.py to awx/main/socket_queue.py to prevent
conflict with the "socket" module from python base
* Revist/gut the Instance manager... not sure if this manager is really
needed anymore
* stable: (24 commits)
Updating changelogs for 3.0.2 release
fixing deprecated_team.organization credential migration
Fix issue when installing bundled ansible on el6
fixed localed date stuff
update test to check org_auditor access
ensure team organizations are assigned to credentials
temporarily pin the pytest version until the ldap error can be fixed
fixed locale
fix date locale angular scheduler
Make sure org admins can see credential after migration, comment updates on related tests add clause in test to verify automatic setting of org of new team credential
Update team admin credential migration test to current state-of-knowledge
fix ng-toast rel, resolves#3197 (#3316)
allow users to edit their first and last name
Revert "Prevent ignored task from being displayed as failing."
Revert "Modify job event save behavior"
fixing old tests for new user creation permissions
Clean venv on 'make clean'
Resolve KeyError by coercing instance_id to a str
Update cloudforms dynamic inventory
Update foreman inventory script
...
