1
0
mirror of https://github.com/ansible/awx.git synced 2024-11-01 08:21:15 +03:00
Commit Graph

385 Commits

Author SHA1 Message Date
Bill Nottingham
ed0bb127e8 Don't use the rsa python lib. 2019-04-02 15:48:30 -04:00
Ryan Petrello
b851e2be4a
don't add hvac as a dependency for hashicorp vault integration
hvac is just based on requests anyways, and it doesn't support half of
what we need (like the SSH secrets engine API)
2019-04-02 11:20:44 -04:00
Ryan Petrello
4ed5bca5e3
add credential plugin support for Azure Key Vault 2019-04-02 11:20:26 -04:00
Jake McDermott
c209955400
add credential plugin system and minimal working hashivault 2019-04-02 11:19:26 -04:00
softwarefactory-project-zuul[bot]
7b16931658
Merge pull request #2736 from AlanCoding/logstash_removal
Remove python-logstash dependency

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-03-31 02:35:45 +00:00
Ryan Petrello
f7cffbfe5c
pin runner 1.3.1 2019-03-27 11:23:25 -04:00
AlanCoding
cd7e358b73
Inventory plugins transition dev finishing work
Bump keystone auth to resolve problem with openstack script

Clarify code path, routing to template vs. managed injector
  behavior is also now reflected in test data files

Refactor test data layout for inventory injector logic

Add developer docs for inventory plugins transition

Memoize only get_ansible_version with no parameters

Make inventory plugin injector enablement a separate
  concept from the initial_version
  switch tests to look for plugin_name as well

Add plugin injectors for tower and foreman.

Add jinja2 native types compat feature

move tower source license compare logic to management command

introduce inventory source compat mode

pin jinja2 for native Ansible types

Add parent group keys, and additional translations

manual dash sanitization for un-region-like ec2 groups

nest zones under regions using Ansible core feature just merged
  implement conditionally only with BOTH group_by options

Make compat mode default be true
  in API models, UI add and edit controllers

Add several additional hostvars to translation
Add Azure tags null case translation

Make Azure group_by key off source_vars
  to be consistent with the script

support top-level ec2 boto_profile setting
2019-03-26 10:29:39 -04:00
AlanCoding
bc5881ad21
Primary development of inventory plugins, partial compat layer
Initialize some inventory plugin test data files
Implement openstack inventory plugin

This may be removed later:
- port non-JSON line strip method from core

Dupliate effort with AWX mainline devel
- Produce ansible_version related to venv

Refactor some of injector management, moving more
  of this overhead into tasks.py, when it comes to
  managing injector kwargs

Upgrade and move openstack inventory script
  sync up parameters

Add extremely detailed logic to inventory file creation
for ec2, Azure, and gce so that they are closer to a
genuine superset of what the contrib script used to give.
2019-03-26 10:29:39 -04:00
Jim Ladd
de808d4911
Only install futures on py2 2019-03-26 10:29:38 -04:00
AlanCoding
6c130fa6c3
Build-in inventory plugin code structure with gce working
supporting and related changes
 - Fix inconsistency between can_update / can_start
 - Avoid creating inventory file twice unnecessarily
 - Non-functional consolidation in Azure injection logic
 - Inject GCE creds as indented JSON for readability
 - Create new injector class structure, add gce
 - Reduce management command overrides of runtime environment
2019-03-26 10:29:35 -04:00
Ryan Petrello
42bfff301c
remove python-memcached as a base dependency for playbook execution 2019-03-25 12:26:44 -04:00
softwarefactory-project-zuul[bot]
8b2fc26219
Merge pull request #3041 from chrismeyersfsu/runnerpy3
ansible-runner integration

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-03-21 22:21:50 +00:00
Ryan Petrello
b02d9ae282
pin apache-libcloud to a version that doesn't use PyCrypto
once a new version lands on PyPI, we'll pin to _it_
2019-03-21 14:21:04 -04:00
chris meyers
0a670e8db1 change from runner master to runner 1.3 2019-03-21 07:46:11 -04:00
AlanCoding
1ae4ed4922
migrate python-logstash license 2019-03-20 15:29:06 -04:00
AlanCoding
017d367749
Remove dependency and insert class 2019-03-20 15:29:05 -04:00
chris meyers
8a04c22b2b point at another runner branch
* revert parent_uuid because it causes problems with unexpected
parameter on event creation for some event type.
2019-03-18 14:21:47 -04:00
chris meyers
8fb65b40de use ansible runner to run playbooks
* Project Updates
* Jobs
* Inventory Updates
* System Jobs
* AdHoc Commands

* Notifications
* Fact Cache
* proot
2019-03-18 14:21:47 -04:00
chris meyers
a7cda95803 init ansible-runner requirements 2019-03-18 14:21:47 -04:00
Ryan Petrello
04622d5786
remove redbaron and update dependencies 2019-02-21 10:08:24 -05:00
Christian Adams
8597670299 upgrade social-auth-core to v3.0.0 2019-01-21 16:35:47 -05:00
Tyler Cross
0da0a8e67b
CORS Support
Added the django-cors-headers app and middleware to make CORS possible.
2019-01-18 12:49:00 -05:00
Ryan Petrello
65641c7edd
add support for custom py3 ansible virtualenvs 2019-01-18 10:55:53 -05:00
Shane McDonald
5aa99b2ca1
Dependency updates for Python 3 2019-01-15 14:09:05 -05:00
Ryan Petrello
af4234556e
remove dm.xmlsec.binding
python-saml uses dm.xmlsec.binding only supports python2
by moving to py3, we now use python3-saml (which uses python-xmlsec
instead)

see: https://github.com/onelogin/python-saml/issues/145#issuecomment-222021691
2019-01-15 14:09:05 -05:00
Ryan Petrello
f223df303f
convert py2 -> py3 2019-01-15 14:09:01 -05:00
Ryan Petrello
f132ce9b64 switch image builds to py3 2019-01-15 13:25:13 -05:00
Wayne Witzel III
1079051b12 update to the latest asgi-amqp 2019-01-03 07:52:16 -05:00
Bill Nottingham
e9c3623dfd specify a version 2018-11-16 15:34:42 -05:00
Bill Nottingham
c65b362841 Add ncclient for use by networking modules. 2018-11-16 15:21:04 -05:00
AlanCoding
d8d710a83d
get rid of decorator dependency 2018-10-31 11:37:10 -04:00
Bill Nottingham
7fc269b65a Re-add markdown, which is used for rendering API help. 2018-10-30 12:10:00 -04:00
Bill Nottingham
a56a231869 Remove some obsolete requirements.
Bump cryptography to latest.
2018-10-23 10:37:36 -04:00
Bill Nottingham
2c01476eca Don't explicitly remove certifi. 2018-10-18 16:41:33 -04:00
Bill Nottingham
8adbc8a026 Update Azure requirements to match Ansible 2.7 requirements.
Add comments for Ansible requirements to note where they're used.

Remove our custom docutils fork, as the fix was merged upstream.
2018-10-18 16:41:33 -04:00
Bill Nottingham
aec3244f52
Update to latest django subminor to pick up assorted fixes. 2018-10-18 09:23:57 -04:00
Ryan Petrello
79002ae563
upgrade to the latest kombu + celery 2018-10-16 16:14:58 -04:00
softwarefactory-project-zuul[bot]
0a964b2bf6
Merge pull request #2266 from ansible/celery-tastes-bad
replace the celery-based task queue with a kombu-based implementation

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2018-10-12 18:40:54 +00:00
Ryan Petrello
517043e209
update to the latest stable 1.1 django-oauth-toolkit
see: https://github.com/jazzband/django-oauth-toolkit/pull/629
2018-10-12 10:21:57 -04:00
Ryan Petrello
ff1e8cc356
replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Bill Nottingham
be01bed34b Purge inventory script requirements from the AWX virtual environment.
boto is still used by AWX itself.
2018-10-11 09:45:41 -04:00
Wayne Witzel III
b8c1724880 use latest asgi_amqp version 2018-10-09 15:34:07 -04:00
Matthew Jones
3a8bacb8ef
Add an initial check and gate job configuration for zuul
Updates for running ui tests and linters
2018-10-05 13:39:59 -04:00
adamscmRH
227960e3ea pin pluggy at 0.6.0 2018-08-27 16:55:40 -04:00
Shane McDonald
1c32df0fcd Move django-oauth-toolkit into requirements_git.txt
All SCM based requirements need to go into this file because of how we built Ansible Tower (offline).
2018-08-06 19:00:54 -04:00
Ryan Petrello
87d6940158
pin django-oauth-toolkit to our fork w/ a security fix
see: https://github.com/jazzband/django-oauth-toolkit/issues/625
2018-08-06 14:54:46 -04:00
Shane McDonald
795f26e7b9 Lock down version of setuptools_scm
This was causing offline pip installs to fail for some weird reason:

The 'setuptools_scm>=1.15.0' distribution was not found and is required by the application

Even though it is there. v2.x still works.
2018-07-27 15:28:41 -04:00
adamscmRH
8cc77234fc revert django-radius from 1.3.3 to 1.1.0 2018-06-22 15:01:11 -04:00
Christian Adams
1b5ea07745
Merge pull request #2171 from rooftopcellist/upgrade_oauth_112
Upgrade Django-oauth-toolkit to 1.1.2
2018-06-15 10:22:20 -04:00
adamscmRH
c5f2b9a471 upgrade django-oauth-toolkit to 1.1.2 2018-06-15 09:50:34 -04:00
adamscmRH
b87d25300a update dev pytest 2018-06-14 10:11:11 -04:00
Pierre-Louis Bonicoli
16cde26468 ansible venv: re-add netaddr Python package
Closes #1763
2018-05-30 10:32:54 -04:00
Ryan Petrello
7f7f635f7b upgrade to the latest pexpect
see: https://github.com/pexpect/pexpect/pull/492
see: https://github.com/ansible/tower/issues/1797
2018-05-29 09:52:59 -04:00
Bill Nottingham
50503f97cc Regen requirements_ansible.txt (minimal update). 2018-05-23 14:11:09 -04:00
Bill Nottingham
7bc28276f2 add note 2018-05-23 10:31:54 -04:00
Fedor Sumkin
a1ed0f47ab adding pexpect support 2018-05-22 16:45:05 -04:00
Chris Meyers
7f214f5ad2
Merge pull request #1766 from chrismeyersfsu/i_like_parallel_tests_and_I_can_not_lie
parallelize test running
2018-05-16 15:13:20 -04:00
chris meyers
97ab6449b9 parallelize test running 2018-05-16 14:29:15 -04:00
Bill Nottingham
6622e6bbd6 Bump pexpect version on isolated node. 2018-05-15 11:24:27 -04:00
Ryan Petrello
f0d3713e99 use select.poll() instead of select.select() for file descriptors
see: https://github.com/ansible/tower/issues/861
see: https://github.com/pexpect/pexpect/pull/474
2018-04-25 16:29:55 -04:00
Ryan Petrello
19d5d71937 unpin botocore now that an upstream dependency issue is resolved
related: 4f585dd09e
related: https://github.com/boto/botocore/pull/1433
2018-04-23 14:43:09 -04:00
Ryan Petrello
a2a246a834 update to a newer python-dateutil with more bug fixes we were vendoring
related: dateutil/dateutil#649
2018-03-26 17:12:01 -04:00
Ben Thomasson
b29a605800
Cleans up feature set for 3.3
This removes the experimental features that were not selected
for 3.3 release.

* Removes dpath requirement
* Removes generated action_plugins
* Removes network UI v1 api
* Removes unused network management commands
* Removes network UI CLI client
* Removes templates
* Removes unused DataBinding models
* Removes obsolete test
* Removes unused admin and tests
* Removes experimental UndoPersistence, RedoPersistence, and auto-layout functions
* Removes API endpoints for network visualization
* Removes unused consumer routes
* Removes group, site, and rack features for 3.3
* Removes unused tables controller
* Removes undo/redo
* Removes group code and scale checks
2018-03-23 17:00:24 -04:00
Ben Thomasson
701150bd1a
Adds configuration for the network-ui websocket
* Configures NGINX for the network-ui websocket.
* Configures supervisor.conf for network_ui websocket.
2018-03-23 17:00:14 -04:00
Ben Thomasson
48d801271c
Imports prototype from ansible-network-ui
The ansible-network-ui prototype project builds a standalone Network UI
outside of Tower as its own Django application. The original prototype
code is located here:
https://github.com/benthomasson/ansible-network-ui.

The prototype provides a virtual canvas that supports placing
networking devices onto 2D plane and connecting those devices together
with connections called links.  The point where the link connects
to the network device is called an interface.  The devices, interfaces,
and links may all have their respective names.  This models physical
networking devices is a simple fashion.

The prototype implements a pannable and zoomable 2D canvas in using SVG
elements and AngularJS directives.   This is done by adding event
listeners for mouse and keyboard events to an SVG element that fills the
entire browser window.

Mouse and keyboard events are handled in a processing pipeline where
the processing units are implemented as finite state machines that
provide deterministic behavior to the UI.

The finite state machines are built in a visual way that makes
the states and transitions clearly evident.  The visual tool for
building FSM is located here:
https://github.com/benthomasson/fsm-designer-svg.   This tool
is a fork of this project where the canvas is the same.  The elements
on the page are FSM states and the directional connections are called
transitions.   The bootstrapping of the FSM designer tool and
network-ui happen in parallel.  It was useful to try experiemental
code in FSM designer and then import it into network-ui.

The FSM designer tool provides a YAML description of the design
which can be used to generate skeleton code and check the implementation
against the design for discrepancies.

Events supported:

* Mouse click
* Mouse scroll-wheel
* Keyboard events
* Touch events

Interactions supported:

* Pan canvas by clicking-and-dragging on the background
* Zooming canvas by scrolling mousewheel
* Adding devices and links by using hotkeys
* Selecting devices, interaces, and links by clicking on their icon
* Editing labels on devices, interfaces, and links by double-clicking on
  their icon
* Moving devices around the canvas by clicking-and-dragging on their
  icon

Device types supported:

* router
* switch
* host
* racks

The database schema for the prototype is also developed with a visual
tool that makes the relationships in the snowflake schema for the models
quickly evident.  This tool makes it very easy to build queries across
multiple tables using Django's query builder.

See: https://github.com/benthomasson/db-designer-svg

The client and the server communicate asynchronously over a websocket.
This allows the UI to be very responsive to user interaction since
the full request/response cycle is not needed for every user
interaction.

The server provides persistence of the UI state in the database
using event handlers for events generated in the UI.  The UI
processes mouse and keyboard events, updates the UI, and
generates new types of events that are then sent to the server
to be persisted in the database.

UI elements are tracked by unique ids generated on the client
when an element is first created.  This allows the elements to
be correctly tracked before they are stored in the database.

The history of the UI is stored in the TopologyHistory model
which is useful for tracking which client made which change
and is useful for implementing undo/redo.

Each message is given a unique id per client and has
a known message type.  Message types are pre-populated
in the MessageType model using a database migration.

A History message containing all the change messages for a topology is
sent when the websocket is connected.  This allows for undo/redo work
across sessions.

This prototype provides a server-side test runner for driving
tests in the user interface.  Events are emitted on the server
to drive the UI.  Test code coverage is measured using the
istanbul library which produces instrumented client code.
Code coverage for the server is is measured by the coverage library.

The test code coverage for the Python code is 100%.
2018-03-23 17:00:14 -04:00
Ryan Petrello
918f372c20 add uwsgitop as a dependency
see: https://github.com/ansible/ansible-tower/issues/7966
2018-03-19 08:53:30 -04:00
Ryan Petrello
4f585dd09e pin botocore to avoid dependency hell re: latest python-dateutil
boto decided to pin python-dateutil on a version _lower than_ what we
need for the TZID= bug fix:
90d7692702 (diff-b4ef698db8ca845e5845c4618278f29a)
2018-03-16 16:08:03 -04:00
adamscmRH
8d460490c1 add xmlsec flag to docker installs 2018-03-14 14:28:35 -04:00
Ryan Petrello
44adab0e9e bump python-dateutil to latest
this change provides support for numerous bug fixes, along with
support for parsing TZINFO= from rrule strings

related: https://github.com/ansible/ansible-tower/issues/823
related: https://github.com/dateutil/dateutil/issues/614
2018-03-12 12:20:03 -04:00
AlanCoding
4a01805a19
add shade back into AWX requirements
Last round of dependency updates showed that AWX
depended on packages which came implicitly from shade
decorator is added as an explicit dependency
and all of the rest of shade requirements are
added back in here.
2018-03-08 10:32:19 -05:00
AlanCoding
556e6c4a11
Dependency Updates
Upgrades of minor dependency upgrades
Inventory scripts were upgraded in separate commit

Major exclusions from this update
- celery was already downgraded for other reasons
- Django / DRF major update already done, minor bumps here
- asgi-amqp has fixes coming independently, not touched
- TACACS plus added features not needed

Removals of note
- remove shade from AWX requirements
- remove kombu from Ansible requirements

Other notes

Add note about pinning setuptools and pip,
done but not mentioned previously

Stop pinning gevent-websocket and twisted

upgrade Azure to Ansible core requirements

more detailed notes
https://gist.github.com/AlanCoding/9442a512ab6977940bc7b5b346d4f70b

upgrade version of Django for Exception
2018-03-06 16:04:01 -05:00
Demin, Petr
f4a241aba2
Constrain requests 2018-03-06 12:47:34 -05:00
Ryan Petrello
13672cc88c
fix busted shippable builds 2018-03-05 14:16:42 -05:00
Wayne Witzel III
73f0a0d147 Update to latest asgi-amqp 2018-02-28 14:43:37 -05:00
Chris Meyers
d551566b4d
Merge pull request #1372 from chrismeyersfsu/old-celery3
celery 4.x to 3.x roll back
2018-02-27 15:26:46 -05:00
Ryan Petrello
d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Aaron Tan
1c2621cd60 Implement session-based and OAuth 2 authentications
Relates #21. Please see acceptance docs for feature details.

Signed-off-by: Aaron Tan <jangsutsr@gmail.com>
2018-02-22 15:18:12 -05:00
Wayne Witzel III
91c6d406c5 Rollback celery 2018-02-22 09:37:14 -05:00
Ryan Petrello
2a168faf6a
upgrade to the latest pexpect
see: https://github.com/ansible/awx/issues/417
2018-02-12 09:18:14 -05:00
Ryan Petrello
57c22c20b2
add support for building swagger/OpenAPI JSON
to build, run `make swagger`; a file named `swagger.json` will be
written to the current working directory
2018-02-06 10:12:57 -05:00
Ryan Petrello
51f7907a01
optimize OutputEventFilter for large stdout streams
update our event data search algorithm to be a bit lazier in event data
discovery; this drastically improves processing speeds for stdout >5MB

see: https://github.com/ansible/awx/issues/417
2018-01-16 14:41:35 -05:00
Chris Meyers
2ed97aeb0c implement multiple ldap servers 2018-01-11 09:03:14 -05:00
Matthew Jones
67d927121d
Merge pull request #940 from ryanpetrello/multivenv
implement support for per-playbook/project/org virtualenvs
2018-01-10 12:15:38 -05:00
Ryan Petrello
1e8c89f536
implement support for per-playbook/project/org virtualenvs
see: https://github.com/ansible/awx/issues/34
2018-01-09 22:47:01 -05:00
Wayne Witzel III
1690938dfb Update asgi-amqp requirement 2018-01-09 23:54:30 +00:00
Matthew Jones
9dbcc5934e
Merge remote-tracking branch 'tower/release_3.2.2' into devel 2017-12-13 12:25:47 -05:00
Matthew Jones
da0b686369
Adding jupyter notebook support to the AWX development environment
* Jupyter starts alongside the other awx services and is available on
  0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
2017-12-05 23:46:18 -05:00
Shane McDonald
ec14ae1930 Add m2r to setup requirements file
We `pip download` this file for offline installs. Automat lists this package as a setup_requires, but `pip download` doesn’t resolve these dependencies (distutils will attempt to install them via easy_install when setup.py is invoked).
2017-12-05 15:26:56 -05:00
Wayne Witzel III
a858093db8 Update to asgi_amqp 1.0.1 2017-11-27 19:41:30 +00:00
Wayne Witzel III
f118e27047
Flake8 fixes and URL updates 2017-11-10 17:04:33 -05:00
Wayne Witzel III
8faf588775
Update package versions, settings, and tooling 2017-11-09 17:17:30 -05:00
Ryan Petrello
e06d4d7734 don't install pycurl from pypi; use a system package instead
the ovirt4 sdk relies on pycurl, which is complicated to install w/ pip;
rely on pycurl to be provided by a system package instead
2017-10-10 14:38:44 -04:00
Ryan Petrello
7438062b97 add ovirt sdk dependency for ovirt4 support 2017-10-10 14:38:44 -04:00
Ryan Petrello
768c7ba3dc
bump azurerm dependencies to support Ansible 2.4
see: https://github.com/ansible/ansible-tower/issues/7470
2017-09-01 15:15:53 -04:00
Shane McDonald
5af04e3164 Add pycrypto to Tower's requirements files
This is needed for GCE inventory syncs to work. Inventory Syncs run through the Tower virtual environment.
2017-08-10 09:10:10 -04:00
Shane McDonald
ac9fcdb63e Bump setuptools version
This addresses a bug I was seeing when running `pip download setuptools --no-binary=:all:`.
2017-08-01 10:13:04 -04:00
Ryan Petrello
ad66ff9938 pin kombu to a known working version (from tower 3.1.4)
we're encountering some *really* gnarly bugs with newer version of
celery and kombu; pin to *exactly* what works in 3.1.4 to try to avoid
them, because we can't upgrade celery to 4.0 (where this bug *may* be
fixed)

see: https://github.com/ansible/ansible-tower/issues/6534
see: https://github.com/celery/celery/issues/3712
see: https://github.com/celery/kombu/pull/760
2017-07-28 10:59:06 -04:00
Ryan Petrello
9784633728 pin websocket-client to a known working version
websocket-client 0.43.0 has a bug in its https client that breaks
requests:
https://github.com/websocket-client/websocket-client/issues/326

slackclient is affected by this bug:
https://github.com/slackapi/python-slackclient/issues/189

this commits also upgrades slackclient to add proxy support

see: #7088
see: #7093
2017-07-17 12:04:30 -04:00
Ryan Petrello
0239cd37fe pin celery to a known version that works
newer celery 3.x seems to a bug which causes forked celery worker
interprocess communication to hang (see: #6534)
2017-07-13 16:24:30 -04:00
Ryan Petrello
5f6ed1bd6c upgrade to the latest version of tacacs_plus 2017-07-12 13:52:37 -04:00