1
0
mirror of https://github.com/ansible/awx.git synced 2024-11-01 08:21:15 +03:00
Commit Graph

24821 Commits

Author SHA1 Message Date
chris meyers
87de0cf0b3
flake8, pytest, license fixes 2020-03-18 16:10:20 -04:00
chris meyers
18f5dd6e04
add websocket backplane documentation 2020-03-18 16:10:20 -04:00
chris meyers
89163f2915
remove redis broker url test
* We use sockets everywhere. Thus, password special characters no longer
are an issue.
2020-03-18 16:10:20 -04:00
chris meyers
59c9de2761
awxkit python2.7 compatible print
* awxkit still supports python2.7 so do not use fancy f"" yet; instead,
use .format()
2020-03-18 16:10:20 -04:00
chris meyers
b58c71bb74
remove broadcast websockets view 2020-03-18 16:10:20 -04:00
Ryan Petrello
1caa2e0287
work around a limitation in postgres notify to properly support copying
postgres has a limitation on its notify message size (8k), and the
messages we generate for deep copying functionality easily go over this
limit; instead of passing a giant nested data structure across the
message bus, this change makes it so that we temporarily store the JSON
structure in memcached, and look it up from *within* the task

see: https://github.com/ansible/tower/issues/4162
2020-03-18 16:10:20 -04:00
chris meyers
770b457430
redis socket support 2020-03-18 16:10:19 -04:00
chris meyers
d58df0f34a
fix sliding window calculation 2020-03-18 16:10:19 -04:00
chris meyers
3f5e2a3cd3
try to make openshift build happy 2020-03-18 16:10:19 -04:00
chris meyers
2b59af3808
safely operate in async or sync context 2020-03-18 16:10:19 -04:00
chris meyers
9e5fe7f5c6
translate Instance hostname to safe analytics name
* More robust translation of Instance hostname to analytics safe name by
replacing all non-alpha numeric characters with _
2020-03-18 16:10:19 -04:00
chris meyers
093d204d19
fix flake8 2020-03-18 16:10:19 -04:00
chris meyers
e25bd931a1
change dispatcher test to make required queue
* No fallback-default queue anymore. Queue must be explicitly provided.
2020-03-18 16:10:19 -04:00
chris meyers
8350bb3371
robust broadcast websocket error hanndling 2020-03-18 16:10:18 -04:00
chris meyers
d6594ab602
add broadcast websocket metrics
* Gather brroadcast websocket metrics and push them into redis every
configurable seconds.
* Pop metrics from redis in web view layer to display via the api on
demand
2020-03-18 16:10:18 -04:00
chris meyers
b6b9802f9e
increase per-channel capacity
* 100 is the default capacity for a channel. If the client doesn't read
the socket fast enough, websocket messages can and will be lost. This
increases the default to 10,000
2020-03-18 16:10:18 -04:00
chris meyers
0da94ada2b
add missing service name to dev env
* Dev env was bringing the wsbroadcast service up but not under the
tower-process dependency. This is cleaner.
2020-03-18 16:10:18 -04:00
chris meyers
3b9e67ed1b
remove channel group model
* Websocket user session <-> group subscription membership now resides
in Redis rather than the database.
2020-03-18 16:10:18 -04:00
chris meyers
14320bc8e6
handle websocket unsubscribe
* Do not return from blocking unsubscribe until _after_ putting the
gotten unsubscribe message on the queue so that it can be read by the
thread of execution that was unblocked.
2020-03-18 16:10:18 -04:00
chris meyers
3c5c9c6fde
move broadcast websocket out into its own process 2020-03-18 16:10:18 -04:00
chris meyers
f5193e5ea5
resolve rebase errors 2020-03-18 16:10:17 -04:00
chris meyers
03b73027e8
websockets aware of Instance changes
* New tower nodes that are (de)registered in the Instance table are seen
by the websocket layer and connected to or disconnected from by the
websocket broadcast backplane using a polling mechanism.
* This is especially useful for openshift and kubernetes. This will be
useful for standalone Tower in the future when the restarting of Tower
services is not required.
2020-03-18 16:10:17 -04:00
chris meyers
c06b6306ab
remove health info
* Sending health about websockets over websockets is not a great idea.
* I tried sending health data via prometheus and encountered problems
that will need PR's to prometheus_client library to solve. Circle back
to this later.
2020-03-18 16:10:17 -04:00
Shane McDonald
45ce6d794e
Initial migration of rabbitmq -> redis for k8s installs 2020-03-18 16:10:17 -04:00
chris meyers
e94bb44082
replace rabbitmq with redis
* local awx docker-compose and image build only.
2020-03-18 16:10:17 -04:00
chris meyers
be58906aed
remove kombu 2020-03-18 16:10:17 -04:00
chris meyers
403e9bbfb5
add websocket health information 2020-03-18 16:10:16 -04:00
chris meyers
ea29f4b91f
account for isolated job status
* We can not query the dispatcher running on isolated nodes to see if
the playbook is still running because that is the nature of isolated
nodes, they don't run the dispatcher nor do they run the message broker.
Therefore, we should query the control node that is arbitrating the
isolated work. If the control node process in the dispatcher is dead,
consider the iso job dead.
2020-03-18 16:10:16 -04:00
chris meyers
3f2d757f4e
update awxkit to use new unsubscribe event
* Instead of waiting an arbitrary number of seconds. We can now wait the
exact amount of time needed to KNOW that we are unsubscribed. This
changeset takes advantage of the new subscribe reply semantic.
2020-03-18 16:10:16 -04:00
chris meyers
feac93fd24
add websocket group unsubscribe reply
* This change adds more than just an unsubscribe reply.
* Websockets canrequest to join/leave groups. They do so using a single
idempotent request. This change replies to group requests over the
websockets with the diff of the group subscription. i.e. what groups the
user currenntly is in, what groups were left, and what groups were
joined.
2020-03-18 16:10:16 -04:00
chris meyers
088373963b
satisfy generic Role code
* User in channels session is a lazy user class. This does not conform
to what the generic Role ancestry code expects. The Role ancestry code
expects a User objects. This change converts the lazy object into a
proper User object before calling the permission code path.
2020-03-18 16:10:16 -04:00
chris meyers
5818dcc980
prefer simple async -> sync
* asgiref async_to_sync was causing a Redis connection _for each_ call
to emit_channel_notification i.e. every event that the callback receiver
processes. This is a "known" issue
https://github.com/django/channels_redis/pull/130#issuecomment-424274470
and the advise is to slow downn the rate at which you call
async_to_sync. That is not an option for us. Instead, we put the async
group_send call onto the event loop for the current thread and wait for
it to be processed immediately.

The known issue has to do with event loop + socket relationship. Each
connection to redis is achieved via a socket. That conection can only be
waiting on by the event loop that corresponds to the calling thread.
async_to_sync creates a _new thread_ for each invocation. Thus, a new
connection to redis is required. Thus, the excess redis connections that
can be observed via netstat | grep redis | wc -l.
2020-03-18 16:10:16 -04:00
chris meyers
dc6c353ecd
remove support for multi-reader dispatch queue
* Under the new postgres backed notify/listen message queue, this never
actually worked. Without using the database to store state, we can not
provide a at-most-once delivery mechanism w/ multi-readers.
* With this change, work is done ONLY on the node that requested for the
work to be done. Under rabbitmq, the node that was first to get the
message off the queue would do the work; presumably the least busy node.
2020-03-18 16:10:16 -04:00
chris meyers
50b56aa8cb
autobahn 20.1.2 released an hour ago
* 20.1.1 no longer available on pypi
2020-03-18 16:10:15 -04:00
chris meyers
3fec69799c
fix websocket job subscription access control 2020-03-18 16:10:15 -04:00
chris meyers
2a2c34f567
combine all the broker replacement pieces
* local redis for event processing
* postgres for message broker
* redis for websockets
2020-03-18 16:10:15 -04:00
chris meyers
558e92806b
POC postgres broker 2020-03-18 16:10:15 -04:00
chris meyers
355fb125cb
redis events 2020-03-18 16:10:15 -04:00
chris meyers
c8eeacacca
POC channels 2 2020-03-18 16:10:12 -04:00
softwarefactory-project-zuul[bot]
d0a3c5a42b
Merge pull request #6323 from AlanCoding/rm_verify_ssl_test
Replace verify_ssl test that did not work right

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 19:33:24 +00:00
softwarefactory-project-zuul[bot]
64139f960f
Merge pull request #6331 from marshmalien/fix-project-schedule-breadcrumb
Add nested project schedule detail breadcrumb

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 19:33:19 +00:00
softwarefactory-project-zuul[bot]
eda494be63
Merge pull request #6330 from rooftopcellist/fix_flakey_workflow_functest
Fix flaky workflow test & set junit family

Reviewed-by: Christian Adams <rooftopcellist@gmail.com>
             https://github.com/rooftopcellist
2020-03-18 19:27:26 +00:00
Christian Adams
4a0c371014 Fix flaky workflow test & set junit family 2020-03-18 14:02:33 -04:00
softwarefactory-project-zuul[bot]
6b43da35e1
Merge pull request #5745 from wenottingham/no-license-and-registration-please
Clean up a few more cases where we checked the license for features.

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 17:59:16 +00:00
softwarefactory-project-zuul[bot]
afa3b500d3
Merge pull request #6273 from AlanCoding/failure_verbosity
Print module standard out in test failure scenarios

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 17:50:16 +00:00
softwarefactory-project-zuul[bot]
c3efb13020
Merge pull request #6325 from AlanCoding/autohack
Automatically hack sys.path to make running tests easier

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 17:46:59 +00:00
Marliana Lara
eb28800082
Fix nested project schedule breadcrumb 2020-03-18 13:17:42 -04:00
softwarefactory-project-zuul[bot]
3219b9b4ac
Merge pull request #6318 from AlexSCorey/6100-ConvertWFJTandJTtoHooks
Moves JT Form to using react hooks and custom hooks

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 15:58:13 +00:00
softwarefactory-project-zuul[bot]
e9a48cceba
Merge pull request #6319 from squidboylan/collection_test_refactor
Collection test refactor

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 14:53:04 +00:00
softwarefactory-project-zuul[bot]
9a7fa1f3a6
Merge pull request #6313 from mabashian/jest-24-25-upgrade
Upgrade jest and babel-jest to latest (v25)

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-03-18 14:33:44 +00:00