1
0
mirror of https://github.com/ansible/awx.git synced 2024-10-31 06:51:10 +03:00
Commit Graph

51 Commits

Author SHA1 Message Date
Yanis Guenane
ca247182df yamllint: Make all files in awx pass yamllint
This commit updates all files that weren't passing yamllint for them to
pass.

A new yamllint target has been added. One can run `tox -e yamllint` or
`yamllint -s .` locally to ensure yaml files are still passing.

This check will be enabled in the CI so it can get on every new
contributions, and prevent merging non-compliant code.

Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2019-12-02 15:12:51 +01:00
Shane McDonald
f78c9f357d
Mount local projects directory inside of dev container.
Yesterday I noticed that we have awx/projects in our .gitignore. I am assuming
this pre-dates our containerized development environment. With this commit, any
project under awx/projects/ will be made available in the dev environment for
selection when creating a Manual project. This comes in super handy when
testing changes to playbooks locally.
2019-10-31 08:01:54 -04:00
Shane McDonald
3b89e894db Stop using PG SCL in dev env 2019-09-16 11:41:13 -04:00
Ryan Petrello
5ed97e0f65
change the default port range for the sdb debugging tool
the current range conflicts w/ a port used by the pycharm editor
2019-09-13 11:55:43 -04:00
Christian Adams
ec1e93cc69 Upgrade to postgres 10.6
- use awx-python in shebang in dev env
  - scl enable where needed for rhel7 & container installs
  - use scram-sha-256 pg user hashing by default
  - ensure psycopg2 is using the correct PG_CONFIG at build time for the right libpq version
2019-09-12 12:52:43 -04:00
Jose OrPa
04ab736f09 #3778 Upgrading postgresql to v10 2019-09-12 12:52:42 -04:00
Shane McDonald
f05bed6366 Use the source tree as the working directory for our dev env 2019-06-25 18:28:01 -04:00
Shane McDonald
5ca0cdb124 Avoid DNS timeout in non-Docker for Mac installs
Shaves 20 seconds off of rebooting the dev environment on Linux.
2019-05-10 12:31:08 -04:00
Shane McDonald
7d8a910be7 Improve dev environment init process
This ensures that /etc/passwd is always written, regardless of how the container starts.
2019-05-10 10:14:51 -04:00
Shane McDonald
b29a9cd86e Fix dev environment when running as root on the host
Without this, CURRENT_UID isnt actually passed in from the host, and wipes out /etc/passwd even when we’re actually running as root.

I tested this as a non-root user on Linux, and on Docker for Mac
2019-03-06 17:08:56 -05:00
AlanCoding
0b32733dc8
set fixed container names 2018-11-26 08:26:57 -05:00
Shane McDonald
a361b5da6e
Fix permissions when running dev container as non-root user
I wanted to pass `—user` to `docker-compose` up, but that option doesnt exist. To get around this, I had to record the uid on the host (CURRENT_UID), interpolate the variable in tools/docker-compose.yml, and detect that inside the container. I then piggy-backed on the /etc/passwd hack we use for scenarios with unpredictable uids.
2018-10-24 10:30:04 -04:00
Ryan Petrello
ff1e8cc356
replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Matthew Jones
5c400cdf79 Add local minishift development tooling
Based on mapping the local development tree through minishift
hostfolder interface.
2018-07-19 10:39:08 -04:00
Matthew Jones
22dd6ddfea
Remove the logstash container from the base dev docker compose
Now with less java running while you code!
2018-06-05 09:42:38 -04:00
Ben Thomasson
701150bd1a
Adds configuration for the network-ui websocket
* Configures NGINX for the network-ui websocket.
* Configures supervisor.conf for network_ui websocket.
2018-03-23 17:00:14 -04:00
Ryan Petrello
d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Matthew Jones
da0b686369
Adding jupyter notebook support to the AWX development environment
* Jupyter starts alongside the other awx services and is available on
  0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
2017-12-05 23:46:18 -05:00
Alan Rominger
dcfcfb6c7b Merge pull request #245 from AlanCoding/fix_supervisor
specify all group queues, get supervisor working
2017-08-14 11:31:00 -04:00
AlanCoding
2385f62311 specify all group queues, get supervisor working 2017-08-10 14:57:25 -04:00
Matthew Jones
67474c8de1 Updating development tooling to be more generic
* Not assuming GCR is being used for image hosting
* Breaking out the dev environment bootstrapping from service starting
2017-08-09 15:53:25 -04:00
Matthew Jones
3892e4e389 Tower -> AWX Tooling Migration
* Switching version number scheme and mechanism
* Refactor development tooling towards 'awx' paths and names
* Purging packaging details from Makefile
2017-07-21 17:06:45 -04:00
Alan Rominger
9fda4eee85 Remove extra_hosts that breaks dev environment
This was triggered by a docker update, and an empty value of DOCKER_HOST_IP resulted in an error running `make docker-compose`
2017-06-29 13:41:33 -04:00
AlanCoding
3cedcf22a9 add instance groups and queues to cluster tooling 2017-06-22 10:40:58 -04:00
Ryan Petrello
422950f45d Support for executing job and adhoc commands on isolated Tower nodes (#6524) 2017-06-14 11:47:30 -04:00
Matthew Jones
1a4a6273a4 Merge branch 'rampart_groups_setup_playbook' into devel
* rampart_groups_setup_playbook:
  Updating changelog for Instance Groups
  Fix an incorrect reference on instance group jobs list
  Purge remaining references to rampart groups
  Simplify can_access for instance groups on job templates
  Adding Instance Group permissions and tests
  Increase test coverage for task scheduler inventory updates
  Exit logic fixes for instance group tools
  View Fixes for instance groups
  new view to allow associations but no creations
  Updating acceptance documentation and system docs
  Updating unit tests for task manager refactoring
  Update views and serializers to support instance group (ramparts)
  Implementing models for instance groups, updating task manager
  Updating the setup playbook to support instance group installation
  Add nginx to server start and switch back to first tmux win
  Fix an issue where the local queue wouldn't use the rabbitmq name
2017-05-12 13:40:30 -04:00
Matthew Jones
4ced911c00 Implementing models for instance groups, updating task manager
* New InstanceGroup model and associative relationship with Instances
* Associative instances between Organizations, Inventory, and Job
  Templates and InstanceGroups
* Migrations for adding fields and tables for Instance Groups
* Adding activity stream reference for instance groups
* Task Manager Refactoring:
** Simplifying task manager relationships and move away from the
   interstitial hash tables
** Simplify dependency determination logic
** Reduce task manager runtime complexity by removing the partial
   references and moving the logic into the task manager directly or
   relying on Job model logic for determinism
2017-05-10 12:32:54 -04:00
Ryan Petrello
3126bfa1a2 add tooling to aid in remote debugging sessions 2017-05-09 15:36:13 -04:00
Matthew Jones
576e3bfc1a Adding logstash to default compose file 2017-03-07 16:48:51 -05:00
Ryan Petrello
4932310ad1 map docker container ports to allow access easier to memcache
this eases memcached interaction from your host, e.g.,
$ echo "get :1:LICENSE" | nc localhost 11211
2017-02-09 11:00:06 -05:00
Ryan Petrello
51b50ce735 map docker container ports to allow usage of celery.contrib.rdb
http://docs.celeryproject.org/en/latest/reference/celery.contrib.rdb.html
allows you to remotely debug running celery tasks with:

    from celery.contrib import rdb
    rdb.set_trace()

this will bind a remote Python debugger on a random TCP port between
6899-6999, which you can telnet into for remote task debugging
2017-01-31 09:13:07 -05:00
Wayne Witzel III
e9be93cd70 Update tower_tools image to run nginx 2016-10-18 11:16:38 -04:00
Wayne Witzel III
90ed95abd1 add https ports 2016-10-14 02:23:12 -04:00
Wayne Witzel III
4220246400 use uwsgi/daphne/nginx for dev 2016-10-14 01:45:08 -04:00
Matthew Jones
3255a17856 Exposing rabbitmq management console interface on compose 2016-09-15 10:46:45 -04:00
Matthew Jones
f5d2c5c18a Merge branch 'ramparts_and_ha' into devel
* ramparts_and_ha: (21 commits)
  Rename database migrations for devel integration
  Integrate callback receiver refactoring
  Fix an issue running jobs in the cluster
  Implement a more dynamic celery queue system
  Purge old munin monitors and tools
  Refactor Tower HA Instance logic and models
  Docker compose improvements
  Initial Docker Compose workflow for Tower cluster
  Add memcached role for setup playbook
  Removing qpid from deb packaging
  Refactor rabbitmq role
  Integrate memcached into setup playbook
  Remove mongodb shutdown task
  Remove dependency on erlang_sd_notify
  Add initial rabbitmq role
  Initial rabbitmq setup playbook integration
  Update development environment for rabbit
  Replace qpid with rabbitmq
  Remove redis role from setup playbook
  Update qpid packaging, remove migrations
  ...
2016-09-15 10:19:57 -04:00
Chris Meyers
c3bb7521de Merge pull request #3474 from chrismeyersfsu/feature-workflows
workflows
2016-09-15 10:12:58 -04:00
Matthew Jones
807cced571 Implement a more dynamic celery queue system
* Meant to be a starting point to more efficiently manage work routing
  and to balance work across all tower nodes
* Integrate flower as a dev tool that starts alongside other nodes.
  Helpful for observing and monitoring the queues/exchanges
* For the moment, force the task manager to only run on one node (not
  sure if this is needed)
* Define queues and routes for all task work
* Bump celery version to 3.1.23
* Expose flower through haproxy
2016-09-09 15:18:18 -04:00
Matthew Jones
eafb6c92b5 Docker compose improvements
* Make sure we explicitly set a hostname for tower nodes
* Switch rabbit vhost to use the root
2016-09-08 11:02:51 -04:00
Matthew Jones
ebf103f345 Initial Docker Compose workflow for Tower cluster
The goal is to share a common pattern with the existing development work
2016-09-08 10:18:14 -04:00
Matthew Jones
27ab6705e7 Merge branch 'ramparts_and_ha' into devel
* ramparts_and_ha:
  Removing qpid from deb packaging
  Refactor rabbitmq role
  Integrate memcached into setup playbook
  Remove mongodb shutdown task
  Remove dependency on erlang_sd_notify
  Add initial rabbitmq role
  Initial rabbitmq setup playbook integration
  Update development environment for rabbit
  Replace qpid with rabbitmq
  Remove redis role from setup playbook
  Update qpid packaging, remove migrations
  Integrate packaging for qpid/memcached
2016-09-06 09:47:11 -04:00
Matthew Jones
f0b7622672 Update development environment for rabbit 2016-08-31 14:47:20 -04:00
Chris Meyers
2cfdee3b21 turn job templates in jobs via launch 2016-08-31 09:00:25 -04:00
Matthew Jones
b507dc53ea Putting a bow on some of the new dev workflow
* Documentation of the developer variety
* Defaulting the COMPOSE_TAG to develop, can be overridden
* Automatic docker login assuming gcr login
* Including a manual build step as an alternative to gcr
* Make qpid container shut the f**k up when it's running
2016-08-25 11:57:31 -04:00
Matthew Jones
c8e2f73d48 Support pulling the dev container image from gcr
This will be branch-relative.  If you are ont he `devel` branch then it
will pull that tag, it will try to pull the branch you are on and can be
overridden with the COMPOSE_TAG environment variable
2016-08-24 22:45:20 -04:00
Matthew Jones
dcb4959443 Initial qpid development work
* Switch base tower devel image from u14.04 to c7
* Switch container image to build python dependencies into itself
  instead of forcing it to be built on startup
* Upgrade venv pip to 8.1.2
* Neuter queue.py which was heavily tied to redis and was basically
  orphaned code
* Alter local_settings to override default cache settings and use
  memcached
* Alter local settings to refer to qpid instead of redis for celery
  broker
* Purge redis python dependencies and add qpid and memcached
* Update docker-compose to purge redis and add qpid and memcached
2016-08-24 16:23:47 -04:00
Matthew Jones
20fe9e9614 Updating docker dev workflow
* Moving to new docker compose file format
* Adding some things that will be needed for python dependencies
2016-06-08 12:25:14 -04:00
Leigh Johnson
097fba690b update docker-compose build, resolves #1377 2016-04-01 10:35:07 -04:00
Joshua "jag" Ginsberg
f25b76aee9 Integrate statsd metrics into ansible playbook execution.
* Add dependencies for pystatsd and django-statsd-mozilla
* Default turned off except for development environment
* Modify docker-compose to install statsd/graphite host
2016-01-26 16:54:24 -05:00
Matthew Jones
99da0965ef Add a docker-ui container to the compose workflow 2015-10-19 12:07:09 -04:00