1
0
mirror of https://github.com/ansible/awx.git synced 2024-11-01 16:51:11 +03:00
Commit Graph

576 Commits

Author SHA1 Message Date
softwarefactory-project-zuul[bot]
12a04a6da6
Merge pull request #2956 from ansible/swap-diff-order-schema-change
swap file order so diff of schema makes more sense

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-01-11 20:11:42 +00:00
AlanCoding
77d2364022
Make docker environment interoperable with supervisorctl commands 2019-01-10 13:41:15 -05:00
Elijah DeLee
e5ad2e44fb
swap file order so diff of schema makes more sense
This way we will get +'s next to new content :)
2018-12-20 15:05:01 -05:00
Ryan Petrello
a810aaf319
fix isolated nodes in the dev environment 2018-11-28 09:54:39 -05:00
Elijah DeLee
ca84d312ce Rename schema job to be more clear about its purpose
The make target fails when it detects schema changes, not when schema is invalid.

Also update CONTRIBUTING.md to include information about zuul jobs.
2018-11-20 07:42:10 -06:00
Elijah DeLee
4ae1fdef05 Ignore differences in whitespace for schema validation 2018-11-16 09:47:33 -05:00
Elijah DeLee
a68e22b114 Add tox target to detect schema changes
Fetches reference schema from public bucket
Still need define method for updating reference schema on merge.
2018-11-15 16:25:13 -05:00
Elijah DeLee
d70cd113e1 Reduce duplicated logic for genschema target 2018-11-15 15:29:35 -05:00
Matthew Jones
f737fc066f Generate schema suitable for comparing for schema changes 2018-11-15 15:29:35 -05:00
Ryan Petrello
a748a272fb
Merge remote-tracking branch 'tower/release_3.3.1' into devel 2018-11-01 12:07:02 -04:00
Shane McDonald
a361b5da6e
Fix permissions when running dev container as non-root user
I wanted to pass `—user` to `docker-compose` up, but that option doesnt exist. To get around this, I had to record the uid on the host (CURRENT_UID), interpolate the variable in tools/docker-compose.yml, and detect that inside the container. I then piggy-backed on the /etc/passwd hack we use for scenarios with unpredictable uids.
2018-10-24 10:30:04 -04:00
Ryan Petrello
ff1e8cc356
replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Bill Nottingham
be01bed34b Purge inventory script requirements from the AWX virtual environment.
boto is still used by AWX itself.
2018-10-11 09:45:41 -04:00
Ryan Petrello
f87a09c46a
build swagger docs as part of CI 2018-10-10 10:27:54 -04:00
Jake McDermott
7d5f6aa49d
don't update lock file by default 2018-10-10 09:23:37 -04:00
Shane McDonald
7a5cfd05a3 Run tests in Docker as non-root user 2018-10-09 15:16:01 -04:00
Ryan Petrello
29b90b700e
minor docker-compose fix 2018-10-05 13:40:10 -04:00
Matthew Jones
f7c5289195
Clean up CI compose test invocation 2018-10-05 13:40:09 -04:00
Matthew Jones
56263a5fea
Force ui cleanup in the test environment
Also allow using the system make
2018-10-05 13:40:07 -04:00
Matthew Jones
3a8bacb8ef
Add an initial check and gate job configuration for zuul
Updates for running ui tests and linters
2018-10-05 13:39:59 -04:00
Shane McDonald
50fe0392ed Updates to versioning system.
https://github.com/ansible/awx/issues?q=%22--first-parent%22
2018-09-28 15:48:33 -04:00
Shane McDonald
62e3b9e3b6 Driveby cleanup: use built-in Make variable 2018-09-26 21:27:07 -04:00
Shane McDonald
90bd27f5a8 Whitespace fix
I’m not actually this pedantic, I just need something to tag.
2018-09-12 13:41:56 -04:00
Ryan Petrello
30fbeb43bb
fail CI if the change includes model changes that are missing migrations 2018-08-16 17:43:32 -04:00
Ryan Petrello
6f5259d017
remove the network UI 2018-07-30 11:03:53 -04:00
Matthew Jones
190525e835 Adding cluster development service definitions 2018-07-19 10:39:08 -04:00
Ryan Petrello
34fe54c6e0
Makefile update. 2018-07-16 14:45:44 -04:00
Ryan Petrello
84eacfc360
fix a few isolated dev issues
the main goal of this change is to make `make docker-isolated` work out
of the box

- specify the proper version for awx-expect --version
- update some deprecated playbook bits
- change the isolated container to privileged so bwrap will work
- fix awx-manage test_isolated_connection
- expedite the first isolated heartbeat so you don't have to wait 10m;
  this is accomplished by _not_ setting Instance.last_isolated_check to
  now() at insertion time (which causes the next check not to happen for
  10 minutes)
- fix a bug that caused isolated node execution to fail when bwrap was
  enabled

see: https://github.com/ansible/tower/issues/2150

This reverts commit 9863fe71dc.
2018-06-13 14:17:58 -04:00
chris meyers
97ab6449b9 parallelize test running 2018-05-16 14:29:15 -04:00
Ben Thomasson
af4367b222
Adds unit tests for network_ui
* Covers 100% of non-migration python lines of code
2018-05-03 14:13:33 -04:00
AlanCoding
ac20aa954a
Replace logging-related restart with dynamic handler
refactor existing handlers to be the related
  "real" handler classes, which are swapped
  out dynamically by external logger "proxy" handler class

real handler swapout only done on setting change

remove restart_local_services method
get rid of uWSGI fifo file

change TCP/UDP return type contract so that it mirrors
  the request futures object
add details to socket error messages
2018-05-02 09:47:22 -04:00
Ryan Petrello
c8e416f0b7 restore the celery hostname 2018-04-27 11:40:58 -04:00
Wayne Witzel III
7151071779 Add awx-link make target 2018-04-25 11:55:57 -04:00
Chris Meyers
a2901a47ee
Merge pull request #1410 from chrismeyersfsu/fix-revert_tower_special_group
send all tower work to a user-hidden queue
2018-04-20 14:21:50 -04:00
chris meyers
a56771c8f0 send all tower work to a user-hidden queue
* Before, we had a special group, tower, that ran any async work that
tower needed done. This allowed users fine grain control over which
nodes did background work. However, this granularity was too complicated
for users. So now, all tower system work goes to a special non-user
exposed celery queue. Tower remains the fallback instance group to
execute jobs on. The tower group will be created upon install and
protected from deletion.
2018-04-20 13:04:36 -04:00
AlanCoding
5eaffb3520
run ansible tests separately 2018-04-16 10:32:51 -04:00
Ben Thomasson
43601be8a7 Removes --fake-initial from awx-manage migrate.
The --fake-initial option is no longer needed and can cause
application with an initial migration to fail as was seen
in the network_ui application.
2018-04-02 15:34:39 -04:00
Ryan Petrello
13672cc88c
fix busted shippable builds 2018-03-05 14:16:42 -05:00
Chris Meyers
d551566b4d
Merge pull request #1372 from chrismeyersfsu/old-celery3
celery 4.x to 3.x roll back
2018-02-27 15:26:46 -05:00
chris meyers
e4470aa4cf remove uneeded celery configs
* Celery routes and queues are set and defined at runtime. Thus, a
static definition of routes and queues is not needed.
2018-02-27 11:36:55 -05:00
Ryan Petrello
d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Ryan Petrello
605c5e3276
fix celery pid restart issues 2018-02-09 11:03:00 -05:00
Ryan Petrello
8b976031cb
use VERSION_TARGET for Swagger doc generation 2018-02-06 10:48:51 -05:00
Ryan Petrello
7ff9f0b7d1
build example Swagger request and response bodies from our API tests 2018-02-06 10:36:25 -05:00
Ryan Petrello
57c22c20b2
add support for building swagger/OpenAPI JSON
to build, run `make swagger`; a file named `swagger.json` will be
written to the current working directory
2018-02-06 10:12:57 -05:00
Matthew Jones
6163cc6b5c
Merge pull request #1058 from ansible/scalable_clustering
Implement Container Cluster-based dynamic scaling
2018-02-02 09:22:06 -05:00
Ryan Petrello
35230eded1
run the celery reload in a shell so the uwsgi hook isn't fatal on fail 2018-02-01 22:32:08 -05:00
Matthew Jones
d9e774c4b6
Updates for automatic triggering of policies
* Switch policy router queue to not be "tower" so that we don't
  fall into a chicken/egg scenario
* Show fixed policy list in serializer so a user can determine if
  an instance is manually managed
* Change IG membership mixin to not directly handle applying topology
  changes. Instead it just makes sure the policy instance list is
  accurate
* Add create/delete hooks for instances and groups to trigger policy
  re-evaluation
* Update policy algorithm for fairer distribution
* Fix an issue where CELERY_ROUTES wasn't renamed after celery/django
  upgrade
* Update unit tests to be more explicit
* Update count calculations used by algorithm to only consider
  non-manual instances
* Adding unit tests and fixture
* Don't propagate logging messages from awx.main.tasks and
  awx.main.scheduler
* Use advisory lock to prevent policy eval conflicts
* Allow updating instance groups from view
2018-02-01 16:56:16 -05:00
Chris Meyers
0e97dc4b84
Beat and celery clustering fixes
* use embedded beat rather than standalone
* dynamically set celeryd hostname at runtime
* add embeded beat flag to celery startup
* Embedded beat mode routes will piggyback off of celery worker setup
signal
2018-02-01 16:47:33 -05:00
Chris Meyers
c9ff3e99b8
celeryd attach to queues dynamically
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process

add support for separate beat process
2018-02-01 16:37:33 -05:00
Ryan Petrello
e7ed4811c1
reload the entire celery worker pool when uwsgi reloads the Python app
this is for the development environment only; when uwsgi notices a code
change, it automatically reloads the uwsgi workers; this patch includes
a hook that sends `SIGHUP` to the celery process, causing it to spawn
a new set of workers as well
2018-01-25 14:59:40 -05:00
Shane McDonald
52e531625c Use first parent commit when determining version from tags
We were having issues where an older tag was being outputed from `git describe`.

From the man page:

Follow only the first parent commit upon seeing a merge commit. This is useful when you wish to not match tags on branches merged in the history of the target commit.
2018-01-15 11:01:47 -05:00
Matthew Jones
9dbcc5934e
Merge remote-tracking branch 'tower/release_3.2.2' into devel 2017-12-13 12:25:47 -05:00
Matthew Jones
34c206fab0
Bump psql-container pg version to 9.6 2017-12-11 12:01:28 -05:00
Matthew Jones
da0b686369
Adding jupyter notebook support to the AWX development environment
* Jupyter starts alongside the other awx services and is available on
  0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
2017-12-05 23:46:18 -05:00
Wayne Witzel III
8faf588775
Update package versions, settings, and tooling 2017-11-09 17:17:30 -05:00
Jake McDermott
77e11fe8fe
collect unit test results for shippable
Signed-off-by: Jake McDermott <jmcdermott@ansible.com>
2017-11-02 01:22:07 -04:00
Jake McDermott
4883876dc5
run both unit test suites and linting tasks 2017-11-01 13:36:34 -04:00
Ryan Petrello
5bb06fdb50
stop hard-coding the awx version in the isolated development environment
see: #296
2017-10-13 12:17:04 -04:00
Matthew Jones
523734a6af
omit old unit tests from "test" make target 2017-09-21 21:33:34 -04:00
Jake McDermott
2a9c97f96d
remove unused config 2017-09-15 17:22:23 -04:00
Wayne Witzel III
f1c9d5a8f4
Add PR template and tweak make VERSION output to be pastable 2017-09-12 09:56:35 -04:00
Matthew Jones
6e4d886ea4
Reduce the size of the production container image 2017-08-22 13:21:53 -04:00
Alan Rominger
dcfcfb6c7b Merge pull request #245 from AlanCoding/fix_supervisor
specify all group queues, get supervisor working
2017-08-14 11:31:00 -04:00
Matthew Jones
dd44f7a640 Merge pull request #236 from matburt/contributing_and_other_docs
CONTRIBUTING.md and other docs
2017-08-11 10:30:55 -04:00
AlanCoding
2385f62311 specify all group queues, get supervisor working 2017-08-10 14:57:25 -04:00
Matthew Jones
67474c8de1 Updating development tooling to be more generic
* Not assuming GCR is being used for image hosting
* Breaking out the dev environment bootstrapping from service starting
2017-08-09 15:53:25 -04:00
AlanCoding
afb267cee9 update awx-manage command to new name 2017-08-08 15:18:07 -04:00
Shane McDonald
f4aacebb86 Fix OpenShift container image builds 2017-08-07 17:31:18 -04:00
Shane McDonald
9c56a18ced Remove PACKAGE_BUILD logic from Makefile
Also fix some whitespace
2017-08-03 16:22:59 -04:00
Matthew Jones
af3652bb2f Updates to production container builds for new repository 2017-08-03 10:39:04 -04:00
Shane McDonald
ac9fcdb63e Bump setuptools version
This addresses a bug I was seeing when running `pip download setuptools --no-binary=:all:`.
2017-08-01 10:13:04 -04:00
Shane McDonald
97b7321920 Cleaning up Makefile
Removing references to and definitions of unused targets
2017-07-27 16:46:40 -04:00
Shane McDonald
1b8b6e1159 Remove default goal from makefile
There is no build target anymore
2017-07-27 16:32:09 -04:00
Shane McDonald
f3130e06c9 Add i18n flag file back to makefile
This was accidentally deleted during the repo split
2017-07-27 16:30:29 -04:00
Shane McDonald
9979513e24 Remove unused variable from setup.py invocations 2017-07-25 11:10:18 -04:00
Shane McDonald
3f3b02dbea Fix whitespace 2017-07-25 11:10:18 -04:00
Shane McDonald
7695cb6419 Fix version when installing from sdist
When installing an sdist, setup.py is invoked on the machine you're installing on. We extract the version from a git tag, but the repo is not included in the sdist. The git describe --long command will silently fail and cause the installed package to report version 0.0.0.0.
2017-07-25 11:09:51 -04:00
Matthew Jones
1a706b6154 Commands to generate a python wheel for packaging 2017-07-24 12:36:52 -04:00
Shane McDonald
17db724932 Make packaging work with our internal build systems 2017-07-21 21:27:02 -04:00
Matthew Jones
3892e4e389 Tower -> AWX Tooling Migration
* Switching version number scheme and mechanism
* Refactor development tooling towards 'awx' paths and names
* Purging packaging details from Makefile
2017-07-21 17:06:45 -04:00
Shane McDonald
4a175222b5 Cache i18n make targets 2017-07-17 09:41:47 -04:00
Shane McDonald
3450937329 Install tower-license in debs 2017-07-14 11:35:24 -04:00
Shane McDonald
d273272d9a Add tower-license as a source to the RPM 2017-07-14 11:35:24 -04:00
AlanCoding
592df0c8e2 new target to make the UI 2017-07-07 11:37:51 -04:00
Matthew Jones
d443f5cf48 Merge branch 'containerization' into devel
* containerization:
  Updating make targets for fixes to bringing cluster up and svcs
  Adding Makefile targets for local builds/deployments
  Moving Openshift work to installer directory to reduce conflicts
  Support deploying Tower into OpenShift
  Implementing containerized tower packaging
2017-06-30 16:21:03 -04:00
Matthew Jones
d82c69680b Updating make targets for fixes to bringing cluster up and svcs 2017-06-30 16:06:57 -04:00
Matthew Jones
0ee3ad8c14 Adding Makefile targets for local builds/deployments
Updating paths
2017-06-30 15:25:42 -04:00
Matthew Jones
d1ec8f93c0 Support deploying Tower into OpenShift 2017-06-29 16:31:07 -04:00
Ryan Petrello
611c42f741 fix the make rdb debugging tool 2017-06-29 15:40:09 -04:00
Michael Foord
f93919a8af Remove deprecated and unneeded -e flag to docker login 2017-06-29 12:40:09 +01:00
Shane McDonald
4a8e8d4fbf ansible-tower-isolated RPM 2017-06-28 15:02:34 -04:00
Shane McDonald
4e24a77370 Clean up rpm-sign 2017-06-28 15:02:34 -04:00
Ryan Petrello
3000f52a92 install a randomized RSA key for controller -> isolated rampart auth
see: #6507
2017-06-27 10:53:44 -04:00
AlanCoding
3cedcf22a9 add instance groups and queues to cluster tooling 2017-06-22 10:40:58 -04:00
Chris Meyers
fee16fe391 Merge pull request #6588 from chrismeyersfsu/feature-fact_cache
initial tower fact cache implementation
2017-06-22 09:58:28 -04:00
Ryan Petrello
45bdd9f747 improve readability of the honcho console logs in the dev environment
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
  are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
2017-06-22 09:19:32 -04:00
Chris Meyers
ec2e537f63 remove fache cache receiver 2017-06-21 13:45:27 -04:00
Alan Rominger
2c92f623d8 Merge pull request #6617 from ansible/isolated_setup
setup playbook and heartbeat for isolated deployments
2017-06-20 17:20:53 -04:00
Ryan Petrello
2ba9e56033 Include the Tower venv in the isolated bubblewrapped arguments.
This moves the container-based code location and venvs.
The goal here is that the paths of Tower source for isolated
vs normal nodes matches (both in prod and local development) so that we
don't have to add a bunch of additional bwrap argument logic for
<location-of-isolated-tower-venv>.
2017-06-19 15:52:54 -04:00