1
0
mirror of https://github.com/ansible/awx.git synced 2024-11-01 08:21:15 +03:00
Commit Graph

529 Commits

Author SHA1 Message Date
Shane McDonald
015234287c Ensure that a root user is always present in development environment
@AlanCoding was seeing errors in the development container when trying to run some commands as root. This fixes that.
2019-05-10 09:16:28 -04:00
AlanCoding
677a8b34ba
Fix bug in dev supervisor reparenting processes 2019-05-10 07:46:53 -04:00
Ryan Petrello
adfdfcdd0a
use make targets for dev supervisor commands 2019-05-09 13:45:49 -04:00
Shane McDonald
0be8fe521a Refactor Dockerfiles
This commit does a few things:

- Add the `--squash` option to the `awx-devel-build` make target. This reduces the resulting image size from 2.12 GB to 1.37 GB. I think we can get this down even more by inspecting the image contents.
- Reorganize commands so that the cache expires less often. Before this commit, any changes to the Makefile would essentially cause the entire image to rebuild.
- Break yum dependencies up into multiple lines. This makes it easier to see what changes in a diff.
- Use `n` to install our required version of node (rather `curl node | bash`). I’ve found this to be easier to maintain / more portable when working with other Dockerfiles.
- General organizational changes to make things easier to parse visually.
2019-05-08 20:36:41 -04:00
Ryan Petrello
766a5c0c3f
remove honcho in the dev environment (just use foregrounded supervisor)
using supervisor gives us the ability to restart entire processes on
code change (like the dispatcher and callback receiver)
2019-05-08 14:44:07 -04:00
softwarefactory-project-zuul[bot]
cf6a103207
Merge pull request #3818 from ryanpetrello/devel
merge in downstream changes

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-05-03 14:36:56 +00:00
softwarefactory-project-zuul[bot]
e5cf5be18d
Merge pull request #3815 from pmoravec/awx-pmoravec-tower-sos-plugin-improvements
sosreport plugin improvements

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-05-03 12:47:39 +00:00
Ryan Petrello
8d75fc5f56
Merge branch 'downstream' into devel 2019-05-03 07:58:25 -04:00
Pavel Moravec
8909a8a8e4 sosreport plugin to stick on sos >= 3.0 API only
As sosreport 3.0 was released 5 years ago, older sos versions can
be ignored / not further supported.

Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
2019-05-03 10:38:34 +02:00
Pavel Moravec
29822ee140 don't collect data from other sosreport plugins
Tower plugin shouldn't collect data that other sosreport plugins collect.

Further, few code optimizations in calling sos API are made.

Signed-off-by: Pavel Moravec <pmoravec@redhat.com>
2019-05-02 21:45:09 +02:00
Ryan Petrello
50f9c70afd
remove references to the (now defunct) fact receiver 2019-05-01 23:48:05 -04:00
Bill Nottingham
5d1346b956 Use AWX python interpreter for failure-event-handler.
python3 isn't in the normal path when using SCLs.
2019-04-29 20:32:50 -04:00
softwarefactory-project-zuul[bot]
aaeb2d6fb9
Merge pull request #3740 from wenottingham/lets-re-compose-ourselves
Sync docker-compose dockerfile with the actual production image build.

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-26 19:05:43 +00:00
Bill Nottingham
aa52e41c02 Kill the rabbitmq sos collection from our plugin.
It exists in upstream sosreport, and this can cause conflicts.
2019-04-24 14:15:17 -04:00
Bill Nottingham
0d18d46ccc Sync docker-compose dockerfile with image build dockerfile.
It was installing an IUS repo, rabbitmq-server, and other unneeded things.
2019-04-18 16:50:38 -04:00
Christian Adams
fb4495b1b5 add prometheus data to gitignore 2019-04-15 10:43:28 -04:00
softwarefactory-project-zuul[bot]
e8de7bc845
Merge pull request #3673 from rooftopcellist/metrics_uuids
add uuids to ping and metrics

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-04-13 11:45:47 +00:00
Christian Adams
40393e201f add uuids to ping and metrics 2019-04-12 18:37:40 -04:00
Ryan Petrello
17a803f49c
remove the old callback plugin import paths and callback-specific tests 2019-04-12 16:11:23 -04:00
Ryan Petrello
fc9da002d2
add an example config file and make target for starting a prometheus 2019-04-08 12:01:46 -04:00
Ryan Petrello
79d580d5b9
update periodic isolated cleanup to match the new paths post-runner 2019-04-05 09:43:27 -04:00
Ryan Petrello
ce2fc1a9dd
fix more py3 centos7 issues 2019-04-04 18:35:59 -04:00
Matthew Jones
fd3a423b07
The python 3.6 package seems to already make this symlink 2019-04-04 18:35:56 -04:00
Jake McDermott
3dee9f0512
add plugin for cyberark aim 2019-04-02 11:23:20 -04:00
Ryan Petrello
2824616ba6
add support for CyberArk Conjur (API v5) 2019-04-02 11:21:44 -04:00
Ryan Petrello
7a43f00a5d
add support for HashiCorp signed SSH certificates 2019-04-02 11:20:29 -04:00
Ryan Petrello
4ed5bca5e3
add credential plugin support for Azure Key Vault 2019-04-02 11:20:26 -04:00
Jake McDermott
c209955400
add credential plugin system and minimal working hashivault 2019-04-02 11:19:26 -04:00
Ryan Petrello
563a0cc2a4
move awx.main.expect to awx.main.isolated 2019-03-29 12:14:40 -04:00
Ryan Petrello
495dc2202f
more iso cleanup and bug fixes 2019-03-25 17:47:58 -04:00
Ryan Petrello
af8e071840
remove old callback plugin code and tests 2019-03-25 12:26:51 -04:00
softwarefactory-project-zuul[bot]
8b2fc26219
Merge pull request #3041 from chrismeyersfsu/runnerpy3
ansible-runner integration

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-03-21 22:21:50 +00:00
Matthew Jones
91cbaa1096 Install supervisor into the dev environment 2019-03-21 15:58:24 -04:00
Ryan Petrello
602ef9750f update isolated task execution for ansible-runner 2019-03-18 14:21:47 -04:00
softwarefactory-project-zuul[bot]
c41068edc4
Merge pull request #3420 from baxeno/https_download_postgresql_rpm
docker: yum: use https for postgresql pgdg rpm download

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-03-15 12:04:21 +00:00
Elijah DeLee
d39cfd1778 Update reference to test playbooks 2019-03-14 14:33:16 -04:00
Bruno Thomsen
2e0edcbabd docker: yum: use https for postgresql rpm download.
Signed-off-by: Bruno Thomsen <bruno.thomsen@gmail.com>
2019-03-14 17:14:17 +01:00
Ryan Petrello
50d8eb30e1
fix a failing test in the unit-test target used to generate swagger docs 2019-03-12 14:59:24 -04:00
Shane McDonald
e09684462c Working out some python3 kinks 2019-03-12 11:27:50 -04:00
softwarefactory-project-zuul[bot]
a9226fc25f
Merge pull request #3375 from shanemcd/devel
Fix dev environment when running as root on the host

Reviewed-by: Matthew Jones <mat@matburt.net>
             https://github.com/matburt
2019-03-07 17:20:32 +00:00
Matthew Jones
5cf7cc21c8
Revert "Fix chrome can not be started with unit-tests due to missing shared libraries"
This reverts commit d558ffd699.
2019-03-07 10:07:08 -05:00
Matthew Jones
499fd7b2f1
Revert "Fix chrome can not be started with unit-tests due to missing shared libraries"
This reverts commit 3e5f328b52.
2019-03-07 10:06:30 -05:00
Shane McDonald
b29a9cd86e Fix dev environment when running as root on the host
Without this, CURRENT_UID isnt actually passed in from the host, and wipes out /etc/passwd even when we’re actually running as root.

I tested this as a non-root user on Linux, and on Docker for Mac
2019-03-06 17:08:56 -05:00
softwarefactory-project-zuul[bot]
658e5f0fc8
Merge pull request #3272 from jladdjr/mo_stats
Add support for new playbook stats

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-28 18:24:03 +00:00
Jim Ladd
b7b0bdaeca Ansible 2.8 deprecates use of -U 2019-02-25 00:42:19 -08:00
Kia Lam
d2c345a374
Fix API lint error. 2019-02-22 13:06:20 -05:00
Kia Lam
0943f989ce
Fix various errors when trying to run make bulk_data.
- Properly import PrimoridialModel.
- Use floor division operator for range() method to avoid float vs int errors.
2019-02-22 11:59:18 -05:00
softwarefactory-project-zuul[bot]
889dae357b
Merge pull request #3235 from ryanpetrello/sql-profiling
add a custom DB backend that provides system-level SQL profiling

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2019-02-15 21:56:28 +00:00
Ryan Petrello
6f43875e80
record profile data in /var/log/tower, not /var/lib/awx 2019-02-15 14:34:55 -05:00
Ryan Petrello
eed94b641e
add a custom DB backend that provides system-level SQL profiling
run this command on _any_ node in an awx cluster:

$ awx-manage profile_sql --threshold=2.0 --minutes=1

...and for 1 minute, the timing for _every_ SQL query in _every_ awx
Python process that uses the Django ORM will be measured

queries that run longer than (in this example) 2 seconds will be
written to a per-process sqlite database in /var/lib/awx/profile, and
the file will contain an EXPLAIN VERBOSE for the query and the full
Python stack that led to that SQL query's execution (this includes not
just WSGI requests, but background processes like the runworker and
dispatcher)

$ awx-manage profile_sql --threshold=0

...can be used to disable profiling again (if you don't want to wait for
the minute to expire)
2019-02-14 15:04:46 -05:00
Ryan Petrello
9bebf3217e
remove usage of import * and enforce F405 in our linter
import * is a scourge upon the earth
2019-02-13 17:10:33 -05:00
Hideki Saito
3e5f328b52 Fix chrome can not be started with unit-tests due to missing shared libraries
- Added installation packages conforming to the unit-tests/Dockerfile

Signed-off-by: Hideki Saito <saito@fgrep.org>
2019-01-31 13:02:23 +09:00
Hideki Saito
d558ffd699 Fix chrome can not be started with unit-tests due to missing shared libraries
- Modify Dockerfile to install necesarry shared libraries for chrome

Signed-off-by: Hideki Saito <saito@fgrep.org>
2019-01-31 09:54:55 +09:00
Jake McDermott
73f617d811
Install dependencies for Chromium in unit test image 2019-01-28 15:57:17 -05:00
Ryan Petrello
daeeaf413a
clean up unnecessary usage of the six library (awx only supports py3) 2019-01-25 00:19:48 -05:00
Shane McDonald
d273472927
Enable py3 SCL if needed 2019-01-15 14:09:05 -05:00
Author: Jim Ladd
2c5bdf3611
fix some isolated py3 bugs 2019-01-15 14:09:05 -05:00
Ryan Petrello
c6482137d1
parametrize PYTHON for Ubuntu py35 support 2019-01-15 14:09:05 -05:00
Ryan Petrello
f223df303f
convert py2 -> py3 2019-01-15 14:09:01 -05:00
Ryan Petrello
f132ce9b64 switch image builds to py3 2019-01-15 13:25:13 -05:00
AlanCoding
77d2364022
Make docker environment interoperable with supervisorctl commands 2019-01-10 13:41:15 -05:00
Ryan Petrello
a810aaf319
fix isolated nodes in the dev environment 2018-11-28 09:54:39 -05:00
AlanCoding
0b32733dc8
set fixed container names 2018-11-26 08:26:57 -05:00
Ryan Petrello
b4f906ceb1
add dispatcher status to the sosreport 2018-11-20 12:12:02 -05:00
Ryan Petrello
a748a272fb
Merge remote-tracking branch 'tower/release_3.3.1' into devel 2018-11-01 12:07:02 -04:00
AlanCoding
19030b9d5f
apply docker-compose fix to cluster target too 2018-10-26 09:36:11 -04:00
Shane McDonald
a361b5da6e
Fix permissions when running dev container as non-root user
I wanted to pass `—user` to `docker-compose` up, but that option doesnt exist. To get around this, I had to record the uid on the host (CURRENT_UID), interpolate the variable in tools/docker-compose.yml, and detect that inside the container. I then piggy-backed on the /etc/passwd hack we use for scenarios with unpredictable uids.
2018-10-24 10:30:04 -04:00
Shane McDonald
7df63830ed
Remove reference to file that doesnt exist anymore 2018-10-24 10:30:03 -04:00
Matthew Jones
119b9475ea
Force openshift user behavior for uids over 2500 2018-10-16 11:30:04 -04:00
Hideki Saito
d5626a4f3e [3.3.1] Add files and output of commands to gather with sosreport
- Fixed issue #3064
2018-10-15 11:40:51 +09:00
softwarefactory-project-zuul[bot]
0a964b2bf6
Merge pull request #2266 from ansible/celery-tastes-bad
replace the celery-based task queue with a kombu-based implementation

Reviewed-by: https://github.com/softwarefactory-project-zuul[bot]
2018-10-12 18:40:54 +00:00
Ryan Petrello
ac80bc874a
more shippable -> zuul cleanup 2018-10-12 11:50:29 -04:00
Ryan Petrello
ff1e8cc356
replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Ryan Petrello
f87a09c46a
build swagger docs as part of CI 2018-10-10 10:27:54 -04:00
Shane McDonald
7a5cfd05a3 Run tests in Docker as non-root user 2018-10-09 15:16:01 -04:00
Ryan Petrello
ee11341430
more make clean tinkering 2018-10-05 13:40:08 -04:00
Matthew Jones
56263a5fea
Force ui cleanup in the test environment
Also allow using the system make
2018-10-05 13:40:07 -04:00
Matthew Jones
3a8bacb8ef
Add an initial check and gate job configuration for zuul
Updates for running ui tests and linters
2018-10-05 13:39:59 -04:00
Shane McDonald
1b4c3f56fa
Merge pull request #2113 from kialam/upgrade-node-lts
Upgrade Node and NPM to LTS
2018-09-18 12:46:30 -04:00
Shane McDonald
1371e394de Update Node version in dev container image 2018-09-18 12:37:20 -04:00
Shane McDonald
580004b395 Merge remote-tracking branch 'downstream/release_3.3.0' into devel
# Conflicts:
#	awx/main/notifications/slack_backend.py
2018-09-18 08:11:50 -04:00
Shane McDonald
07cb2aa9bb Fix / improve minishift dev env playbook
- Redo how we detect / set the minishift path
- Log into the correct admin account, once.
- Make sure commands that fail cause tasks to fail (s/;/&&/)
2018-08-15 15:01:40 -04:00
Ryan Petrello
18f6f68540
Merge remote-tracking branch 'tower/release_3.3.0' into devel 2018-08-10 11:54:34 -04:00
Ryan Petrello
6f5259d017
remove the network UI 2018-07-30 11:03:53 -04:00
Christian Adams
95a32aaa6e
Merge pull request #2606 from rooftopcellist/update_logging_doc
Update logging doc
2018-07-22 16:15:28 -04:00
chris meyers
7d24566120 more robustness 2018-07-20 15:02:44 -04:00
adamscmRH
ef29dab013 update logging readme 2018-07-19 11:41:39 -04:00
Matthew Jones
190525e835 Adding cluster development service definitions 2018-07-19 10:39:08 -04:00
Matthew Jones
5c400cdf79 Add local minishift development tooling
Based on mapping the local development tree through minishift
hostfolder interface.
2018-07-19 10:39:08 -04:00
Rob Ruma
6378479ee7 Updates to callback scripts that include retry functionality for bash and extra_vars handling for PowerShell
Signed-off-by: Rob Ruma <robruma@users.noreply.github.com>
2018-07-09 13:00:10 -04:00
Rob Ruma
6514c338a2 Adding ability to ignore SSL certificate validation errors, cleanup parameters and usage
Signed-off-by: Rob Ruma <robruma@users.noreply.github.com>
2018-07-06 23:18:12 -04:00
Bill Nottingham
0d03036fb1
Merge pull request #1629 from robruma/feature/update_callback_scripts
Updates to callback scripts
2018-06-25 15:47:25 -04:00
Guoqiang Zhang
530860c386 Force the python IO encoding to be utf_8 when using honcho 2018-06-22 09:35:28 -04:00
Ryan Petrello
84eacfc360
fix a few isolated dev issues
the main goal of this change is to make `make docker-isolated` work out
of the box

- specify the proper version for awx-expect --version
- update some deprecated playbook bits
- change the isolated container to privileged so bwrap will work
- fix awx-manage test_isolated_connection
- expedite the first isolated heartbeat so you don't have to wait 10m;
  this is accomplished by _not_ setting Instance.last_isolated_check to
  now() at insertion time (which causes the next check not to happen for
  10 minutes)
- fix a bug that caused isolated node execution to fail when bwrap was
  enabled

see: https://github.com/ansible/tower/issues/2150

This reverts commit 9863fe71dc.
2018-06-13 14:17:58 -04:00
Ryan Petrello
1733a20094
make sdb-listen work for docker-compose-cluster
use a different port range for each container, because docker can't map
them all to the same port range
2018-06-12 13:57:28 -04:00
Matthew Jones
22dd6ddfea
Remove the logstash container from the base dev docker compose
Now with less java running while you code!
2018-06-05 09:42:38 -04:00
chris meyers
9863fe71dc do not require privileged iso container
* The init call w/ privileged was causing my laptop to wig out. This
changeset still functions w/ out requiring privileged access.
2018-06-04 10:06:59 -04:00
Ryan Petrello
3abdf66794 run network ui tests in shippable and Jenkins 2018-05-24 09:07:53 -04:00
Ben Thomasson
89cabf7ca0
Fixes timeout when exporting YAML from network UI
Exporting YAML on dev envs with honcho and in production environments
would timeout.  This was due to daphne handling the export request
in dev but not in production.  This fixes network_ui to use uwsgi instead
of daphne to handle the request.
2018-05-02 10:31:36 -04:00
Wayne Witzel III
7151071779 Add awx-link make target 2018-04-25 11:55:57 -04:00