1
0
mirror of https://github.com/ansible/awx.git synced 2024-11-01 08:21:15 +03:00
Commit Graph

436 Commits

Author SHA1 Message Date
Ryan Petrello
84eacfc360
fix a few isolated dev issues
the main goal of this change is to make `make docker-isolated` work out
of the box

- specify the proper version for awx-expect --version
- update some deprecated playbook bits
- change the isolated container to privileged so bwrap will work
- fix awx-manage test_isolated_connection
- expedite the first isolated heartbeat so you don't have to wait 10m;
  this is accomplished by _not_ setting Instance.last_isolated_check to
  now() at insertion time (which causes the next check not to happen for
  10 minutes)
- fix a bug that caused isolated node execution to fail when bwrap was
  enabled

see: https://github.com/ansible/tower/issues/2150

This reverts commit 9863fe71dc.
2018-06-13 14:17:58 -04:00
Ryan Petrello
1733a20094
make sdb-listen work for docker-compose-cluster
use a different port range for each container, because docker can't map
them all to the same port range
2018-06-12 13:57:28 -04:00
Matthew Jones
22dd6ddfea
Remove the logstash container from the base dev docker compose
Now with less java running while you code!
2018-06-05 09:42:38 -04:00
chris meyers
9863fe71dc do not require privileged iso container
* The init call w/ privileged was causing my laptop to wig out. This
changeset still functions w/ out requiring privileged access.
2018-06-04 10:06:59 -04:00
Ryan Petrello
3abdf66794 run network ui tests in shippable and Jenkins 2018-05-24 09:07:53 -04:00
Ben Thomasson
89cabf7ca0
Fixes timeout when exporting YAML from network UI
Exporting YAML on dev envs with honcho and in production environments
would timeout.  This was due to daphne handling the export request
in dev but not in production.  This fixes network_ui to use uwsgi instead
of daphne to handle the request.
2018-05-02 10:31:36 -04:00
Wayne Witzel III
7151071779 Add awx-link make target 2018-04-25 11:55:57 -04:00
AlanCoding
4036f64cc4
update the dev supervisor file to match recent change 2018-04-20 14:53:21 -04:00
Alan Rominger
c0fd56c654
Merge pull request #1381 from AlanCoding/test_docker
Run ansible tests separately
2018-04-19 13:04:14 -04:00
Bill Nottingham
ea9278a3ea Drop /var/lib/awx/job_status listing from sosreport.
This directory is not persistently populated any more.
2018-04-19 10:47:45 -04:00
Bill Nottingham
40cf80db1d Add rabbitmq status commands as well. 2018-04-17 12:25:26 -04:00
Bill Nottingham
ade51c93d8 Add some more information to sosreports.
- tower cluster configuration
- list of all venvs (including custom ones)
2018-04-17 12:25:26 -04:00
AlanCoding
5eaffb3520
run ansible tests separately 2018-04-16 10:32:51 -04:00
Ben Thomasson
701150bd1a
Adds configuration for the network-ui websocket
* Configures NGINX for the network-ui websocket.
* Configures supervisor.conf for network_ui websocket.
2018-03-23 17:00:14 -04:00
Ben Thomasson
48d801271c
Imports prototype from ansible-network-ui
The ansible-network-ui prototype project builds a standalone Network UI
outside of Tower as its own Django application. The original prototype
code is located here:
https://github.com/benthomasson/ansible-network-ui.

The prototype provides a virtual canvas that supports placing
networking devices onto 2D plane and connecting those devices together
with connections called links.  The point where the link connects
to the network device is called an interface.  The devices, interfaces,
and links may all have their respective names.  This models physical
networking devices is a simple fashion.

The prototype implements a pannable and zoomable 2D canvas in using SVG
elements and AngularJS directives.   This is done by adding event
listeners for mouse and keyboard events to an SVG element that fills the
entire browser window.

Mouse and keyboard events are handled in a processing pipeline where
the processing units are implemented as finite state machines that
provide deterministic behavior to the UI.

The finite state machines are built in a visual way that makes
the states and transitions clearly evident.  The visual tool for
building FSM is located here:
https://github.com/benthomasson/fsm-designer-svg.   This tool
is a fork of this project where the canvas is the same.  The elements
on the page are FSM states and the directional connections are called
transitions.   The bootstrapping of the FSM designer tool and
network-ui happen in parallel.  It was useful to try experiemental
code in FSM designer and then import it into network-ui.

The FSM designer tool provides a YAML description of the design
which can be used to generate skeleton code and check the implementation
against the design for discrepancies.

Events supported:

* Mouse click
* Mouse scroll-wheel
* Keyboard events
* Touch events

Interactions supported:

* Pan canvas by clicking-and-dragging on the background
* Zooming canvas by scrolling mousewheel
* Adding devices and links by using hotkeys
* Selecting devices, interaces, and links by clicking on their icon
* Editing labels on devices, interfaces, and links by double-clicking on
  their icon
* Moving devices around the canvas by clicking-and-dragging on their
  icon

Device types supported:

* router
* switch
* host
* racks

The database schema for the prototype is also developed with a visual
tool that makes the relationships in the snowflake schema for the models
quickly evident.  This tool makes it very easy to build queries across
multiple tables using Django's query builder.

See: https://github.com/benthomasson/db-designer-svg

The client and the server communicate asynchronously over a websocket.
This allows the UI to be very responsive to user interaction since
the full request/response cycle is not needed for every user
interaction.

The server provides persistence of the UI state in the database
using event handlers for events generated in the UI.  The UI
processes mouse and keyboard events, updates the UI, and
generates new types of events that are then sent to the server
to be persisted in the database.

UI elements are tracked by unique ids generated on the client
when an element is first created.  This allows the elements to
be correctly tracked before they are stored in the database.

The history of the UI is stored in the TopologyHistory model
which is useful for tracking which client made which change
and is useful for implementing undo/redo.

Each message is given a unique id per client and has
a known message type.  Message types are pre-populated
in the MessageType model using a database migration.

A History message containing all the change messages for a topology is
sent when the websocket is connected.  This allows for undo/redo work
across sessions.

This prototype provides a server-side test runner for driving
tests in the user interface.  Events are emitted on the server
to drive the UI.  Test code coverage is measured using the
istanbul library which produces instrumented client code.
Code coverage for the server is is measured by the coverage library.

The test code coverage for the Python code is 100%.
2018-03-23 17:00:14 -04:00
Rob Ruma
e8b026648f Updates to callback scripts that include retry functionality for bash and extra_vars handling for PowerShell
Signed-off-by: Rob Ruma <robruma@users.noreply.github.com>
2018-03-21 08:22:08 -04:00
adamscmRH
8d460490c1 add xmlsec flag to docker installs 2018-03-14 14:28:35 -04:00
Ryan Petrello
13672cc88c
fix busted shippable builds 2018-03-05 14:16:42 -05:00
chris meyers
b69315f2eb fix up the config map watcher script
* invoke main() in config watcher script
* correctly call hash update by passing the filename
2018-03-01 17:06:07 -05:00
Ryan Petrello
d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Bill Nottingham
c1203942e0 Remove obsolete ansible_awx.egg-info. 2018-02-26 15:04:37 -05:00
Chris Church
4ec683efcb
Fix dummy data generator for WFJT node credentials. 2018-02-21 08:55:43 -05:00
Matthew Jones
0d2daecf49
Merge pull request #1243 from matburt/fix_clustering_isolated
Fix isolated instance clustering implementation
2018-02-14 08:32:24 -05:00
Matthew Jones
925d9efecf
Fixing up isolated node execution after cluster changes
* Rework queue detection to include control groups and isolated instances
* Fix up development tooling around isolated nodes
* Update unit tests
2018-02-13 21:51:38 -05:00
cclauss
260aec543e Misc Python 3 changes: execfile(), file(), reduce(), StandardError 2018-02-09 17:17:05 +01:00
Alan Rominger
b3dcfc8c18
Merge pull request #903 from ansible/item_copy
Implement item copy feature
2018-02-08 15:51:16 -05:00
cclauss
e18838a4b7 Modernize Python 2 code to get ready for Python 3 2018-02-08 17:26:22 +01:00
Matthew Jones
c819560d39
Add automatic deprovisioning support, only enabled for openshift
* Implement a config watcher for service restarts
* If the configmap bind point changes then restart all services
2018-02-01 16:51:40 -05:00
Chris Meyers
0e97dc4b84
Beat and celery clustering fixes
* use embedded beat rather than standalone
* dynamically set celeryd hostname at runtime
* add embeded beat flag to celery startup
* Embedded beat mode routes will piggyback off of celery worker setup
signal
2018-02-01 16:47:33 -05:00
Chris Meyers
c9ff3e99b8
celeryd attach to queues dynamically
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process

add support for separate beat process
2018-02-01 16:37:33 -05:00
Shane McDonald
67c6591f6f Use newer version of git in dev image
More fallout from #982
2018-01-22 13:57:44 -05:00
Aaron Tan
a2fd78add4 Implement item copy feature
See acceptance doc for implement details.

Signed-off-by: Aaron Tan <jangsutsr@gmail.com>
2018-01-02 10:20:44 -05:00
Matthew Jones
9dbcc5934e
Merge remote-tracking branch 'tower/release_3.2.2' into devel 2017-12-13 12:25:47 -05:00
Matthew Jones
da0b686369
Adding jupyter notebook support to the AWX development environment
* Jupyter starts alongside the other awx services and is available on
  0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
2017-12-05 23:46:18 -05:00
AlanCoding
67867cf0c8
flake8: comply with new E722 rule 2017-12-01 12:16:44 -05:00
Hideki Saito
febfcf709d
fix incorrect virtualenv path for sosreport plugin 2017-11-29 09:57:41 -05:00
Jake McDermott
34195a1b35
add pycrypto distro package to tools awx container image 2017-11-28 12:45:53 -05:00
Ryan Petrello
28ce9b700e
replace all Job/JT relations with a single M2M credentials relation
Includes backwards compatibility for now-deprecated .credential,
.vault_credential, and .extra_credentials

This is a building block for multi-vault implementation and Alan's saved
launch configurations (both coming soon)

see: https://github.com/ansible/awx/issues/352
see: https://github.com/ansible/awx/issues/169
2017-11-14 12:49:12 -05:00
AlanCoding
805170ffd7
get development supervisor use working again 2017-11-13 20:31:32 -05:00
Wayne Witzel III
8faf588775
Update package versions, settings, and tooling 2017-11-09 17:17:30 -05:00
Jake McDermott
77e11fe8fe
collect unit test results for shippable
Signed-off-by: Jake McDermott <jmcdermott@ansible.com>
2017-11-02 01:22:07 -04:00
AlanCoding
90f63774f4
flake8: comply with new E722 rule 2017-10-23 14:36:48 -04:00
AlanCoding
341ef411a4
update isolated container requirements 2017-10-10 14:46:41 -04:00
Ryan Petrello
74f2509482 support ovirt4 as a built-in inventory source
see: https://github.com/ansible/ansible-tower/issues/6522
2017-10-10 14:38:44 -04:00
Jake McDermott
573d35feff
use dataset name as naming prefix 2017-09-15 17:22:13 -04:00
Jake McDermott
ab87f81ad8
add data generator preset 2017-09-15 17:22:03 -04:00
Ryan Petrello
44702c5cfd
fix broken shippable test suite 2017-09-06 16:12:43 -07:00
Matthew Jones
67d1a86d81
Merge branch 'release_3.2.0' into devel
* release_3.2.0: (138 commits)
  Pull Dutch and Spanish translations
  Increase verbosity of CTiT Logging test error handling
  fix to console error of conditional toggle showing
  Fix error message when calling remove on undefined DOM element
  fix ctit logging toggle from being showed for log types other than https
  Remove delete and edit buttons from smart inventory host list.  Only option should be view.
  feedback from PR
  Enhance query string in ad hoc command event save to consider smart inventory
  Fixed host filter clearall
  fuller validation for host_filter
  On JT form, Show credential tags from summary_fields if user doesn't have view permission on the credential
  Align key toggle button to role dropdown in user team permissions modal
  Removed rogue console.logs
  Removed extra refresh call
  Enhace query string in job event save to consider smart inventory
  Fix typo in scan_packages plugin
  Switch running_jobs and capacity table columns
  Disable insights cred when user doesn't have edit permissions
  Disallow changing credential_type of an existing credential
  fix bug with host_filter RBAC check
  ...
2017-09-06 16:10:08 -04:00
AlanCoding
1f2a7801fd
fix flake8 and data gen functionality 2017-08-18 15:30:20 -04:00
Ryan Petrello
c4a1dfc4e8
only run shippable UI for devel and release branches 2017-08-18 14:21:48 -04:00
Shane McDonald
da18033a46 Make parent image tag for unit tests a build arg 2017-08-17 09:16:52 -04:00
Shane McDonald
7ac2376a01 Make parent image tag for unit tests a build arg 2017-08-16 22:31:15 -04:00
AlanCoding
d48c1bd71d
data generator schema update for 3.2 2017-08-16 17:14:56 -04:00
Matthew Jones
de2aa2792a
Merge branch 'release_3.2.0' into devel
* release_3.2.0: (342 commits)
  fail all jobs on an offline node
  filtering out super users from permissions lists
  removing vars from schedules for project syncs and inv syncs
  update license page if user inputs a new type of license
  Show IG name on job results if it comes from the socket
  rename isolated->expect in script tooling
  Center survey maker delete dialog in browser window
  Fix job details right panel content from overflowing in Firefox
  graceful killing of receiver worker processes
  change imports to reflect isolated->expect move
  Update smart inventory host popover content
  Fix extra variable textarea scroll in Firefox
  initial commit to move folder isolated->expect
  Add missing super call in NotificationTemplateSerializer
  Various workflow maker bug fixes
  Style nodes with deleted unified job templates
  Fixed job template view for user with read-only access
  presume 401 from insights means invalid credential
  only reap non-netsplit nodes
  import os, fixing bug that forced SIGKILL
  ...
2017-08-15 22:22:26 -04:00
AlanCoding
86be5cf2cf
rename isolated->expect in script tooling 2017-08-15 13:25:23 -04:00
AlanCoding
6944ffa010
change imports to reflect isolated->expect move 2017-08-15 12:08:58 -04:00
Alan Rominger
dcfcfb6c7b Merge pull request #245 from AlanCoding/fix_supervisor
specify all group queues, get supervisor working
2017-08-14 11:31:00 -04:00
Matthew Jones
dd44f7a640 Merge pull request #236 from matburt/contributing_and_other_docs
CONTRIBUTING.md and other docs
2017-08-11 10:30:55 -04:00
AlanCoding
2385f62311 specify all group queues, get supervisor working 2017-08-10 14:57:25 -04:00
Matthew Jones
67474c8de1 Updating development tooling to be more generic
* Not assuming GCR is being used for image hosting
* Breaking out the dev environment bootstrapping from service starting
2017-08-09 15:53:25 -04:00
Alan Rominger
d7f869f566 Merge pull request #155 from AlanCoding/dev_ig_supervisor
functionality for dev + IG + supervisor configuration
2017-08-08 09:22:39 -04:00
Matthew Jones
fc00c220eb Minor update to compilemessages utility for better print syntax
This also lets it run under python3
2017-08-03 23:33:07 -04:00
AlanCoding
4d5bedce74 functionality for dev + IG + supervisor configuration 2017-08-02 14:32:35 -04:00
Shane McDonald
c8b0142929 Fix test failures 2017-07-27 11:39:00 -04:00
Matthew Jones
7231537841 Remove ansible reference from dev PKG-INFO 2017-07-26 14:18:19 -04:00
Matthew Jones
c7a85d9738 Mass rename from ansible_(awx|tower) -> (awx|tower) 2017-07-26 13:33:26 -04:00
Shane McDonald
e904d47122 Fix unit test runs 2017-07-25 14:01:43 -04:00
Shane McDonald
1249a8b30f Rename tower-python to awx-python 2017-07-25 13:31:52 -04:00
Ryan Petrello
0ce3152e6f fix busted test runs 2017-07-25 12:47:46 -04:00
Ryan Petrello
e29492a259 more tower -> awx for task execution and isolated tooling 2017-07-25 10:36:06 -04:00
Matthew Jones
1e27bad010 Don't tie awx-manage to a particular dev environment version 2017-07-25 09:41:20 -04:00
Matthew Jones
eb64d6e808 Update cluster compose for repo migration 2017-07-24 21:03:55 -04:00
Ryan Petrello
8ce1421c6a fix tower-expect -> awx-expect for isolated tower builds 2017-07-24 16:03:58 -04:00
Matthew Jones
f175fbba23 Fix up some tower-manage -> awx-manage commands 2017-07-24 09:38:36 -04:00
Matthew Jones
e9dde459c7 Updating some dev environment tooling for version numbers 2017-07-22 00:08:38 -04:00
Matthew Jones
f0d68f429c Updating some dev environment tooling for version numbers 2017-07-22 00:06:44 -04:00
Shane McDonald
17db724932 Make packaging work with our internal build systems 2017-07-21 21:27:02 -04:00
Matthew Jones
3892e4e389 Tower -> AWX Tooling Migration
* Switching version number scheme and mechanism
* Refactor development tooling towards 'awx' paths and names
* Purging packaging details from Makefile
2017-07-21 17:06:45 -04:00
AlanCoding
459c33d272 More consistently provide fields in job_events logger
* Change scheme from using event dict to JobEvent object
* Add processing to grok object fields
* Allow override of provided formatter in case of future issues
2017-07-10 10:34:04 -04:00
Ryan Petrello
611c42f741 fix the make rdb debugging tool 2017-06-29 15:40:09 -04:00
Alan Rominger
9fda4eee85 Remove extra_hosts that breaks dev environment
This was triggered by a docker update, and an empty value of DOCKER_HOST_IP resulted in an error running `make docker-compose`
2017-06-29 13:41:33 -04:00
Ryan Petrello
ac9bc166a0 properly invoke the byte-compiled isolated job runner in production
see: #5903
2017-06-29 11:11:28 -04:00
Shane McDonald
4a8e8d4fbf ansible-tower-isolated RPM 2017-06-28 15:02:34 -04:00
Ryan Petrello
3000f52a92 install a randomized RSA key for controller -> isolated rampart auth
see: #6507
2017-06-27 10:53:44 -04:00
AlanCoding
3cedcf22a9 add instance groups and queues to cluster tooling 2017-06-22 10:40:58 -04:00
Chris Meyers
fee16fe391 Merge pull request #6588 from chrismeyersfsu/feature-fact_cache
initial tower fact cache implementation
2017-06-22 09:58:28 -04:00
Ryan Petrello
45bdd9f747 improve readability of the honcho console logs in the dev environment
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
  are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
2017-06-22 09:19:32 -04:00
adamscmRH
e0581af9ff Missing Procfile docker-compose fix 2017-06-21 14:29:49 -04:00
Chris Meyers
ec2e537f63 remove fache cache receiver 2017-06-21 13:45:27 -04:00
Alan Rominger
b9ecc775dc Update developer documentation for isolated jobs 2017-06-20 15:09:45 -04:00
Ryan Petrello
2ba9e56033 Include the Tower venv in the isolated bubblewrapped arguments.
This moves the container-based code location and venvs.
The goal here is that the paths of Tower source for isolated
vs normal nodes matches (both in prod and local development) so that we
don't have to add a bunch of additional bwrap argument logic for
<location-of-isolated-tower-venv>.
2017-06-19 15:52:54 -04:00
Ryan Petrello
44e0c8621a isolated ramparts: replace systemd unit with a tower-expect binary
instead of launching isolated tasks via `systemctl`, treat
`awx.main.isolated.run` as an executable that knows how to daemonize

additionally, add `setup.py isolated_build` for isolated Tower source
distribution
2017-06-16 09:59:21 -04:00
Ryan Petrello
422950f45d Support for executing job and adhoc commands on isolated Tower nodes (#6524) 2017-06-14 11:47:30 -04:00
Aaron Tan
2ed3be0aab Add the ability to programmatically generate named URLs on client-side. 2017-06-14 10:37:40 -04:00
Matthew Jones
da5ef30b6a Swap out dumb-init for tini
This seems to be less intrusive and less noisy of a pid 1
2017-05-26 11:33:14 -04:00
Matthew Jones
cf7fb8ddd7 Fix system jobs not starting due to task manager and cred changes
* Task Manager logic wasn't assigning default instance group on system
  jobs
* Task credential changes assumed the model would have a credential
* Fix up an innocuous error symlinking rdb.py if it already exists
2017-05-18 13:10:48 -04:00
Matthew Jones
1a4a6273a4 Merge branch 'rampart_groups_setup_playbook' into devel
* rampart_groups_setup_playbook:
  Updating changelog for Instance Groups
  Fix an incorrect reference on instance group jobs list
  Purge remaining references to rampart groups
  Simplify can_access for instance groups on job templates
  Adding Instance Group permissions and tests
  Increase test coverage for task scheduler inventory updates
  Exit logic fixes for instance group tools
  View Fixes for instance groups
  new view to allow associations but no creations
  Updating acceptance documentation and system docs
  Updating unit tests for task manager refactoring
  Update views and serializers to support instance group (ramparts)
  Implementing models for instance groups, updating task manager
  Updating the setup playbook to support instance group installation
  Add nginx to server start and switch back to first tmux win
  Fix an issue where the local queue wouldn't use the rabbitmq name
2017-05-12 13:40:30 -04:00
Matthew Jones
705f8af440 Update views and serializers to support instance group (ramparts)
* includes top level views for instances and instance groups and
  extending those views to be able to view running jobs
* Associative endpoints on Organizations, Inventories, and Job
  Templates
* Related and summary field entries where appropriate
* Adding job model references to executing instance group
* Fix up default queue properties for clustering from the settings file
* Update production and default settings for instance queues in settings
2017-05-10 12:33:03 -04:00
Matthew Jones
4ced911c00 Implementing models for instance groups, updating task manager
* New InstanceGroup model and associative relationship with Instances
* Associative instances between Organizations, Inventory, and Job
  Templates and InstanceGroups
* Migrations for adding fields and tables for Instance Groups
* Adding activity stream reference for instance groups
* Task Manager Refactoring:
** Simplifying task manager relationships and move away from the
   interstitial hash tables
** Simplify dependency determination logic
** Reduce task manager runtime complexity by removing the partial
   references and moving the logic into the task manager directly or
   relying on Job model logic for determinism
2017-05-10 12:32:54 -04:00
Ryan Petrello
3126bfa1a2 add tooling to aid in remote debugging sessions 2017-05-09 15:36:13 -04:00