1
0
mirror of https://github.com/ansible/awx.git synced 2024-10-31 15:21:13 +03:00
Commit Graph

25271 Commits

Author SHA1 Message Date
Ryan Petrello
bb5136cdae properly escape URL paths and querystrings for paths in logging settings 2020-04-13 11:44:00 -04:00
Ryan Petrello
b0db2b7bec add some exception handling for dealing with logging connection resets
when rsyslogd restarts due to config changes, there's a brief moment
where the socket will refuse connections on teardown; exception handling
is needed here to deal with that
2020-04-13 11:44:00 -04:00
Ryan Petrello
1000dc10fb an an rsyslogd config check to the logging test endpoint 2020-04-13 11:44:00 -04:00
Ryan Petrello
2a4b009f04 rsyslogd: use %rawmsg-after-pri% instead of %msg%
after some prolonged RFC reading and tinkering w/ rsyslogd...

cpython's SysLogHandler doesn't emit RFC3164 formatted messages
in the format you'd expect; it's missing the ISO date, hostname, etc...
along with other header values; the handler implementation relies on you
to specify a syslog-like formatter (we've replaced all of this with our
own *custom* logstash-esque formatter that effectively outputs valid JSON
- without dates and other syslog header values prepended)

because of this unanticipated format, rsyslogd chokes when trying to
parse the message's parts;  AWX is emitting:

<priority>RAWJSON

...so the usage of `%msg%` isn't going to work for us, because rsyslog
tries to parse *all* of the possible headers (and yells, because it
can't find a date to parse):

see: https://www.rsyslog.com/files/temp/doc-indent/configuration/properties.html#message-properties

this is fine, because we don't *need* any of that message parsing
anyways; in the end, we're *just* interested in forwarding the raw
JSON/text content to the third party log handler
2020-04-13 11:44:00 -04:00
Ryan Petrello
8cdd42307c clarify that logging username/password is only valid for HTTP/s 2020-04-13 11:44:00 -04:00
Ryan Petrello
269558876e only use a basic auth password for external logging if username is set 2020-04-13 11:44:00 -04:00
Ryan Petrello
bba680671b when writing the rsyslog config, do it post-commit
there's a race condition if we do this pre-commit where the correct
value isn't actually *persisted* to the database yet, and we end up
saving the *prior* setting values
2020-04-13 11:44:00 -04:00
Ryan Petrello
f70a76109c make rsyslog fall back to no-op if logging is disabled 2020-04-13 11:44:00 -04:00
Christian Adams
5d54877183 Add action to default rsyslog.conf so supervisor starts correctly the first time 2020-04-13 11:44:00 -04:00
Ryan Petrello
f7dac8e68d more external logging unit test fixups 2020-04-13 11:44:00 -04:00
Ryan Petrello
39648b4f0b fix up a few test and lint errors related to external logging 2020-04-13 11:44:00 -04:00
Christian Adams
b942fde59a Ensure log messages have valid json
- Fix messages getting contatenated at 8k
 - Fix rsyslog cutting off the opening brace of log messages
 - Make valid default conf and emit logs based on prescence of .sock and
 settings
2020-04-13 11:44:00 -04:00
Ryan Petrello
ce82b87d9f rsyslog hardening (fixing a few weird things we noticed) 2020-04-13 11:44:00 -04:00
Christian Adams
70391f96ae Revert rsyslog valid config to one that fails intentionally 2020-04-13 11:43:59 -04:00
Christian Adams
2329c1b797 Add rsyslog config to container from file for consistency 2020-04-13 11:43:59 -04:00
Christian Adams
470159b4d7 Enable innocuous but valid config for rsyslog if disabled 2020-04-13 11:43:59 -04:00
Christian Adams
e740340793 ConfigMap rsyslog conf files for k8 2020-04-13 11:43:59 -04:00
Christian Adams
4d5507d344 Add default rsyslog.conf without including /etc/rsyslog.conf 2020-04-13 11:43:59 -04:00
Christian Adams
d350551547 Tweaks to Test Button logic and cleans up flake8 and test failures 2020-04-13 11:43:59 -04:00
Christian Adams
7fd79b8e54 Remove unneeded logging sock variable 2020-04-13 11:43:59 -04:00
John Mitchell
eb12f45e8e add ngToast disable on timeout for log agg notifications, and disable test button until active test completes. 2020-04-13 11:43:59 -04:00
Christian Adams
fb047b1267 Add unit tests for reconfiguring rsyslog & for test endpoint 2020-04-13 11:43:59 -04:00
Christian Adams
d31c528257 Fix Logging settings "Test" button functionality 2020-04-13 11:43:59 -04:00
Christian Adams
996d7ce054 Move supervisor and rsyslog sock files to their own dirs under /var/run 2020-04-13 11:43:59 -04:00
Christian Adams
7040fcfd88 Fix container rsyslog dir permissions 2020-04-13 11:43:59 -04:00
John Mitchell
88ca4b63e6 update configure tower in tower test ui for log aggregator form 2020-04-13 11:43:59 -04:00
Shane McDonald
c0af3c537b Configure rsyslog to listen over a unix domain socket instead of a port
- Add a placeholder rsyslog.conf so it doesn't fail on start
 - Create access restricted directory for unix socket to be created in
 - Create RSyslogHandler to exit early when logging socket doesn't exist
 - Write updated logging settings when dispatcher comes up and restart rsyslog so they  take effect
 - Move rsyslogd to the web container and create rpc supervisor.sock
 - Add env var for supervisor.conf path
2020-04-13 11:43:59 -04:00
Christian Adams
f8afae308a Add rsyslog to supervisor for the task container
- Add proper paths for rsyslog's supervisor logs
 - Do not enable debug mode for rsyslogd
 - Include system rsyslog.conf, and specify tower logging conf when
   starting rsyslog.
2020-04-13 11:43:59 -04:00
Christian Adams
4cd0d60711 Properly handle logger paths and https/http configuration
- log aggregator url paths were not being passed to rsyslog
 - http log services like loggly will now truly use http and port 80
 - add rsyslog.pid to .gitignore
2020-04-13 11:43:59 -04:00
Christian Adams
955d57bce6 Upstream rsyslog packaging changes
- add rsyslog repo to Dockerfile for AWX installation
 - Update Library Notes for requests-futures removal
2020-04-13 11:43:59 -04:00
Ryan Petrello
589d27c88c POC: replace our external log aggregation feature with rsyslog
- this change adds rsyslog (https://github.com/rsyslog/rsyslog) as
  a new service that runs on every AWX node (managed by supervisord)
  in particular, this feature requires a recent version (v8.38+) of
  rsyslog that supports the omhttp module
  (https://github.com/rsyslog/rsyslog-doc/pull/750)
- the "external_logger" handler in AWX is now a SysLogHandler that ships
  logs to the local UDP port where rsyslog is configured to listen (by
  default, 51414)
- every time a LOG_AGGREGATOR_* setting is changed, every AWX node
  reconfigures and restarts its local instance of rsyslog so that its
  fowarding settings match what has been configured in AWX
- unlike the prior implementation, if the external logging aggregator
  (splunk/logstash) goes temporarily offline, rsyslog will retain the
  messages and ship them when the log aggregator is back online
- 4xx or 5xx level errors are recorded at /var/log/tower/external.err
2020-04-13 11:43:59 -04:00
softwarefactory-project-zuul[bot]
eafb751ecc
Merge pull request #6679 from ryanpetrello/fix-6675
skip non-files when consuming events synced from isolated hosts

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-13 15:36:42 +00:00
softwarefactory-project-zuul[bot]
30ea66023f
Merge pull request #6671 from wenottingham/even-moar-data-plz
Collect task timing, warnings, and deprecations from job events

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-13 15:06:46 +00:00
Ryan Petrello
9843e21632
skip non-files when consuming events synced from isolated hosts
see: https://github.com/ansible/awx/issues/6675
2020-04-13 10:14:10 -04:00
softwarefactory-project-zuul[bot]
6002beb231
Merge pull request #6677 from chrismeyersfsu/fix-spelling
fix spelling mistake in wsbroadcast status output

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-13 14:13:24 +00:00
chris meyers
9c6e42fd1b fix spelling mistake in wsbroadcast status output 2020-04-13 09:37:32 -04:00
softwarefactory-project-zuul[bot]
eeab4b90a5
Merge pull request #6568 from AlanCoding/whoops_not_changed
Do not set changed=True if the object did not change

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-10 00:16:02 +00:00
softwarefactory-project-zuul[bot]
a7f1a36ed8
Merge pull request #6670 from ryanpetrello/redis-fixup
work around redis connection failures in the callback receiver

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-09 21:41:08 +00:00
Bill Nottingham
d651786206 Collect task timing, warnings, and deprecations from job events
Timing information requires ansible-runner >= 1.4.6.
2020-04-09 17:27:19 -04:00
softwarefactory-project-zuul[bot]
19e4758be1
Merge pull request #6637 from john-westcott-iv/tower_workflow_job_lanch_update
Initial commit of tests for tower_workflow_launch

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-09 19:53:35 +00:00
softwarefactory-project-zuul[bot]
fe9de0d4cc
Merge pull request #6658 from mabashian/6655-job-redirect
Fixes issue where job type redirects weren't firing correctly

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-09 19:47:20 +00:00
Ryan Petrello
80147acc1c
work around redis connection failures in the callback receiver
if redis stops/starts, sometimes the callback receiver doesn't recover
without a restart; this fixes that
2020-04-09 15:38:03 -04:00
beeankha
4acdf8584b Update workflow_launch module and test playbook 2020-04-09 15:12:49 -04:00
beeankha
cf607691ac Pass data and errors more clearly, change extra_vars to be a dict, update test playbook to have a task utilizing extra_vars 2020-04-09 12:40:13 -04:00
beeankha
d7adcfb119 Revert unnecessary changes made to test playbook during rebase 2020-04-09 12:38:06 -04:00
beeankha
97d26728e4 Fix one more linter issue 2020-04-09 12:38:06 -04:00
John Westcott IV
6403895eae Puting tasks back to natural order 2020-04-09 12:38:06 -04:00
beeankha
8b26ff1fe6 Fix linter errors 2020-04-09 12:38:06 -04:00
beeankha
9ddd020348 Fix sanity tests and edit test playbook 2020-04-09 12:38:06 -04:00
John Westcott IV
a2d1c32da3 Initial commit of tests for tower_workflow_launch 2020-04-09 12:38:06 -04:00