darktemplar/auditd-plugin-clickhouse
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
57 Commits 2 Branches 0 Tags 94 KiB
237f504efe
Commit Graph

57 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aleksei Nikiforov
237f504efe Only send signal for data writing when reaching specified amount of data 2020-01-28 11:20:45 +03:00
Aleksei Nikiforov
3f295f3e05 auparse: flush feed and release memory 
This might produce some bad data if lines of data in input feed is incorrectly split
 2020-01-28 11:20:45 +03:00
Aleksei Nikiforov
d8bde76fd2 Make audit data buffer size unconfigurable 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
3479a02ac2 Reduce objects copying 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
5dfb70a328 Don't stop writer thread before all data is processed 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
972da5776a Fix memory leak from auparse_get_node 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
62adea5afd Send data to database only in chunks exceeding limit count if one is specified 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
f0fe0f5a61 Don't store logs internally if logging to file is disabled 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
1520830a97 Don't pollute log while emptying data pipe and no data is present 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
7ae97f57d7 20200116-alt1 
- New fields added to database table.
- All unknown fields encountered are saved into database.
- Fixed crash when processing NULL strings.
- Fixed warnings for some audit data types.
- Implemented logging.
- Implemented audit data serialization and temporary storing before sending
  to database.
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
1dc7f65269 Add new fields to known database fields 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
6fc2fe5320 Add serialization test 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
bb98d8809a Get rid of some dynamic_pointer_casts by updating return types 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
7a04b13645 Add information about date and time into log 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
ddcdfa9c1c Improve logging messages for POLLHUP and POLLERR conditions 
Don't report EINTR from poll
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
6677b661c2 Save audit record filename into audit record class 
This will allow to remove files even if file naming scheme is changed
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
e54ea41414 Read previously saved data when application is launched 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
aaf2d3b886 Implement audit data removal 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
e9cb80077d Implement audit record deserialization 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
a42ec11de1 Serialize and save data before sending it to write thread 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
be7e252971 Record all unknown fields into special "unknown_field" array 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
b1287924f2 Make logging compatible with logrotate tool and provide logrotate config 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
149a7e7a44 Move some functions to utils file 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
8e2910ab94 Add bufferization of data and separate writer thread 
Add time and count limit and configuration options for them.
If no limits are set, separate thread is not used.
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
36891f1273 Move audit data convertion into separate functions 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
1895289a87 Add buffered logging and new option for logfile location 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
0ea79509b8 Add conversion from field type to string and use it to improve logging 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
559944866f Fix AUPARSE_TYPE_ARCH and AUPARSE_TYPE_CAP_BITMAP checks 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
c99829625f Unclassified field type can be either string or integer 
Don't produce warnings for such types
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
517d5d87c4 Properly process NULL strings 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
ba27894d20 20191217-alt1 
- Initial build for ALT
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
c811c95797 Fix column names after tables flattening 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
a0ea1c1f26 Revert "Process nested structures as arrays" 
This reverts commit 0f41d6a19c18a2ead8b709fab515cdb9770422ae.
 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
0e852e9ad3 Don't use Nested clickhouse type, flatten tables. 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
853818b7ab Also exit if some error with stdin is encountered 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
f2c0a405dd Process nested structures as arrays 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
9906c2e5a4 Sanitize column names 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
10b9a9c159 Enable reconnect to clickhouse server by default 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
9d5a803603 Fix column types 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
c6b1330fb5 Write all data into database 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
ef3b71bf27 Catch and process exceptions in audit callback 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
575ca79ff1 Populate empty fields 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
7ed7d49a25 Add function for columns generation from record field class 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
508ca46a67 Update string_array type to use nested arrays instead of single array 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
98bc48492f audit record: create and reuse fields 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
0aca43be6f Rework code for type match checking 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
4de04f70b9 Improve audit callback: start creating audit record to be added to DB later 2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
3c09000edc Implement string_array type 
Update AbstractRecordField interface to match new use-case
 2019-12-17 14:34:19 +03:00
Aleksei Nikiforov
3ba3561664 Implement warnings and mark all optional fields as Nullable 2019-12-17 14:34:19 +03:00
Aleksei Nikiforov
ef888ec74b Update json datatypes structure to match one expected by application 2019-12-17 14:34:18 +03:00