Aleksei Nikiforov
8e2910ab94
Add bufferization of data and separate writer thread
...
Add time and count limit and configuration options for them.
If no limits are set, separate thread is not used.
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
36891f1273
Move audit data convertion into separate functions
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
1895289a87
Add buffered logging and new option for logfile location
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
0ea79509b8
Add conversion from field type to string and use it to improve logging
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
559944866f
Fix AUPARSE_TYPE_ARCH and AUPARSE_TYPE_CAP_BITMAP checks
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
c99829625f
Unclassified field type can be either string or integer
...
Don't produce warnings for such types
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
517d5d87c4
Properly process NULL strings
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
ba27894d20
20191217-alt1
...
- Initial build for ALT
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
c811c95797
Fix column names after tables flattening
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
a0ea1c1f26
Revert "Process nested structures as arrays"
...
This reverts commit 0f41d6a19c18a2ead8b709fab515cdb9770422ae.
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
0e852e9ad3
Don't use Nested clickhouse type, flatten tables.
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
853818b7ab
Also exit if some error with stdin is encountered
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
f2c0a405dd
Process nested structures as arrays
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
9906c2e5a4
Sanitize column names
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
10b9a9c159
Enable reconnect to clickhouse server by default
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
9d5a803603
Fix column types
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
c6b1330fb5
Write all data into database
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
ef3b71bf27
Catch and process exceptions in audit callback
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
575ca79ff1
Populate empty fields
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
7ed7d49a25
Add function for columns generation from record field class
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
508ca46a67
Update string_array type to use nested arrays instead of single array
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
98bc48492f
audit record: create and reuse fields
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
0aca43be6f
Rework code for type match checking
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
4de04f70b9
Improve audit callback: start creating audit record to be added to DB later
2020-01-27 13:07:59 +03:00
Aleksei Nikiforov
3c09000edc
Implement string_array type
...
Update AbstractRecordField interface to match new use-case
2019-12-17 14:34:19 +03:00
Aleksei Nikiforov
3ba3561664
Implement warnings and mark all optional fields as Nullable
2019-12-17 14:34:19 +03:00
Aleksei Nikiforov
ef888ec74b
Update json datatypes structure to match one expected by application
2019-12-17 14:34:18 +03:00
Aleksei Nikiforov
d7ac138531
Implement clickhouse database creation
2019-12-17 14:33:32 +03:00
Aleksei Nikiforov
fef1385176
Add datatypes file parsing function
2019-12-17 14:33:31 +03:00
Aleksei Nikiforov
2875907d35
Add file with audit types mapping
2019-12-17 14:32:09 +03:00
Aleksei Nikiforov
826ae34d18
Modify integer record field to also store interpreted string value
2019-12-17 14:32:09 +03:00
Aleksei Nikiforov
cfd18dd0d9
Initial processing of audit types added
2019-12-17 14:32:09 +03:00
Aleksei Nikiforov
7f1ed9da8f
Add clickhouse config parsing and initial audit setup
2019-12-16 16:42:52 +03:00
Aleksei Nikiforov
35d96682eb
Initial commit
2019-12-16 16:42:40 +03:00
