MINOR: log: add +bin logformat node option

Support '+bin' option argument on logformat nodes to try to preserve
binary output type with binary sample expressions.

For this, we rely on the log/sink API which is capable of conveying binary
data since all related functions don't search for a terminating NULL byte
in provided log payload as they take a string pointer and a string length
as argument.

Example:
  log-format "%{+bin}o %[bin(00AABB)]"

Will produce:
  00aabb

(output was piped to `hexdump  -ve '1/1 "%.2x"'` to dump raw bytes as HEX
characters)

This should be used carefully, because many syslog endpoints don't expect
binary data (especially NULL bytes). This is mainly intended for use with
set-var-fmt actions or with ring/udp log endpoints that know how to deal
with such binary payloads.

Also, this option is only supported globally (for use with '%o'), it will
not have any effect when set on an individual node. (it makes no sense to
have binary data in the middle of log payload that was started without
binary data option)
This commit is contained in:
Aurelien DARRAGON 2024-04-25 16:29:01 +02:00
parent 162e311a0e
commit b7c3d8c87c
3 changed files with 36 additions and 7 deletions

View File

@ -25739,6 +25739,14 @@ Flags are :
* X: hexadecimal representation (IPs, Ports, %Ts, %rt, %pid)
* E: escape characters '"', '\' and ']' in a string with '\' as prefix
(intended purpose is for the RFC5424 structured-data log formats)
* bin: try to preserve binary data, this can be useful with sample
expressions that output binary data in order to preserve the original
data. Be careful however, because it can obviously generate non-
printable chars, including NULL-byte, which most syslog endpoints
don't expect. Thus it is mainly intended for use with set-var-fmt,
rings and binary-capable log endpoints.
This option can only be set globally (with %o), it will be ignored
if set on an individual node's options.
Example:

View File

@ -47,6 +47,7 @@
#define LOG_OPT_HTTP 0x00000020
#define LOG_OPT_ESC 0x00000040
#define LOG_OPT_MERGE_SPACES 0x00000080
#define LOG_OPT_BIN 0x00000100
/* Fields that need to be extracted from the incoming connection or request for

View File

@ -327,6 +327,7 @@ struct logformat_tag_args tag_args_list[] = {
{ "Q", LOG_OPT_QUOTE },
{ "X", LOG_OPT_HEXA },
{ "E", LOG_OPT_ESC },
{ "bin", LOG_OPT_BIN },
{ 0, 0 }
};
@ -1748,8 +1749,10 @@ static inline void lf_buildctx_prepare(struct lf_buildctx *ctx,
if (node) {
/* per-node options are only considered if not already set
* globally
*
* Also, ignore LOG_OPT_BIN since it is a global-only option
*/
ctx->options |= node->options;
ctx->options |= (node->options & ~LOG_OPT_BIN);
/* consider node's typecast setting */
ctx->typecast = node->typecast;
@ -3237,18 +3240,35 @@ int sess_build_logline(struct session *sess, struct stream *s, char *dst, size_t
type = SMP_T_STR; // default
if (key && key->data.type == SMP_T_BIN &&
(ctx.options & LOG_OPT_BIN)) {
/* output type is binary, and binary option is set:
* preserve output type unless typecast is set to
* force output type to string
*/
if (ctx.typecast != SMP_T_STR)
type = SMP_T_BIN;
}
if (key && !sample_convert(key, type))
key = NULL;
if (ctx.options & LOG_OPT_HTTP)
ret = lf_encode_chunk(tmplog, dst + maxsize,
'%', http_encode_map, key ? &key->data.u.str : &empty, &ctx);
else
ret = lf_text_len(tmplog,
key ? key->data.u.str.area : NULL,
key ? key->data.u.str.data : 0,
dst + maxsize - tmplog,
&ctx);
else {
if (key && type == SMP_T_BIN)
ret = lf_encode_chunk(tmplog, dst + maxsize,
0, no_escape_map,
&key->data.u.str,
&ctx);
else
ret = lf_text_len(tmplog,
key ? key->data.u.str.area : NULL,
key ? key->data.u.str.data : 0,
dst + maxsize - tmplog,
&ctx);
}
if (ret == NULL)
goto out;
tmplog = ret;