14848 Commits

Author SHA1 Message Date
Remi Tricot-Le Breton
476462010e BUG/MINOR: proxy: Missing calloc return value check in chash_init_server_tree
A memory allocation failure happening in chash_init_server_tree while
trying to allocate a server's lb_nodes item used in consistent hashing
would have resulted in a crash. This function is only called during
configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:55:51 +02:00
Remi Tricot-Le Breton
17acbab0ac BUG/MINOR: http: Missing calloc return value check in make_arg_list
A memory allocation failure happening in make_arg_list when trying to
allocate the argument list would have resulted in a crash. This function
is only called during configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:09 +02:00
Remi Tricot-Le Breton
b6864a5b6f BUG/MINOR: http: Missing calloc return value check while parsing redirect rule
A memory allocation failure happening in http_parse_redirect_rule when
trying to allocate a redirect_rule structure would have resulted in a
crash. This function is only called during configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:08 +02:00
Remi Tricot-Le Breton
1f4fa906c7 BUG/MINOR: worker: Missing calloc return value check in mworker_env_to_proc_list
A memory allocation failure happening in mworker_env_to_proc_list when
trying to allocate a mworker_proc would have resulted in a crash. This
function is only called during init.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:06 +02:00
Remi Tricot-Le Breton
6443bcc2e1 BUG/MINOR: compression: Missing calloc return value check in comp_append_type/algo
A memory allocation failure happening in comp_append_type or
comp_append_algo called while parsing compression options would have
resulted in a crash. These functions are only called during
configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:04 +02:00
Remi Tricot-Le Breton
8cb033643f BUG/MINOR: http: Missing calloc return value check while parsing tcp-request rule
A memory allocation failure happening in tcp_parse_request_rule while
processing the "capture" keyword and trying to allocate a cap_hdr
structure would have resulted in a crash. This function is only called
during configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:02 +02:00
Remi Tricot-Le Breton
2ca42b4656 BUG/MINOR: http: Missing calloc return value check while parsing tcp-request/tcp-response
A memory allocation failure happening in tcp_parse_tcp_req or
tcp_parse_tcp_rep when trying to allocate an act_rule structure would
have resulted in a crash. These functions are only called during
configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:00 +02:00
Remi Tricot-Le Breton
18a82ba690 BUG/MINOR: proxy: Missing calloc return value check in proxy_defproxy_cpy
A memory allocation failure happening in proxy_defproxy_cpy while
copying the default compression options would have resulted in a crash.
This function is called for every new proxy found while parsing the
configuration.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:50:59 +02:00
Remi Tricot-Le Breton
55ba0d6865 BUG/MINOR: proxy: Missing calloc return value check in proxy_parse_declare
A memory allocation failure happening during proxy_parse_declare while
processing the "capture" keyword and allocating a cap_hdr structure
would have resulted in a crash. This function is only called during
configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:50:57 +02:00
Remi Tricot-Le Breton
a4bf8a059d BUG/MINOR: http: Missing calloc return value check in parse_http_req_capture
A memory allocation failure happening in parse_http_req_capture while
processing a "len" keyword and allocating a cap_hdr structure would
have resulted in a crash. This function is only called during
configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:50:55 +02:00
Remi Tricot-Le Breton
612b2c37be BUG/MINOR: ssl: Missing calloc return value check in ssl_init_single_engine
A memory allocation failure happening during ssl_init_single_engine
would have resulted in a crash. This function is only called during
init.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:50:49 +02:00
Remi Tricot-Le Breton
208ff01b23 BUG/MINOR: peers: Missing calloc return value check in peers_register_table
A memory allocation failure happening during peers_register_table would
have resulted in a crash. This function is only called during init.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:50:46 +02:00
Remi Tricot-Le Breton
f1800e64ef BUG/MINOR: server: Missing calloc return value check in srv_parse_source
Two calloc calls were not checked in the srv_parse_source function.
Considering that this function could be called at runtime through a
dynamic server creation via the CLI, this could lead to an unfortunate
crash.

It was raised in GitHub issue #1233.
It could be backported to all stable branches even though the runtime
crash could only happen on branches where dynamic server creation is
possible.
2021-05-31 10:50:32 +02:00
Mark Mullan
e1ecfdda81 DOC: intro: Fix typo in starter guide
s/sned/send.
2021-05-27 08:26:43 +02:00
Tim Duesterhus
5546c8bdce MINOR: cfgparse: Fail when encountering extra arguments in macro
This resolves GitHub issue #1124.

This change should be backported as a *warning* to 2.4.
2021-05-27 07:54:21 +02:00
Christopher Faulet
1f08bffe0c MINOR: http-ana: Perform L7 retries because of status codes in response analyser
L7 retries because of status codes are now performed in the response
analyser. This way, it is no longer required to handle L7 retries in
si_cs_recv(). It is also useless to set CF_READ_ERROR on the response
channel to be able to trigger such retries.

In addition, if no L7 retries are performed when the response is received,
the L7 buffer is immediately released. Before in this case, it was only
released with the stream.
2021-05-26 13:56:06 +02:00
Christopher Faulet
d976923ab2 BUG/MINOR: http-ana: Handle L7 retries on refused early data before K/A aborts
When a network error occurred on the server side, if it is not the first
request (in case of keep-alive), nothing is returned to the client and its
connexion is closed to be sure it may retry. However L7 retries on refused
early data (0rtt-rejected) must be performed first.

In addition, such L7 retries must also be performed before incrementing the
failed responses counter.

This patch must be backported as far as 2.0.
2021-05-26 13:56:06 +02:00
Christopher Faulet
552601d5fd BUG/MINOR: http-ana: Send the right error if max retries is reached on L7 retry
This bug was introduced by the previous commit (9f5382e45 Revert "MEDIUM:
http-ana: Deal with L7 retries in HTTP analysers") because I failed the
revert.

On L7 retry, if the maximum connection retries is reached, an error must be
return to the client. Depending the situation, it may be a 502-Bad-Gateway
(empty-response or junk-response), a 504-Gateway-Timeout (response-timeout)
or a 425-Too-Early (0rtt-rejected). But contrary to what the comment says,
the do_l7_retry() function always returns a success.

Note it is not a problem for L7 retries on the response status code because
the stream-interface already takes care to have not reached the maximum
connection retries counter to trigger a L7 retry.

This patch must be backported to 2.4 because the commit must also be
backported to 2.4.
2021-05-26 10:31:11 +02:00
Christopher Faulet
9f5382e452 Revert "MEDIUM: http-ana: Deal with L7 retries in HTTP analysers"
This reverts commit 5b82cc5b5c350c7cfa194cc6bc16ad9308784541. The purpose of
this commit was to fully handle L7 retries in HTTP analysers and stop to
deal with the L7 buffer in si_cs_send()/si_cs_recv(). It is of course
cleaner this way. But there is a huge drawback. The L7 buffer is reserved
from the time the request analysis is finished until the moment the response
is received. For a small request, the analysis is finished before the
connection to the server. Thus for the L7 buffer will be kept for queued
sessions while it is not mandatory.

So, for now, the commit is reverted to go back to the less expensive
solution. This patch must be backported to 2.4.
2021-05-25 10:51:20 +02:00
Christopher Faulet
44c0dcfe90 CLEANUP: mux-h1: Rename functions parsing input buf and filling output buf
Main functions are renamed h1_process_demux() and h1_process_mux() to be
consistent with the H2 mux. For the same reason,
h1_process_header/data/tralers) functions, responsible to parse incoming
data are renamed with "h1_handle_" prefix.
2021-05-25 10:41:50 +02:00
Christopher Faulet
00d7cde551 MINOR: muxes/h1-htx: Realign input buffer using b_slow_realign_ofs()
Input buffers have never output data. So, use b_slow_realign_ofs() function
instead of b_slow_realign(). It is a slighly simpler function. And in the H1
mux, it allows a realign by setting the input buffer head to permit
zero-copies.
2021-05-25 10:41:50 +02:00
Christopher Faulet
4fc51a73e6 MINOR: buf: Add function to realign a buffer with a specific head position
b_slow_realign() function may be used to realign a buffer with a given
amount of output data, eventually 0. In such case, the head is set to
0. This function is not designed to be used with input only buffers, like
those used in the muxes. It is the purpose of b_slow_realign_ofs()
function. It does almost the same, realign a buffer. But it do so by setting
the buffer head to a specific offset.
2021-05-25 10:41:50 +02:00
Christopher Faulet
7a835f3cb0 MINOR: h1-htx: Use a correlation table to speed-up small chunks parsing
Instead of using hex2i() to convert an hexa digit to an integer in the
function parsing small chunks, we now use a table because it is faster.
2021-05-25 10:41:50 +02:00
Christopher Faulet
bdcefe58b7 MEDIUM: h1-htx: Add a function to parse contiguous small chunks
Add h1_parse_full_contig_chunks() function to parse full contiguous chunks.
This function neither handles incomplete chunks nor wrapping buffers. It is
designed to efficiently parse a buffer with several small chunks. Of course,
there is no zero copy here because it is not possible. This function is a
bit tricky and all changes may a have a impact. This one may probably be
optimized, but it is good enough for now and not too complex.

The main function (h1_parse_msg_chunks) always tries to use this function
when the HTTP parser is waiting for a chunk size. In this case, there is no
zero-copy, so there is no reason to call the generic version to parse the
chunk. However, if some unparsed data remain after this step, the generic
function is called. This way, wrapping data and incomplete chunks may be
parsed.

Quick tests show it is now slightly faster in all cases than the legacy
mode.
2021-05-25 10:41:50 +02:00
Christopher Faulet
0d4c924c34 MEDIUM: h1-htx: Split function to parse a chunk and the loop on the buffer
A generic function is now used to only parse the current chunk (h1_parse_chunk)
and the main one (h1_parse_msg_chunks) is used to loop on the buffer and relies
on the first one. This change is mandatory to be able to use an optimized
function to parse contiguous small chunks.
2021-05-25 10:41:50 +02:00
Christopher Faulet
140691baf9 MINOR: h1-htx: Move HTTP chunks parsing into a dedicated function
Chunked data are now parsed in a dedicated function. This way, it will be
possible to have two functions to parse chunked messages. The current one
for messages with large chunks and an other one to parse messages with small
chunks.

The parsing of small chunks is really sensitive because it may be used as a
DoS attack. So we must be carefull to have an optimized function to parse
such messages.
2021-05-25 10:41:50 +02:00
Christopher Faulet
16a524c9ea MINOR: mux-h1/mux-fcgi: Don't needlessly loop on data parsing
Because the function parsing H1 data is now able to handle wrapping input
buffers, there is no reason to loop anymore in the muxes to be sure to parse
wrapping data.
2021-05-25 10:41:50 +02:00
Christopher Faulet
f7c2044f8f MEDIUM: h1-htx: Adapt H1 data parsing to copy wrapping data in one call
Since the beginning, wrapping input data are parsed and copied in 2 steps to
not deal with the wrapping in H1 parsing functions. But there is no reason
to do so. This needs 2 calls to parsing functions. This also means, most of
time, when the input buffer does not wrap, there is an extra call for
nothing.

Thus, now, the data parsing functions try to copy as much data as possible,
handling wrapping buffer if necessary.
2021-05-25 10:41:50 +02:00
Christopher Faulet
de471a4a8d MINOR: h1-htx: Update h1 parsing functions to return result as a size_t
h1 parsing functions (h1_parse_msg_*) returns the number of bytes parsed or
0 if nothing is parsed because an error occurred or some data are
missing. But they never return negative values. Thus, instead of a signed
integer, these function now return a size_t value.

The H1 and FCGI muxes are updated accordingly. Note that h1_parse_msg_data()
has been slightly adapted because the parsing of chunked messages still need
to handle negative values when a parsing error is reported by
h1_parse_chunk_size() or h1_skip_chunk_crlf().
2021-05-25 10:41:50 +02:00
Dragan Dosen
3e6690a555 CLEANUP: pattern: remove export of non-existent function pattern_delete() 2021-05-25 08:44:48 +02:00
Dragan Dosen
a75eea78e2 MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
The output of "show map/acl" now contains the 'entry_cnt' value that
represents the count of all the entries for each map/acl, not just the
active ones, which means that it also includes entries currently being
added.
2021-05-25 08:44:45 +02:00
Christopher Faulet
acfd71b97a BUG/MINOR: http-comp: Preserve HTTP_MSGF_COMPRESSIONG flag on the response
This flag is set on the response when its payload is compressed by HAProxy.
It must be preserved because it may be used when the log message is emitted.

When the compression filter was refactored to support the HTX, an
optimization was added to not perform extra proessing on the trailers.
HTTP_MSGF_COMPRESSIONG flag is removed when the last data block is
compressed. It is not required, it is just an optimization and unfortunately
a bug. This optimization must be removed to preserve the flag.

This patch must be backported as far as 2.0. On the HTX is affected.
2021-05-21 09:59:00 +02:00
Christopher Faulet
a6d3704e38 BUG/MEDIUM: filters: Exec pre/post analysers only one time per filter
For each filter, pre and post callback functions must only be called one
time. To do so, when one of them is finished, the corresponding analyser bit
must be removed from pre_analyzers or post_analyzers bit field. It is only
an issue with pre-analyser callback functions if the corresponding analyser
yields. It may happens with lua action for instance. In this case, the
filters pre analyser callback function is unexpectedly called several times.

This patch should fix the issue #1263. It must be backported is all stable
versions.
2021-05-21 09:59:00 +02:00
Miroslav Zagorac
65b86685e1 BUILD/MINOR: opentracing: fixed build when using clang
The arguments of the snprintf() function are now consistent with the
format used.

This should fix the github issue #1242.
2021-05-20 05:25:02 +02:00
Amaury Denoyelle
79a88ba3d0 BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'
A deadlock is possible with 'set maxconn server' command, if there is
pending connection ready to be dequeued. This is caused by the locking
of server spinlock in both cli_parse_set_maxconn_server and
process_srv_queue.

Fix this by reducing the scope of the server lock into
server_parse_maxconn_change_request. If connection are dequeued, the
lock is taken a second time. This can be seen as suboptimal but as it
happens only during 'set maxconn server' it can be considered as
tolerable.

This issue was reported on the mailing list, for the 1.8.x branch.
It must be backported up to the 1.8.
2021-05-19 17:52:05 +02:00
Remi Tricot-Le Breton
2608e348be BUG/MEDIUM: ebtree: Invalid read when looking for dup entry
The first item inserted into an ebtree will be inserted directly below
the root, which is a simple struct eb_root which only holds two branch
pointers (left and right).
If we try to find a duplicated entry to this first leaf through a
ebmb_next_dup, our leaf_p pointer will point to the eb_root instead of a
complete eb_node so we cannot look for the bit part of our leaf_p since
it would try to cast our eb_root into an eb_node and perform an out of
bounds access when reading "eb_root_to_node(eb_untag(t,EB_LEFT)))->bit".
This bug was found by address sanitizer running on a CRL hot update VTC
test.

Note that the bug has been there since the import of the eb_next_dup()
and eb_prev_dup() function in 1.5-dev19 by commit 2b5702030 ("MINOR:
ebtree: add new eb_next_dup/eb_prev_dup() functions to visit duplicates").

It can be backported to all stable branches.
2021-05-18 19:26:21 +02:00
Ilya Shipitsin
75c95aa649 CI: github actions: add OpenTracing builds
let us add OpenTracing module to "all features" builds
2021-05-18 18:37:37 +02:00
Remi Tricot-Le Breton
a6b2784099 CLEANUP: ssl: Fix coverity issues found in CA file hot update code
Coverity found a few uninitialized values and some dead code in the
CA/CRL file hot update code as well as a missing return value check.
2021-05-18 10:52:54 +02:00
Ilya Shipitsin
0b03895620 CI: introduce scripts/build-vtest.sh for installing VTest
we install VTest for all CI systems, let us unify instalation
2021-05-18 10:48:30 +02:00
Remi Tricot-Le Breton
18c7d83934 BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
The following functions used in CA/CRL file hot update were not defined
in OpenSSL 1.0.2 so they need to be defined in openssl-compat :
- X509_CRL_get_signature_nid
- X509_CRL_get0_lastUpdate
- X509_CRL_get0_nextUpdate
- X509_REVOKED_get0_serialNumber
- X509_REVOKED_get0_revocationDate
2021-05-18 00:28:31 +02:00
Remi Tricot-Le Breton
d75b99e69c BUILD/MINOR: ssl: Fix compilation with SSL enabled
The CA/CRL hot update patches did not compile on some targets of the CI
(mainly gcc + ssl). This patch should fix almost all of them. It adds
missing variable initializations and return value checks to the
BIO_reset calls in show_crl_detail.
2021-05-17 11:53:21 +02:00
Remi Tricot-Le Breton
3c222bd99c DOC: ssl: Add documentation about CRL file hot update commands
Add description for new "abort/commit/del/new/set/show ssl crl-file" CLI
commands.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
a3b2e099c2 REGTESTS: ssl: Add "set/commit ssl crl-file" test
This file adds tests for the new "set ssl crl-file" and "commit ssl
crl-file" commands which allow the hot update of CRL file through CLI
commands.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
f615070bcc REGTESTS: ssl: Add "new/del ssl crl-file" tests
This vtc tests the "new ssl crl-file" which allows to create a new empty
CRL file that can then be set through a "set+commit ssl crl-file"
command pair. It also tests the "del ssl crl-file" command which allows
to delete an unused CRL file.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
51e28b6bee MEDIUM: ssl: Add "show ssl crl-file" CLI command
This patch adds the "show ssl crl-file [<crlfile>]" CLI command. This
command can be used to display the list of all the known CRL files when
no specific file name is specified, or to display the details of a
specific CRL file when a name is given.
The details displayed for a specific CRL file are inspired by the ones
shown by a "openssl crl -text -noout -in <filename>".
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
eef8e7b8bc MINOR: ssl: Add "abort ssl crl-file" CLI command
The "abort" command aborts an ongoing transaction started by a "set ssl
crl-file" command. Since the updated CRL file data is not pushed into
the CA file tree until a "commit ssl crl-file" call is performed, the
abort command simply deleted the new cafile_entry (storing the new CRL
file data) stored in the transaction.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
720e3b9f33 MEDIUM: ssl: Add "new+del crl-file" CLI commands
This patch adds the "new ssl crl-file" and "del ssl crl-file" CLI
commands.
The "new" command can be used to create a new empty CRL file that can be
filled in thanks to a "set ssl crl-file" command. It can then be used in
a new crt-list line.
The newly created CRL file is added to the CA file tree so any call to
"show ssl crl-file" will display its name.
The "del" command allows to delete an unused CRL file. A CRL file will
be considered unused if its list of ckch instances is empty. It does not
work on an uncommitted CRL file transaction created via a "set ssl
crl-file" command call.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
a51b339d95 MEDIUM: ssl: Add "set+commit ssl crl-file" CLI commands
This patch adds the "set ssl crl-file" and "commit ssl crl-file"
commands, following the same logic as the certificate and CA file update
equivalents.
When trying to update a Certificate Revocation List (CRL) file via a
"set" command, we start by looking for the entry in the CA file tree and
then building a new cafile_entry out of the payload, without adding it
to the tree yet. It will only be added when a "commit" command is
called.
During a "commit" command, we insert the newly built cafile_entry in the
CA file tree while keeping the previous entry. We then iterate over all
the instances that used the CRL file and rebuild a new one and its
dedicated SSL context for every one of them.
When all the contexts are properly created, the old instances get
replaced by the new ones and the old CRL file is removed from the tree.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
f81c70ceec MINOR: ssl: Chain instances in ca-file entries
In order for crl-file hot update to be possible, we need to add an extra
link between the CA file tree entries that hold Certificate Revocation
Lists and the instances that use them. This way we will be able to
rebuild each instance upon CRL modification.
This mechanism is similar to what was made for the actual CA file update
since both the CA files and the CRL files are stored in the same CA file
tree.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
653a16769a DOC: internals: update the SSL architecture schema
Add the new relation between the CA file tree entries and the ckch
instances introduced during CA file hot update implementation (see
GitHub #1057).
2021-05-17 10:50:24 +02:00