18431 Commits

Author SHA1 Message Date
Amaury Denoyelle
da684b9d2c BUG/MINOR: mux-quic: handle properly alloc error in qcs_new()
Use qcs_free() on allocation failure in qcs_new() This ensures that all
qcs content is properly deallocated and prevent memleaks. Most notably,
qcs instance is now removed from qcc tree.

This bug is labelled as MINOR as it occurs only on qcs allocation
failure due to memory exhaustion.

This must be backported up to 2.6.

(cherry picked from commit 4b167006fd574d6377cfc59afb31b167d8694c11)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 164acf2d8a03ad068e7ca1de0964f5f0f07375df)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:25:21 +01:00
Amaury Denoyelle
9ac19a6b3b BUG/MINOR: mux-quic: remove qcs from opening-list on free
qcs instances for bidirectional streams are inserted in
<qcc.opening_list>. It is removed from the list once a full HTTP request
has been parsed. This is required to implement http-request timeout.

If a qcs instance is freed before receiving a full HTTP request, it must
be removed from the <qcc.opening_list>. Else a segfault will occur in
qcc_refresh_timeout() when accessing a dangling pointer.

For the moment this bug was not reproduced in production. This is
because there exists only few rare cases where a qcs is freed before
HTTP request parsing. However, as error detection will be improved on
H3, this will occur more frequently in the near future.

This must be backported up to 2.6.

(cherry picked from commit 641a65ff3cccd394eed49378c6ccdb8ba0a101a7)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 252b67c4722ff2d4131e7875879364087f27a2fa)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:25:17 +01:00
Amaury Denoyelle
e78bb0f6b1 BUG/MINOR: quic: properly handle alloc failure in qc_new_conn()
qc_new_conn() is used to allocate a quic_conn instance and its various
internal members. If one allocation fails, quic_conn_release() is used
to cleanup things.

For the moment, pool_zalloc() is used which ensures that all content is
null. However, some members must be initialized to a special values
to be able to use quic_conn_release() safely. This is the case for
quic_conn lists and its tasklet.

Also, some quic_conn internal allocation functions were doing their own
cleanup on failure without reset to NULL. This caused an issue with
quic_conn_release() which also frees this members. To fix this, these
functions now only return an error without cleanup. It is the caller
responsibility to free the allocated content, which is done via
quic_conn_release().

Without this patch, allocation failure in qc_new_conn() would often
result in segfault. This was reproduced easily using fail-alloc at 10%.

This should be backported up to 2.6.

(cherry picked from commit dbf6ad470b3206f64254141e7cf80a980261be29)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit d35d46916d8ff53b13c08862297f49b5d881d738)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:25:07 +01:00
Ilya Shipitsin
ac1504b040 CI: github: split matrix for development and stable branches
ML ref: https://www.mail-archive.com/haproxy@formilux.org/msg42934.html

we agreed to use "latest" images for development branches and fixed
images for stable branches

Can be backported to 2.6.

(cherry picked from commit f5994fc69268603b686c37fb38e5086f98ec1880)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit e557ae9bac049e1a239510cc77c1812404c4d2ea)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:24:34 +01:00
Ilya Shipitsin
5f493cbe57 CI: github: remove redundant ASAN loop
it was there because we only ran ASAN for clang, now no need to separate loop

Can be backported to 2.6.

(cherry picked from commit 6dedeb70dacf50250cf808872e6b548026b07ccc)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit a468a38c3cc49b8d8876b05da534654134c38fda)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:24:29 +01:00
Amaury Denoyelle
dec6435195 BUG/MEDIUM: h3: parse content-length and reject invalid messages
Ensure that if a request contains a content-length header it matches
with the total size of following DATA frames. This is conformance with
HTTP/3 RFC 9114.

For the moment, this kind of errors triggers a connection close. In the
future, it should be handled only with a stream reset. To reduce
backport surface, this will be implemented in another commit.

This must be backported up to 2.6. It relies on the previous commit :
  MINOR: http: extract content-length parsing from H2

(cherry picked from commit d2c5ee665ea2d812056cb24edbca31dc42e5d246)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 43bb85f88d4a0273f90fa9d41ed52dbcb8c52abb)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:24:11 +01:00
Amaury Denoyelle
d1975e2fc7 MINOR: http: extract content-length parsing from H2
Extract function h2_parse_cont_len_header() in the generic HTTP module.
This allows to reuse it for all HTTP/x parsers. The function is now
available as http_parse_cont_len_header().

Most notably, this will be reused in the next bugfix for the H3 parser.
This is necessary to check that content-length header match the length
of DATA frames.

Thus, it must be backported to 2.6.

(cherry picked from commit 15f3cc4b389d1e92f7d537a2321ad027cf3b5a15)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 76d3becee5c10aacabb5cb26b6776c00ca5b9ae6)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:24:03 +01:00
Amaury Denoyelle
1e4d26aee5 BUG/MEDIUM: h3: reject request with invalid pseudo header
RFC 9114 dictates several requirements for pseudo header usage in H3
request. Previously only minimal checks were implemented. Enforce all
the following requirements with this patch :
* reject request with undefined or invalid pseudo header
* reject request with duplicated pseudo header
* reject non-CONNECT request with missing mandatory pseudo header
* reject request with pseudo header after standard ones

For the moment, this kind of errors triggers a connection close. In the
future, it should be handled only with a stream reset. To reduce
backport surface, this will be implemented in another commit.

This must be backported up to 2.6.

(cherry picked from commit 7b5a671fb8914aa2a2af113d69a80d5dc7ceb841)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit d2938a95c987534b40ebf3a7b51cadc4f3f60867)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:23:59 +01:00
Amaury Denoyelle
22b44d5f2c BUG/MEDIUM: h3: reject request with invalid header name
Reject request containing invalid header name. This concerns every
header containing uppercase letter or a non HTTP token such as a space.

For the moment, this kind of errors triggers a connection close. In the
future, it should be handled only with a stream reset. To reduce
backport surface, this will be implemented in another commit.

Thanks to Yuki Mogi from FFRI Security, Inc. for having reported this.

This must be backported up to 2.6.

(cherry picked from commit d6fb7a0e0f3a79afa1f4b6fc7b62053c3955dc4a)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 3ca4223c5e1f18a19dc93b0b09ffdbd295554d46)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:23:54 +01:00
William Lallemand
a55ada3b0e REGTESTS: startup: add alternatives values in automatic_maxconn.vtc
The calculated maxconn could produce other values when compiled with
debug options.

Must be backported where 6b6f082 was backported (as far as 2.5).

(cherry picked from commit f98b3b1107208499d8b8d70f63356507c81edecd)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 20bd4a8d1507e3ee6d52cc5af6c23a006b0e3a75)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:23:29 +01:00
William Lallemand
daab6c2a35 REGTESTS: startup: change the expected maxconn to 11000
change the expected maxconn from 10000 to 11000 in
automatic_maxconn.vtc

To be backported only if the test failed, the value might be the right
one in previous versions.

(cherry picked from commit 2a225390eb78c1b5fb0fc6c5352974e16fbdd952)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit e191844b64bdc894f424a6e30858c7c55d4fd7dc)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:23:17 +01:00
Christopher Faulet
cba3678307 BUG/MEDIUM: resolvers: Use tick_first() to update the resolvers task timeout
In resolv_update_resolvers_timeout(), the resolvers task timeout is updated
by checking running and waiting resolutions. However, to find the next
wakeup date, MIN() operator is used to compare ticks. Ticks must never be
compared with such operators, tick helper functions must be used, to
properly handled TICK_ETERNITY value. In this case, tick_first() must be
used instead of MIN().

It is an old bug but it is pretty visible since the commit fdecaf6ae4
("BUG/MINOR: resolvers: do not run the timeout task when there's no
resolution"). Because of this bug, the resolvers task timeout may be set to
TICK_ETERNITY, stopping periodic resolutions.

This patch should solve the issue #1962. It must be backported to all stable
versions.

(cherry picked from commit 819d48b14e7edcebc737e5fc7d4f1b269c07bc5d)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit d94ca04f965fd5a2ad7ee500b8bbf46acd722206)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:22:11 +01:00
William Lallemand
27103c3c6d REGTESTS: startup: activate automatic_maxconn.vtc
Check if USE_OBSOLETE_LINK=1 was used so it could run this test when
ASAN is not built, since ASAN require this option.

For this test to work, the ulimit -n value must be big enough.

Could be backported at least to 2.5.

(cherry picked from commit 6b6f082969acf2694ac9d688d408e3ab6586b1ec)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit b6bfe7b905a4fb8197c30db7fe937840506812af)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:20:34 +01:00
William Lallemand
fb4957c4fe CI: github: set ulimit -n to a greater value
Set ulimit -n to 65536 to limit less the maxconn computation.

Could be backported at least to 2.5.

(cherry picked from commit 2cb1493748c4f434af084e3d027679b17580bf89)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit c7ff3f0419d8ddb09b633f8aa50c167e45cc081e)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:20:25 +01:00
William Lallemand
d3c486f567 BUG/MINOR: startup: don't use internal proxies to compute the maxconn
With internal proxies using the SSL activated (httpclient for example)
the automatic computation of the maxconn is wrong because these proxies
are always activated by default.

This patch fixes the issue by not counting these internal proxies during
the computation.

Must be backported as far as 2.5.

(cherry picked from commit 0adafb307eacd75ad0305f996c07668f470da1e4)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit b1005c0ba1db639c15d4fee17820af40039c1894)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:20:02 +01:00
William Lallemand
a2ab8f902d REGTESTS: startup: check maxconn computation
Check the maxconn computation with multiple -m parameters.

Broken with ASAN for now.

Could be backported as far as 2.2.

(cherry picked from commit 38c5b6ea971952e2fd5ca6949d2f4076d9c1f6ff)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 8ffe3f24e889c8406cfd29eb6807cb4f45cfad25)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:19:57 +01:00
Christopher Faulet
315ab77aec REGTESTS: fix the race conditions in iff.vtc
A "Connection: close" header is added to responses to avoid any connection
reuse. This should avoid any "HTTP header incomplete" errors.

(cherry picked from commit e1b866a28a53035106cb6d1b49b5951e26215d76)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 50339568f9aed04dda6955129e11f92164da30b7)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:19:50 +01:00
Youfu Zhang
f988992d16 BUG/MAJOR: fcgi: Fix uninitialized reserved bytes
The output buffer is not zero-initialized. If we don't clear reserved
bytes, fcgi requests sent to backend will leak sensitive data.

This patch must be backported as far as 2.2.

(cherry picked from commit 2e6bf0a2722866ae0128a4392fa2375bd1f03ff8)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit db03179fee55c60a92ce6b86a0f04dbb9ba0328b)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:19:44 +01:00
Christopher Faulet
02495c9e7d DOC: promex: Add missing backend metrics
"haproxy_backend_agg_server_status" and "haproxy_backend_agg_check_status"
were not referenced in promex README.

"haproxy_backend_agg_server_check_status" is also missing but it is a
deprecated metric. Thus, it is better to not reference it.

(cherry picked from commit 7edec90c002740df26d13f8f780f5ccd762fb876)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit e41897cad6400ca2a9de6d63af4ee7363563ac16)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:19:39 +01:00
Cedric Paillet
d8567a9fea MINOR: promex: introduce haproxy_backend_agg_check_status
This patch introduces haproxy_backend_agg_check_status metric
as we wanted in 42d7c402d but with the right data source.

This patch could be backported as far as 2.4.

(cherry picked from commit e06e31ea3b62ef8ccb911ac3969ae70f7bbb7574)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit f0319e0f56581873f906f79dc218bf6f10b8f6c2)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:19:28 +01:00
Cedric Paillet
7962dcc093 BUG/MINOR: promex: create haproxy_backend_agg_server_status
haproxy_backend_agg_server_check_status currently aggregates
haproxy_server_status instead of haproxy_server_check_status.
We deprecate this and create a new one,
haproxy_backend_agg_server_status to clarify what it really
does.

This patch could be backported as far as 2.4.

(cherry picked from commit 7d6644e689f15b329789a355ea2812ea0223fe4f)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 2c0d7982e7612b2e7157170aa7109f20b780bb64)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 09:19:24 +01:00
William Lallemand
7b6a3c0575 BUG/MEDIUM: httpclient/lua: double LIST_DELETE on end of lua task
The lua httpclient cleanup can be called in 2 places, the
hlua_httpclient_gc() and the hlua_httpclient_destroy_all().

A LIST_DELETE() is performed to remove the hlua_hc struct of the list.
However, when the lua task ends and call hlua_ctx_destroy(), it does a
LIST_DELETE() first, and then the gc tries to do a LIST_DELETE() again
in hlua_httpclient_gc(), provoking a crash.

This patch fixes the issue by doing a LIST_DEL_INIT() instead of
LIST_DELETE() in both cases.

Should fix issue #1958.

Must be backported where bb58142 is backported.

(cherry picked from commit 94dbfedec138101f314d746dab87caa3a1760b59)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit c177de37d8f68a9434530e4f5706efdaa2b934b5)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 08:57:40 +01:00
Willy Tarreau
abb8b15c9b BUILD: makefile/da: also clean Os/ in Device Atlas dummy lib dir
Commit b81483cf2 ("MEDIUM: da: update doc and build for new scheduler
mode service.") added a new directory to the Device Atlas dummy lib,
but this one is not cleaned during "make clean", causing build failures
sometimes when switching between compiler versions during development.

This should be backported to 2.6.

(cherry picked from commit 46676d44e04d4348d6d17317597692f674407038)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 3e815946e14487cc1318f3c78c7dd15c0c28de5c)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 08:57:19 +01:00
William Lallemand
360b26a15d BUG/MEDIUM: mworker: fix segv in early failure of mworker mode with peers
During an early failure of the mworker mode, the
mworker_cleanlisteners() function is called and tries to cleanup the
peers, however the peers are in a semi-initialized state and will use
NULL pointers.

The fix check the variable before trying to use them.

Bug revealed in issue #1956.

Could be backported as far as 2.0.

(cherry picked from commit 035058e8bfe4fc731b2ede8fcdc2b8d14c4936ce)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 661557989e3a8d84d18c997ebdabb26146ebe8ad)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 08:56:42 +01:00
William Lallemand
06dc13be6b MINOR: mworker: display an alert upon a wait-mode exit
When the mworker wait mode fails it does an exit, but there is no
error message which says it exits.

Add a message which specify that the error is non-recoverable.

Could be backported in 2.7 and possibly earlier branch.

(cherry picked from commit 40db4ae8bb864adfebbd4529a3bae187974b3698)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit bb0ab9833adf1c871143d8555fedbb9ec1823f8a)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2023-01-20 08:56:23 +01:00
Remi Tricot-Le Breton
0ce2ac2a72 BUG/MINOR: ssl: Fix potential overflow
Coverity raised a potential overflow issue in these new functions that
work on unsigned long long objects. They were added in commit 9b25982
"BUG/MEDIUM: ssl: Verify error codes can exceed 63".

This patch needs to be backported alongside 9b25982.

(cherry picked from commit e239e4938d89956e7820be4a0f26e782a86bcf6d)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2023-01-17 15:11:27 +01:00
Remi Tricot-Le Breton
64fa46abcc BUG/MEDIUM: ssl: Verify error codes can exceed 63
The CRT and CA verify error codes were stored in 6 bits each in the
xprt_st field of the ssl_sock_ctx meaning that only error code up to 63
could be stored. Likewise, the ca-ignore-err and crt-ignore-err options
relied on two unsigned long longs that were used as bitfields for all
the ignored error codes. On the latest OpenSSL1.1.1 and with OpenSSLv3
and newer, verify errors have exceeded this value so these two storages
must be increased. The error codes will now be stored on 7 bits each and
the ignore-err bitfields are replaced by a big enough array and
dedicated bit get and set functions.

It can be backported on all stable branches.

[wla: let it be tested a little while before backport]
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit 9b25982716f0416c28f8fc894c58eb40885cf9e5)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2023-01-17 15:11:22 +01:00
William Lallemand
f0994946ae BUILD: peers: peers-t.h depends on stick-table-t.h
peers-t.h uses "struct stktable" as well as STKTABLE_DATA_TYPES which
are defined in stick-table-t.h. It works by accident because
stick-table-t.h was always included before. But could provoke build
issue with EXTRA code.

To be backported as far as 2.2.

(cherry picked from commit 46bea1c6163731a45749e4429fbd1294441a7c68)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit 5c89a0c0484b706cfa10398be8539f39c7b311e9)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2022-12-16 16:03:01 +01:00
Ilya Shipitsin
28784f7b8c CI: github: split ssl lib selection based on git branch
when *SSL_VERSION="latest" behaviour was introduced, it seems to be fine
for development branches, but too intrusive for stable branches.

let us limit "latest" semantic only for development builds, if branch name
contains "haproxy-" it is supposed to be stable branch, no latest openssl
should be taken

[wla: must be backported as far as 2.6]
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit 4a04cd35ae89bf6a3bb7620f7a49804de3240ac4)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit 91490ad7e928af4abfc8b9fc7493f10ce05f5ac4)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2022-12-12 17:00:58 +01:00
William Lallemand
dd7c7da99b CI: github: reintroduce openssl 1.1.1
OpenSSL 1.1.1 is not tested anymore since github updated "ubuntu-latest"
to 22.04, let's reintroduce this version.

(cherry picked from commit 393e4e4dd155a9553aaa7d6e0870d6cbfcaec9c0)
[wla: only need to be backported for 2.7 and 2.6, prior versions are
still build with ubuntu-20.04]
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
(cherry picked from commit 45d1f973d5a8d5af3061959e35af422fe7746909)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2022-12-12 09:06:33 +01:00
Christopher Faulet
585d4687cd BUG/MEDIIM: stconn: Flush output data before forwarding close to write side
In process_stream(), we wait to have an empty output channel to forward a
close to the write side (a shutw). However, at the stream-connector level,
when a close is detected on one side and we don't want to keep half-close
connections, the shutw is unconditionally forwarded to the write side. This
typically happens on server side.

At first glance, this bug may truncate messages. But depending on the muxes
and the stream states, the bug may be more visible. On recent versions
(2.8-dev and 2.7) and on 2.2 and 2.0, the stream may be freezed, waiting for
the client timeout, if the client mux is unable to forward data because the
client is too slow _AND_ the response channel is not empty _AND_ the server
closes its connection _AND_ the server mux has forwarded all data to the
upper layer _AND_ the client decides to send some data and to close its
connection. On 2.6 and 2.4, it is worst. Instead of a freeze, the client mux
is woken up in loop.

Of course, conditions are pretty hard to meet. Especially because it is highly
time dependent. For what it's worth, I reproduce it with tcploop on client and
server sides and a basic HTTP configuration for HAProxy:

  * client: tcploop -v 8889 C S:"GET / HTTP/1.1\r\nConnection: upgrade\r\n\r\n" P5000 S:"1234567890" K
  * server: tcploop -v 8000 L A R S:"HTTP/1.1 101 ok\r\nConnection: upgrade\r\n\r\n" P2000 S2660000 F R

On 2.8-dev, without this patch, the stream is freezed and when the client
connection timed out, client data are truncated and '--cL' is reported in
logs. With the patch, the client data are forwarded to the server and the
connection is closed. A '--CD' is reported in logs.

It is an old bug. It was probably introduced with the multiplexers. To fix
it, in stconn (Formerly the stream-interface), we must wait all output data
be flushed before forwarding close to write side.

This patch must be backported as far as 2.2 and must be evaluated for 2.0.

(cherry picked from commit 7f59d68fe272c677e577523b3fe866f2b22a56ab)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 4c667c94f575e18a10558ea7fea97b3427eb9d66)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-06 11:03:13 +01:00
Christopher Faulet
72be5e4efd BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action
An abosulte URI is marked as normalized if it comes from an H2 client. This
way, we know we can send a relative URI to an H1 server. But, after a
set-uri action, the URI must no longer be considered as normalized.
Otherwise there is no way to send an absolute URI on the server side.

If it is important to update a normalized absolute URI without altering this
property, the host, path and/or query-string must be set separatly.

This patch should fix the issue #1938. It should be backported as far as
2.4.

(cherry picked from commit 84cdbe478a82afdcaf4f049e8ed431ca349c6ba2)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-05 07:39:16 +01:00
Christopher Faulet
c55bfdb6e5 [RELEASE] Released version 2.6.7
Released version 2.6.7 with the following main changes :
    - REGTESTS: 4be_1srv_smtpchk_httpchk_layer47errors: Return valid SMTP replies
    - BUG/MINOR: hlua: Remove \n in Lua error message built with memprintf
    - BUG/MINOR: stream: Perform errors handling in right order in stream_new()
    - BUG/MEDIUM: stconn: Reset SE descriptor when we fail to create a stream
    - BUG/MEDIUM: resolvers: Remove aborted resolutions from query_ids tree
    - BUG/MINOR: hlua: fixing hlua_http_msg_del_data behavior
    - BUG/MINOR: hlua: fixing hlua_http_msg_insert_data behavior
    - BUG/MINOR: hlua: _hlua_http_msg_delete incorrect behavior when offset is used
    - DOC: management: httpclient can resolve server names in URLs
    - BUG/MAJOR: conn-idle: fix hash indexing issues on idle conns
    - BUG/MINOR: backend: only enforce turn-around state when not redispatching
    - BUG/MINOR: checks: update pgsql regex on auth packet
    - DOC: config: Fix pgsql-check documentation to make user param mandatory
    - CLEANUP: mux-quic: remove usage of non-standard ull type
    - CLEANUP: quic: remove global var definition in quic_tls header
    - BUG/MINOR: quic: adjust quic_tls prototypes
    - CLEANUP: quic: fix headers
    - CLEANUP: quic: remove unused function prototype
    - CLEANUP: quic: remove duplicated varint code from xprt_quic.h
    - CLEANUP: quic: create a dedicated quic_conn module
    - BUG/MINOR: mux-quic: ignore STOP_SENDING for locally closed stream
    - BUG/MEDIUM: lua: Don't crash in hlua_lua2arg_check on failure
    - BUG/MEDIUM: lua: handle stick table implicit arguments right.
    - BUILD: h1: silence an initiialized warning with gcc-4.7 and -Os
    - MINOR: fd: add a new function to only raise RLIMIT_NOFILE
    - MINOR: init: do not try to shrink existing RLIMIT_NOFIlE
    - BUG/MINOR: http-fetch: Update method after a prefetch in smp_fetch_meth()
    - BUILD: http_fetch: silence an uninitiialized warning with gcc-4/5/6 at -Os
    - BUG/MINOR: hlua: hlua_channel_insert_data() behavior conflicts with documentation
    - MINOR: quic: limit usage of ssl_sock_ctx in favor of quic_conn
    - MINOR: mux-quic: check quic-conn return code on Tx
    - CLEANUP: quic: fix indentation
    - BUG/MINOR: mux-h1: Account consumed output data on synchronous connection error
    - MINOR: smtpchk: Update expect rule to fully match replies to EHLO commands
    - BUG/MINOR: smtpchk: SMTP Service check should gracefully close SMTP transaction
    - BUG/MINOR: config: don't count trailing spaces as empty arg (v2)
    - BUG/MEDIUM: config: count line arguments without dereferencing the output
    - MEDIUM: quic: retrieve frontend destination address
    - CLEANUP: quic/receiver: remove the now unused tx_qring list
    - BUG/MINOR: quic: set IP_PKTINFO socket option for QUIC receivers only
    - DOC: configuration: missing 'if' in tcp-request content example
    - BUG/MAJOR: stick-tables: do not try to index a server name for applets
    - BUG/MINOR: server: make sure "show servers state" hides private bits
    - MINOR: quic: New quic_cstream object implementation
    - MINOR: quic: Extract CRYPTO frame parsing from qc_parse_pkt_frms()
    - MINOR: quic: Use a non-contiguous buffer for RX CRYPTO data
    - BUG/MINOR: quic: Stalled 0RTT connections with big ClientHello TLS message
    - MINOR: quic: Split the secrets key allocation in two parts
    - CLEANUP: quic: remove unused rxbufs member in receiver
    - CLEANUP: quic: improve naming for rxbuf/datagrams handling
    - MINOR: quic: implement datagram cleanup for quic_receiver_buf
    - BUILD: ssl_sock: bind_conf uninitialized in ssl_sock_bind_verifycbk()
    - BUG/MEDIUM: httpclient: Don't set EOM flag on an empty HTX message
    - MINOR: httpclient/lua: Don't set req_payload callback if body is empty
    - CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in matrix.py
    - CI: Replace the deprecated `::set-output` command by writing to $GITHUB_OUTPUT in workflow definition
    - BUILD: quic: QUIC mux build fix for 32-bit build
    - BUG/MEDIUM: httpclient: segfault when the httpclient parser fails
    - BUILD: ssl_sock: fix null dereference for QUIC build
    - BUILD: quic: Fix build for m68k cross-compilation
    - BUG/MINOR: quic: fix buffer overflow on retry token generation
    - MINOR: quic: add version field on quic_rx_packet
    - MINOR: quic: extend pn_offset field from quic_rx_packet
    - MINOR: quic: define first packet flag
    - MINOR: quic: extract connection retrieval
    - MINOR: quic: split and rename qc_lstnr_pkt_rcv()
    - MINOR: quic: refactor packet drop on reception
    - MINOR: quic: extend Retry token check function
    - BUG/MINOR: log: Preserve message facility when the log target is a ring buffer
    - BUG/MINOR: ring: Properly parse connect timeout
    - BUG/MEDIUM: httpclient/lua: crash when the lua task timeout before the httpclient
    - BUG/MEDIUM: httpclient: check if the httpclient was released in the IO handler
    - REGTESTS: httpclient/lua: test the lua task timeout with the httpclient
    - CI: github: dump the backtrace of coredumps in the alpine container
    - BUILD: Makefile: add "USE_SHM_OPEN" on the linux-musl target
    - BUG/MINOR: mux-quic: complete flow-control for uni streams
    - BUG/MEDIUM: compression: handle rewrite errors when updating response headers
    - MINOR: quic: do not crash on unhandled sendto error
    - MINOR: quic: display unknown error sendto counter on stat page
    - BUG/MINOR: sink: Only use backend capability for the sink proxies
    - BUG/MINOR: sink: Set default connect/server timeout for implicit ring buffers
    - CI: SSL: use proper version generating when "latest" semantic is used
    - CI: SSL: temporarily stick to LibreSSL=3.5.3
    - DOC: management: add forgotten "show startup-logs"
    - DOC: lua: add a note about compression w/ httpclient
    - BUG/MAJOR: stick-table: don't process store-response rules for applets
    - BUG/MEDIUM: stick-table: fix a race condition when updating the expiration task
    - MINOR: quic: remove unnecessary quic_session_accept()
    - BUG/MINOR: quic: fix subscribe operation
    - BUG/MINOR: log: fixing bug in tcp syslog_io_handler Octet-Counting
    - BUG/MINOR: quic: fix race condition on datagram purging
    - CI: add monthly gcc cross compile jobs
    - BUG/MINOR: httpclient: fixed memory allocation for the SSL ca_file
    - BUG/MINOR: ssl: Memory leak of DH BIGNUM fields
    - BUG/MINOR: ssl: Memory leak of AUTHORITY_KEYID struct when loading issuer
    - BUG/MINOR: ssl: ocsp structure not freed properly in case of error
    - CI: switch to the "latest" LibreSSL
    - CI: enable QUIC for LibreSSL builds
    - CI: emit the compiler's version in the build reports
    - BUG/MEDIUM: wdt/clock: properly handle early task hangs
    - BUG/MINOR: http-htx: Fix error handling during parsing http replies
    - BUG/MINOR: resolvers: Don't wait periodic resolution on healthcheck failure
    - BUG/MINOR: resolvers: Set port before IP address when processing SRV records
    - BUG/MINOR: mux-fcgi: Be sure to send empty STDING record in case of zero-copy
    - BUG/MEDIUM: mux-fcgi: Avoid value length overflow when it doesn't fit at once
    - BUG/MINOR: mux-h1: Do not send a last null chunk on body-less answers
    - REG-TESTS: cache: Remove T-E header for 304-Not-Modified responses
    - DOC: config: fix alphabetical ordering of global section
    - BUG/MEDIUM: ring: fix creation of server in uninitialized ring
    - BUILD: quic: fix dubious 0-byte overflow on qc_release_lost_pkts
    - BUG/MINOR: pool/cli: use ullong to report total pool usage in bytes
    - BUG/MEDIUM: listener: Fix race condition when updating the global mngmt task
    - BUG/MINOR: http_ana/txn: don't re-initialize txn and req var lists
    - BUG/MEDIUM: raw-sock: Don't report connection error if something was received
    - BUG/MINOR: ssl: don't initialize the keylog callback when not required
    - BUG/MEDIUM: peers: messages about unkown tables not correctly ignored
    - BUILD: peers: Remove unused variables
    - MINOR: ncbuf: complete doc for ncb_advance()
    - BUG/MEDIUM: quic: fix unsuccessful handshakes on ncb_advance error
    - BUG/MEDIUM: quic: fix memleak for out-of-order crypto data
    - MINOR: quic: complete traces/debug for handshake
    - BUG/MAJOR: quic: Crash upon retransmission of dgrams with several packets
    - BUG/MAJOR: quic: Crash after discarding packet number spaces
    - DOC: configuration: fix quic prefix typo
    - MINOR: quic: report error if force-retry without cluster-secret
    - MINOR: global: generate random cluster.secret if not defined
    - BUG/MINOR: server/idle: at least use atomic stores when updating max_used_conns
    - BUILD: listener: fix build warning on global_listener_rwlock without threads
    - DOC: quic: add note on performance issue with listener contention
    - BUG/MINOR: cfgparse-listen: fix ebpt_next_dup pointer dereference on proxy "from" inheritance
    - BUG/MINOR: log: fix parse_log_message rfc5424 size check
    - BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action
    - BUILD: http-htx: Silent build error about a possible NULL start-line
    - DOC: configuration.txt: add default_value for table_idle signature
    - BUILD: ssl-sock: Silent error about NULL deref in ssl_sock_bind_verifycbk()
    - BUG/MINOR: mux-h1: Fix handling of 408-Request-Time-Out
    - DOC: configuration.txt: fix typo in table_idle signature
    - BUG/MEDIUM: quic: fix datagram dropping on queueing failed
    - MINOR: ssl: enhance ca-file error emitting
    - MINOR: ssl: forgotten newline in error messages on ca-file
    - BUG/MINOR: ssl: shut the ca-file errors emitted during httpclient init
    - Revert "BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action"
    - DOC: config: provide some configuration hints for "http-reuse"
    - DOC: config: refer to section about quoting in the "add_item" converter
    - DOC: config: clarify the fact that SNI should not be used in HTTP scenarios
    - DOC: config: mention that a single monitor-uri rule is supported
    - DOC: config: explain how default matching method for ACL works
    - DOC: config: clarify the fact that "retries" is not just for connections
    - DOC: config: clarify the -m dir and -m dom pattern matching methods
    - SCRIPTS: announce-release: add a link to the data plane API
    - CLEANUP: ncbuf: remove ncb_blk args by value
    - CLEANUP: ncbuf: inline small functions
    - CLEANUP: ncbuf: use standard BUG_ON with DEBUG_STRICT
    - BUG/MINOR: quic: Endless loop during retransmissions
    - MINOR: mux-h2: add the expire task and its expiration date in "show fd"
    - MINOR: mux-h1: add the expire task and its expiration date in "show fd"
2022-12-02 18:10:22 +01:00
Christopher Faulet
28c3938c76 MINOR: mux-h1: add the expire task and its expiration date in "show fd"
Just like for the H2 multiplexer, info about the H1 connection task is now
displayed in "show fd" output. The task pointer is displayed and, if not
null, its expiration date.

It may be useful to backport it.

(cherry picked from commit 38f61351c390f139c694e40d69ebdccf1fab3792)
[cf: changes applied in h1_show_fd()]
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:35:51 +01:00
Willy Tarreau
37a325bdc9 MINOR: mux-h2: add the expire task and its expiration date in "show fd"
Some issues such as #1929 seem to involve a task without timeout but we
can't find the condition to reproduce this in the code. However, not having
this info in the output doesn't help, so this patch adds the task pointer
and its timeout (when the task is non-null). It may be useful to backport
it.

(cherry picked from commit f8c7709013929391c507b57a25134ea96ffa1c6c)
[cf: changes applied in h2_show_fd()]
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:33:46 +01:00
Frédéric Lécaille
3da16eabd4 BUG/MINOR: quic: Endless loop during retransmissions
qc_dgrams_retransmit() could reuse the same local list and could splice it two
times to the packet number space list of frame to be send/resend. This creates a
loop in this list and makes qc_build_frms() possibly endlessly loop when trying
to build frames from the packet number space list of frames. Then haproxy aborts.

This issue could be easily reproduced patching qc_build_frms() function to set <dlen>
variable value to 0 after having built at least 10 CRYPTO frames and using ngtcp2
as client with 30% packet loss in both direction.

Thank you to @gabrieltz for having reported this issue in GH #1903.

Must be backported to 2.6.

(cherry picked from commit 7b5d9b1f03cef92bda6cd2a3be93b9bbbfd61734)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:31:34 +01:00
Amaury Denoyelle
6ce0ffdb3d CLEANUP: ncbuf: use standard BUG_ON with DEBUG_STRICT
ncbuf can be compiled for haproxy or standalone to run unit test suite.
For the latest mode, BUG_ON() macro has been re-implemented in a simple
version.

The inclusion of the default or the redefined macro relied on DEBUG_DEV.
Change this to now rely on DEBUG_STRICT as this is activated for the
default build.

This change is safe as only BUG_ON_HOT() macro is used in ncbuf code,
which is activated only with the default value DEBUG_STRICT=2.

This should be backported up to 2.6.

(cherry picked from commit 2526a6aca5c6f453571f2ccdcea7e2ab66aeea67)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:31:31 +01:00
Amaury Denoyelle
8083a52d73 CLEANUP: ncbuf: inline small functions
ncbuf API relies on lot of small functions. Mark these functions as
inline to reduce call invocations and facilitate compiler optimizations
to reduce code size.

This should be backported up to 2.6.

(cherry picked from commit d64a26f0238f386065e26654e6a8a925f96c8baa)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:31:28 +01:00
Amaury Denoyelle
213d39921a CLEANUP: ncbuf: remove ncb_blk args by value
ncb_blk structure is used to represent a block of data or a gap in a
non-contiguous buffer. This is used in several functions for ncbuf
implementation. Before this patch, ncb_blk was passed by value, which is
sub-optimal. Replace this by const pointer arguments.

This has the side-effect of suppressing a compiler warning reported in
older GCC version :
  CC      src/http_conv.o
  src/ncbuf.c: In function 'ncb_blk_next':
  src/ncbuf.c:170: warning: 'blk.end' may be used uninitialized in this function

This should be backported up to 2.6.

(cherry picked from commit 17e20e8cefbdd6518f362325cd528aee2bcdb277)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:31:24 +01:00
Willy Tarreau
d4223c17e7 SCRIPTS: announce-release: add a link to the data plane API
Since Marko announced at HAProxyConf 2022 that the data plane API is
mostly complete and will now follow the same release cycle as haproxy
starting with 2.7, it's probably the right moment to encourage users
to start trying it so that we can hope to migrate all the painful
discovery stuff there in a not too distant future.

Let's just point to the latest release for now. We'll see in the future
if we need to adapt the link depending on the branch.

(cherry picked from commit e3a02d5e08e4ee4f70bb4553694bc32adb7931fc)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:31:03 +01:00
Willy Tarreau
541ef40850 DOC: config: clarify the -m dir and -m dom pattern matching methods
There's regularly some confusion about them (do they match at the
beginning, end ? do they support multiple components etc). Tim
suggested to improve the doc in issue #61, it's never too late, so
let's do it now wih a few examples.

(cherry picked from commit f386a2de92702386ff73aa9da662ebed2766bfbb)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:29:48 +01:00
Willy Tarreau
e29a4d1547 DOC: config: clarify the fact that "retries" is not just for connections
In issue #412 it was rightfully reported that the wording in "retries"
still exclusively speaks about connection attempts, while since L7
retries with "retry-on" it's no longer a limitation. Let's update the
text.

(cherry picked from commit 0b4a622b49437e9f41f764a3909f43b7cbd04f61)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:29:38 +01:00
Willy Tarreau
73f69e72a7 DOC: config: explain how default matching method for ACL works
In issue #698, it's made apparent that the default matching method for
ACL keywords can be confusing when a converter is applied, because
depending on the converters used, users may think that the default
matching method from the sample fetch name might apply to the whole
expression. It's easier to understand that this doesn't make sense
when thinking about converters turning to completely different types
(e.g. hdr_beg(host),do_resolve() returns an IP, thus it's obvious
that _beg makes no sense at all).  This patch states this in the
doc to avoid future confusion.

(cherry picked from commit 4f4fea417b2c80b2db738a8cf2ea0a3c67ccfc7b)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:29:28 +01:00
Willy Tarreau
104cf6ed1a DOC: config: mention that a single monitor-uri rule is supported
It was reported in issue #1059 that when multiple monitor-uri rules are
specified, only the last one is used. While this was done on purpose
since a single URI is used, it was not clearly mentioned in the doc,
possibly leading to confusion or wasted time trying to establish a
working setup. Let's clarify this point.

(cherry picked from commit 7fe0c62516ac0cb54290f8b6febe351038792e98)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:29:18 +01:00
Willy Tarreau
0b41d6be9b DOC: config: clarify the fact that SNI should not be used in HTTP scenarios
As reported by Tim in issue #1373 some warnings are deserved to explain
why using the frontend SNI for routing or connecting to a server is
usually not correct, especially since it can be tempting and used to
make sense in pure TCP scenarios.

(cherry picked from commit d26fb57e817da6cb587c0b61baf60a011e756c49)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:29:11 +01:00
Willy Tarreau
0b80fa9093 DOC: config: refer to section about quoting in the "add_item" converter
As requested by Nick in issue #1719, let's add a reference to the section
about quoting there, since add_item() will often be used with commas and
it's easy to mess up.

(cherry picked from commit b143d110bf6b5fa65ba864e153f7cc0ee1c5ebac)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:28:29 +01:00
Willy Tarreau
e3bafd5980 DOC: config: provide some configuration hints for "http-reuse"
This adds some configuration hints regarding various workloads that do
not manage to achieve high reuse rates due to too low a global maxconn
or thread groups.

This fixes github issue #1472.

(cherry picked from commit 44fce8bd73cc743afc7e7f3f70a5a177a9557e62)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
2022-12-02 15:28:15 +01:00
Christopher Faulet
f7acbd8407 Revert "BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri action"
This reverts commit 7533b98b8a4023f815eb7374fa3118bed67a3fc5.

This fix is reverted for now because it may introduce issues with some
config. So we want to announce the possible breakage in the 2.6.7 and
include the fix into the 2.6.8. This way, users will have some time to
modify their configuration.
2022-12-02 15:22:28 +01:00
William Lallemand
b1351c1a05 BUG/MINOR: ssl: shut the ca-file errors emitted during httpclient init
With an OpenSSL library which use the wrong OPENSSLDIR, HAProxy tries to
load the OPENSSLDIR/certs/ into @system-ca, but emits a warning when it
can't.

This patch fixes the issue by allowing to shut the error when the SSL
configuration for the httpclient is not explicit.

Must be backported in 2.6.

(cherry picked from commit 0a2d63236c4ada9a33f7e9495aa332fdcd9f5f82)
[wla: context changed in httpclient_precheck()]
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2022-11-25 09:58:29 +01:00
William Lallemand
d371520001 MINOR: ssl: forgotten newline in error messages on ca-file
Add forgotten newlines in ssl_store_load_ca_from_buf() error messages.

(cherry picked from commit 3992f55ff390b3db423ffbc7fd322e84a7256ab4)
Signed-off-by: William Lallemand <wlallemand@haproxy.org>
2022-11-25 09:52:15 +01:00