13484 Commits

Author SHA1 Message Date
Tim Duesterhus
54182ec9d7 CLEANUP: Apply the coccinelle patch for XXXcmp() on include/
Compare the various `XXXcmp()` functions against zero.
2021-01-04 10:09:02 +01:00
Tim Duesterhus
e5ff14100a CLEANUP: Compare the return value of XXXcmp() functions with zero
According to coding-style.txt it is recommended to use:

`strcmp(a, b) == 0` instead of `!strcmp(a, b)`

So let's do this.

The change was performed by running the following (very long) coccinelle patch
on src/:

    @@
    statement S;
    expression E;
    expression F;
    @@

      if (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
      )
    (
      S
    |
      { ... }
    )

    @@
    statement S;
    expression E;
    expression F;
    @@

      if (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
      )
    (
      S
    |
      { ... }
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G &&
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G ||
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    && G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    || G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G &&
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G ||
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    && G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    || G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )
2021-01-04 10:09:02 +01:00
Tim Duesterhus
f89d43a381 MINOR: lua: Use consistent error message 'memory allocation failed'
Other locations in the configuration parser use 'memory allocation failed', so
use this one as well.
2021-01-03 20:37:16 +01:00
Tim Duesterhus
621e74afd1 BUG/MINOR: lua: Fix memory leak error cases in hlua_config_prepend_path
In case of an error `p` is not properly freed.

Minor leak during configuration parsing in out of memory situations, no
backport needed.
2021-01-03 20:37:16 +01:00
Tim Duesterhus
a7ebffef66 BUG/MINOR: sink: Return an allocation failure in __sink_new if strdup() fails
This patch fixes GitHub issue #1023.

The function was introduced in commit 99c453d ("MEDIUM: ring: new
section ring to declare custom ring buffers."), which first appeared
in 2.2-dev9. The fix should be backported to 2.2+.
2021-01-03 20:35:45 +01:00
Willy Tarreau
5e8c35da1b BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11
Ilya reported in issue #1010 that gcc 11 (still in development phase 3)
sees out-of-bounds accesses to tlskeys_reference. After a deep analysis
it turns out that the warning is erroneous and Tim could set up a
reproducer that was further simplified to file a gcc bug report (98503).
In order not to break builds on this version, let's temporarily disable
the warning, since no acceptable workaround could be defined. Once gcc
gets it right, the warning should be re-enabled as it is likely to catch
real errors (it's the first false positive here for us even though
roughly one hundred issues were reported around this one alone).
2021-01-03 20:27:13 +01:00
Thayne McCombs
23cc52d34b REGTESTS: add test for stickiness using "srvkey addr"
This tests the new "srvkey addr" stick-table statement, which sticks based
on the server's address.
2020-12-31 10:08:54 +01:00
Thayne McCombs
92149f9a82 MEDIUM: stick-tables: Add srvkey option to stick-table
This allows using the address of the server rather than the name of the
server for keeping track of servers in a backend for stickiness.

The peers code was also extended to support feeding the dictionary using
this key instead of the name.

Fixes #814
2020-12-31 10:04:54 +01:00
Tim Duesterhus
dc38bc4a1a BUG/MEDIUM: cache: Fix hash collision in accept-encoding handling for Vary
This patch fixes GitHub Issue #988. Commit ce9e7b25217c46db1ac636b2c885a05bf91ae57e
was not sufficient, because it fell back to a hash comparison if the bitmap
of known encodings was not acceptable instead of directly returning the the
cached response is not compatible.

This patch also extends the reg-test to test the hash collision that was
mentioned in #988.

Vary handling is 2.4, no backport needed.
2020-12-31 09:39:08 +01:00
Ilya Shipitsin
64b6f36788 CI: GitHub Actions: enable daily Coverity scan
That scan was previously implemented on Travis. Let us migrate
it to GitHub Actions.

Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
2020-12-28 12:02:09 +01:00
Remi Tricot-Le Breton
e6cc5b5974 MINOR: cache: Replace the "process-vary" option's expected values
Replace the <0/1> expected values of the process-vary option by a more
usual <on/off> pair.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
42efffd7f6 MINOR: cache: Remove redundant test in http_action_req_cache_use
The suppressed check is fully covered by the next one and can then be
removed.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
14df1e1526 REGTESTS: cache: Add a specific test for the accept-encoding normalizer
Those tests check the explicit support of a subset of encodings added to
the accept-encoding normalizer.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
b054b6dec6 REGTESTS: cache: Simplify vary.vtc file
This test used chunked responses but it is not needed. All the chunked
responses are replaced by more simple 'bodylen' directives.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
ce9e7b2521 MEDIUM: cache: Manage a subset of encodings in accept-encoding normalizer
The accept-encoding normalizer now explicitely manages a subset of
encodings which will all have their own bit in the encoding bitmap
stored in the cache entry. This way two requests with the same primary
key will be served the same cache entry if they both explicitely accept
the stored response's encoding, even if their respective secondary keys
are not the same and do not match the stored response's one.
The actual hash of the accept-encoding will still be used if the
response's encoding is unmanaged.
The encoding matching and the encoding weight parsing are done for every
subpart of the accept-encoding values, and a bitmap of accepted
encodings is built for every request. It is then tested upon any stored
response that has the same primary key until one with an accepted
encoding is found.
The specific "identity" and "*" accept-encoding values are managed too.
When storing a response in the key, we also parse the content-encoding
header in order to only set the response's corresponding encoding's bit
in its cache_entry encoding bitmap.

This patch fixes GitHub issue #988.
It does not need to be backported.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
56e46cb393 MINOR: http: Add helper functions to trim spaces and tabs
Add two helper functions that trim leading or trailing spaces and
horizontal tabs from an ist string.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
6a34b2b65d MINOR: cache: Add specific secondary key comparison mechanism
Add the possibility to define custom comparison functions for every
sub-part of the secondary key hash instead of using a global memcmp.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
e4421dec7e BUG/MINOR: cache: Manage multiple headers in accept-encoding normalization
The accept-encoding part of the secondary key (vary) was only built out
of the first occurrence of the header. So if a client had two
accept-encoding headers, gzip and br for instance, the key would have
been built out of the gzip string. So another client that only managed
gzip would have been sent the cached resource, even if it was a br resource.
The http_find_header function is now called directly by the normalizers
so that they can manage multiple headers if needed.
A request that has more than 16 encodings will be considered as an
illegitimate request and its response will not be stored.

This fixes GitHub issue #987.

It does not need any backport.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
2b5c5cbef6 MINOR: cache: Avoid storing responses whose secondary key was not correctly calculated
If any of the secondary hash normalizing functions raises an error, the
secondary hash will be unusable. In this case, the response will not be
stored anymore.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
bba2912758 MINOR: cache: Refactoring of secondary_key building functions
The two secondary_key building functions (prebuild_full_key
and build_key) have roughly the same content so their code can
be mutualized.
2020-12-24 17:18:00 +01:00
Frdric Lcaille
f63921fc24 MINOR: quic: Add traces for quic_packet_encrypt().
Add traces to have an idea why this function may fail. In fact
in never fails when the passed parameters are correct, especially the
lengths. This is not the case when a packet is not correctly built
before being encrypted.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
133e8a7146 MINOR: quic: make a packet build fails when qc_build_frm() fails.
Even if the size of frames built by qc_build_frm() are computed so that
not to overflow a buffer, do not rely on this and always makes a packet
build fails if we could not build a frame.
Also add traces to have an idea where qc_build_frm() fails.
Fixes a memory leak in qc_build_phdshk_apkt().
2020-12-23 11:57:26 +01:00
Frdric Lcaille
f7e0b8d6ae MINOR: quic: Add traces for in flght ack-eliciting packet counter.
Add trace for this counter. Also shorten its variable name (->ifae_pkts).
2020-12-23 11:57:26 +01:00
Frdric Lcaille
b4e17386cb MINOR: quic: Update the initial salt to that of draft-29.
This salt is ued at leat up to draft-32. At this date ngtcp2 always
uses this salt even if it started the draft-33 development.

Note that when the salt is not correct, we cannot remove the header
protection. In this case the packet number length is wrong.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
47c433fdcb MINOR: quic: Display the SSL alert in ->ssl_send_alert() callback.
At least displays the SSL alert error code passed to ->ssl_send_alert()
QUIC BIO method and the SSL encryption level. This function is newly called
when using picoquic client with a recent version of BoringSSL (Nov 19 2020).
This is not the case with OpenSSL with 32 as QUIC draft implementation.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
26c49d9eb0 MINOR: quic: Add traces to congestion avoidance NewReno callback.
These traces are missing and are useful do diagnose issue in the congestion
avoidance callback for NewReno algorithm.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
0c14020f11 MINOR: quic: Code reordering to help in reviewing/modifying.
Reorder by increasing type the switch/case in qc_parse_pkt_frms()
which is the high level frame parser.
Add new STREAM_X frame types to support some tests with ngtcp2 client.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
f7fe9659f0 MINOR: quic: Flag RX packet as ack-eliciting from the generic parser.
Add ->flags to the QUIC frame parser as this has been done for the builder so
that to flag RX packets as ack-eliciting at low level. This should also be
helpful to maintain the code if we have to add new flags to RX packets.
Remove the statements which does the same thing as higher level in
qc_parse_pkt_frms().
2020-12-23 11:57:26 +01:00
Frdric Lcaille
04ffb66bc9 MINOR: quic: Make usage of the congestion control window.
Remove ->ifcdata which was there to control the CRYPTO data sent to the
peer so that not to saturate its reception buffer. This was a sort
of flow control.
Add ->prep_in_flight counter to the QUIC path struct to control the
number of bytes prepared to be sent so that not to saturare the
congestion control window. This counter is increased each time a
packet was built. This has nothing to see with ->in_flight which
is the real in flight number of bytes which have really been sent.
We are olbiged to maintain two such counters to know how many bytes
of data we can prepared before sending them.
Modify traces consequently which were useful to diagnose issues about
the congestion control window usage.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
c5e72b9868 MINOR: quic: Attempt to make trace more readable
As there is a lot of information in this protocol, this is not
easy to make the traces readable. We remove here a few of them and
shorten some line shortening the variable names.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
8090b51e92 MAJOR: quic: Make usage of ebtrees to store QUIC ACK ranges.
Store QUIC ACK ranges in ebtrees in place of lists with a 0(n) time complexity
for insertion.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
b83b0105e8 MINOR: quic: Enable the compilation of QUIC modules.
Adds the QUIC C object files to the list to compile to support QUIC protocol.
USE_QUIC compilation option must be used for that.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
0a76901926 MINOR: cfgparse: QUIC default server transport parameters init.
This patch is there to initialize the default transport parameters for QUIC
as a preparation for one of the QUIC next steps to come: fully support QUIC
protocol for haproxy servers.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
026a7921a5 MINOR: quic: QUIC socket management finalization.
Implement ->accept_conn() callback for QUIC listener sockets.
Note that this patch also implements quic_session_accept() function
which is similar to session_accept_fd() without calling conn_complete_session()
at this time because we do not have any real QUIC mux.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
e9473c7833 MINOR: ssl: QUIC transport parameters parsing.
This patch modifies the TLS ClientHello message callback so that to parse the QUIC
client transport parameters.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
ec216523f7 MINOR: ssl: SSL CTX initialization modifications for QUIC.
Makes TLS/TCP and QUIC share the same CTX initializer so that not to modify the
caller which is an XPRT callback used both by the QUIC xprt and the SSL xprt over
TCP.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
f46c10cfb1 MINOR: server: Add QUIC definitions to servers.
This patch adds QUIC structs to server struct so that to make the QUIC code
compile. Also initializes the ebtree to store the connections by connection
IDs.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
884f2e9f43 MINOR: listener: Add QUIC info to listeners and receivers.
This patch adds a quic_transport_params struct to bind_conf struct
used for the listeners. This is to store the QUIC transport parameters
for the listeners. Also initializes them when calling str2listener().
Before str2sa_range() it's too early to figure we're going to speak QUIC,
and after it's too late as listeners are already created. So it seems that
doing it in str2listener() when the protocol is discovered is the best
place.

Also adds two ebtrees to the underlying receivers to store the connection
by connections IDs (one for the original connection IDs, and another
one for the definitive connection IDs which really identify the connections.

However it doesn't seem normal that it is stored in the receiver nor the
listener. There should be a private context in the listener so that
protocols can store internal information. This element should in
fact be the listener handle.

Something still feels wrong, and probably we'll have to make QUIC and
SSL co-exist: a proof of this is that there's some explicit code in
bind_parse_ssl() to prevent the "ssl" keyword from replacing the xprt.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
a7e7ce957d MINOR: quic: Import C source code files for QUIC protocol.
This patch imports all the C files for QUIC protocol implementation with few
modifications from 20200720-quic branch of quic-dev repository found at
https://github.com/haproxytech/quic-dev.

Traces were implemented to help with the development.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
0c4e3b09b0 MINOR: quic: Add definitions for QUIC protocol.
This patch imports all the definitions for QUIC protocol with few modifications
from 20200720-quic branch of quic-dev repository found at
https://github.com/haproxytech/quic-dev.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
10caf65634 MINOR: tools: Add support for QUIC addresses parsing.
Add "quic4" and "quic6" keywords to str2sa_range() to parse QUIC IPv4
and IPv6 addresses respectively.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
e50afbd4e4 MINOR: cfgparse: Do not modify the QUIC xprt when parsing "ssl".
When parsing "ssl" keyword for TLS bindings, we must not use the same xprt as the one
for TLS/TCP connections. So, do not modify the QUIC xprt which will be initialized
when parsing QUIC addresses wich "ssl" bindings.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
901ee2f37b MINOR: ssl: Export definitions required by QUIC.
QUIC needs to initialize its BIO and SSL session the same way as for SSL over TCP
connections. It needs also to use the same ClientHello callback.
This patch only exports functions and variables shared between QUIC and SSL/TCP
connections.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
5e3d83a221 MINOR: connection: Add a new xprt to connection.
Simply adds XPRT_QUIC new enum to integrate QUIC transport protocol.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
5aa92411fb MINOR: ssl_sock: Initialize BIO and SSL objects outside of ssl_sock_init()
This patch extraces the code which initializes the BIO and SSL session
objects so that to reuse it elsewhere later for QUIC conections which
only needs SSL and BIO objects at th TLS layer stack level to work.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
70da889d57 MINOR: quic: Redefine control layer callbacks which are QUIC specific.
We add src/quic_sock.c QUIC specific socket management functions as callbacks
for the control layer: ->accept_conn, ->default_iocb and ->rx_listening.
accept_conn() will have to be defined. The default I/O handler only recvfrom()
the datagrams received. Furthermore, ->rx_listening callback always returns 1 at
this time but should returns 0 when reloading the processus.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
72f7cb170a MINOR: connection: Attach a "quic_conn" struct to "connection" struct.
This is a simple patch to prepare the integration of QUIC support to come.
quic_conn struct is supposed to embed any QUIC specific information for a QUIC
connection.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
ca42b2c9d3 MINOR: protocol: Create proto_quic QUIC protocol layer.
As QUIC is a connection oriented protocol, this file is almost a copy of
proto_tcp without TCP specific features. To suspend/resume a QUIC receiver
we proceed the same way as for proto_udp receivers.

With the recent updates to the listeners, we don't need a specific set of
quic*_add_listener() functions, the default ones are sufficient. The fields
declaration were reordered to make the various layers more visible like in
other protocols.

udp_suspend_receiver/udp_resume_receiver are up-to-date (the check for INHERITED
is present) and the code being UDP-specific, it's normal to use UDP here.
Note that in the future we might more reasily reference stacked layers so that
there's no more need for specifying the pointer here.
2020-12-23 11:57:26 +01:00
Dragan Dosen
9eea56009d REGTESTS: add tests for the xxh3 converter 2020-12-23 06:39:21 +01:00
Dragan Dosen
04bf0cc086 MINOR: sample: add the xxh3 converter
This patch adds support for the XXH3 variant of hash function that
generates a 64-bit hash.
2020-12-23 06:39:21 +01:00