Commit Graph

13511 Commits

Author SHA1 Message Date
Ilya Shipitsin
1e9a66603f CLEANUP: assorted typo fixes in the code and comments
This is 14th iteration of typo fixes
2021-01-06 16:26:50 +01:00
Willy Tarreau
96d5368202 SCRIPTS: make announce release support preparing announces before tag exists
It takes so much time to write an announce message that sometimes it's
annoying not being able to start the work while a fix is being finished.
With the new "-p" argument, announce-release will allow to prepare the
announce message for the current HEAD and with no tag yet. It will
restart from the last tag and automatically increment the version using
the same algorithm as create-release so that everything is accurate. It
should then be easier at the last moment to just include the final entry
by hand when the last fix finally arrives. For convenience, this argument
also allows to create an announce from another branch than master.
2021-01-06 15:49:44 +01:00
Willy Tarreau
0c612936b2 SCRIPTS: improve announce-release to support different tag and versions
By having three variables it will be easier to preset the version and
the tag separately. One contains the announced version, another one the
associated tag and the last one the final commit ID (used as the ending
point before the release). This initially allows to check for the HEAD
matching the tag only when the version was not forced, hence re-announce
already tagged versions after some extra commits were added for example.
2021-01-06 15:45:15 +01:00
Frdric Lcaille
153d4a89d0 BUG/MINOR: quic: NULL pointer dereferences when building post handshake frames.
The second one was detected by cppcheck contrary to the first one.
Fixes issue #1032.
Thank you to Ilya for having reported this.
2021-01-06 13:59:05 +01:00
Christopher Faulet
de79cd28ec BUG/MINOR: stats: Make stat_l variable used to dump a stat line thread local
Since ee63d4bd6 ("MEDIUM: stats: integrate static proxies stats in new
stats"), all dumped stats for a given domain, the default ones and the
modules ones, are merged in a signle array to dump them in a generic way.
For this purpose, the stat_l global variable is allocated at startup to
store a line of stats before the dump, i.e. all stats of an entity
(frontend, backend, listener, server or dns nameserver). But this variable
is not thread safe.  If stats are retrieved concurrently by several clients
on different threads, the same variable is used. This leads to corrupted
stats output.

To fix the bug, the stat_l variable is now thread local.

This patch should probably solve issues #972 and #992. It must be backported
to 2.3.
2021-01-06 10:34:12 +01:00
Willy Tarreau
fae9a4cd2c REGTESTS: add unresolvable servers to srvkey-addr
This ensures that the previous fix remains operational, as without it
an unresolvable server causes a crash when it's been indexed.
2021-01-06 09:20:22 +01:00
Thayne McCombs
24da7e1aa6 BUG/MEDIUM: server: srv_set_addr_desc() crashes when a server has no address
GitHub Issue #1026 reported a crash during configuration check for the
following example config:

    backend 0
    server 0 0
    server 0 0

HAProxy crashed in srv_set_addr_desc() due to a NULL pointer dereference
caused by `sa2str` returning NULL for an `AF_UNSPEC` address (`0`).

Check to make sure the address key is non-null before using it for
comparison or inserting it into the tree.

The crash was introduced in commit 92149f9a8 ("MEDIUM: stick-tables: Add
srvkey option to stick-table") which not in any released version so no
backport is needed.

Cc: Tim Duesterhus <tim@bastelstu.be>
2021-01-06 09:19:15 +01:00
Tim Duesterhus
1ab76dd956 DOC: Improve the message printed when running make w/o TARGET
Rephrase the message to no longer talk about something that "is no longer
supported", but about what actually *is* supported.

Adjustments include:

- Removal of rare targets to make it easier to find the proper one.
- Reformatting to be easier to read (more newlines)
- Explanation of common non-default feature flags.
2021-01-05 19:32:48 +01:00
Ilya Shipitsin
ce94a948fb CI: github actions: build several popular "contrib" tools
this adds "halog", "flags" and "poll" builds. builds are done in
separate steps for better failure identification
2021-01-05 17:44:00 +01:00
Christopher Faulet
8d4977ae86 BUG/MINOR: tcpcheck: Report a L7OK if the last evaluated rule is a send rule
When all rules of a tcpcheck ruleset are successfully evaluated, the right
check status must always be reported. It is true if the last evaluated rule
is an expect or a connect rule. But not if it is a send rule. In this
situation, nothing more is done until the check timeout expiration and a
L7TOUT is reported instead of a L7OK.

Now, by default, when all rules were successfully evaluated, a L7OK is
reported. When the last evaluated rule is an expect or a connect, the
behavior remains unchanged.

This patch should fix the issue #1027. It must be backported as far as 2.2.
2021-01-05 17:31:49 +01:00
William Dauchy
afb9368221 CLEANUP: spoe: fix typo on var_check_arg comment
there was an extra `s` added to the `var_check_arg` function

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-05 17:23:32 +01:00
Tim Duesterhus
c294284e33 CLEANUP: Reduce scope of hdr_age in http_action_store_cache()
This is only required to process the `age` header.
2021-01-05 17:05:58 +01:00
Tim Duesterhus
e2fff10a19 CLEANUP: Reduce scope of header_name in http_action_store_cache()
This variable is only needed deeply nested in a single location and clang's
static analyzer complains about a dead initialization. Reduce the scope to
satisfy clang and the human that reads the function.
2021-01-05 17:05:58 +01:00
Willy Tarreau
8f7efcddd6 CLEANUP: mworker: remove duplicate pointer tests in cfg_parse_program()
As reported in issue #1017, there are two harmless duplicate tests in
cfg_parse_program(), one made of a "if" using the same condition as the
loop it's in, and the other one being a null test before a free. This
just removes them. No backport is needed.
2021-01-05 15:58:37 +01:00
Tim Duesterhus
5ce5a1586d BUG/MINOR: cfgparse: Fail if the strdup() for rule->be.name for use_backend fails
This patch fixes GitHub issue #1024.

I could track the `strdup` back to commit
3a1f5fda10 which is 1.9-dev8. It's probably not
worth the effort to backport it across this refactoring.

This patch should be backported to 1.9+.
2021-01-05 11:37:41 +01:00
Willy Tarreau
782e9be1b3 Revert "BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11"
This reverts commit 5e8c35da1b.

While the issue is being discussed with gcc folks, a reasonable workaround
could be found for the tls_keys_ref list usage which doesn't significantly
complicate the code. Since it was the only place triggering the warning and
I don't feel very comfortable leaving this one disabled for too long, let's
re-enable it right now. This definitely closes issue #1010.
2021-01-05 11:17:28 +01:00
Willy Tarreau
b6fc524f05 MINOR: ssl: make tlskeys_list_get_next() take a list element
As reported in issue #1010, gcc-11 as of 2021-01-05 is overzealous in
its -Warray-bounds check as it considers that a cast of a global struct
accesses the entire struct even if only one specific element is accessed.
This instantly breaks all lists making use of container_of() to build
their iterators as soon as the starting point is known if the next
element is retrieved from the list head in a way that is visible to the
compiler's optimizer, because it decides that accessing the list's next
element dereferences the list as a larger struct (which it does not).

The temporary workaround consisted in disabling -Warray-bounds, but this
warning is traditionally quite effective at spotting real bugs, and we
actually have is a single occurrence of this issue in the whole code.

By changing the tlskeys_list_get_next() function to take a list element
as the starting point instead of the current element, we can avoid
the starting point issue but this requires to change all call places
to write hideous casts made of &((struct blah*)ref)->list. At the
moment we only have two such call places, the first one being used to
initialize the list (which is the one causing the warning) and which
is thus easy to simplify, and the second one for which we already have
an aliased pointer to the reference that is still valid at the call
place, and given the original pointer also remained unchanged, we can
safely use this alias, and this is safer than leaving a cast there.

Let's make this change now while it's still easy.

The generated code only changed in function cli_io_handler_tlskeys_files()
due to register allocation and the change of variable scope between the
old one and the new one.
2021-01-05 11:15:45 +01:00
Tim Duesterhus
cb8b281c02 CLEANUP: ssl: Remove useless local variable in tlskeys_list_get_next()
`getnext` was only used to fill `ref` at the beginning of the function. Both
have the same type. Replace the parameter name by `ref` to remove the useless
local variable.
2021-01-05 10:25:20 +01:00
Tim Duesterhus
2c7bb33144 CLEANUP: ssl: Remove useless loop in tlskeys_list_get_next()
This loop was always exited in the first iteration by `return`.
2021-01-05 10:24:36 +01:00
Frdric Lcaille
242fb1b639 MINOR: quic: Drop packets with STREAM frames with wrong direction.
A server initiates streams with odd-numbered stream IDs.
Also add useful traces when parsing STREAM frames.
2021-01-04 12:31:28 +01:00
Frdric Lcaille
129a351a3f BUG/MINOR: quic: Wrong STREAM frames parsing.
After having re-read the RFC, we noticed there are two bugs in the STREAM
frame parser. When the OFF bit (0x04) in the frame type is not set
we must set the offset to 0 (it was not set at all). When the LEN bit (0x02)
is not set we must extend the length of the data field to the end of the packet
(it was not set at all).
2021-01-04 12:31:28 +01:00
Frdric Lcaille
50044adc60 MINOR: quic: Pass quic_conn struct to frame parsers.
This is only for debugging purposes.
2021-01-04 12:31:28 +01:00
Frdric Lcaille
ea60499912 BUG/MINOR: quic: Possible CRYPTO frame building errors.
This is issue is due to the fact that when we call the function
responsible of building CRYPTO frames to fill a buffer, the Length
field of this packet did not take into an account the trailing 16 bytes for
the AEAD tag. Furthermore, the remaining <room> available in this buffer
was not decremented by the CRYPTO frame length, but only by the CRYPTO data length
of this frame.
2021-01-04 12:31:28 +01:00
Frdric Lcaille
6c1e36ce55 CLEANUP: quic: Remove useless QUIC event trace definitions.
Remove QUIC_EV_CONN_E* event trace macros which were defined for  errors.
Replace QUIC_EV_CONN_ECHPKT by QUIC_EV_CONN_BCFRMS used in qc_build_cfrms()
2021-01-04 12:31:28 +01:00
Frdric Lcaille
d341fc3609 CLEANUP: qpack: Wrong comment about the draft for QPACK static header table.
This came with a "copy and paste" from the definition for HPACK.
2021-01-04 12:31:28 +01:00
Frdric Lcaille
164096eb76 MINOR: qpack: Add static header table definitions for QPACK.
As HPACK, QPACK makes usage of a static header table.
2021-01-04 12:31:28 +01:00
Tim Duesterhus
8cb12a8292 CLEANUP: Apply the coccinelle patch for XXXcmp() on contrib/
Compare the various `cmp()` functions against zero.
2021-01-04 10:09:02 +01:00
Tim Duesterhus
54182ec9d7 CLEANUP: Apply the coccinelle patch for XXXcmp() on include/
Compare the various `XXXcmp()` functions against zero.
2021-01-04 10:09:02 +01:00
Tim Duesterhus
e5ff14100a CLEANUP: Compare the return value of XXXcmp() functions with zero
According to coding-style.txt it is recommended to use:

`strcmp(a, b) == 0` instead of `!strcmp(a, b)`

So let's do this.

The change was performed by running the following (very long) coccinelle patch
on src/:

    @@
    statement S;
    expression E;
    expression F;
    @@

      if (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
      )
    (
      S
    |
      { ... }
    )

    @@
    statement S;
    expression E;
    expression F;
    @@

      if (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
      )
    (
      S
    |
      { ... }
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G &&
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G ||
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    && G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) != 0
    || G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G &&
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    G ||
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    && G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    || G
    )

    @@
    expression E;
    expression F;
    expression G;
    @@

    (
    - !
    (
    dns_hostname_cmp
    |
    eb_memcmp
    |
    memcmp
    |
    strcasecmp
    |
    strcmp
    |
    strncasecmp
    |
    strncmp
    )
    -  (E, F)
    +  (E, F) == 0
    )
2021-01-04 10:09:02 +01:00
Tim Duesterhus
f89d43a381 MINOR: lua: Use consistent error message 'memory allocation failed'
Other locations in the configuration parser use 'memory allocation failed', so
use this one as well.
2021-01-03 20:37:16 +01:00
Tim Duesterhus
621e74afd1 BUG/MINOR: lua: Fix memory leak error cases in hlua_config_prepend_path
In case of an error `p` is not properly freed.

Minor leak during configuration parsing in out of memory situations, no
backport needed.
2021-01-03 20:37:16 +01:00
Tim Duesterhus
a7ebffef66 BUG/MINOR: sink: Return an allocation failure in __sink_new if strdup() fails
This patch fixes GitHub issue #1023.

The function was introduced in commit 99c453d ("MEDIUM: ring: new
section ring to declare custom ring buffers."), which first appeared
in 2.2-dev9. The fix should be backported to 2.2+.
2021-01-03 20:35:45 +01:00
Willy Tarreau
5e8c35da1b BUILD: Makefile: disable -Warray-bounds until it's fixed in gcc 11
Ilya reported in issue #1010 that gcc 11 (still in development phase 3)
sees out-of-bounds accesses to tlskeys_reference. After a deep analysis
it turns out that the warning is erroneous and Tim could set up a
reproducer that was further simplified to file a gcc bug report (98503).
In order not to break builds on this version, let's temporarily disable
the warning, since no acceptable workaround could be defined. Once gcc
gets it right, the warning should be re-enabled as it is likely to catch
real errors (it's the first false positive here for us even though
roughly one hundred issues were reported around this one alone).
2021-01-03 20:27:13 +01:00
Thayne McCombs
23cc52d34b REGTESTS: add test for stickiness using "srvkey addr"
This tests the new "srvkey addr" stick-table statement, which sticks based
on the server's address.
2020-12-31 10:08:54 +01:00
Thayne McCombs
92149f9a82 MEDIUM: stick-tables: Add srvkey option to stick-table
This allows using the address of the server rather than the name of the
server for keeping track of servers in a backend for stickiness.

The peers code was also extended to support feeding the dictionary using
this key instead of the name.

Fixes #814
2020-12-31 10:04:54 +01:00
Tim Duesterhus
dc38bc4a1a BUG/MEDIUM: cache: Fix hash collision in accept-encoding handling for Vary
This patch fixes GitHub Issue #988. Commit ce9e7b2521
was not sufficient, because it fell back to a hash comparison if the bitmap
of known encodings was not acceptable instead of directly returning the the
cached response is not compatible.

This patch also extends the reg-test to test the hash collision that was
mentioned in #988.

Vary handling is 2.4, no backport needed.
2020-12-31 09:39:08 +01:00
Ilya Shipitsin
64b6f36788 CI: GitHub Actions: enable daily Coverity scan
That scan was previously implemented on Travis. Let us migrate
it to GitHub Actions.

Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
2020-12-28 12:02:09 +01:00
Remi Tricot-Le Breton
e6cc5b5974 MINOR: cache: Replace the "process-vary" option's expected values
Replace the <0/1> expected values of the process-vary option by a more
usual <on/off> pair.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
42efffd7f6 MINOR: cache: Remove redundant test in http_action_req_cache_use
The suppressed check is fully covered by the next one and can then be
removed.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
14df1e1526 REGTESTS: cache: Add a specific test for the accept-encoding normalizer
Those tests check the explicit support of a subset of encodings added to
the accept-encoding normalizer.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
b054b6dec6 REGTESTS: cache: Simplify vary.vtc file
This test used chunked responses but it is not needed. All the chunked
responses are replaced by more simple 'bodylen' directives.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
ce9e7b2521 MEDIUM: cache: Manage a subset of encodings in accept-encoding normalizer
The accept-encoding normalizer now explicitely manages a subset of
encodings which will all have their own bit in the encoding bitmap
stored in the cache entry. This way two requests with the same primary
key will be served the same cache entry if they both explicitely accept
the stored response's encoding, even if their respective secondary keys
are not the same and do not match the stored response's one.
The actual hash of the accept-encoding will still be used if the
response's encoding is unmanaged.
The encoding matching and the encoding weight parsing are done for every
subpart of the accept-encoding values, and a bitmap of accepted
encodings is built for every request. It is then tested upon any stored
response that has the same primary key until one with an accepted
encoding is found.
The specific "identity" and "*" accept-encoding values are managed too.
When storing a response in the key, we also parse the content-encoding
header in order to only set the response's corresponding encoding's bit
in its cache_entry encoding bitmap.

This patch fixes GitHub issue #988.
It does not need to be backported.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
56e46cb393 MINOR: http: Add helper functions to trim spaces and tabs
Add two helper functions that trim leading or trailing spaces and
horizontal tabs from an ist string.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
6a34b2b65d MINOR: cache: Add specific secondary key comparison mechanism
Add the possibility to define custom comparison functions for every
sub-part of the secondary key hash instead of using a global memcmp.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
e4421dec7e BUG/MINOR: cache: Manage multiple headers in accept-encoding normalization
The accept-encoding part of the secondary key (vary) was only built out
of the first occurrence of the header. So if a client had two
accept-encoding headers, gzip and br for instance, the key would have
been built out of the gzip string. So another client that only managed
gzip would have been sent the cached resource, even if it was a br resource.
The http_find_header function is now called directly by the normalizers
so that they can manage multiple headers if needed.
A request that has more than 16 encodings will be considered as an
illegitimate request and its response will not be stored.

This fixes GitHub issue #987.

It does not need any backport.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
2b5c5cbef6 MINOR: cache: Avoid storing responses whose secondary key was not correctly calculated
If any of the secondary hash normalizing functions raises an error, the
secondary hash will be unusable. In this case, the response will not be
stored anymore.
2020-12-24 17:18:00 +01:00
Remi Tricot-Le Breton
bba2912758 MINOR: cache: Refactoring of secondary_key building functions
The two secondary_key building functions (prebuild_full_key
and build_key) have roughly the same content so their code can
be mutualized.
2020-12-24 17:18:00 +01:00
Frdric Lcaille
f63921fc24 MINOR: quic: Add traces for quic_packet_encrypt().
Add traces to have an idea why this function may fail. In fact
in never fails when the passed parameters are correct, especially the
lengths. This is not the case when a packet is not correctly built
before being encrypted.
2020-12-23 11:57:26 +01:00
Frdric Lcaille
133e8a7146 MINOR: quic: make a packet build fails when qc_build_frm() fails.
Even if the size of frames built by qc_build_frm() are computed so that
not to overflow a buffer, do not rely on this and always makes a packet
build fails if we could not build a frame.
Also add traces to have an idea where qc_build_frm() fails.
Fixes a memory leak in qc_build_phdshk_apkt().
2020-12-23 11:57:26 +01:00
Frdric Lcaille
f7e0b8d6ae MINOR: quic: Add traces for in flght ack-eliciting packet counter.
Add trace for this counter. Also shorten its variable name (->ifae_pkts).
2020-12-23 11:57:26 +01:00