5152 Commits

Author SHA1 Message Date
William Lallemand
722180aca8 BUILD: make tune.ssl.keylog available again
Since commit 04a5a44 ("BUILD: ssl: use HAVE_OPENSSL_KEYLOG instead of
OpenSSL versions") the "tune.ssl.keylog" feature is broken because
HAVE_OPENSSL_KEYLOG does not exist.

Replace this by a HAVE_SSL_KEYLOG which is defined in openssl-compat.h.
Also add an error when not built with the right openssl version.

Must be backported as far as 2.3.
2021-06-09 17:10:13 +02:00
Amaury Denoyelle
846830e47d BUG: errors: remove printf positional args for user messages context
Change the algorithm for the generation of the user messages context
prefix. Remove the dubious API relying on optional printf positional
arguments. This may be non portable, and in fact the CI glibc crashes
with the following error when some arguments are not present in the
format string :

"invalid %N$ use detected".

Now, a fixed buffer attached to the context instance is allocated once
for the program lifetime. Then call repeatedly snprintf with the
optional arguments of context if present to build the context string.
The buffer is deallocated via a per-thread free handler.

This does not need to be backported.
2021-06-08 11:40:44 +02:00
Maximilian Mader
fc0cceb08a MINOR: haproxy: Add -cc argument
This patch adds the `-cc` (check condition) argument to evaluate conditions on
startup and return the result as the exit code.

As an example this can be used to easily check HAProxy's version in scripts:

    haproxy -cc 'version_atleast(2.4)'

This resolves GitHub issue #1246.

Co-authored-by: Tim Duesterhus <tim@bastelstu.be>
2021-06-08 11:17:19 +02:00
Maximilian Mader
29c6cd7d8a CLEANUP: tools: Make errptr const in parse_line()
This change is for consistency with `cfg_eval_condition()`.
2021-06-08 10:56:10 +02:00
Amaury Denoyelle
6af81f80fb MEDIUM: errors: implement parsing context type
Create a parsing_ctx structure. This type is used to store information
about the current file/line parsed. A global context is created and
can be manipulated when haproxy is in STARTING mode. When starting is
over, the context is resetted and should not be accessed anymore.
2021-06-07 16:58:16 +02:00
Amaury Denoyelle
1833e43c3e MEDIUM: errors: implement user messages buffer
The user messages buffer is used to store the stderr output after the
starting is over. Each thread has it own user messages buffer. Add some
functions to add a new message, retrieve and clear the content.

The user messages buffer primary goal is to be consulted by CLI
handlers. Each handlers using it must clear the buffer before starting
its operation.
2021-06-07 16:58:16 +02:00
Amaury Denoyelle
ce986e1ce8 REORG: errors: split errors reporting function from log.c
Move functions related to errors output on stderr from log.c to a newly
created errors.c file. It targets print_message and
ha_alert/warning/notice/diag functions and related startup_logs feature.
2021-06-07 16:58:15 +02:00
Amaury Denoyelle
01b3c3d4fb MINOR: errors: allow empty va_args for diag variadic macro
Use the '##' operator to allow the usage of HA_DIAG_WARNING_COND macro
without extra arguments.

This must be backported up to 2.4.
2021-06-07 16:58:15 +02:00
Remi Tricot-Le Breton
476462010e BUG/MINOR: proxy: Missing calloc return value check in chash_init_server_tree
A memory allocation failure happening in chash_init_server_tree while
trying to allocate a server's lb_nodes item used in consistent hashing
would have resulted in a crash. This function is only called during
configuration parsing.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:55:51 +02:00
Remi Tricot-Le Breton
1f4fa906c7 BUG/MINOR: worker: Missing calloc return value check in mworker_env_to_proc_list
A memory allocation failure happening in mworker_env_to_proc_list when
trying to allocate a mworker_proc would have resulted in a crash. This
function is only called during init.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:51:06 +02:00
Remi Tricot-Le Breton
208ff01b23 BUG/MINOR: peers: Missing calloc return value check in peers_register_table
A memory allocation failure happening during peers_register_table would
have resulted in a crash. This function is only called during init.

It was raised in GitHub issue #1233.
It could be backported to all stable branches.
2021-05-31 10:50:46 +02:00
Remi Tricot-Le Breton
f1800e64ef BUG/MINOR: server: Missing calloc return value check in srv_parse_source
Two calloc calls were not checked in the srv_parse_source function.
Considering that this function could be called at runtime through a
dynamic server creation via the CLI, this could lead to an unfortunate
crash.

It was raised in GitHub issue #1233.
It could be backported to all stable branches even though the runtime
crash could only happen on branches where dynamic server creation is
possible.
2021-05-31 10:50:32 +02:00
Christopher Faulet
4fc51a73e6 MINOR: buf: Add function to realign a buffer with a specific head position
b_slow_realign() function may be used to realign a buffer with a given
amount of output data, eventually 0. In such case, the head is set to
0. This function is not designed to be used with input only buffers, like
those used in the muxes. It is the purpose of b_slow_realign_ofs()
function. It does almost the same, realign a buffer. But it do so by setting
the buffer head to a specific offset.
2021-05-25 10:41:50 +02:00
Christopher Faulet
de471a4a8d MINOR: h1-htx: Update h1 parsing functions to return result as a size_t
h1 parsing functions (h1_parse_msg_*) returns the number of bytes parsed or
0 if nothing is parsed because an error occurred or some data are
missing. But they never return negative values. Thus, instead of a signed
integer, these function now return a size_t value.

The H1 and FCGI muxes are updated accordingly. Note that h1_parse_msg_data()
has been slightly adapted because the parsing of chunked messages still need
to handle negative values when a parsing error is reported by
h1_parse_chunk_size() or h1_skip_chunk_crlf().
2021-05-25 10:41:50 +02:00
Dragan Dosen
3e6690a555 CLEANUP: pattern: remove export of non-existent function pattern_delete() 2021-05-25 08:44:48 +02:00
Dragan Dosen
a75eea78e2 MINOR: map/acl: print the count of all the map/acl entries in "show map/acl"
The output of "show map/acl" now contains the 'entry_cnt' value that
represents the count of all the entries for each map/acl, not just the
active ones, which means that it also includes entries currently being
added.
2021-05-25 08:44:45 +02:00
Remi Tricot-Le Breton
2608e348be BUG/MEDIUM: ebtree: Invalid read when looking for dup entry
The first item inserted into an ebtree will be inserted directly below
the root, which is a simple struct eb_root which only holds two branch
pointers (left and right).
If we try to find a duplicated entry to this first leaf through a
ebmb_next_dup, our leaf_p pointer will point to the eb_root instead of a
complete eb_node so we cannot look for the bit part of our leaf_p since
it would try to cast our eb_root into an eb_node and perform an out of
bounds access when reading "eb_root_to_node(eb_untag(t,EB_LEFT)))->bit".
This bug was found by address sanitizer running on a CRL hot update VTC
test.

Note that the bug has been there since the import of the eb_next_dup()
and eb_prev_dup() function in 1.5-dev19 by commit 2b5702030 ("MINOR:
ebtree: add new eb_next_dup/eb_prev_dup() functions to visit duplicates").

It can be backported to all stable branches.
2021-05-18 19:26:21 +02:00
Remi Tricot-Le Breton
18c7d83934 BUILD/MINOR: ssl: Fix compilation with OpenSSL 1.0.2
The following functions used in CA/CRL file hot update were not defined
in OpenSSL 1.0.2 so they need to be defined in openssl-compat :
- X509_CRL_get_signature_nid
- X509_CRL_get0_lastUpdate
- X509_CRL_get0_nextUpdate
- X509_REVOKED_get0_serialNumber
- X509_REVOKED_get0_revocationDate
2021-05-18 00:28:31 +02:00
Remi Tricot-Le Breton
a51b339d95 MEDIUM: ssl: Add "set+commit ssl crl-file" CLI commands
This patch adds the "set ssl crl-file" and "commit ssl crl-file"
commands, following the same logic as the certificate and CA file update
equivalents.
When trying to update a Certificate Revocation List (CRL) file via a
"set" command, we start by looking for the entry in the CA file tree and
then building a new cafile_entry out of the payload, without adding it
to the tree yet. It will only be added when a "commit" command is
called.
During a "commit" command, we insert the newly built cafile_entry in the
CA file tree while keeping the previous entry. We then iterate over all
the instances that used the CRL file and rebuild a new one and its
dedicated SSL context for every one of them.
When all the contexts are properly created, the old instances get
replaced by the new ones and the old CRL file is removed from the tree.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
0bb482436c MINOR: ssl: Add a cafile_entry type field
The CA files and CRL files are stored in the same cafile_tree so this
patch adds a new field the the cafile_entry structure that specifies the
type of the entry. Since a ca-file can also have some CRL sections, the
type will be based on the option used to load the file and not on its
content (ca-file vs crl-file options).
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
a32a68bd3b MEDIUM: ssl: Add "set+commit ssl ca-file" CLI commands
This patch adds the "set ssl ca-file" and "commit ssl ca-file" commands,
following the same logic as the certificate update equivalents.
When trying to update a ca-file entry via a "set" command, we start by
looking for the entry in the cafile_tree and then building a new
cafile_entry out of the given payload. This new object is not added to
the cafile_tree until "commit" is called.
During a "commit" command, we insert the newly built cafile_entry in the
cafile_tree, while keeping the previous entry as well. We then iterate
over all the instances linked in the old cafile_entry and rebuild a new
ckch instance for every one of them. The newly inserted cafile_entry is
used for all those new instances and their respective SSL contexts.
When all the contexts are properly created, the old instances get
replaced by the new ones and the old cafile_entry is removed from the
tree.

This fixes a subpart of GitHub issue #1057.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
38c999b11c MINOR: ssl: Add helper function to add cafile entries
Adds a way to insert a new uncommitted cafile_entry in the tree. This
entry will be the one fetched by any lookup in the tree unless the
oldest cafile_entry is explicitely looked for. This way, until a "commit
ssl ca-file" command is completed, there could be two cafile_entries
with the same path in the tree, the original one and the newly updated
one.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
383fb1472e MEDIUM: ssl: Add a way to load a ca-file content from memory
The updated CA content coming from the CLI during a ca-file update will
directly be in memory and not on disk so the way CAs are loaded in a
cafile_entry for now (via X509_STORE_load_locations calls) cannot be
used.
This patch adds a way to fill a cafile_entry directly from memory and to
load the contained certificate and CRL sections into an SSL store.
CRL sections are managed as well as certificates in order to mimic the
way CA files are processed when specified in an option. Indeed, when
parsing a CA file given through a ca-file or ca-verify-file option, we
iterate over the different sections in ssl_set_cert_crl_file and load
them regardless of their type. This ensures that a file that was
properly parsed when given as an option will also be accepted by the
CLI.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
5daff3c8ab MINOR: ssl: Add helper functions to create/delete cafile entries
Add ssl_store_create_cafile_entry and ssl_store_delete_cafile_entry
functions.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
40ddea8222 MINOR: ssl: Add reference to default ckch instance in bind_conf
In order for the link between the cafile_entry and the default ckch
instance to be built, we need to give a pointer to the instance during
the ssl_sock_prepare_ctx call.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
4458b9732d MEDIUM: ssl: Chain ckch instances in ca-file entries
Each ca-file entry of the tree will now hold a list of the ckch
instances that use it so that we can iterate over them when updating the
ca-file via a cli command. Since the link between the SSL contexts and
the CA file tree entries is only built during the ssl_sock_prepare_ctx
function, which are called after all the ckch instances are created, we
need to add a little post processing after each ssl_sock_prepare_ctx
that builds the link between the corresponding ckch instance and CA file
tree entries.
In order to manage the ca-file and ca-verify-file options, any ckch
instance can be linked to multiple CA file tree entries and any CA file
entry can link multiple ckch instances. This is done thanks to a
dedicated list of ckch_inst references stored in the CA file tree
entries over which we can iterate (during an update for instance). We
avoid having one of those instances go stale by keeping a list of
references to those references in the instances.
When deleting a ckch_inst, we can then remove all the ckch_inst_link
instances that reference it, and when deleting a cafile_entry, we
iterate over the list of ckch_inst reference and clear the corresponding
entry in their own list of ckch_inst_link references.
2021-05-17 10:50:24 +02:00
Remi Tricot-Le Breton
af8820a9a5 CLEANUP: ssl: Move ssl_store related code to ssl_ckch.c
This patch moves all the ssl_store related code to ssl_ckch.c since it
will mostly be used there once the CA file update CLI commands are all
implemented. It also makes the cafile_entry structure visible as well as
the cafile_tree.
2021-05-17 10:50:24 +02:00
Willy Tarreau
1f97306ecc [RELEASE] Released version 2.5-dev0
Released version 2.5-dev0 with the following main changes :
    - MINOR: version: it's development again
2021-05-14 09:36:37 +02:00
Willy Tarreau
1cb9fe7a75 MINOR: version: it's development again
this essentially reverts 46fb37c70cd43f39de533f13639c29855dfec91e.
2021-05-14 09:36:08 +02:00
Willy Tarreau
46fb37c70c MINOR: version: mention that it's LTS now.
The version will be maintained up to around Q2 2026. Let's
also update the INSTALL file to mention this.
2021-05-14 09:02:22 +02:00
Willy Tarreau
388fc25915 IMPORT: slz: use inttypes.h instead of stdint.h
stdint.h is not as portable as inttypes.h. It doesn't exist at least
on AIX 5.1 and Solaris 7, while inttypes.h is present there and does
include stdint.h on platforms supporting it.

This is equivalent to libslz upstream commit e36710a ("slz: use
inttypes.h instead of stdint.h")
2021-05-14 08:44:52 +02:00
Willy Tarreau
9e274280a4 IMPORT: slz: do not produce the crc32_fast table when CRC is natively supported
On ARM with native CRC support, no need to inflate the executable with
a 4kB CRC table, let's just drop it.

This is slz upstream commit d8715db20b2968d1f3012a734021c0978758f911.
2021-05-12 09:29:33 +02:00
Tim Duesterhus
dec1c36b3a MINOR: uri_normalizer: Add fragment-encode normalizer
This normalizer encodes '#' as '%23'.

See GitHub Issue #714.
2021-05-11 17:24:32 +02:00
Tim Duesterhus
c9e05ab2de MINOR: uri_normalizer: Add fragment-strip normalizer
This normalizer strips the URI's fragment component which should never be sent
to the server.

See GitHub Issue #714.
2021-05-11 17:23:46 +02:00
Willy Tarreau
da7f11bfb5 CLEANUP: pattern: remove the unused and dangerous pat_ref_reload()
This function was not used anymore after the atomic updates were
implemented in 2.3, and it must not be used given that it does not
yield and can easily make the process hang for tens of seconds on
large acls/maps. Let's remove it before someone uses it as an
example to implement something else!
2021-05-11 16:49:55 +02:00
Willy Tarreau
9bc457f0ea BUILD: compat: include malloc_np.h for USE_MEMORY_PROFILING on FreeBSD
This include is needed for malloc_usable_size(). It's also important to
think about disabling global pools.
2021-05-09 23:46:45 +02:00
Willy Tarreau
92fbbcc4c6 MINOR: cli: sort the output of the "help" keywords
It's still very difficult to find all commands starting with a given
keyword like "set", "show" etc. Let's sort the lines by usage message,
this is much more convenient.
2021-05-09 22:39:07 +02:00
Willy Tarreau
6b86d9e485 BUILD: errors: include stdarg in errors.h
It's needed for va_list as defined in ha_vdiag_warning().
2021-05-09 12:11:41 +02:00
Willy Tarreau
2a8a2f0223 BUILD: ssl: define HAVE_CRYPTO_memcmp() based on the library version
The build fails on versions older than 1.0.1d which is the first one
introducing CRYPTO_memcmp(), so let's have a define for this instead
of enabling it whenever USE_OPENSSL is set. One could also wonder why
we're relying on openssl for such a trivial thing, and a simple local
implementation could also allow to restore lexicographic ordering.
2021-05-09 12:10:36 +02:00
Willy Tarreau
714f34580e DOC: fix a few remainig cases of "Haproxy" and "HAproxy" in doc and comments
Some of the Lua doc and a few places still used "Haproxy" or "HAproxy".
There was even one "HA proxy". A few of them were in an example of VTest
output, indicating that VTest ought to be fixed as well. No big deal but
better address all the remaining ones so that these inconsistencies stop
spreading around.
2021-05-09 06:50:46 +02:00
Willy Tarreau
a219ec5cb2 BUILD: config: do not include proxy.h nor errors.h anymore in cfgparse.h
These ones induce a long dependency chain and are not needed anymore.
2021-05-08 20:35:39 +02:00
Willy Tarreau
32840b77a5 BUILD: connection: stop including listener-t.h
listener-t comes with openssl just due to the SSL_CTX type that is
declred as a typedef in openssl hence cannot be abstracted at this
level. However connection-t.h doen't need all that just to know that
bind_conf is a struct. Let's declare it with other external types
instead..
2021-05-08 20:27:08 +02:00
Willy Tarreau
08138612a4 REORG: config: uninline warnifnotcap() and failifnotcap()
These ones are used by virtually every config parser. Not only they
provide no benefit in being inlined, but they imply a very deep
dependency starting at proxy.h, which results for example in task.c
including openssl.

Let's move these two functions to cfgparse.c.
2021-05-08 20:27:08 +02:00
Willy Tarreau
6ec1f25bc5 REORG: stick-table: move composite address functions to stick_table.h
These caddr_* functions were once placed into tools.h in the hope they
would be useful but nobody knows they exist. They could deserve being
moved to their own file with other pointer manipulation functions maybe,
but for now they're the only reason left for stick_table.h to include
tools.h, so let's move them directly there since it's its only user.
This allows to remove tools.h from stick_table.h and slightly reduce
the overall build time.
2021-05-08 20:24:09 +02:00
Willy Tarreau
3b63ca20f4 REORG: stick-table: uninline stktable_alloc_data_type()
This function has no business being inlined in stick_table.h since it's
only used at boot time by the config parser. In addition it causes an
undesired dependency on tools.h because it uses parse_time_err(). Let's
move it to stick_table.c.
2021-05-08 20:24:09 +02:00
Willy Tarreau
e59b5169b3 BUILD: connection: move list_mux_proto() to connection.c
No idea why this was put inlined into connection.h, it's used only once
for haproxy -vv, and requires tools.h, causing an undesired dependency
from connection.h. Let's move it to connection.c instead where it ought
to have been.
2021-05-08 20:24:09 +02:00
Willy Tarreau
5703a38a06 BUILD: stick-table: include freq_ctr.h from stick_table.h
It's needed for update_freq_ctr_period() which is used there.
2021-05-08 19:37:41 +02:00
Willy Tarreau
15f9ac3c59 REORG: mworker: move proc_self from global to mworker
Only mworker uses proc_self, and it was declared in global.h, forcing
users of global.h to include mworker and its dependencies.

Moving it to mworker reduces the preprocessed size of version.c from
170 to 125kB by shrinking the number of local includes from 30 to 16
and the number of system includes from 147 to 132.
2021-05-08 12:34:44 +02:00
Willy Tarreau
29c460bc07 REORG: threads: move all_thread_mask() to thread.h
It was declared in global.h, forcing plenty of source files to include
it only for this while it's only based on definitions from thread.h.
2021-05-08 12:26:10 +02:00
Willy Tarreau
cfc4f24d80 REORG: vars: move the "proc" scope variables out of the global struct
The presence of this field causes a long dependency chain because almost
everyone includes global-t.h, and vars include sample_data which include
some system includes as well as HTTP parts.

There is absolutely no reason for having the process-wide variables in
the global struct, let's just move them into vars.c and vars.h. This
reduces from ~190k to ~170k the preprocessed output of version.c.
2021-05-08 12:11:29 +02:00