2026 Commits

Author SHA1 Message Date
Willy Tarreau
84c4e7451e DOC: peers: fix the protocol tag name in the doc
The peers protocol has been using "HAProxyS" as a binary tag sent on the
wire since day one in 1.5-dev3 with commit 2b920a1af ("[MAJOR] Add new
files src/peer.c, include/proto/peers.h and include/types/peers.h for
sync stick table management"), regardless, the doc says the protocol
identifier is "HaproxyS". It is likely this got fixed in the code before
merging and not in the doc.

This should be backported to any release as the doc is wrong.
2021-05-09 06:38:07 +02:00
Willy Tarreau
58000fe16d DOC: remove last occurrences of "HA-Proxy" syntax
There were only a few more used as output examples and comments in a few
docs, it was the right moment to get rid of them. The file intro.txt
which explains how to parse the version also got a hint about the possible
presence of a hyphen in the name in older versions.
2021-05-09 06:29:40 +02:00
Daniel Corbett
9f0843f4e2 DOC: Fix a few grammar/spelling issues and casing of HAProxy
This patch fixes a few grammar and spelling issues in configuration.txt.
It was also noted that there was a wide range of case usage
(i.e. haproxy, HAproxy, HAProxy, etc... ). This patch updates them
all to be consistently "HAProxy" except where a binary is mentioned.
2021-05-09 05:59:49 +02:00
Willy Tarreau
2745620240 MINOR: stats: support an optional "float" option to "show info"
This will allow some fields to be produced with a higher accuracy when
the requester indicates being able to parse floats. Rates and times are
among the elements which can make sense.
2021-05-08 10:52:12 +02:00
Willy Tarreau
589722ed2c DOC: management: mention that some fields may be emitted as floats
Floats in stats output are currently not emitted but are technically
possible, so let's mention that.
2021-05-08 07:48:07 +02:00
Remi Tricot-Le Breton
02bd68431b DOC: ssl: Add information about crl-file option
When using the crl-file option with multiple Certificate Authority
levels in the CA chain, there must be one CRL per CA or the verify
function on the backend side will raise an "unagle to get certificate
CRL" error (error code 3).

This was required by GitHub issue #1201.
2021-05-07 18:31:11 +02:00
Remi Tricot-Le Breton
7c980dffad DOC: ssl: Extra files loading now works for backends too
When implementing the server side certificate hot update, the ckch
mechanism was used on the backend side in order to mimic the frontend
certificate management and to enable server line certificate update via
the CLI (see GitHub issue #427). As an unexpected side effect, we now
also look for ssl extra files (cert.pem.key, cert.pem.ocsp ...) for the
backend side.
This patch updates the documentation accordingly.

This answers to GitHub issue #845.
2021-05-07 15:37:12 +02:00
Amaury Denoyelle
a9e639afe2 MINOR: http_act: mark normalize-uri as experimental
normalize-uri http rule is marked as experimental, so it cannot be
activated without the global 'expose-experimental-directives'. The
associated vtc is updated to be able to use it.
2021-05-07 14:35:02 +02:00
Amaury Denoyelle
d2e53cd47e MINOR: cfgparse: implement experimental config keywords
Add a new flag to mark a keyword as experimental. An experimental
keyword cannot be used if the global 'expose-experimental-directives' is
not present first.

Only keywords parsed through a standard cfg_keywords lists in
global/proxies section will be automatically detected if declared
experimental. To support a keyword outside of these lists,
check_kw_experimental must be called manually during its parsing.

If an experimental keyword is present in the config, the tainted flag is
updated.

For the moment, no keyword is marked as experimental.
2021-05-07 14:34:41 +02:00
Willy Tarreau
7190b987ab MINOR: config: add a new message directive: .diag
This one works just like .notice/.warning/.alert except that it prints
the message at level "DIAG" only when haproxy runs in diagnostic mode
(-dD). This can be convenient for example to pass a few hints to help
locate certain config parts or to leave messages about certain temporary
workarounds.

Example:

  .diag "WTA/2021-05-07: $.LINE: replace 'redirect' with 'return' after final switch to 2.4"
         http-request redirect location /goaway if ABUSE
2021-05-07 09:06:40 +02:00
Willy Tarreau
9f903af510 MEDIUM: log: slightly refine the output format of alerts/warnings/etc
For about 20 years we've been emitting cryptic messages on warnings and
alerts, that nobody knows how to parse:

  [NOTICE] 126/080118 (3115) : haproxy version is 2.4-dev18-0b7c78-49
  [NOTICE] 126/080118 (3115) : path to executable is ./haproxy
  [WARNING] 126/080119 (3115) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
  [ALERT] 126/080119 (3115) : backend 'default' has no server available!

Hint: the first 3-digit number is the day of year, and the 6 digits
after it represent the time of day in format HHMMSS, then the pid in
parenthesis. These are not quite user-friendly and such cryptic into
are not useful at all.

This patch slightly adjusts the output by performing these minimal changes:
  - removing the date/time, as they were added very early when haproxy
    was meant to be used in foreground as a debugging tool, and they're
    provided in more details in logs nowadays ;

  - better aligning the fields by padding the severity tag to 10 chars.
    The diag output was renamed to "DIAG" only.

Now the output provides this:

  [NOTICE]   (4563) : haproxy version is 2.4-dev18-75a428-51
  [NOTICE]   (4563) : path to executable is ./haproxy
  [WARNING]  (4563) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
  [ALERT]    (4563) : backend 'default' has no server available!

The useless space before the colon was kept so as not to confuse any
possible output parser.

The few entries in the doc referring to this format were adjusted to
reflect the new one.

The change was tagged "MEDIUM" as it may have visible consequences on
home-grown monitoring tools, though it is extremely unlikely due to the
limited extent of these changes.
2021-05-07 08:55:11 +02:00
Willy Tarreau
0b7c78aa05 MINOR: config: add predicates "version_atleast" and "version_before" to cond blocks
These predicates respectively verify that the current version is at least
a given version or is before a specific one. The syntax is exactly the one
reported by "haproxy -v", though each component is optional, so both "1.5"
and "2.4-dev18-88910-48" are supported. Missing components equal zero, and
"dev" is below "pre" or "rc", which are both inferior to no such mention
(i.e. they are negative). Thus "2.4-dev18" is older than "2.4-rc1" which
is older than "2.4".
2021-05-06 17:04:45 +02:00
Willy Tarreau
58ca706e16 MINOR: config: add predicate "feature" to detect certain built-in features
The "feature(name)" predicate will return true if <name> corresponds to
a name listed after a '+' in the features list, that is it was enabled at
build time with USE_<name>=1. Typical use cases will include OPENSSL, LUA
and LINUX_SPLICE. But maybe it will also be convenient to use with optional
addons such as PROMEX and the device detection modules to help keeping the
same configs across various deployments.
2021-05-06 17:02:36 +02:00
Willy Tarreau
6492e87b0e MINOR: config: add predicates "streq()" and "strneq()" to conditional expressions
"streq(str1,str2)" will return true if the two strings match while
"strneq(str1,str2)" will return true only if they differ. This is
convenient to match an environment variable against a predefined value.
2021-05-06 17:02:36 +02:00
Willy Tarreau
42ed14b529 MINOR: config: add predicate "defined()" to conditional expression blocks
"defined(name)" will return true if <name> is a defined environment variable
otherwise false, regardless of its contents.
2021-05-06 17:02:36 +02:00
Willy Tarreau
a46f1af2b1 MINOR: config: support some pseudo-variables for file/line/section
The new pseudo-variables ".FILE", ".LINE" and ".SECTION" will be resolved
on the fly by the config parser and will respectively retrieve the current
configuration file name, the current line number and the current section
being parsed. This may help emit logs, errors, and debugging information
(e.g. which rule matched).

The '.' in the first char was reserved for such pseudo-variables and no
other variable is permitted. This will allow to add support for new ones
in the future if they prove to be useful (e.g. randoms/uuid for secret
keying or automatic naming of configuration objects).
2021-05-06 10:36:38 +02:00
Willy Tarreau
ca3afc2456 MINOR: activity: add the profiling.memory global setting
This allows to enable/disable memory usage profiling very early, which
can be convenient to trace the memory usage in maps, certificates, Lua
etc.
2021-05-05 19:09:19 +02:00
Willy Tarreau
42712cb6d4 MINOR: activity: make "show profiling" support a few arguments
These ones allow to limit the output to only certain sections and/or
a number of lines per dump.
2021-05-05 19:09:19 +02:00
Willy Tarreau
00dd44f67f MINOR: activity: add a "memory" entry to "profiling"
This adds the necessary flags to permit run-time enabling/disabling of
memory profiling. For now this is disabled.

A few words were added to the management doc about it and recalling that
this is limited to certain OSes.
2021-05-05 18:55:02 +02:00
Willy Tarreau
080347fe2a [RELEASE] Released version 2.4-dev18
Released version 2.4-dev18 with the following main changes :
    - DOC: Fix indentation for `path-strip-dot` normalizer
    - DOC: Fix RFC reference for the percent-to-uppercase normalizer
    - DOC: Add RFC references for the path-strip-dot(dot)? normalizers
    - MINOR: uri_normalizer: Add a `percent-decode-unreserved` normalizer
    - BUG/MINOR: mux-fcgi: Don't send normalized uri to FCGI application
    - REORG: htx: Inline htx functions to add HTX blocks in a message
    - CLEANUP: assorted typo fixes in the code and comments
    - DOC: general: fix white spaces for HTML converter
    - BUG/MINOR: ssl: ssl_sock_prepare_ssl_ctx does not return an error code
    - BUG/MINOR: cpuset: move include guard at the very beginning
    - BUG/MAJOR: fix build on musl with cpu_set_t support
    - BUG/MEDIUM: cpuset: fix build on MacOS
    - BUG/MINOR: htx: Preserve HTX flags when draining data from an HTX message
    - MEDIUM: htx: Refactor htx_xfer_blks() to not rely on hdrs_bytes field
    - CLEANUP: htx: Remove unsued hdrs_bytes field from the HTX start-line
    - BUG/MINOR: mux-h2: Don't encroach on the reserve when decoding headers
    - MEDIUM: http-ana: handle read error on server side if waiting for response
    - MINOR: htx: Limit length of headers name/value when a HTX message is dumped
    - BUG/MINOR: applet: Notify the other side if data were consumed by an applet
    - BUG/MINOR: hlua: Don't consume headers when starting an HTTP lua service
    - BUG/MEDIUM: mux-h2: Handle EOM flag when sending a DATA frame with zero-copy
    - CLEANUP: channel: No longer notify the producer in co_skip()/co_htx_skip()
    - DOC: general: fix example in set-timeout
    - CLEANUP: cfgparse: de-uglify early file error handling in readcfgfile()
    - MINOR: config: add a new "default-path" global directive
    - BUG/MEDIUM: peers: initialize resync timer to get an initial full resync
    - BUG/MEDIUM: peers: register last acked value as origin receiving a resync req
    - BUG/MEDIUM: peers: stop considering ack messages teaching a full resync
    - BUG/MEDIUM: peers: reset starting point if peers appears longly disconnected
    - BUG/MEDIUM: peers: reset commitupdate value in new conns
    - BUG/MEDIUM: peers: re-work updates lookup during the sync on the fly
    - BUG/MEDIUM: peers: reset tables stage flags stages on new conns
    - MINOR: peers: add informative flags about resync process for debugging
    - BUG/MEDIUM: time: fix updating of global_now upon clock drift
    - CLEANUP: freq_ctr: make arguments of freq_ctr_total() const
    - CLEANUP: hlua: rename hlua_appctx* appctx to luactx
    - MINOR: server: fix doc/trace on lb algo for dynamic server creation
    - REGTESTS: server: fix cli_add_server due to previous trace update
    - REGTESTS: add minimal CLI "add map" tests
    - DOC: management: move "set var" to the proper place
    - CLEANUP: map: slightly reorder the add map function
    - MINOR: map: get rid of map_add_key_value()
    - MINOR: map: show the current and next pattern version in "show map"
    - MINOR: map/acl: add the possibility to specify the version in "show map/acl"
    - MINOR: pattern: support purging arbitrary ranges of generations
    - MINOR: map/acl: add the possibility to specify the version in "clear map/acl"
    - MINOR: map/acl: add the "prepare map/acl" CLI command
    - MINOR: map/acl: add the "commit map/acl" CLI command
    - MINOR: map/acl: make "add map/acl" support an optional version number
    - CLEANUP: map/cli: properly align the map/acl help
    - BUILD: compiler: do not use already defined __read_mostly on dragonfly
2021-05-01 08:25:15 +02:00
Willy Tarreau
bb51c44d64 MINOR: map/acl: make "add map/acl" support an optional version number
By passing a version number to "add map/acl", it becomes possible to
atomically replace maps and ACLs. The principle is that a new version
number is first retrieved by calling"prepare map/acl", and this version
number is used with "add map" and "add acl". Newly added entries then
remain invisible to the matching mechanism but are visible in "show
map/acl" when the version number is specified, or may be cleard with
"clear map/acl". Finally when the insertion is complete, a
"commit map/acl" command must be issued, and the version is atomically
updated so that there is no intermediate state with incomplete entries.
2021-04-30 15:36:31 +02:00
Willy Tarreau
7a562ca809 MINOR: map/acl: add the "commit map/acl" CLI command
The command is used to atomically replace a map/acl with the pending
contents of the designated version. The new version must have been
allocated by "prepare map/acl" prior to this. At the moment it is not
possible to force the version when adding new entries, so this may only
be used to atomically clear an ACL/map.
2021-04-30 15:36:31 +02:00
Willy Tarreau
97218ce3a9 MINOR: map/acl: add the "prepare map/acl" CLI command
This command allocates a new version for the map/acl, that will be usable
later to prepare the addition of new values to atomically replace existing
ones. Technically speaking the operation consists in atomically incrementing
the next version. There's no "undo" operation here, if a version is not
committed, it will automatically be trashed when committing a newer version.
2021-04-30 15:36:31 +02:00
Willy Tarreau
ff3feeb5cf MINOR: map/acl: add the possibility to specify the version in "clear map/acl"
This will ease maintenance of versionned maps by allowing to clear old or
failed updates instead of the current version. Nothing was done to allow
clearing everyhing, though if there was a need for this, implementing "@all"
or something equivalent wouldn't require more than 3 lines of code.
2021-04-30 15:36:31 +02:00
Willy Tarreau
95f753e403 MINOR: map/acl: add the possibility to specify the version in "show map/acl"
The maps and ACLs internally all have two versions, the "current" one,
which is the one being matched against, and the "next" one, the one being
filled during an atomic replacement. Till now the "show" commands only used
to show the current one but it can be convenient to be able to show other
ones as well, so let's add the ability to do this with "show map" and
"show acl". The method used here consists in passing the version number
as "@<ver>" before the map/acl name or ID. It would have been better after
it but that could create confusion with keys already using such a format.
2021-04-30 15:36:31 +02:00
Willy Tarreau
4000ff0448 DOC: management: move "set var" to the proper place
Commit b8bd1ee89 ("MEDIUM: cli: add a new experimental "set var" command")
added "get var" and "set var" but "set var" was misplaced in the doc,
breaking the alphabetic ordering.
2021-04-30 15:36:31 +02:00
Amaury Denoyelle
eafd701dc5 MINOR: server: fix doc/trace on lb algo for dynamic server creation
The text mentionned that only backends with consistent hash method were
supported for dynamic servers. In fact, it is only required that the lb
algorith is dynamic.
2021-04-29 14:59:42 +02:00
Willy Tarreau
8a022d5049 MINOR: config: add a new "default-path" global directive
By default haproxy loads all files designated by a relative path from the
location the process is started in. In some circumstances it might be
desirable to force all relative paths to start from a different location
just as if the process was started from such locations. This is what this
directive is made for. Technically it will perform a temporary chdir() to
the designated location while processing each configuration file, and will
return to the original directory after processing each file. It takes an
argument indicating the policy to use when loading files whose path does
not start with a slash ('/').

A few options are offered, "current" (the default), "config" (files
relative to config file's dir), "parent" (files relative to config file's
parent dir), and "origin" with an absolute path.

This should address issue #1198.
2021-04-28 11:30:13 +02:00
Alex
59c5335dbe DOC: general: fix example in set-timeout
The alternative arguments are always in curly brackets, let's fix it for
set-timeout.
The Example in set-timeout does not have the one of the required argument.

This commit makes the PR https://github.com/cbonte/haproxy-dconv/pull/34
obsolete.
2021-04-28 11:08:35 +02:00
Alex
bf1bd5a2a8 DOC: general: fix white spaces for HTML converter
The HTML converter expects some formats to recognize if a keyword is a
keyword.
2021-04-26 10:43:51 +02:00
Ilya Shipitsin
b2be9a1ea9 CLEANUP: assorted typo fixes in the code and comments
This is 22nd iteration of typo fixes
2021-04-26 10:42:58 +02:00
Tim Duesterhus
2e4a18e04a MINOR: uri_normalizer: Add a percent-decode-unreserved normalizer
This normalizer decodes percent encoded characters within the RFC 3986
unreserved set.

See GitHub Issue #714.
2021-04-23 19:43:45 +02:00
Tim Duesterhus
d6d33deaea DOC: Add RFC references for the path-strip-dot(dot)? normalizers
This is RFC 3986#6.2.2.3.
2021-04-23 19:43:45 +02:00
Tim Duesterhus
c315efda60 DOC: Fix RFC reference for the percent-to-uppercase normalizer
The section is 6.2.2.1, not 6.2.21 (missing dot).
2021-04-23 19:43:45 +02:00
Tim Duesterhus
7a95f41fd6 DOC: Fix indentation for path-strip-dot normalizer
The long explanation should be indented two additional spaces.
2021-04-23 19:43:45 +02:00
Willy Tarreau
bfd19d68f0 [RELEASE] Released version 2.4-dev17
Released version 2.4-dev17 with the following main changes :
    - MINOIR: mux-pt/trace: Register a new trace source with its events
    - BUG/MINOR: mux-pt: Fix a possible UAF because of traces in mux_pt_io_cb
    - CI: travis: Drastically clean up .travis.yml
    - CLEANUP: pattern: make all pattern tables read-only
    - MINOR: trace: replace the trace() inline function with an equivalent macro
    - MINOR: initcall: uniformize the section names between MacOS and other unixes
    - CLEANUP: initcall: rename HA_SECTION to HA_INIT_SECTION
    - MINOR: compiler: add macros to declare section names
    - CLEANUP: initcall: rely on HA_SECTION_* instead of defining its own
    - MINOR: global: declare a read_mostly section
    - MINOR: fd: move a few read-mostly variables to their own section
    - MINOR: epoll: move epoll_fd to read_mostly
    - MINOR: kqueue: move kqueue_fd to read_mostly
    - MINOR: pool: move pool declarations to read_mostly
    - MINOR: threads: mark all_threads_mask as read_mostly
    - MINOR: server: move idle_conn_task to read_mostly
    - MINOR: protocol: move __protocol_by_family to read_mostly
    - MINOR: pattern: make the pat_lru_seed read_mostly
    - MINOR: trace: make trace sources read_mostly
    - MINOR: freq_ctr: add a generic function to report the total value
    - MEDIUM: freq_ctr: make read_freq_ctr_period() use freq_ctr_total()
    - MEDIUM: freq_ctr: reimplement freq_ctr_remain_period() from freq_ctr_total()
    - MINOR: freq_ctr: add the missing next_event_delay_period()
    - MINOR: freq_ctr: unify freq_ctr and freq_ctr_period into freq_ctr
    - MEDIUM: freq_ctr: replace the per-second counters with the generic ones
    - MINOR: freq_ctr: add cpu_relax in the rotation loop of update_freq_ctr_period()
    - MINOR: freq_ctr: simplify and improve the update function
    - CLEANUP: time: remove the now unused ms_left_scaled
    - MINOR: time: move the time initialization out of tv_update_date()
    - MINOR: time: remove useless variable copies in tv_update_date()
    - MINOR: time: change the global timeval and the the global tick at once
    - MEDIUM: time: make the clock offset global and no per-thread
    - MINOR: atomic: reimplement the relaxed version of x86 BTS/BTR
    - MINOR: trace: Add the checks as a possible trace source
    - MINOIR: checks/trace: Register a new trace source with its events
    - MINOR: hlua: Add function to release a lua function
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a task
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a converter
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a fetch
    - BUG/MINOR: hlua: Fix memory leaks on error path when parsing a lua action
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering an action
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a service
    - BUG/MINOR: hlua: Fix memory leaks on error path when registering a cli keyword
    - BUG/MINOR: cfgparse/proxy: Fix some leaks during proxy section parsing
    - BUG/MINOR: listener: Handle allocation error when allocating a new bind_conf
    - BUG/MINOR: cfgparse/proxy: Hande allocation errors during proxy section parsing
    - MINOR: cfgparse/proxy: Group alloc error handling during proxy section parsing
    - DOC: internals: update the SSL architecture schema
    - BUG/MEDIUM: sample: Fix adjusting size in field converter
    - MINOR: sample: add ub64dec and ub64enc converters
    - CLEANUP: sample: align samples list in sample.c
    - MINOR: ist: Add `istclear(struct ist*)`
    - CI: cirrus: install "pcre" package
    - MINOR: opentracing: correct calculation of the number of arguments in the args[]
    - MINOR: opentracing: transfer of context names without prefix
    - MINOR: sample: converter: Add mjson library.
    - MINOR: sample: converter: Add json_query converter
    - CI: travis-ci: enable weekly graviton2 builds
    - DOC: ssl: Certificate hot update only works on fronted certificates
    - DOC: ssl: Certificate hot update works on server certificates
    - BUG/MEDIUM: threads: Ignore current thread to end its harmless period
    - MINOR: threads: Only consider running threads to end a thread harmeless period
    - BUG/MINOR: checks: Set missing id to the dummy checks frontend
    - MINOR: logs: Add support of checks as session origin to format lf strings
    - BUG/MINOR: connection: Fix fc_http_major and bc_http_major for TCP connections
    - MINOR: connection: Make bc_http_major compatible with tcp-checks
    - BUG/MINOR: ssl-samples: Fix ssl_bc_* samples when called from a health-check
    - BUG/MINOR: http-fetch: Make method smp safe if headers were already forwarded
    - MINOR: tcp_samples: Add samples to get src/dst info of the backend connection
    - MINOR: tcp_samples: Be able to call bc_src/bc_dst from the health-checks
    - BUG/MINOR: http_htx: Remove BUG_ON() from http_get_stline() function
    - BUG/MINOR: logs: Report the true number of retries if there was no connection
    - BUILD: makefile: Redirect stderr to /dev/null when probing options
    - MINOR: uri_normalizer: Add uri_normalizer module
    - MINOR: uri_normalizer: Add `enum uri_normalizer_err`
    - MINOR: uri_normalizer: Add `http-request normalize-uri`
    - MINOR: uri_normalizer: Add a `merge-slashes` normalizer to http-request normalize-uri
    - MINOR: uri_normalizer: Add a `dotdot` normalizer to http-request normalize-uri
    - MINOR: uri_normalizer: Add support for supressing leading `../` for dotdot normalizer
    - MINOR: uri_normalizer: Add a `sort-query` normalizer
    - MINOR: uri_normalizer: Add a `percent-upper` normalizer
    - MEDIUM: http_act: Rename uri-normalizers
    - DOC: Add introduction to http-request normalize-uri
    - DOC: Note that URI normalization is experimental
    - BUG/MINOR: pools: maintain consistent ->allocated count on alloc failures
    - BUG/MINOR: pools/buffers: make sure to always reserve the required buffers
    - MINOR: pools: drop the unused static history of artificially failed allocs
    - CLEANUP: pools: remove unused arguments to pool_evict_from_cache()
    - MEDIUM: pools: move the cache into the pool header
    - MINOR: pool: remove the size field from pool_cache_head
    - MINOR: pools: rename CONFIG_HAP_LOCAL_POOLS to CONFIG_HAP_POOLS
    - MINOR: pools: enable the fault injector in all allocation modes
    - MINOR: pools: make the basic pool_refill_alloc()/pool_free() update needed_avg
    - MEDIUM: pools: unify pool_refill_alloc() across all models
    - CLEANUP: pools: re-merge pool_refill_alloc() and __pool_refill_alloc()
    - MINOR: pools: call pool_alloc_nocache() out of the pool's lock
    - CLEANUP: pools: move the lock to the only __pool_get_first() that needs it
    - CLEANUP: pools: rename __pool_get_first() to pool_get_from_shared_cache()
    - CLEANUP: pools: rename pool_*_{from,to}_cache() to *_local_cache()
    - CLEANUP: pools: rename __pool_free() to pool_put_to_shared_cache()
    - MINOR: tools: add statistical_prng_range() to get a random number over a range
    - MINOR: pools: use cheaper randoms for fault injections
    - MINOR: pools: move the fault injector to __pool_alloc()
    - MINOR: pools: split the OS-based allocator in two
    - MINOR: pools: always use atomic ops to maintain counters
    - MINOR: pools: move pool_free_area() out of the lock in the locked version
    - MINOR: pools: factor the release code into pool_put_to_os()
    - MEDIUM: pools: make CONFIG_HAP_POOLS control both local and shared pools
    - MINOR: pools: create unified pool_{get_from,put_to}_cache()
    - MINOR: pools: evict excess objects using pool_evict_from_local_cache()
    - MEDIUM: pools: make pool_put_to_cache() always call pool_put_to_local_cache()
    - CLEANUP: pools: make the local cache allocator fall back to the shared cache
    - CLEANUP: pools: merge pool_{get_from,put_to}_local_caches with generic ones
    - CLEANUP: pools: uninline pool_put_to_cache()
    - CLEANUP: pools: declare dummy pool functions to remove some ifdefs
    - BUILD: pools: fix build with DEBUG_FAIL_ALLOC
    - BUG/MINOR: server: make srv_alloc_lb() allocate lb_nodes for consistent hash
    - CONTRIB: mod_defender: import the minimal number of includes
    - CONTRIB: mod_defender: make the code build with the embedded includes
    - CONTRIB: modsecurity: import the minimal number of includes
    - CONTRIB: modsecurity: make the code build with the embedded includes
    - CLEANUP: sample: Improve local variables in sample_conv_json_query
    - CLEANUP: sample: Explicitly handle all possible enum values from mjson
    - CLEANUP: sample: Use explicit return for successful `json_query`s
    - CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
    - CONTRIB: move spoa_example out of the tree
    - BUG/MINOR: server: free srv.lb_nodes in free_server
    - BUG/MINOR: logs: free logsrv.conf.file on exit
    - BUG/MEDIUM: server: ensure thread-safety of server runtime creation
    - MINOR: server: add log on dynamic server creation
    - MINOR: server: implement delete server cli command
    - CONTRIB: move spoa_server out of the tree
    - CONTRIB: move modsecurity out of the tree
    - BUG/MINOR: server: fix potential null gcc error in delete server
    - BUG/MAJOR: mux-h2: Properly detect too large frames when decoding headers
    - BUG/MEDIUM: mux-h2: Fix dfl calculation when merging CONTINUATION frames
    - BUG/MINOR: uri_normalizer: Use delim parameter when building the sorted query in uri_normalizer_query_sort
    - CLEANUP: uri_normalizer: Remove trailing whitespace
    - MINOR: uri_normalizer: Add a `strip-dot` normalizer
    - CONTRIB: move mod_defender out of the tree
    - CLEANUP: contrib: remove the last references to the now dead contrib/ directory
    - BUG/MEDIUM: config: fix cpu-map notation with both process and threads
    - MINOR: config: add a diag for invalid cpu-map statement
    - BUG/MINOR: mworker/init: don't reset nb_oldpids in non-mworker cases
    - BUG/MINOR: mworker: don't use oldpids[] anymore for reload
    - BUILD: makefile: fix the "make clean" target on strict bourne shells
    - IMPORT: slz: import slz into the tree
    - BUILD: compression: switch SLZ from out-of-tree to in-tree
    - CI: github: do not build libslz any more
    - CLEANUP: compression: remove calls to SLZ init functions
    - BUG/MEDIUM: mux-h2: Properly handle shutdowns when received with data
    - MINOR: cpuset: define a platform-independent cpuset type
    - MINOR: cfgparse: use hap_cpuset for parse_cpu_set
    - MEDIUM: config: use platform independent type hap_cpuset for cpu-map
    - MINOR: thread: implement the detection of forced cpu affinity
    - MINOR: cfgparse: support the comma separator on parse_cpu_set
    - MEDIUM: cfgparse: detect numa and set affinity if needed
    - MINOR: global: add option to disable numa detection
    - BUG/MINOR: haproxy: fix compilation on macOS
    - BUG/MINOR: cpuset: fix compilation on platform without cpu affinity
    - MINOR: time: avoid unneeded updates to now_offset
    - MINOR: time: avoid overwriting the same values of global_now
    - CLEANUP: time: use __tv_to_ms() in tv_update_date() instead of open-coding
    - MINOR: time: avoid u64 needlessly expensive computations for the 32-bit now_ms
    - BUG/MINOR: peers: remove useless table check if initial resync is finished
    - BUG/MEDIUM: peers: re-work connection to new process during reload.
    - BUG/MEDIUM: peers: re-work refcnt on table to protect against flush
    - BUG/MEDIUM: config: fix missing initialization in numa_detect_topology()
2021-04-23 19:11:10 +02:00
Amaury Denoyelle
0f50cb9c73 MINOR: global: add option to disable numa detection
Render numa detection optional with a global configuration statement
'no numa-cpu-mapping'. This can be used if the applied affinity of the
algorithm is not optimal. Also complete the documentation with this new
keyword.
2021-04-23 16:06:49 +02:00
Amaury Denoyelle
982fb53390 MEDIUM: config: use platform independent type hap_cpuset for cpu-map
Use the platform independent type hap_cpuset for the cpu-map statement
parsing. This allow to address CPU index greater than LONGBITS.

Update the documentation to reflect the removal of this limit except for
platforms without cpu_set_t type or equivalent.
2021-04-23 16:06:49 +02:00
Maximilian Mader
ff3bb8b609 MINOR: uri_normalizer: Add a strip-dot normalizer
This normalizer removes "/./" segments from the path component.
Usually the dot refers to the current directory which renders those segments redundant.

See GitHub Issue #714.
2021-04-21 12:15:14 +02:00
Amaury Denoyelle
e558043e13 MINOR: server: implement delete server cli command
Implement a new CLI command 'del server'. It can be used to removed a
dynamically added server. Only servers in maintenance mode can be
removed, and without pending/active/idle connection on it.

Add a new reg-test for this feature. The scenario of the reg-test need
to first add a dynamic server. It is then deleted and a client is used
to ensure that the server is non joinable.

The management doc is updated with the new command 'del server'.
2021-04-21 11:00:31 +02:00
Willy Tarreau
8695199aa8 CONTRIB: move spoa_example out of the tree
As previously mentioned SPOA code has nothing to do in the haproxy core
since they're not dependent on haproxy's version. This one was moved to
its own repository here with complete history:

     https://github.com/haproxy/spoa-example
2021-04-21 09:39:06 +02:00
Willy Tarreau
2b71810cb3 CLEANUP: lists/tree-wide: rename some list operations to avoid some confusion
The current "ADD" vs "ADDQ" is confusing because when thinking in terms
of appending at the end of a list, "ADD" naturally comes to mind, but
here it does the opposite, it inserts. Several times already it's been
incorrectly used where ADDQ was expected, the latest of which was a
fortunate accident explained in 6fa922562 ("CLEANUP: stream: explain
why we queue the stream at the head of the server list").

Let's use more explicit (but slightly longer) names now:

   LIST_ADD        ->       LIST_INSERT
   LIST_ADDQ       ->       LIST_APPEND
   LIST_ADDED      ->       LIST_INLIST
   LIST_DEL        ->       LIST_DELETE

The same is true for MT_LISTs, including their "TRY" variant.
LIST_DEL_INIT keeps its short name to encourage to use it instead of the
lazier LIST_DELETE which is often less safe.

The change is large (~674 non-comment entries) but is mechanical enough
to remain safe. No permutation was performed, so any out-of-tree code
can easily map older names to new ones.

The list doc was updated.
2021-04-21 09:20:17 +02:00
Tim Duesterhus
2963fd34f8 DOC: Note that URI normalization is experimental
Add a paragraph to the URI normalization documentation that URI normalization
is currently considered to be experimental.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
b918a4a564 DOC: Add introduction to http-request normalize-uri
This patch adds an introduction to the http-request normalize-uri section,
explaining what to expect from the normalizers and possible issues that might
arise when not being careful.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
5be6ab269e MEDIUM: http_act: Rename uri-normalizers
This patch renames all existing uri-normalizers into a more consistent naming
scheme:

1. The part of the URI that is being touched.
2. The modification being performed as an explicit verb.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
a407193376 MINOR: uri_normalizer: Add a percent-upper normalizer
This normalizer uppercases the hexadecimal characters used in percent-encoding.

See GitHub Issue #714.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
d7b89be30a MINOR: uri_normalizer: Add a sort-query normalizer
This normalizer sorts the `&` delimited query parameters by parameter name.

See GitHub Issue #714.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
560e1a6352 MINOR: uri_normalizer: Add support for supressing leading ../ for dotdot normalizer
This adds an option to supress `../` at the start of the resulting path.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
9982fc2bbd MINOR: uri_normalizer: Add a dotdot normalizer to http-request normalize-uri
This normalizer merges `../` path segments with the predecing segment, removing
both the preceding segment and the `../`.

Empty segments do not receive special treatment. The `merge-slashes` normalizer
should be executed first.

See GitHub Issue #714.
2021-04-19 09:05:57 +02:00
Tim Duesterhus
d371e99d1c MINOR: uri_normalizer: Add a merge-slashes normalizer to http-request normalize-uri
This normalizer merges adjacent slashes into a single slash, thus removing
empty path segments.

See GitHub Issue #714.
2021-04-19 09:05:57 +02:00