1194 Commits

Author SHA1 Message Date
Willy Tarreau
1a753a2ead DOC: remove obsolete section about header manipulation
Cyril Bont reported that the doc contains two chapters number 6,
one of which is a leftover of the section about old header manipulation
directives that were removed by commit a6a56e6 ("MEDIUM: config: Remove
parsing of req* and rsp* directives"). This patch removes this.
2019-10-28 07:03:13 +01:00
Willy Tarreau
c70df53250 [RELEASE] Released version 2.1-dev3
Released version 2.1-dev3 with the following main changes :
    - MINOR: mux-h2/trace: missing conn pointer in demux full message
    - MINOR: mux-h2: add a per-connection list of blocked streams
    - BUILD: ebtree: make eb_is_empty() and eb_is_dup() take a const
    - BUG/MEDIUM: mux-h2: do not enforce timeout on long connections
    - BUG/MEDIUM: tasks: Don't forget to decrement tasks_run_queue.
    - BUG/MINOR: peers: crash on reload without local peer.
    - BUG/MINOR: mux-h2/trace: Fix traces on h2c initialization
    - MINOR: h1-htx: Update h1_copy_msg_data() to ease the traces in the mux-h1
    - MINOR: htx: Adapt htx_dump() to be used from traces
    - MINOR: mux-h1/trace: register a new trace source with its events
    - MINOR: proxy: Store http-send-name-header in lower case
    - MINOR: http: Remove headers matching the name of http-send-name-header option
    - BUG/MINOR: mux-h1: Adjust header case when the server name is add to a request
    - BUG/MINOR: mux-h1: Adjust header case when chunked encoding is add to a message
    - MINOR: mux-h1: Try to wakeup the stream on output buffer allocation
    - MINOR: fcgi: Add function to get the string representation of a record type
    - MINOR: mux-fcgi/trace: Register a new trace source with its events
    - BUG/MEDIUM: cache: make sure not to cache requests with absolute-uri
    - DOC: clarify some points around http-send-name-header's behavior
    - MEDIUM: mux-h2: support emitting CONTINUATION frames after HEADERS
    - BUG/MINOR: mux-h1/mux-fcgi/trace: Fix position of the 4th arg in some traces
    - DOC: fix typo in Prometheus exporter doc
    - MINOR: h2: clarify the rules for how to convert an H2 request to HTX
    - MINOR: htx: Add 2 flags on the start-line to have more info about the uri
    - MINOR: http: Add a function to get the authority into a URI
    - MINOR: h1-htx: Set the flag HTX_SL_F_HAS_AUTHORITY during the request parsing
    - MEDIUM: http-htx: Keep the Host header and the request start-line synchronized
    - MINOR: h1-htx: Only use the path of a normalized URI to format a request line
    - MEDIUM: h2: make the request parser rebuild a complete URI
    - MINOR: h2: report in the HTX flags when the request has an authority
    - MEDIUM: mux-h2: do not map Host to :authority on output
    - MEDIUM: h2: use the normalized URI encoding for absolute form requests
    - MINOR: stats: mention in the help message support for "json" and "typed"
    - MINOR: stats: get rid of the ST_CONVDONE flag
    - MINOR: stats: replace the ST_* uri_auth flags with STAT_*
    - MINOR: stats: always merge the uri_auth flags into the appctx flags
    - MINOR: stats: set the appctx flags when initializing the applet only
    - MINOR: stats: get rid of the STAT_SHOWADMIN flag
    - MINOR: stats: make stats_dump_fields_json() directly take flags
    - MINOR: stats: uniformize the calling convention of the dump functions
    - MINOR: stats: support the "desc" output format modifier for info and stat
    - MINOR: stats: prepare to add a description with each stat/info field
    - MINOR: stats: make "show stat" and "show info"
    - MINOR: stats: fill all the descriptions for "show info" and "show stat"
    - BUG/MEDIUM: applet: always check a fast running applet's activity before killing
    - BUILD: stats: fix missing '=' sign in array declaration
    - MINOR: lists: add new macro LIST_SPLICE_END_DETACHED
    - MINOR: list: add new macro MT_LIST_BEHEAD
    - MEDIUM: task: Split the tasklet list into two lists.
    - MINOR: h2: Document traps to be avoided on multithread.
    - MINOR: lists: Try to use local variables instead of macro arguments.
    - MINOR: lists: Fix alignement of \ when relevant.
    - MINOR: mux-h2: also support emitting CONTINUATION on trailers
    - MINOR: ssl: crt-list do ckchn_lookup
    - REORG: ssl: rename ckch_node to ckch_store
    - REORG: ssl: move structures to ssl_sock.h
    - MINOR: ssl: initialize the sni_keytypes_map as EB_ROOT
    - MINOR: ssl: initialize explicitly the sni_ctx trees
    - BUG/MINOR: ssl: abort on sni allocation failure
    - BUG/MINOR: ssl: free the sni_keytype nodes
    - BUG/MINOR: ssl: abort on sni_keytypes allocation failure
    - MEDIUM: ssl: introduce the ckch instance structure
    - MEDIUM: ssl: split ssl_sock_add_cert_sni()
    - MINOR: ssl: ssl_sock_load_ckchn() can properly fail
    - MINOR: ssl: ssl_sock_load_multi_ckchs() can properly fail
    - MEDIUM: ssl: ssl_sock_load_ckchs() alloc a ckch_inst
    - MINOR: ssl: ssl_sock_load_crt_file_into_ckch() is filling from a BIO
    - MEDIUM: ssl/cli: 'set ssl cert' updates a certificate from the CLI
    - MINOR: ssl: load the sctl in/from the ckch
    - MINOR: ssl: load the ocsp in/from the ckch
    - BUG/MEDIUM: ssl: NULL dereference in ssl_sock_load_cert_sni()
    - BUG/MINOR: ssl: fix build without SSL
    - BUG/MINOR: ssl: fix build without multi-cert bundles
    - BUILD: ssl: wrong #ifdef for SSL engines code
    - BUG/MINOR: ssl: fix OCSP build with BoringSSL
    - BUG/MEDIUM: htx: Catch chunk_memcat() failures when HTX data are formatted to h1
    - BUG/MINOR: chunk: Fix tests on the chunk size in functions copying data
    - BUG/MINOR: mux-h1: Mark the output buffer as full when the xfer is interrupted
    - MINOR: mux-h1: Xfer as much payload data as possible during output processing
    - CLEANUP: h1-htx: Move htx-to-h1 formatting functions from htx.c to h1_htx.c
    - BUG/MINOR: mux-h1: Capture ignored parsing errors
    - MINOR: h1: Reject requests with different occurrences of the header host
    - MINOR: h1: Reject requests if the authority does not match the header host
    - REGTESTS: Send valid URIs in peers reg-tests and fix HA config to avoid warnings
    - REGTESTS: Adapt proxy_protocol_random_fail.vtc to match normalized URI too
    - BUG/MINOR: WURFL: fix send_log() function arguments
    - BUG/MINOR: ssl: fix error messages for OCSP loading
    - BUG/MINOR: ssl: can't load ocsp files
    - MINOR: version: make the version strings variables, not constants
    - BUG/MINOR: http-htx: Properly set htx flags on error files to support keep-alive
    - MINOR: htx: Add a flag on HTX to known when a response was generated by HAProxy
    - MINOR: mux-h1: Force close mode for proxy responses with an unfinished request
    - BUILD: travis-ci: limit build to branches "master" and "next"
    - BUILD/MEDIUM: threads: rename thread_info struct to ha_thread_info
    - BUILD/SMALL: threads: enable threads on osx
    - BUILD/MEDIUM: threads: enable cpu_affinity on osx
    - MINOR: istbuf: add b_fromist() to make a buffer from an ist
    - BUG/MINOR: cache: also cache absolute URIs
    - BUG/MINOR: mworker/ssl: close openssl FDs unconditionally
    - BUG/MINOR: tcp: Don't alter counters returned by tcp info fetchers
    - BUG/MEDIUM: lists: Handle 1-element-lists in MT_LIST_BEHEAD().
    - BUG/MEDIUM: mux_pt: Make sure we don't have a conn_stream before freeing.
    - BUG/MEDIUM: tasklet: properly compute the sleeping threads mask in tasklet_wakeup()
    - BUG/MAJOR: idle conns: schedule the cleanup task on the correct threads
    - BUG/MEDIUM: task: make tasklets either local or shared but not both at once
    - Revert e8826ded5fea3593d89da2be5c2d81c522070995.
    - BUG/MEDIUM: mux_pt: Don't destroy the connection if we have a stream attached.
    - BUG/MEDIUM: mux_pt: Only call the wake emthod if nobody subscribed to receive.
    - REGTEST: mcli/mcli_show_info: launch a 'show info' on the master CLI
    - CLEANUP: ssl: make ssl_sock_load_cert*() return real error codes
    - CLEANUP: ssl: make ssl_sock_load_ckchs() return a set of ERR_*
    - CLEANUP: ssl: make cli_parse_set_cert handle errcode and warnings.
    - CLEANUP: ssl: make ckch_inst_new_load_(multi_)store handle errcode/warn
    - CLEANUP: ssl: make ssl_sock_put_ckch_into_ctx handle errcode/warn
    - CLEANUP: ssl: make ssl_sock_load_dh_params handle errcode/warn
    - CLEANUP: bind: handle warning label on bind keywords parsing.
    - BUG/MEDIUM: ssl: 'tune.ssl.default-dh-param' value ignored with openssl > 1.1.1
    - BUG/MINOR: mworker/cli: reload fail with inherited FD
    - BUG/MINOR: ssl: Fix fd leak on error path when a TLS ticket keys file is parsed
    - BUG/MINOR: stick-table: Never exceed (MAX_SESS_STKCTR-1) when fetching a stkctr
    - BUG/MINOR: cache: alloc shctx after check config
    - BUG/MINOR: sample: Make the `field` converter compatible with `-m found`
    - BUG/MINOR: server: check return value of fopen() in apply_server_state()
    - REGTESTS: make seamless-reload depend on 1.9 and above
    - REGTESTS: server/cli_set_fqdn requires version 1.8 minimum
    - BUG/MINOR: dns: allow srv record weight set to 0
    - BUG/MINOR: ssl: fix memcpy overlap without consequences.
    - BUG/MINOR: stick-table: fix an incorrect 32 to 64 bit key conversion
    - BUG/MEDIUM: pattern: make the pattern LRU cache thread-local and lockless
    - BUG/MINOR: mux-h2: do not emit logs on backend connections
    - CLEANUP: ssl: remove old TODO commentary
    - CLEANUP: ssl: fix SNI/CKCH lock labels
    - MINOR: ssl: OCSP functions can load from file or buffer
    - MINOR: ssl: load sctl from buf OR from a file
    - MINOR: ssl: load issuer from file or from buffer
    - MINOR: ssl: split ssl_sock_load_crt_file_into_ckch()
    - BUG/MINOR: ssl/cli: fix looking up for a bundle
    - MINOR: ssl/cli: update ocsp/issuer/sctl file from the CLI
    - MINOR: ssl: update ssl_sock_free_cert_key_and_chain_contents
    - MINOR: ssl: copy a ckch from src to dst
    - MINOR: ssl: new functions duplicate and free a ckch_store
    - MINOR: ssl/cli: assignate a new ckch_store
    - MEDIUM: cli/ssl: handle the creation of SSL_CTX in an IO handler
    - BUG/MINOR: ssl/cli: fix build of SCTL and OCSP
    - BUG/MINOR: ssl/cli: out of bounds when built without ocsp/sctl
    - BUG/MINOR: ssl: fix build with openssl < 1.1.0
    - BUG/MINOR: ssl: fix build of X509_chain_up_ref() w/ libreSSL
    - MINOR: tcp: avoid confusion in time parsing init
    - MINOR: debug: add a new "debug dev stream" command
    - MINOR: cli/debug: validate addresses using may_access() in "debug dev stream"
    - REORG: move CLI access level definitions to cli.h
    - MINOR: cli: add an expert mode to hide dangerous commands
    - MINOR: debug: make most debug CLI commands accessible in expert mode
    - MINOR: stats/debug: maintain a counter of debug commands issued
    - BUG/MEDIUM: debug: address a possible null pointer dereference in "debug dev stream"
2019-10-25 15:48:53 +02:00
Willy Tarreau
403bfbb130 BUG/MEDIUM: pattern: make the pattern LRU cache thread-local and lockless
As reported in issue #335, a lot of contention happens on the PATLRU lock
when performing expensive regex lookups. This is absurd since the purpose
of the LRU cache was to have a fast cache for expressions, thus the cache
must not be shared between threads and must remain lockless.

This commit makes the LRU cache thread-local and gets rid of the PATLRU
lock. A test with 7 threads on 4 cores climbed from 67kH/s to 369kH/s,
or a scalability factor of 5.5.

Given the huge performance difference and the regression caused to
users migrating from processes to threads, this should be backported at
least to 2.0.

Thanks to Brian Diekelman for his detailed report about this regression.
2019-10-23 07:27:25 +02:00
Christopher Faulet
ba0c53ef71 BUG/MINOR: tcp: Don't alter counters returned by tcp info fetchers
There are 2 kinds of tcp info fetchers. Those returning a time value (fc_rtt and
fc_rttval) and those returning a counter (fc_unacked, fc_sacked, fc_retrans,
fc_fackets, fc_lost, fc_reordering). Because of a bug, the counters were handled
as time values, and by default, were divided by 1000 (because of an invalid
conversion from us to ms). To work around this bug and have the right value, the
argument "us" had to be specified.

So now, tcp info fetchers returning a counter don't support any argument
anymore. To not break old configurations, if an argument is provided, it is
ignored and a warning is emitted during the configuration parsing.

In addition, parameter validiation is now performed during the configuration
parsing.

This patch must be backported as far as 1.7.
2019-10-17 15:20:06 +02:00
Willy Tarreau
81bef7e899 DOC: clarify some points around http-send-name-header's behavior
The directive causes existing an header to be removed, which is not
explicitly mentioned though already being relied on, and also mention
the fast that it should not be used to modify transport level headers
and that doing it on Host is more than border-line and definitely not
a supported long-term option eventhough it currently works.
2019-10-07 14:58:02 +02:00
Willy Tarreau
cb8f03fc46 [RELEASE] Released version 2.1-dev2
Released version 2.1-dev2 with the following main changes :
    - DOC: management: document reuse and connect counters in the CSV format
    - DOC: management: document cache_hits and cache_lookups in the CSV format
    - BUG/MINOR: dns: remove irrelevant dependency on a client connection
    - MINOR: applet: make appctx use their own pool
    - BUG/MEDIUM: checks: Don't attempt to receive data if we already subscribed.
    - BUG/MEDIUM: http/htx: unbreak option http_proxy
    - BUG/MINOR: backend: do not try to install a mux when the connection failed
    - MINOR: mux-h2: Don't adjust anymore the amount of data sent in h2_snd_buf()
    - BUG/MINOR: http_fetch: Fix http_auth/http_auth_group when called from TCP rules
    - BUG/MINOR: http_htx: Initialize HTX error messages for TCP proxies
    - BUG/MINOR: cache/htx: Make maxage calculation HTX aware
    - BUG/MINOR: hlua: Make the function txn:done() HTX aware
    - MINOR: proto_htx: Directly call htx_check_response_for_cacheability()
    - MINOR: proto_htx: Rely on the HTX function to apply a redirect rules
    - MINOR: proto_htx: Add the function htx_return_srv_error()
    - MINOR: backend/htx: Don't rewind output data to set the sni on a srv connection
    - MINOR: proto_htx: Don't stop forwarding when there is a post-connect processing
    - DOC: htx: Update comments in HTX files
    - CLEANUP: htx: Remove the unsued function htx_add_blk_type_size()
    - MINOR: htx: Deduce the number of used blocks from tail and head values
    - MINOR: htx: Use an array of char to store HTX blocks
    - MINOR: htx: Slightly update htx_dump() to report better messages
    - DOC: htx: Add internal documentation about the HTX
    - MAJOR: http: Deprecate and ignore the option "http-use-htx"
    - MEDIUM: mux-h2: Remove support of the legacy HTTP mode
    - CLEANUP: h2: Remove functions converting h2 requests to raw HTTP/1.1 ones
    - MINOR: connection: Remove the multiplexer protocol PROTO_MODE_HTX
    - MINOR: stream: Rely on HTX analyzers instead of legacy HTTP ones
    - MEDIUM: http_fetch: Remove code relying on HTTP legacy mode
    - MINOR: config: Remove tests on the option 'http-use-htx'
    - MINOR: stream: Remove tests on the option 'http-use-htx' in stream_new()
    - MINOR: proxy: Remove tests on the option 'http-use-htx' during H1 upgrade
    - MINOR: hlua: Remove tests on the option 'http-use-htx' to reject TCP applets
    - MINOR: cache: Remove tests on the option 'http-use-htx'
    - MINOR: contrib/prometheus-exporter: Remove tests on the option 'http-use-htx'
    - CLEANUP: proxy: Remove the flag PR_O2_USE_HTX
    - MINOR: proxy: Don't adjust connection mode of HTTP proxies anymore
    - MEDIUM: backend: Remove code relying on the HTTP legacy mode
    - MEDIUM: hlua: Remove code relying on the legacy HTTP mode
    - MINOR: http_act: Remove code relying on the legacy HTTP mode
    - MEDIUM: cache: Remove code relying on the legacy HTTP mode
    - MEDIUM: compression: Remove code relying on the legacy HTTP mode
    - MINOR: flt_trace: Remove code relying on the legacy HTTP mode
    - MINOR: stats: Remove code relying on the legacy HTTP mode
    - MAJOR: filters: Remove code relying on the legacy HTTP mode
    - MINOR: stream: Remove code relying on the legacy HTTP mode
    - MAJOR: http: Remove the HTTP legacy code
    - MINOR: hlua: Remove useless test on TX_CON_WANT_* flags
    - MINOR: proto_http: Remove unused http txn flags
    - MINOR: proto_http: Remove the unused flag HTTP_MSGF_WAIT_CONN
    - CLEANUP: proto_http: Group remaining flags of the HTTP transaction
    - CLEANUP: channel: Remove the unused flag CF_WAKE_CONNECT
    - CLEANUP: proto_http: Remove unecessary includes and comments
    - CLEANUP: proto_http: Move remaining code from proto_http.c to proto_htx.c
    - REORG: proto_htx: Move HTX analyzers & co to http_ana.{c,h} files
    - BUG/MINOR: debug: Remove flags CO_FL_SOCK_WR_ENA/CO_FL_SOCK_RD_ENA
    - MINOR: proxy: Remove support of the option 'http-tunnel'
    - DOC: config: Update as a result of the legacy HTTP removal
    - MEDIUM: config: Remove parsing of req* and rsp* directives
    - MINOR: proxy: Remove the unused list of block rules
    - MINOR: proxy/http_ana: Remove unused req_exp/rsp_exp and req_add/rsp_add lists
    - DOC: config: Remove unsupported req* and rsp* keywords
    - MINOR: global: Preset tune.max_http_hdr to its default value
    - MINOR: http: Don't store raw HTTP errors in chunks anymore
    - BUG/MINOR: session: Emit an HTTP error if accept fails only for H1 connection
    - BUG/MINOR: session: Send a default HTTP error if accept fails for a H1 socket
    - CLEANUP: mux-h2: Remove unused flags H2_SF_CHNK_*
    - BUG/MINOR: checks: do not exit tcp-checks from the middle of the loop
    - MINOR: config: Warn only if the option http-use-htx is used with "no" prefix
    - BUG/MEDIUM: mux-h1: Trim excess server data at the end of a transaction
    - MINOR: connection: add conn_get_src() and conn_get_dst()
    - MINOR: frontend: switch to conn_get_{src,dst}() for logging and debugging
    - MINOR: backend: switch to conn_get_{src,dst}() for port and address mapping
    - MINOR: ssl: switch to conn_get_dst() to retrieve the destination address
    - MINOR: tcp: replace various calls to conn_get_{from,to}_addr with conn_get_{src,dst}
    - MINOR: stream-int: use conn_get_{src,dst} in conn_si_send_proxy()
    - MINOR: stream/cli: use conn_get_{src,dst} in "show sess" and "show peers" output
    - MINOR: log: use conn_get_{dst,src}() to retrieve the cli/frt/bck/srv/ addresses
    - MINOR: http/htx: use conn_get_dst() to retrieve the destination address
    - MINOR: lua: use conn_get_{src,dst} to retrieve connection addresses
    - MINOR: http: check the source address via conn_get_src() in sample fetch functions
    - CLEANUP: connection: remove the now unused conn_get_{from,to}_addr()
    - MINOR: connection: add new src and dst fields
    - MINOR: connection: use conn->{src,dst} instead of &conn->addr.{from,to}
    - MINOR: ssl-sock: use conn->dst instead of &conn->addr.to
    - MINOR: lua: switch to conn->dst for a connection's target address
    - MINOR: peers: use conn->dst for the peer's target address
    - MINOR: htx: switch from conn->addr.{from,to} to conn->{src,dst}
    - MINOR: stream: switch from conn->addr.{from,to} to conn->{src,dst}
    - MINOR: proxy: switch to conn->src in error snapshots
    - MINOR: session: use conn->src instead of conn->addr.from
    - MINOR: tcp: replace conn->addr.{from,to} with conn->{src,dst}
    - MINOR: unix: use conn->dst for the target address in ->connect()
    - MINOR: sockpair: use conn->dst for the target address in ->connect()
    - MINOR: log: use conn->{src,dst} instead of conn->addr.{from,to}
    - MINOR: checks: replace conn->addr.to with conn->dst
    - MINOR: frontend: switch from conn->addr.{from,to} to conn->{src,dst}
    - MINOR: http: convert conn->addr.from to conn->src in sample fetches
    - MEDIUM: backend: turn all conn->addr.{from,to} to conn->{src,dst}
    - MINOR: connection: create a new pool for struct sockaddr_storage
    - MEDIUM: connection: make sure all address producers allocate their address
    - MAJOR: connection: remove the addr field
    - MINOR: connection: don't use clear_addr() anymore, just release the address
    - MINOR: stream: add a new target_addr entry in the stream structure
    - MAJOR: stream: store the target address into s->target_addr
    - MINOR: peers: now remove the remote connection setup code
    - MEDIUM: lua: do not allocate the remote connection anymore
    - MEDIUM: backend: always release any existing prior connection in connect_server()
    - MEDIUM: backend: remove impossible cases from connect_server()
    - BUG/MINOR: mux-h1: Close server connection if input data remains in h1_detach()
    - BUG/MEDIUM: tcp-checks: do not dereference inexisting conn_stream
    - BUG/MINOR: http_ana: Be sure to have an allocated buffer to generate an error
    - BUG/MINOR: http_htx: Support empty errorfiles
    - BUG/CRITICAL: http_ana: Fix parsing of malformed cookies which start by a delimiter
    - BUG/MEDIUM: protocols: add a global lock for the init/deinit stuff
    - BUG/MINOR: proxy: always lock stop_proxy()
    - MEDIUM: mux-h1: Add the support of headers adjustment for bogus HTTP/1 apps
    - BUILD: threads: add the definition of PROTO_LOCK
    - BUG/MEDIUM: lb-chash: Fix the realloc() when the number of nodes is increased
    - BUG/MEDIUM: streams: Don't switch the SI to SI_ST_DIS if we have data to send.
    - BUG/MINOR: log: make sure writev() is not interrupted on a file output
    - DOC: improve the wording in CONTRIBUTING about how to document a bug fix
    - MEDIUM: h1: Don't try to subscribe if we managed to read data.
    - MEDIUM: h1: Don't wake the H1 tasklet if we got the whole request.
    - REGTESTS: checks: exclude freebsd target for tcp-check_multiple_ports.vtc
    - BUG/MINOR: hlua/htx: Reset channels analyzers when txn:done() is called
    - BUG/MEDIUM: hlua: Check the calling direction in lua functions of the HTTP class
    - MINOR: hlua: Don't set request analyzers on response channel for lua actions
    - MINOR: hlua: Add a flag on the lua txn to know in which context it can be used
    - BUG/MINOR: hlua: Only execute functions of HTTP class if the txn is HTTP ready
    - BUG/MINOR: htx: Fix free space addresses calculation during a block expansion
    - MINOR: ssl: merge ssl_sock_load_cert_file() and ssl_sock_load_cert_chain_file()
    - MEDIUM: ssl: use cert_key_and_chain struct in ssl_sock_load_cert_file()
    - MEDIUM: ssl: split the loading of the certificates
    - MEDIUM: ssl: lookup and store in a ckch_node tree
    - MEDIUM: ssl: load DH param in struct cert_key_and_chain
    - BUG/MAJOR: queue/threads: avoid an AB/BA locking issue in process_srv_queue()
    - MINOR: ssl: use STACK_OF for chain certs
    - MINOR: ssl: add extra chain compatibility
    - MINOR: ssl: check private key consistency in loading
    - MINOR: ssl: do not look at DHparam with OPENSSL_NO_DH
    - CLEANUP: ssl: ssl_sock_load_crt_file_into_ckch
    - MINOR: ssl: clean ret variable in ssl_sock_load_ckchn
    - MAJOR: fd: Get rid of the fd cache.
    - MEDIUM: pollers: Remember the state for read and write for each threads.
    - MEDIUM: mux-h2: don't try to read more than needed
    - BUG/BUILD: ssl: fix build with openssl < 1.0.2
    - BUG/MEDIUM: ssl: does not try to free a DH in a ckch
    - BUG/MINOR: debug: fix a small race in the thread dumping code
    - MINOR: wdt: also consider that waiting in the thread dumper is normal
    - REGTESTS: checks: make 4be_1srv_health_checks more reliable
    - BUILD: ssl: BoringSSL add EVP_PKEY_base_id
    - BUG/MEDIUM: ssl: don't free the ckch in multi-cert bundle
    - BUG/MINOR: ssl: fix ressource leaks on error
    - BUG/MEDIUM: lb-chash: Ensure the tree integrity when server weight is increased
    - BUG/MAJOR: http/sample: use a static buffer for raw -> htx conversion
    - BUG/MINOR: stream-int: make sure to always release empty buffers after sending
    - BUG/MEDIUM: ssl: open the right path for multi-cert bundle
    - BUG/MINOR: stream-int: also update analysers timeouts on activity
    - BUG/MEDIUM: mux-h2: unbreak receipt of large DATA frames
    - BUG/MEDIUM: mux-h2: split the stream's and connection's window sizes
    - BUG/MEDIUM: proxy: Make sure to destroy the stream on upgrade from TCP to H2
    - DOC: Add 'Question.md' issue template, discouraging asking questions
    - BUG/MEDIUM: fd: Always reset the polled_mask bits in fd_dodelete().
    - BUG/MEDIUM: pollers: Clear the poll_send bits as well.
    - BUILD: travis-ci: enable daily Coverity scan
    - BUG/MINOR: mux-h2: don't refrain from sending an RST_STREAM after another one
    - BUG/MINOR: mux-h2: use CANCEL, not STREAM_CLOSED in h2c_frt_handle_data()
    - BUG/MINOR: mux-h2: do not send REFUSED_STREAM on aborted uploads
    - BUG/MEDIUM: mux-h2: do not recheck a frame type after a state transition
    - BUG/MINOR: mux-h2: always send stream window update before connection's
    - BUG/MINOR: mux-h2: always reset rcvd_s when switching to a new frame
    - BUG/MEDIUM: checks: make sure to close nicely when we're the last to speak
    - BUG/MEDIUM: stick-table: Wrong stick-table backends parsing.
    - CLEANUP: mux-h2: move the demuxed frame check code in its own function
    - MINOR: cache: add method to cache hash
    - MINOR: cache: allow caching of OPTIONS request
    - BUG/MINOR: ssl: fix 0-RTT for BoringSSL
    - MINOR: ssl: ssl_fc_has_early should work for BoringSSL
    - BUG/MINOR: pools: don't mark the thread harmless if already isolated
    - BUG/MINOR: buffers/threads: always clear a buffer's head before releasing it
    - CLEANUP: buffer: replace b_drop() with b_free()
    - CLEANUP: task: move the cpu_time field to the task-only part
    - MINOR: cli: add two new states to print messages on the CLI
    - MINOR: cli: add cli_msg(), cli_err(), cli_dynmsg(), cli_dynerr()
    - CLEANUP: cli: replace all occurrences of manual handling of return messages
    - BUG/MEDIUM: proxy: Don't forget the SF_HTX flag when upgrading TCP=>H1+HTX.
    - BUG/MEDIUM: proxy: Don't use cs_destroy() when freeing the conn_stream.
    - BUG/MINOR: lua: fix setting netfilter mark
    - BUG/MINOR: Fix prometheus '# TYPE' and '# HELP' headers
    - BUG/MEDIUM: lua: Fix test on the direction to set the channel exp timeout
    - BUG/MINOR: stats: Wait the body before processing POST requests
    - MINOR: fd: make sure to mark the thread as not stuck in fd_update_events()
    - BUG/MEDIUM: mux_pt: Don't call unsubscribe if we did not subscribe.
    - BUILD: travis-ci: trigger non-mainstream configurations only on daily builds.
    - MINOR: debug: indicate the applet name when the task is task_run_applet()
    - MINOR: tools: add append_prefixed_str()
    - MINOR: lua: export applet and task handlers
    - MEDIUM: debug: make the thread dump code show Lua backtraces
    - BUG/MEDIUM: h1: Always try to receive more in h1_rcv_buf().
    - MINOR: list: add LIST_SPLICE() to merge one list into another
    - MINOR: tools: add a DEFNULL() macro to use NULL for empty args
    - REORG: trace: rename trace.c to calltrace.c and mention it's not thread-safe
    - MINOR: sink: create definitions a minimal code for event sinks
    - MINOR: sink: add a support for file descriptors
    - MINOR: trace: start to create a new trace subsystem
    - MINOR: trace: add allocation of buffer-sized trace buffers
    - MINOR: trace/cli: register the "trace" CLI keyword to list the sources
    - MINOR: trace/cli: parse the "level" argument to configure the trace verbosity
    - MINOR: trace/cli: add "show trace" to report trace state and statistics
    - MINOR: trace: implement a very basic trace() function
    - MINOR: trace: add the file name and line number in the prefix
    - MINOR: trace: make trace() now also take a level in argument
    - MINOR: trace: implement a call to a decode function
    - MINOR: trace: add per-level macros to produce traces
    - MINOR: trace: add a definition of typed arguments to trace()
    - MINOR: trace: make sure to always stop the locking when stopping or pausing
    - MINOR: trace: add the possibility to lock on some arguments
    - MINOR: trace: parse the "lock" argument to trace
    - MINOR: trace: retrieve useful pointers and enforce lock-on
    - DOC: management: document the "trace" and "show trace" commands
    - BUILD: trace: make the lockon_ptr const to silence a warning without threads
    - BUG/MEDIUM: mux-h1: do not truncate trailing 0CRLF on buffer boundary
    - BUG/MEDIUM: mux-h1: do not report errors on transfers ending on buffer full
    - DOC: fixed typo in management.txt
    - BUG/MINOR: mworker: disable SIGPROF on re-exec
    - BUG/MEDIUM: listener/threads: fix an AB/BA locking issue in delete_listener()
    - BUG/MEDIUM: url32 does not take the path part into account in the returned hash.
    - MINOR: backend: Add srv_queue converter
    - MINOR: sink: set the fd-type sinks to non-blocking
    - MINOR: tools: add a function varint_bytes() to report the size of a varint
    - MINOR: buffer: add functions to read/write varints from/to buffers
    - MINOR: fd: add fd_write_frag_line() to send a fragmented line to an fd
    - MINOR: sink: now call the generic fd write function
    - MINOR: ring: add a new mechanism for retrieving/storing ring data in buffers
    - MINOR: ring: add a ring_write() function
    - MINOR: ring: add a generic CLI io_handler to dump a ring buffer
    - MINOR: sink: add support for ring buffers
    - MINOR: sink: implement "show events" to show supported sinks and dump the rings
    - MINOR: sink: now report the number of dropped events on output
    - MINOR: trace: support a default callback for the source
    - MINOR: trace: extend the source location to 13 chars
    - MINOR: trace: show thread number and source name in the trace
    - MINOR: trace: change the TRACE() calling convention to put the args and cb last
    - MINOR: connection: add the fc_pp_authority fetch -- authority TLV, from PROXYv2
    - MINOR: tools: add a generic struct "name_desc" for name-description pairs
    - MINOR: trace: replace struct trace_lockon_args with struct name_desc
    - MINOR: trace: change the "payload" level to "data" and move it
    - MINOR: trace: prepend the function name for developer level traces
    - MINOR: trace: also report the trace level in the output
    - MINOR: trace: change the detail_level to per-source verbosity
    - MINOR: mux-h2/trace: register a new trace source with its events
    - MINOR: mux-h2/trace: add the default decoding callback
    - MEDIUM: mux-h2/trace: add lots of traces all over the code
    - MINOR: mux-h2: add functions to convert an h2c/h2s state to a string
    - MINOR: mux-h2/trace: add a new verbosity level "clean"
    - MINOR: mux-h2/trace: only decode the start-line at verbosity other than "minimal"
    - MINOR: mux-h2/trace: always report the h2c/h2s state and flags
    - MINOR: mux-h2/trace: report h2s->id before h2c->dsi for the stream ID
    - CLEANUP: mux-h2/trace: reformat the "received" messages for better alignment
    - CLEANUP: mux-h2/trace: lower-case event names
    - MINOR: trace: extend default event names to 12 chars
    - BUG/MINOR: ring: fix the way watchers are counted
    - MINOR: cli: extend the CLI context with a list and two offsets
    - MINOR: mux-h2/trace: report the connection pointer and state before FRAME_H
    - MEDIUM: ring: implement a wait mode for watchers
    - BUG/MEDIUM: mux-h2/trace: do not dereference h2c->conn after failed idle
    - BUG/MEDIUM: mux-h2/trace: fix missing braces added with traces
    - BUG/MINOR: ring: b_peek_varint() returns a uint64_t, not a size_t
    - CLEANUP: fd: remove leftovers of the fdcache
    - MINOR: fd: add a new "initialized" bit in the fdtab struct
    - MINOR: fd/log/sink: make the non-blocking initialization depend on the initialized bit
    - MEDIUM: log: use the new generic fd_write_frag_line() function
    - MINOR: log: add a target type instead of hacking the address family
    - MEDIUM: log: add support for logging to a ring buffer
    - MINOR: send-proxy-v2: sends authority TLV according to TLV received
    - MINOR: build: add linux-glibc-legacy build TARGET
    - BUG/MEDIUM: peers: local peer socket not bound.
    - BUILD: connection: silence gcc warning with extra parentheses
    - BUG/MINOR: http-ana: Reset response flags when 1xx messages are handled
    - BUG/MINOR: h1: Properly reset h1m when parsing is restarted
    - BUG/MINOR: mux-h1: Fix size evaluation of HTX messages after headers parsing
    - BUG/MINOR: mux-h1: Don't stop anymore input processing when the max is reached
    - BUG/MINOR: mux-h1: Be sure to update the count before adding EOM after trailers
    - BUG/MEDIUM: cache: Properly copy headers splitted on several shctx blocks
    - BUG/MEDIUM: cache: Don't cache objects if the size of headers is too big
    - BUG/MINOR: mux-h1: Fix a possible null pointer dereference in h1_subscribe()
    - MEDIUM: fd: remove the FD_EV_POLLED status bit
    - MEDIUM: fd: simplify the fd_*_{recv,send} functions using BTS/BTR
    - MINOR: fd: make updt_fd_polling() a normal function
    - CONTRIB: debug: add new program "poll" to test poll() events
    - BUG/MINOR: checks: stop polling for write when we have nothing left to send
    - BUG/MINOR: checks: start sending the request right after connect()
    - BUG/MINOR: checks: make __event_chk_srv_r() report success before closing
    - BUG/MINOR: checks: do not uselessly poll for reads before the connection is up
    - BUG/MINOR: mux-h1: Fix a UAF in cfg_h1_headers_case_adjust_postparser()
    - BUILD: CI: add basic CentOS 6 cirrus build
    - MINOR: contrib/prometheus-exporter: Report DRAIN/MAINT/NOLB status for servers
    - BUG/MINOR: lb/leastconn: ignore the server weights for empty servers
    - BUG/MAJOR: ssl: ssl_sock was not fully initialized.
    - MEDIUM: fd: mark the FD as ready when it's inserted
    - MINOR: fd: add two new calls fd_cond_{recv,send}()
    - MEDIUM: connection: enable reading only once the connection is confirmed
    - MINOR: fd: add two flags ERR and SHUT to describe FD states
    - MEDIUM: fd: do not use the FD_POLL_* flags in the pollers anymore
    - BUG/MEDIUM: connection: don't keep more idle connections than ever needed
    - MINOR: stats: report the number of idle connections for each server
    - BUILD: CI: skip reg-tests/connection/proxy_protocol_random_fail.vtc on CentOS 6
    - BUILD/MINOR: auth: enabling for osx
    - BUG/MINOR: listener: Fix a possible null pointer dereference
    - BUG/MINOR: ssl: always check for ssl connection before getting its XPRT context
    - MINOR: stats: Add JSON export from the stats page
    - BUG/MINOR: filters: Properly set the HTTP status code on analysis error
    - MINOR: sample: Add UUID-fetch
    - CLEANUP: mux-h2: Remove unused flag H2_SF_DATA_CHNK
    - BUG/MINOR: acl: Fix memory leaks when an ACL expression is parsed
    - BUG/MINOR: backend: Fix a possible null pointer dereference
    - BUG/MINOR: Missing stat_field_names (since f21d17bb)
    - BUG/MEDIUM: stick-table: Properly handle "show table" with a data type argument
    - BUILD: CI: temporarily disable ASAN
    - MINOR: htx: Add a flag on HTX message to report processing errors
    - MINOR: mux-h1: Report a processing error during output processing
    - MINOR: http-ana: Handle HTX errors first during message analysis
    - MINOR: http-ana: Remove err_state field from http_msg
    - MINOR: config: Support per-proxy and per-server deinit functions callbacks
    - MINOR: config: Support per-proxy and per-server post-check functions callbacks
    - MINOR: http_fetch: Add sample fetches to get auth method/user/pass
    - MINOR: istbuf: Add the function b_isteqi()
    - MINOR: log: Provide a function to emit a log for an application
    - MINOR: http: Add function to parse value of the header Status
    - MEDIUM: mux-h1/h1-htx: move HTX convertion of H1 messages in dedicated file
    - MINOR: h1-htx: Use the same function to copy message payload in all cases
    - MINOR: muxes/htx: Ignore pseudo header during message formatting
    - MINOR: fcgi: Add code related to FCGI protocol
    - MEDIUM: fcgi-app: Add FCGI application and filter
    - MEDIUM: mux-fcgi: Add the FCGI multiplexer
    - MINOR: doc: Add documentation about the FastCGI support
    - BUG/MINOR: build: Fix compilation of mux_fcgi.c when compiled without SSL
    - BUILD: CI: install golang-1.13 when building BoringSSL
    - BUG/MINOR: mux-h2: Be sure to have a connection to unsubcribe
    - BUG/MINOR: mux-fcgi: Be sure to have a connection to unsubcribe
    - CLEANUP: fcgi-app: Remove useless test on fcgi_conf pointer
    - BUG/MINOR: mux-fcgi: Don't compare the filter name in its parsing callback
    - BUG/MAJOR: mux-h2: Handle HEADERS frames received after a RST_STREAM frame
    - BUG/MEDIUM: check/threads: make external checks run exclusively on thread 1
    - MEDIUM: list: Separate "locked" list from regular list.
    - MINOR: mt_lists: Add new macroes.
    - MEDIUM: servers: Use LIST_DEL_INIT() instead of LIST_DEL().
    - MINOR: mt_lists: Do nothing in MT_LIST_ADD/MT_LIST_ADDQ if already in list.
    - MINOR: mt_lists: Give MT_LIST_ADD, MT_LIST_ADDQ and MT_LIST_DEL a return value.
    - MEDIUM: tasklets: Make the tasklet list a struct mt_list.
    - TESTS: Add a stress-test for mt_lists.
    - BUILD: travis-ci: add PCRE2, SLZ build
    - BUG/MINOR: build: fix event ports (Solaris)
    - BUG/MEDIUM: namespace: fix fd leak in master-worker mode
    - OPTIM: listeners: use tasklets for the multi-queue rings
    - BUILD: makefile: work around yet another GCC fantasy (-Wstring-plus-int)
    - BUG/MINOR: stream-int: Process connection/CS errors first in si_cs_send()
    - BUG/MEDIUM: stream-int: Process connection/CS errors during synchronous sends
    - BUG/MEDIUM: checks: make sure the connection is ready before trying to recv
    - CLEANUP: task: remove impossible test
    - CLEANUP: task: cache the task_per_thread pointer
    - MINOR: task: split the tasklet vs task code in process_runnable_tasks()
    - MINOR: task: introduce a thread-local "sched" variable for local scheduler stuff
    - CLEANUP: mux-fcgi: Remove the unused function fcgi_strm_id()
    - BUG/MINOR: mux-fcgi: Use a literal string as format in app_log()
    - BUG/MEDIUM: tasklets: Make sure we're waking the target thread if it sleeps.
    - MINOR: h2/trace: indicate 'F' or 'B' to locate the side of an h2c in traces
    - MINOR: h2/trace: report the frame type when known
    - BUG/MINOR: mux-h2: do not wake up blocked streams before the mux is ready
    - BUG/MEDIUM: namespace: close open namespaces during soft shutdown
    - MINOR: time: add timeofday_as_iso_us() to return instant time as ISO
    - MINOR: sink: finally implement support for SINK_FMT_{TIMED,ISO}
    - MINOR: sink: change ring buffer "buf0"'s format to "timed"
    - BUG/MEDIUM: mux-h2: don't reject valid frames on closed streams
    - BUG/MINOR: mux-fcgi: silence a gcc warning about null dereference
    - BUG/MINOR: mux-h2: Fix missing braces because of traces in h2_detach()
    - BUG/MINOR: mux-h2: Use the dummy error when decoding headers for a closed stream
    - BUG/MAJOR: mux_h2: Don't consume more payload than received for skipped frames
    - BUG/MINOR: mux-h1: Do h2 upgrade only on the first request
    - BUG/MEDIUM: spoe: Use a different engine-id per process
    - MINOR: spoe: Improve generation of the engine-id
    - MINOR: spoe: Support the async mode with several threads
    - MINOR: http: Add server name header from HTTP multiplexers
    - CLEANUP: http-ana: Remove the unused function http_send_name_header()
    - MINOR: stats: Add the support of float fields in stats
    - BUG/MINOR: contrib/prometheus-exporter: Return the time averages in seconds
    - DOC: Fix documentation about the cli command to get resolver stats
    - BUG/MEDIUM: fcgi: fix missing list tail in sample fetch registration
    - BUG/MINOR: stats: Add a missing break in a switch statement
    - BUG/MINOR: lua: Properly initialize the buffer's fields for string samples in hlua_lua2(smp|arg)
    - CLEANUP: lua: Get rid of obsolete (size_t *) cast in hlua_lua2(smp|arg)
    - BUG/MEDIUM: lua: Store stick tables into the sample's `t` field
    - CLEANUP: proxy: Remove `proxy_tbl_by_name`
    - BUILD: ssl: fix a warning when built with openssl < 1.0.2
    - DOC: replace utf-8 quotes by ascii ones
    - BUG/MEDIUM: fd: HUP is an error only when write is active
    - BUG/MINOR: action: do-resolve does not yield on requests with body
    - Revert "MINOR: cache: allow caching of OPTIONS request"
2019-10-01 18:13:09 +02:00
William Lallemand
93e548ea26 DOC: replace utf-8 quotes by ascii ones
947771b ("MINOR: doc: Add documentation about the FastCGI support")
introduced some UTF-8 in configuration.txt.

Replace the UTF-8 quotes by ASCII.
2019-09-30 13:57:10 +02:00
Christopher Faulet
b30b310f68 MINOR: doc: Add documentation about the FastCGI support 2019-09-17 10:38:35 +02:00
Christopher Faulet
a406356255 MINOR: http_fetch: Add sample fetches to get auth method/user/pass
Now, following sample fetches may be used to get information about
authentication:

 * http_auth_type : returns the auth method as supplied in Authorization header
 * http_auth_user : returns the auth user as supplied in Authorization header
 * http_auth_pass : returns the auth pass as supplied in Authorization header

Only Basic authentication is supported.
2019-09-17 10:18:54 +02:00
Luca Schimweg
8a694b859c MINOR: sample: Add UUID-fetch
Adds the fetch uuid(int). It returns a UUID following the format of
version 4 in the RFC4122 standard.

New feature, but could be backported.
2019-09-13 04:43:33 +02:00
Willy Tarreau
c046d167e4 MEDIUM: log: add support for logging to a ring buffer
Now by prefixing a log server with "ring@<name>" it's possible to send
the logs to a ring buffer. One nice thing is that it allows multiple
sessions to consult the logs in real time in parallel over the CLI, and
without requiring file system access. At the moment, ring0 is created as
a default sink for tracing purposes and is available. No option is
provided to create new rings though this is trivial to add to the global
section.
2019-08-30 15:24:59 +02:00
Geoff Simmons
7185b789f9 MINOR: connection: add the fc_pp_authority fetch -- authority TLV, from PROXYv2
Save the authority TLV in a PROXYv2 header from the client connection,
if present, and make it available as fc_pp_authority.

The fetch can be used, for example, to set the SNI for a backend TLS
connection.
2019-08-28 17:16:20 +02:00
Nenad Merdanovic
177adc9e57 MINOR: backend: Add srv_queue converter
The converter can be useful to look up a server queue from a dynamic value.

It takes an input value of type string, either a server name or
<backend>/<server> format and returns the number of queued sessions
on that server. Can be used in places where we want to look up
queued sessions from a dynamic name, like a cookie value (e.g.
req.cook(SRVID),srv_queue) and then make a decision to break
persistence or direct a request elsewhere.

Signed-off-by: Nenad Merdanovic <nmerdan@haproxy.com>
2019-08-27 04:32:06 +02:00
Christopher Faulet
98fbe9531a MEDIUM: mux-h1: Add the support of headers adjustment for bogus HTTP/1 apps
There is no standard case for HTTP header names because, as stated in the
RFC7230, they are case-insensitive. So applications must handle them in a
case-insensitive manner. But some bogus applications erroneously rely on the
case used by most browsers. This problem becomes critical with HTTP/2
because all header names must be exchanged in lowercase. And HAProxy uses the
same convention. All header names are sent in lowercase to clients and servers,
regardless of the HTTP version.

This design choice is linked to the HTX implementation. So, for previous
versions (2.0 and 1.9), a workaround is to disable the HTX mode to fall
back to the legacy HTTP mode.

Since the legacy HTTP mode was removed, some users reported interoperability
issues because their application was not able anymore to handle HTTP/1 message
received from HAProxy. So, we've decided to add a way to change the case of some
headers before sending them. It is now possible to define a "mapping" between a
lowercase header name and a version supported by the bogus application. To do
so, you must use the global directives "h1-case-adjust" and
"h1-case-adjust-file". Then options "h1-case-adjust-bogus-client" and
"h1-case-adjust-bogus-server" may be used in proxy sections to enable the
conversion. See the configuration manual for more info.

Of course, our advice is to urgently upgrade these applications for
interoperability concerns and because they may be vulnerable to various types of
content smuggling attacks. But, if your are really forced to use an unmaintained
bogus application, you may use these directive, at your own risks.

If it is relevant, this feature may be backported to 2.0.
2019-07-24 18:32:47 +02:00
Christopher Faulet
87f1f3d60b DOC: config: Remove unsupported req* and rsp* keywords
As a result, the section 6 ( HTTP header manipulation) was removed and replace
by the section 10 (Cache).
2019-07-19 09:24:12 +02:00
Christopher Faulet
159e667256 DOC: config: Update as a result of the legacy HTTP removal
Doc about the options 'http-tunnel' and 'http-use-htx' were removed.
2019-07-19 09:24:12 +02:00
Willy Tarreau
db5140741d [RELEASE] Released version 2.1-dev1
Released version 2.1-dev1 with the following main changes :
    - BUG/MEDIUM: h2/htx: Update data length of the HTX when the cookie list is built
    - DOC: this is a development branch again.
    - MEDIUM: Make 'block' directive fatal
    - MEDIUM: Make 'redispatch' directive fatal
    - MEDIUM: Make '(cli|con|srv)timeout' directive fatal
    - MEDIUM: Remove 'option independant-streams'
    - MINOR: sample: Add sha2([<bits>]) converter
    - MEDIUM: server: server-state global file stored in a tree
    - BUG/MINOR: lua/htx: Make txn.req_req_* and txn.res_rep_* HTX aware
    - BUG/MINOR: mux-h1: Add the header connection in lower case in outgoing messages
    - BUG/MEDIUM: compression: Set Vary: Accept-Encoding for compressed responses
    - MINOR: htx: Add the function htx_change_blk_value_len()
    - BUG/MEDIUM: htx: Fully update HTX message when the block value is changed
    - BUG/MEDIUM: mux-h2: Reset padlen when several frames are demux
    - BUG/MEDIUM: mux-h2: Remove the padding length when a DATA frame size is checked
    - BUG/MEDIUM: lb_fwlc: Don't test the server's lb_tree from outside the lock
    - BUG/MAJOR: sample: Wrong stick-table name parsing in "if/unless" ACL condition.
    - BUILD: mworker: silence two printf format warnings around getpid()
    - BUILD: makefile: use :space: instead of digits to count commits
    - BUILD: makefile: adjust the sed expression of "make help" for solaris
    - BUILD: makefile: do not rely on shell substitutions to determine git version
    - BUG/MINOR: mworker-prog: Fix segmentation fault during cfgparse
    - BUG/MINOR: spoe: Fix memory leak if failing to allocate memory
    - BUG/MEDIUM: mworker: don't call the thread and fdtab deinit
    - BUG/MEDIUM: stream_interface: Don't add SI_FL_ERR the state is < SI_ST_CON.
    - BUG/MEDIUM: connections: Always add the xprt handshake if needed.
    - BUG/MEDIUM: ssl: Don't do anything in ssl_subscribe if we have no ctx.
    - BUG/MEDIUM: mworker/cli: command pipelining doesn't work anymore
    - BUG/MINOR: htx: Save hdrs_bytes when the HTX start-line is replaced
    - BUG/MAJOR: mux-h1: Don't crush trash chunk area when outgoing message is formatted
    - BUG/MINOR: memory: Set objects size for pools in the per-thread cache
    - BUG/MINOR: log: Detect missing sampling ranges in config
    - BUG/MEDIUM: proto_htx: Don't add EOM on 1xx informational messages
    - BUG/MEDIUM: mux-h1: Use buf_room_for_htx_data() to detect too large messages
    - BUG/MINOR: mux-h1: Make format errors during output formatting fatal
    - BUG/MEDIUM: ssl: Don't attempt to set alpn if we're not using SSL.
    - BUG/MEDIUM: mux-h1: Always release H1C if a shutdown for writes was reported
    - BUG/MINOR: mworker/cli: don't output a \n before the response
    - BUG/MEDIUM: checks: unblock signals in external checks
    - BUG/MINOR: mux-h1: Skip trailers for non-chunked outgoing messages
    - BUG/MINOR: mux-h1: Don't return the empty chunk on HEAD responses
    - BUG/MEDIUM: connections: Always call shutdown, with no linger.
    - BUG/MEDIUM: checks: Make sure the tasklet won't run if the connection is closed.
    - BUG/MINOR: contrib/prometheus-exporter: Don't use channel_htx_recv_max()
    - BUG/MINOR: hlua: Don't use channel_htx_recv_max()
    - BUG/MEDIUM: channel/htx: Use the total HTX size in channel_htx_recv_limit()
    - BUG/MINOR: hlua/htx: Respect the reserve when HTX data are sent
    - BUG/MINOR: contrib/prometheus-exporter: Respect the reserve when data are sent
    - BUG/MEDIUM: connections: Make sure we're unsubscribe before upgrading the mux.
    - BUG/MEDIUM: servers: Authorize tfo in default-server.
    - BUG/MEDIUM: sessions: Don't keep an extra idle connection in sessions.
    - MINOR: server: Add "no-tfo" option.
    - BUG/MINOR: contrib/prometheus-exporter: Don't try to add empty data blocks
    - MINOR: action: Add the return code ACT_RET_DONE for actions
    - BUG/MEDIUM: http/applet: Finish request processing when a service is registered
    - BUG/MEDIUM: lb_fas: Don't test the server's lb_tree from outside the lock
    - BUG/MEDIUM: mux-h1: Handle TUNNEL state when outgoing messages are formatted
    - BUG/MINOR: mux-h1: Don't process input or ouput if an error occurred
    - MINOR: stream-int: Factorize processing done after sending data in si_cs_send()
    - BUG/MEDIUM: stream-int: Don't rely on CF_WRITE_PARTIAL to unblock opposite si
    - DOC: contrib: spoa_server Add some hints for building spoa_server
    - DOC: Fix typo in intro.txt
    - BUG/MEDIUM: servers: Don't forget to set srv_cs to NULL if we can't reuse it.
    - BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2
    - MINOR: pools: release the pool's lock during the malloc/free calls
    - MINOR: pools: always pre-initialize allocated memory outside of the lock
    - MINOR: pools: make the thread harmless during the mmap/munmap syscalls
    - BUG/MEDIUM: fd/threads: fix excessive CPU usage on multi-thread accept
    - BUG/MINOR: server: Be really able to keep "pool-max-conn" idle connections
    - BUG/MEDIUM: checks: Don't attempt to read if we destroyed the connection.
    - BUG/MEDIUM: da: cast the chunk to string.
    - DOC: Fix typos and grammer in configuration.txt
    - CLEANUP: proto_tcp: Remove useless header inclusions.
    - BUG/MEDIUM: servers: Fix a race condition with idle connections.
    - MINOR: task: introduce work lists
    - BUG/MAJOR: listener: fix thread safety in resume_listener()
    - BUG/MEDIUM: mux-h1: Don't release h1 connection if there is still data to send
    - BUG/MINOR: mux-h1: Correctly report Ti timer when HTX and keepalives are used
    - BUG/MEDIUM: streams: Don't give up if we couldn't send the request.
    - BUG/MEDIUM: streams: Don't redispatch with L7 retries if redispatch isn't set.
    - BUG/MINOR: mux-pt: do not pretend there's more data after a read0
    - BUG/MEDIUM: tcp-check: unbreak multiple connect rules again
    - MEDIUM: mworker-prog: Add user/group options to program section
    - REGTESTS: checks: tcp-check connect to multiple ports
    - BUG/MEDIUM: threads: cpu-map designating a single thread/process are ignored
2019-07-16 19:15:28 +02:00
Andrew Heberle
9723696759 MEDIUM: mworker-prog: Add user/group options to program section
This patch adds "user" and "group" config options to the "program"
section so the configured command can be run as a different user.
2019-07-15 16:43:16 +02:00
John Roesler
fb2fce1680 DOC: Fix typos and grammer in configuration.txt 2019-07-11 10:25:53 +02:00
Frédéric Lécaille
1b9423d214 MINOR: server: Add "no-tfo" option.
Simple patch to add "no-tfo" option to "default-server" and "server" lines
to disable any usage of TCP fast open.

Must be backported to 2.0.
2019-07-04 14:45:52 +02:00
Tim Duesterhus
d437630237 MINOR: sample: Add sha2([<bits>]) converter
This adds a converter for the SHA-2 family, supporting SHA-224, SHA-256
SHA-384 and SHA-512.

The converter relies on the OpenSSL implementation, thus only being available
when HAProxy is compiled with USE_OPENSSL.

See GitHub issue #123. The hypothetical `ssl_?_sha256` fetch can then be
simulated using `ssl_?_der,sha2(256)`:

  http-response set-header Server-Cert-FP %[ssl_f_der,sha2(256),hex]
2019-06-17 13:36:42 +02:00
Tim Duesterhus
24915a55da MEDIUM: Remove 'option independant-streams'
It is deprecated with HAProxy 1.5. Time to remove it.
2019-06-17 13:35:54 +02:00
Tim Duesterhus
86e6b6ebf8 MEDIUM: Make '(cli|con|srv)timeout' directive fatal
They were deprecated with HAProxy 1.5. Time to remove them.
2019-06-17 13:35:54 +02:00
Tim Duesterhus
dac168bc15 MEDIUM: Make 'redispatch' directive fatal
It was deprecated with HAProxy 1.5. Time to remove it.
2019-06-17 13:35:54 +02:00
Tim Duesterhus
7b7c47f05c MEDIUM: Make 'block' directive fatal
It was deprecated with HAProxy 1.5. Time to remove it.
2019-06-17 13:35:54 +02:00
Willy Tarreau
9dc6b97429 [RELEASE] Released version 2.1-dev0
Released version 2.1-dev0 with the following main changes :
    - exact copy of 2.0.0
2019-06-16 21:49:47 +02:00
Willy Tarreau
ba23630ad0 [RELEASE] Released version 2.0.0
Released version 2.0.0 with the following main changes :
    - MINOR: fd: Don't use atomic operations when it's not needed.
    - DOC: mworker-prog: documentation for the program section
    - MINOR: http: add a new "http-request replace-uri" action
    - BUG/MINOR: 51d/htx: The _51d_fetch method, and the methods it calls are now HTX aware.
    - MINOR: 51d: Added dummy libraries for the 51Degrees module for testing.
    - MINOR: mworker: change formatting in uptime field of "show proc"
    - MINOR: mworker: add the HAProxy version in "show proc"
    - MINOR: doc: Remove -Ds option in man page
    - MINOR: doc: add master-worker in the man page
    - MINOR: doc: mention HAPROXY_LOCALPEER in the man
    - BUILD: Silence gcc warning about unused return value
    - CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real lib
    - BUILD: travis-ci: add 51Degree device detection, update openssl to 1.1.1c
    - MINOR: doc: update the manpage and usage message about -S
    - BUILD/MINOR: 51d: Updated build registration output to indicate thatif the library is a dummy one or not.
    - BUG/MEDIUM: h1: Don't wait for handshake if we had an error.
    - BUG/MEDIUM: h1: Wait for the connection if the handshake didn't complete.
    - BUG/MINOR: task: prevent schedulable tasks from starving under high I/O activity
    - BUG/MINOR: fl_trace/htx: Be sure to always forward trailers and EOM
    - BUG/MINOR: channel/htx: Call channel_htx_full() from channel_full()
    - BUG/MINOR: http: Use the global value to limit the number of parsed headers
    - BUG/MINOR: htx: Detect when tail_addr meet end_addr to maximize free rooms
    - BUG/MEDIUM: htx: Don't change position of the first block during HTX analysis
    - CLEANUP: channel: Remove channel_htx_fwd_payload() and channel_htx_fwd_all()
    - BUG/MEDIUM: proto_htx: Introduce the state ENDING during forwarding
    - MINOR: htx: Add 3 flags on the start-line to deal with the request schemes
    - MINOR: h2: Set flags about the request's scheme on the start-line
    - MINOR: mux-h1: Set flags about the request's scheme on the start-line
    - MINOR: mux-h2: Forward clients scheme to servers checking start-line flags
    - MEDIUM: server: server-state only rely on server name
    - CLEANUP: connection: rename the wait_event.task field to .tasklet
    - CLEANUP: tasks: rename task_remove_from_tasklet_list() to tasklet_remove_*
    - BUG/MEDIUM: connections: Don't call shutdown() if we want to disable linger.
    - DOC: add some environment variables in section 2.3
    - BUILD: makefile: clarify the "help" output and list options
    - BUG/MINOR: mux-h1: Wake busy mux for I/O when message is fully sent
    - BUG: tasks: fix bug introduced by latest scheduler cleanup
    - BUG/MEDIUM: mux-h2: fix early close with option abortonclose
    - BUG/MEDIUM: connections: Don't use ALPN to pick mux when in mode TCP.
    - BUG/MEDIUM: connections: Don't try to send early data if we have no mux.
    - BUG/MEDIUM: mux-h2: properly account for the appended data in HTX
    - BUILD: makefile: further clarify the "help" output and list targets
    - BUILD: makefile: rename "linux2628" to "linux-glibc" and remove older targets
    - BUILD: travis-ci: switch to linux-glibc instead of linux2628
    - DOC: update few references to the linux* targets and change them to linux-glibc
    - BUILD: makefile: detect and reject recently removed linux targets
    - BUILD: makefile: enable linux namespaces by default on linux
    - BUILD: makefile: enable TFO on linux platforms
    - BUILD: makefile: enable getaddrinfo on the linux-glibc target
    - DOC: small updates to the CONTRIBUTING file
    - BUG/MEDIUM: ssl: Make sure we initiate the handshake after using early data.
    - CLEANUP: removed obsolete examples an move a few to better places
    - DOC: Fix typos in CONTRIBUTING
    - DOC: update the outdated ROADMAP file
    - DOC: create a BRANCHES file to explain the life cycle
    - DOC: mention in INSTALL haproxy 2.0 is a long-term supported stable version
    - BUILD: travis-ci: TFO and GETADDRINFO are now enabled by default
    - BUILD: makefile: make the obsolete target detection compatible with make-3.80
    - BUILD: tools: work around an internal compiler bug in gcc-3.4
    - BUILD: pattern: work around an internal compiler bug in gcc-3.4
    - BUILD: makefile: enable USE_RT on Solaris
    - BUILD: makefile: do not use echo -n
    - DOC: mention a few common build errors in the INSTALL file
2019-06-16 20:00:26 +02:00
William Lallemand
4d03e431b3 DOC: add some environment variables in section 2.3
Add the missing environment variable in the 2.3 section.
2019-06-14 15:38:05 +02:00
Willy Tarreau
3381022d88 MINOR: http: add a new "http-request replace-uri" action
This action is particularly convenient to replace some deprecated usees
of "reqrep". It takes a match and a format string including back-
references. The reqrep warning was updated to suggest it as well.
2019-06-12 18:06:59 +02:00
William Lallemand
c9515529a0 DOC: mworker-prog: documentation for the program section
This patch documents the program feature.
2019-06-12 18:06:59 +02:00
Willy Tarreau
ca3551f005 [RELEASE] Released version 2.0-dev7
Released version 2.0-dev7 with the following main changes :
    - BUG/MEDIUM: mux-h2: make sure the connection timeout is always set
    - MINOR: tools: add new bitmap manipulation functions
    - MINOR: logs: use the new bitmap functions instead of fd_sets for encoding maps
    - MINOR: chunks: Make sure trash_size is only set once.
    - Revert "MINOR: chunks: Make sure trash_size is only set once."
    - MINOR: threads: serialize threads initialization
    - MINOR peers: data structure simplifications for server names dictionary cache.
    - DOC: peers: Update for dictionary cache entries for peers protocol.
    - MINOR: dict: Store the length of the dictionary entries.
    - MINOR: peers: A bit of optimization when encoding cached server names.
    - MINOR: peers: Optimization for dictionary cache lookup.
    - MEDIUM: tools: improve time format error detection
    - BUG/MEDIUM: H1: When upgrading, make sure we don't free the buffer too early.
    - BUG/MEDIUM: stream_interface: Make sure we call si_cs_process() if CS_FL_EOI.
    - MINOR: threads: avoid clearing harmless twice in thread_release()
    - MEDIUM: threads: add thread_sync_release() to synchronize steps
    - BUG/MEDIUM: init/threads: prevent initialized threads from starting before others
    - OPTIM/MINOR: init/threads: only call protocol_enable_all() on first thread
    - BUG/MINOR: dict: race condition fix when inserting dictionary entries.
    - MEDIUM: init/threads: don't use spinlocks during the init phase
    - BUG/MINOR: cache/htx: Fix the counting of data already sent by the cache applet
    - BUG/MEDIUM: compression/htx: Fix the adding of the last data block
    - MINOR: flt_trace: Don't scrash the original offset during the random forwarding
    - MAJOR: htx: Rework how free rooms are tracked in an HTX message
    - MINOR: htx: Add the function htx_move_blk_before()
    - Revert "BUG/MEDIUM: H1: When upgrading, make sure we don't free the buffer too early."
    - BUG/MINOR: http-rules: mention "deny_status" for "deny" in the error message
    - MINOR: http: turn default error files to HTTP/1.1
    - BUG/MEDIUM: h1: Don't try to subscribe if we had a connection error.
    - BUG/MEDIUM: h1: Don't consider we're connected if the handshake isn't done.
    - MINOR: contrib/spoa_server: Upgrade SPOP to 2.0
    - BUG/MEDIUM: contrib/spoa_server: Set FIN flag on agent frames
    - MINOR: contrib/spoa_server: Add random IP score
    - DOC/MINOR: contrib/spoa_server: Fix typo in README
2019-06-11 19:28:00 +02:00
Willy Tarreau
b57f109966 [RELEASE] Released version 2.0-dev6
Released version 2.0-dev6 with the following main changes :
    - BUG/MEDIUM: connection: fix multiple handshake polling issues
    - MINOR: connection: also stop receiving after a SOCKS4 response
    - MINOR: mux-h1: don't try to recv() before the connection is ready
    - BUG/MEDIUM: mux-h1: only check input data for the current stream, not next one
    - MEDIUM: mux-h1: don't use CS_FL_REOS anymore
    - CLEANUP: connection: remove the now unused CS_FL_REOS flag
    - CONTRIB: debug: add 4 missing connection/conn_stream flags
    - MEDIUM: stream: make a full process_stream() loop when completing I/O on exit
    - MINOR: server: increase the default pool-purge-delay to 5 seconds
    - BUILD: tools: do not use the weak attribute for trace() on obsolete linkers
    - BUG/MEDIUM: vars: make sure the scope is always valid when accessing vars
    - BUG/MEDIUM: vars: make the tcp/http unset-var() action support conditions
    - BUILD: task: fix a build warning when threads are disabled
    - CLEANUP: peers: Remove tabs characters.
    - CLEANUP: peers: Replace hard-coded values by macros.
    - BUG/MINOR: peers: Wrong stick-table update message building.
    - MINOR: dict: Add dictionary new data structure.
    - MINOR: peers: Add a LRU cache implementation for dictionaries.
    - MINOR: stick-table: Add "server_name" new data type.
    - MINOR: cfgparse: Space allocation for "server_name" stick-table data type.
    - MINOR: proxy: Add a "server by name" tree to proxy.
    - MINOR: server: Add a dictionary for server names.
    - MINOR: stream: Stickiness server lookup by name.
    - MINOR: peers: Make peers protocol support new "server_name" data type.
    - MINOR: stick-table: Make the CLI stick-table handler support dictionary entry data type.
    - REGTEST: Add a basic server by name stickiness reg test.
    - MINOR: peers: Add dictionary cache information to "show peers" CLI command.
    - MINOR: peers: Replace hard-coded for peer protocol 64-bits value encoding by macros.
    - MINOR: peers: Replace hard-coded values for peer protocol messaging by macros.
    - CLEANUP: ssl: remove unneeded defined(OPENSSL_IS_BORINGSSL)
    - BUILD: travis-ci improvements
    - MINOR: SSL: add client/server random sample fetches
    - BUG/MINOR: channel/htx: Don't alter channel during forward for empty HTX message
    - BUG/MINOR: contrib/prometheus-exporter: Add HTX data block in one time
    - BUG/MINOR: mux-h1: errflag must be set on H1S and not H1M during output processing
    - MEDIUM: mux-h1: refactor output processing
    - MINOR: mux-h1: Add the flag HAVE_O_CONN on h1s
    - MINOR: mux-h1: Add h1_eval_htx_hdrs_size() to estimate size of the HTX headers
    - MINOR: mux-h1: Don't count the EOM in the estimated size of headers
    - MEDIUM: cache/htx: Always store info about HTX blocks in the cache
    - MEDIUM: htx: Add the parsing of trailers of chunked messages
    - MINOR: htx: Don't use end-of-data blocks anymore
    - BUG/MINOR: mux-h1: Don't send more data than expected
    - BUG/MINOR: flt_trace/htx: Only apply the random forwarding on the message body.
    - BUG/MINOR: peers: Wrong "server_name" decoding.
    - BUG/MEDIUM: servers: Don't attempt to destroy idle connections if disabled.
    - MEDIUM: checks: Make sure we unsubscribe before calling cs_destroy().
    - MEDIUM: connections: Wake the upper layer even if sending/receiving is disabled.
    - MEDIUM: ssl: Handle subscribe by itself.
    - MINOR: ssl: Make ssl_sock_handshake() static.
    - MINOR: connections: Add a new xprt method, remove_xprt.
    - MINOR: connections: Add a new xprt method, add_xprt().
    - MEDIUM: connections: Introduce a handshake pseudo-XPRT.
    - MEDIUM: connections: Remove CONN_FL_SOCK*
    - BUG/MEDIUM: ssl: Don't forget to initialize ctx->send_recv and ctx->recv_wait.
    - BUG/MINOR: peers: Wrong server name parsing.
    - MINOR: server: really increase the pool-purge-delay default to 5 seconds
    - BUG/MINOR: stream: don't emit a send-name-header in conn error or disconnect states
    - MINOR: stream-int: use bit fields to match multiple stream-int states at once
    - MEDIUM: stream-int: remove dangerous interval checks for stream-int states
    - MEDIUM: stream-int: introduce a new state SI_ST_RDY
    - MAJOR: stream-int: switch from SI_ST_CON to SI_ST_RDY on I/O
    - MEDIUM: stream-int: make idle-conns switch to ST_RDY
    - MEDIUM: stream: re-arrange the connection setup status reporting
    - MINOR: stream-int: split si_update() into si_update_rx() and si_update_tx()
    - MINOR: stream-int: make si_sync_send() from the send code of si_update_both()
    - MEDIUM: stream: rearrange the events to remove the loop
    - MEDIUM: stream: only loop on flags relevant to the analysers
    - MEDIUM: stream: don't abusively loop back on changes on CF_SHUT*_NOW
    - BUILD: stream-int: avoid a build warning in dev mode in si_state_bit()
    - BUILD: peers: fix a build warning about an incorrect intiialization
    - BUG/MINOR: time: make sure only one thread sets global_now at boot
    - BUG/MEDIUM: tcp: Make sure we keep the polling consistent in tcp_probe_connect.
2019-06-07 06:12:59 +02:00
Patrick Hemmer
65674662b4 MINOR: SSL: add client/server random sample fetches
This adds 4 sample fetches:
- ssl_fc_client_random
- ssl_fc_server_random
- ssl_bc_client_random
- ssl_bc_server_random

These fetches retrieve the client or server random value sent during the
handshake.

Their use is to be able to decrypt traffic sent using ephemeral ciphers. Tools
like wireshark expect a TLS log file with lines in a few known formats
(https://code.wireshark.org/review/gitweb?p=wireshark.git;a=blob;f=epan/dissectors/packet-tls-utils.c;h=28a51fb1fb029eae5cea52d37ff5b67d9b11950f;hb=HEAD#l5209).
Previously the only format supported using data retrievable from HAProxy state
was the one utilizing the Session-ID. However an SSL/TLS session ID is
optional, and thus cannot be relied upon for this purpose.

This change introduces the ability to extract the client random instead which
can be used for one of the other formats. The change also adds the ability to
extract the server random, just in case it might have some other use, as the
code change to support this was trivial.
2019-06-05 10:07:44 +02:00
Willy Tarreau
fb55365f9e MINOR: server: increase the default pool-purge-delay to 5 seconds
The default used to be a very aggressive delay of 1 second before starting
to purge idle connections, but tests show that with bursty traffic it's a
bit short. Let's increase this to 5 seconds.
2019-06-04 14:06:31 +02:00
Willy Tarreau
abc874ea45 [RELEASE] Released version 2.0-dev5
Released version 2.0-dev5 with the following main changes :
    - BUILD: watchdog: use si_value.sival_int, not si_int for the timer's value
    - BUILD: signals: FreeBSD has SI_LWP instead of SI_TKILL
    - BUILD: watchdog: condition it to USE_RT
    - MINOR: raw_sock: report global traffic statistics
    - MINOR: stats: report the global output bit rate in human readable form
    - BUG/MINOR: proto-htx: Try to keep connections alive on redirect
    - BUG/MEDIUM: spoe: Don't use the SPOE applet after releasing it
    - BUG/MINOR: lua: Set right direction and flags on new HTTP objects
    - BUG/MINOR: mux-h2: Count EOM in bytes sent when a HEADERS frame is formatted
    - BUG/MINOR: mux-h1: Report EOI instead EOS on parsing error or H2 upgrade
    - BUG/MEDIUM: proto-htx: Not forward too much data when 1xx reponses are handled
    - BUG/MINOR: htx: Remove a forgotten while loop in htx_defrag()
    - DOC: fix typos
    - BUG/MINOR: ssl_sock: Fix memory leak when disabling compression
    - OPTIM: freq-ctr: don't take the date lock for most updates
    - MEDIUM: mux-h2: avoid doing expensive buffer realigns when not absolutely needed
    - CLEANUP: debug: remove the TRACE() macro
    - MINOR: buffer: introduce b_make() to make a buffer from its parameters
    - MINOR: buffer: add a new buffer ring API to manipulate rings of buffers
    - MEDIUM: mux-h2: replace all occurrences of mbuf with a buffer ring
    - MEDIUM: mux-h2: make the conditions to send based on mbuf, not just its tail
    - MINOR: mux-h2: introduce h2_release_mbuf() to release all buffers in the mbuf ring
    - MEDIUM: mux-h2: make the send() function iterate over all mux buffers
    - CLEANUP: mux-h2: consistently use a local variable for the mbuf
    - MINOR: mux-h2: report the mbuf's head and tail in "show fd"
    - MAJOR: mux-h2: switch to next mux buffer on buffer full condition.
    - BUILD: connections: shut up gcc about impossible out-of-bounds warning
    - BUILD: ssl: fix latest LibreSSL reg-test error
    - MINOR: cli/activity: remove "fd_del" and "fd_skip" from show activity
    - MINOR: cli/activity: add 3 general purpose counters in development mode
    - BUG/MAJOR: lb/threads: make sure the avoided server is not full on second pass
    - BUG/MEDIUM: queue: fix the tree walk in pendconn_redistribute.
    - BUG/MEDIUM: threads: fix double-word CAS on non-optimized 32-bit platforms
    - MEDIUM: config: now alert when two servers have the same name
    - MINOR: htx: Remove the macro IS_HTX_SMP() and always use IS_HTX_STRM() instead
    - MINOR: htx: Move the macro IS_HTX_STRM() in proto/stream.h
    - MINOR: htx: Store the head position instead of the wrap one
    - MINOR: htx: Store start-line block's position instead of address of its payload
    - MINOR: htx: Add functions to get the first block of an HTX message
    - MINOR: mux-h2/htx: Get the start-line from the head when HEADERS frame is built
    - MINOR: htx: Replace the function http_find_stline() by http_get_stline()
    - CLEANUP: htx: Remove unused function htx_get_stline()
    - MINOR: http/htx: Use sl_pos directly to replace the start-line
    - MEDIUM: http/htx: Perform analysis relatively to the first block
    - MINOR: channel/htx: Call channel_htx_recv_max() from channel_recv_max()
    - MINOR: htx: Add function htx_get_max_blksz()
    - BUG/MINOR: htx: Change htx_xfer_blk() to also count metadata
    - MEDIUM: mux-h1: Use the count value received from the SI in h1_rcv_buf()
    - MINOR: mux-h2: Use the count value received from the SI in h2_rcv_buf()
    - MINOR: stream-int: Don't use the flag CO_RFL_KEEP_RSV anymore in si_cs_recv()
    - MINOR: connection: Remove the unused flag CO_RFL_KEEP_RSV
    - MINOR: mux-h2/htx: Support zero-copy when possible in h2_rcv_buf()
    - MINOR: htx: Add a field to set the memory used by headers in the HTX start-line
    - MINOR: h2/htx: Set hdrs_bytes on the SL when an HTX message is produced
    - MINOR: mux-h1: Set hdrs_bytes on the SL when an HTX message is produced
    - MINOR: htx: Be sure to xfer all headers in one time in htx_xfer_blks()
    - MEDIUM: htx: 1xx messages are now part of the final reponses
    - MINOR: channel/htx: Add function to forward headers of an HTX message
    - MINOR: filters/htx: Use channel_htx_fwd_headers() after headers filtering
    - MINOR: proto-htx: Use channel_htx_fwd_headers() to forward 1xx responses
    - MEDIUM: htx: Store the first block position instead of the start-line one
    - MINOR: stats/htx: don't use the first block position but the head one
    - MINOR: channel/htx: Add functions to forward a part or all HTX payload
    - MINOR: proto-htx: Use channel_htx_fwd_all() when unfiltered body are forwarded
    - MEDIUM: filters/htx: Filter body relatively to the first block
    - MINOR: htx: Optimize htx_drain() when all data are drained
    - MINOR: htx: don't rely on htx_find_blk() anymore in the function htx_truncate()
    - MINOR: htx: remove the unused function htx_find_blk()
    - MINOR: htx: Remove support of pseudo headers because it is unused
    - BUG/MEDIUM: http: fix "http-request reject" when not final
    - MINOR: ssl: Make sure the underlying xprt's init method doesn't fail.
    - MINOR: ssl: Don't forget to call the close method of the underlying xprt.
    - MINOR: htx: rename htx_append_blk_value() to htx_add_data_atonce()
    - MINOR: htx: make htx_add_data() return the transmitted byte count
    - MEDIUM: htx: make htx_add_data() never defragment the buffer
    - MINOR: activity: write totals on the "show activity" output
    - MINOR: activity: report totals and average separately
    - MEDIUM: poller: separate the wait time from the wake events
    - MINOR: activity: report the number of failed pool/buffer allocations
    - MEDIUM: buffers: relax the buffer lock a little bit
    - MINOR: task: turn the WQ lock to an RW_LOCK
    - MEDIUM: task: don't grab the WR lock just to check the WQ
    - BUG/MEDIUM: mux-h1: Don't skip the TCP splicing when there is no more data to read
    - MEDIUM: sessions: Introduce session flags.
    - BUG/MEDIUM: h2: Don't forget to set h2s->cs to NULL after having free'd cs.
    - BUG/MEDIUM: mux-h2: fix the conditions to end the h2_send() loop
    - BUG/MEDIUM: mux-h2: don't refrain from offering oneself a used buffer
    - BUG/MEDIUM: connection: Use the session to get the origin address if needed.
    - MEDIUM: tasks: Get rid of active_tasks_mask.
    - MEDIUM: connection: Upstream SOCKS4 proxy support
    - BUILD: contrib/prometheus: fix build breakage caused by move of idle_pct
    - BUG/MINOR: deinit/threads: make hard-stop-after perform a clean exit
2019-06-02 12:06:08 +02:00
Alexander Liu
2a54bb74cd MEDIUM: connection: Upstream SOCKS4 proxy support
Have "socks4" and "check-via-socks4" server keyword added.
Implement handshake with SOCKS4 proxy server for tcp stream connection.
See issue #82.

I have the "SOCKS: A protocol for TCP proxy across firewalls" doc found
at "https://www.openssh.com/txt/socks4.protocol". Please reference to it.

[wt: for now connecting to the SOCKS4 proxy over unix sockets is not
 supported, and mixing IPv4/IPv6 is discouraged; indeed, the control
 layer is unique for a connection and will be used both for connecting
 and for target address manipulation. As such it may for example report
 incorrect destination addresses in logs if the proxy is reached over
 IPv6]
2019-05-31 17:24:06 +02:00
Michael Prokop
4438c6061d DOC: fix typos
s/accidently/accidentally/
s/any ot these messages/any of theses messages/
s/catched/caught/
s/completly/completely/
s/convertor/converter/
s/desribing/describing/
s/developper/developer/
s/eventhough/even though/
s/exectution/execution/
s/functionnality/functionality/
s/If it receive a/If it receives a/
s/In can even/It can even/
s/informations/information/
s/it will be remove /it will be removed /
s/langage/language/
s/mentionned/mentioned/
s/negociated/negotiated/
s/Optionnaly/Optionally/
s/ouputs/outputs/
s/outweights/outweighs/
s/ressources/resources/
2019-05-25 07:34:24 +02:00
Willy Tarreau
567406949b [RELEASE] Released version 2.0-dev4
Released version 2.0-dev4 with the following main changes :
    - BUILD: enable freebsd builds on cirrus-ci
    - BUG/MINOR: http_fetch: Rely on the smp direction for "cookie()" and "hdr()"
    - MEDIUM: Make 'option forceclose' actually warn
    - MEDIUM: Make 'resolution_pool_size' directive fatal
    - DOC: management: place "show activity" at the right place
    - MINOR: cli/activity: show the dumping thread ID starting at 1
    - MINOR: task: export global_task_mask
    - MINOR: cli/debug: add a thread dump function
    - BUG/MEDIUM: streams: Don't use CF_EOI to decide if the request is complete.
    - BUG/MEDIUM: streams: Try to L7 retry before aborting the connection.
    - BUG/MINOR: debug: make ha_task_dump() always check the task before dumping it
    - BUG/MINOR: debug: make ha_task_dump() actually dump the requested task
    - MINOR: debug: make ha_thread_dump() and ha_task_dump() take a buffer
    - BUG/MINOR: debug: don't check the call date on tasklets
    - MINOR: thread: implement ha_thread_relax()
    - MINOR: task: put barriers after each write to curr_task
    - MINOR: task: always reset curr_task when freeing a task or tasklet
    - MINOR: stream: detach the stream from its own task on stream_free()
    - MEDIUM: debug/threads: implement an advanced thread dump system
    - REGTEST: extend the check duration on tls_health_checks and mark it slow
    - DOC: fix "successful" typo
    - MINOR: init: setenv HAPROXY_CFGFILES
    - MINOR: threads/init: synchronize the threads startup
    - MEDIUM: init/mworker: make the pipe register function a regular initcall
    - CLEANUP: memory: make the fault injection code use the OTHER_LOCK label
    - CLEANUP: threads: remove the now unused START_LOCK label
    - MINOR: init/threads: make the global threads an array of structs
    - MINOR: threads: add each thread's clockid into the global thread_info
    - CLEANUP: stream: remove an obsolete debugging test
    - MINOR: tools: add dump_hex()
    - MINOR: debug: implement ha_panic()
    - MINOR: debug/cli: add some debugging commands for developers
    - MINOR: tools: provide a may_access() function and make dump_hex() use it
    - MINOR: debug: make ha_panic() report threads starting at 1
    - REORG: compat: move some integer limit definitions from standard.h to compat.h
    - REORG: threads: move the struct thread_info from global.h to hathreads.h
    - MINOR: compat: make sure to always define clockid_t
    - MINOR: threads: always place the clockid in the struct thread_info
    - MINOR: threads: add a thread-local thread_info pointer "ti"
    - MINOR: time: move the cpu, mono, and idle time to thread_info
    - MINOR: time: add a function to retrieve another thread's cputime
    - MINOR: debug: report each thread's cpu usage in "show thread"
    - BUILD: threads: only assign the clock_id when supported
    - BUILD: makefile: use USE_OBSOLETE_LINKER for solaris
    - BUILD: makefile: remove -fomit-frame-pointer optimisation (solaris)
    - MAJOR: polling: add event ports support (Solaris)
    - BUG/MEDIUM: streams: Don't switch from SI_ST_CON to SI_ST_DIS on read0.
    - CLEANUP: time: refine the test on _POSIX_TIMERS
    - MINOR: compat: define a new empty type empty_t for non-implemented fields
    - CLEANUP: time: switch clockid_t to empty_t when not available
    - BUG/MINOR: mworker: Fix memory leak of mworker_proc members
    - CLEANUP: objtype: make obj_type() and obj_type_name() take consts
    - MINOR: debug: switch to SIGURG for thread dumps
    - CLEANUP: threads: really move thread_info to hathreads.c
    - MINOR: threads: make threads_{harmless|want_rdv}_mask constant 0 without threads
    - CLEANUP: debug: always report harmless/want_rdv even without threads
    - MINOR: threads: implement ha_tkill() and ha_tkillall()
    - CLEANUP: debug: make use of ha_tkill() and remove ifdefs
    - MINOR: stream: introduce a stream_dump() function and use it in stream_dump_and_crash()
    - MINOR: debug: dump streams when an applet, iocb or stream is known
    - MINOR: threads: add a "stuck" flag to the thread_info struct
    - MINOR: threads: add a timer_t per thread in thread_info
    - MAJOR: watchdog: implement a thread lockup detection mechanism
    - MINOR: stream: remove the cpu time detection from process_stream()
    - MINOR: connection: report the mux names in "haproxy -vv"
    - CLEANUP: mux-h1: use "H1" and not "h1" as the mux's name
    - BUG/MEDIUM: WURFL: segfault in wurfl-get() with missing info.
    - MINOR: WURFL: call header_retireve_callback() in dummy library
    - MINOR: WURFL: fixed Engine load failed error when wurfl-information-list contains wurfl_root_id
    - MINOR: WURFL: shows log messages during module initialization
    - MINOR: WURFL: removes heading wurfl-information-separator from wurfl-get-all() and wurfl-get() results
    - MINOR: WURFL: wurfl_get() and wurfl_get_all() now return an empty string if device detection fails
    - MEDIUM: WURFL: HTX awareness.
    - MINOR: WURFL: module version bump to 2.0
    - MINOR: WURFL: do not emit warnings when not configured
    - CONTRIB: wurfl: address 3 build issues in the wurfl dummy library
    - BUG/MEDIUM: init/threads: provide per-thread alloc/free function callbacks
    - BUILD: travis: add sanitizers to travis-ci builds
    - BUILD: time: remove the test on _POSIX_C_SOURCE
    - CLEANUP: build: rename some build macros to use the USE_* ones
    - CLEANUP: raw_sock: remove support for very old linux splice bug workaround
    - BUG/MEDIUM: dns: make the port numbers unsigned
    - MEDIUM: config: deprecate the antique req* and rsp* commands
2019-05-22 20:48:33 +02:00
Willy Tarreau
96d5195862 MEDIUM: config: deprecate the antique req* and rsp* commands
These commands don't follow the same flow as the rest of the commands,
each of them iterates over all header lines before switching to the
next directive. In addition they make no distinction between start
line and headers and can lead to unparsable rewrites which are very
difficult to deal with internally.

Most of them are still occasionally found in configurations, mainly
because of the usual "we've always done this way". By marking them
deprecated and emitting a warning and recommendation on first use of
each of them, we will raise users' awareness of users regarding the
cleaner, faster and more reliable alternatives.

Some use cases of "reqrep" still appear from time to time for URL
rewriting that is not so convenient with other rules. But at least
users facing this requirement will explain their use case so that we
can best serve them. Some discussion started on this subject in a
thread linked to from github issue #100.

The goal is to remove them in 2.1 since they require to reparse the
result before indexing it and we don't want this hack to live long.
The following directives were marked deprecated :

  -reqadd
  -reqallow
  -reqdel
  -reqdeny
  -reqiallow
  -reqidel
  -reqideny
  -reqipass
  -reqirep
  -reqitarpit
  -reqpass
  -reqrep
  -reqtarpit
  -rspadd
  -rspdel
  -rspdeny
  -rspidel
  -rspideny
  -rspirep
  -rsprep
2019-05-22 20:43:45 +02:00
Emmanuel Hocdet
0ba4f483d2 MAJOR: polling: add event ports support (Solaris)
Event ports are kqueue/epoll polling class for Solaris. Code is based
on https://github.com/joyent/haproxy-1.8/tree/joyent/dev-v1.8.8.
Event ports are available only on SunOS systems derived from
Solaris 10 and later (including illumos systems).
2019-05-21 15:16:45 +02:00
Tim Duesterhus
3506dae342 MEDIUM: Make 'resolution_pool_size' directive fatal
This directive never appeared in a stable release and instead was
introduced and deprecated within 1.8-dev. While it technically could
be outright removed we detect it and error out for good measure.
2019-05-16 18:02:03 +02:00
Tim Duesterhus
10c6c16cde MEDIUM: Make 'option forceclose' actually warn
It is deprecated since 315b39c3914f4c2301ce19a93564566caa2ede50 (1.9-dev),
but only was deprecated in the docs.

Make it warn when being used and remove it from the docs.
2019-05-16 18:02:03 +02:00
Willy Tarreau
a257a9b015 [RELEASE] Released version 2.0-dev3
Released version 2.0-dev3 with the following main changes :
    - BUG/MINOR: peers: Really close the sessions with no heartbeat.
    - CLEANUP: peers: remove useless annoying tabulations.
    - CLEANUP: peers: replace timeout constants by macros.
    - REGTEST: Enable again reg tests with HEAD HTTP method usage.
    - DOC: The option httplog is no longer valid in a backend.
    - DOC: peers: Peers protocol documentation update.
    - REGTEST: remove unexpected "nbthread" statement from Lua test cases
    - BUILD: Makefile: remove 11-years old workarounds for deprecated options
    - BUILD: remove 10-years old error message for obsolete option USE_TCPSPLICE
    - BUILD: Makefile: remove outdated support for dlmalloc
    - BUILD: Makefile: consider a variable's origin and not its value for the options list
    - BUILD: Makefile: also report disabled options in the BUILD_OPTIONS variable
    - BUILD: Makefile: shorten default settings declaration
    - BUILD: Makefile: clean up the target declarations
    - BUILD: report the whole feature set with their status in haproxy -vv
    - BUILD: pass all "USE_*" variables as -DUSE_* to the compiler
    - REGTEST: script: make the script use the new features list
    - REGTEST: script: remove platform-specific assigments of OPTIONS
    - BUG/MINOR: peers: Missing initializations after peer session shutdown.
    - BUG/MINOR: contrib/prometheus-exporter: Fix applet accordingly to recent changes
    - BUILD/MINOR: listener: Silent a few signedness warnings.
    - BUG/MINOR: mux-h1: Only skip invalid C-L headers on output
    - BUG/MEDIUM: mworker: don't free the wrong child when not found
    - BUG/MEDIUM: checks: Don't bother subscribing if we have a connection error.
    - BUG/MAJOR: checks: segfault during tcpcheck_main
    - BUILD: makefile: work around an old bug in GNU make-3.80
    - BUILD: makefile: work around another bug in make 3.80
    - BUILD: http: properly mark some struct as extern
    - BUILD: chunk: properly declare pool_head_trash as extern
    - BUILD: cache: avoid a build warning with some compilers/linkers
    - MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
    - MINOR: tools: add an unsetenv() implementation
    - BUILD: re-implement an initcall variant without using executable sections
    - BUILD: use inttypes.h instead of stdint.h
    - BUILD: connection: fix naming of ip_v field
    - BUILD: makefile: fix build of IPv6 header on aix51
    - BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
    - BUILD: define unsetenv on AIX 5.1
    - BUILD: Makefile: disable shared cache on AIX 5.1
    - MINOR: ssl: Add aes_gcm_dec converter
    - REORG: mworker: move serializing functions to mworker.c
    - REORG: mworker: move signals functions to mworker.c
    - REORG: mworker: move IPC functions to mworker.c
    - REORG: mworker: move signal handlers and related functions
    - REORG: mworker: move mworker_cleanlisteners to mworker.c
    - MINOR: mworker: calloc mworker_proc structures
    - MINOR: mworker: don't use children variable anymore
    - MINOR: cli: export cli_parse_default() definition in cli.h
    - REORG: mworker/cli: move CLI functions to mworker.c
    - MEDIUM: mworker-prog: implement program for master-worker
    - MINOR: mworker/cli: show programs in 'show proc'
    - BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
    - MINOR: cli: start addresses by a prefix in 'show cli sockets'
    - MINOR: cli: export HAPROXY_CLI environment variable
    - BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing error
    - BUG/MINOR: proto_htx: Reset to_forward value when a message is set to DONE
    - REGTEST: http-capture/h00000: Relax a regex matching the log message
    - REGTEST: http-messaging/h00000: Fix the test when the HTX is enabled
    - REGTEST: http-rules/h00003: Use a different client for requests expecting a 301
    - REGTEST: log/b00000: Be sure the client always hits its timeout
    - REGTEST: lua/b00003: Relax the regex matching the log message
    - REGTEST: lua/b00003: Specify the HAProxy pid when the command ss is executed
    - BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on release.
    - BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we subscribed.
    - BUG/MEDIUM: htx: fix random premature abort of data transfers
    - BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
    - BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
    - BUG/MEDIUM: stream: Don't clear the stream_interface flags in si_update_both.
    - MINOR: initcall: Don't forget to define the __start/stop_init_##stg symbols.
    - MINOR: threads: Implement thread_cpus_enabled() for FreeBSD.
    - BUG/MEDIUM: pattern: assign pattern IDs after checking the config validity
    - MINOR: skip get_gmtime where tm is unused
    - MINOR: ssl: Activate aes_gcm_dec converter for BoringSSL
    - BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected state.
    - BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not connected.
    - BUG/MEDIUM: task/threads: address a fairness issue between local and global tasks
    - BUG/MINOR: tasks: make sure the first task to be queued keeps its nice value
    - BUG/MINOR: listener: renice the accept ring processing task
    - MINOR: cli/listener: report the number of accepts on "show activity"
    - MINOR: cli/activity: report the accept queue sizes in "show activity"
    - BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to the stream
    - BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented messages
    - BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is encoded
    - BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads wrap
    - BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a data block
    - MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
    - MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
    - BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
    - MINOR: mux-h1: Simplify handling of 1xx responses
    - MINOR: stats/htx: Don't add "Connection: close" header anymore in stats responses
    - MEDIUM: h1: Add an option to sanitize connection headers during parsing
    - MEDIUM: mux-h1: Simplify the connection mode management by sanitizing headers
    - MINOR: mux-h1: Don't release the conn_stream anymore when h1s is destroyed
    - BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown read/write
    - MINOR: mux-h2: Add a mux_ops dedicated to the HTX mode
    - MINOR: muxes: Add a flag to specify a multiplexer uses the HTX
    - MINOR: stream: Set a flag when the stream uses the HTX
    - MINOR: http: update the macro IS_HTX_STRM() to check the stream flag SF_HTX
    - MINOR: http_fetch/htx: Use stream flags instead of px mode in smp_prefetch_htx
    - MINOR: filters/htx: Use stream flags instead of px mode to instanciate a filter
    - MINOR: muxes: Rely on conn_is_back() during init to handle front/back conn
    - MEDIUM: muxes: Add an optional input buffer during mux initialization
    - MINOR: muxes: Pass the context of the mux to destroy() instead of the connection
    - MEDIUM: muxes: Be prepared to don't own connection during the release
    - MEDIUM: connection: Add conn_upgrade_mux_fe() to handle mux upgrades
    - MEDIUM: htx: Allow the option http-use-htx to be used on TCP proxies too
    - MAJOR: proxy/htx: Handle mux upgrades from TCP to HTTP in HTX mode
    - MAJOR: muxes/htx: Handle inplicit upgrades from h1 to h2
    - MAJOR: htx: Enable the HTX mode by default for all proxies
    - REGTEST: Use HTX by default and add '--no-htx' option to disable it
    - BUG/MEDIUM: muxes: Don't dereference mux context if null in release functions
    - CLEANUP: task: do not export rq_next anymore
    - MEDIUM: tasks: improve fairness between the local and global queues
    - MEDIUM: tasks: only base the nice offset on the run queue depth
    - MINOR: tasks: restore the lower latency scheduling when niced tasks are present
    - BUG/MEDIUM: map: Fix memory leak in the map converter
    - BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation
    - BUILD: task/thread: fix single-threaded build of task.c
    - BUILD: cli/threads: fix build in single-threaded mode
    - BUG/MEDIUM: muxes: Make sure we unsubcribed when destroying mux ctx.
    - BUG/MEDIUM: h2: Make sure we're not already in the send_list in h2_subscribe().
    - BUG/MEDIUM: h2: Revamp the way send subscriptions works.
    - MINOR: connections: Remove the SUB_CALL_UNSUBSCRIBE flag.
    - BUG/MEDIUM: Threads: Only use the gcc >= 4.7 builtins when using gcc >= 4.7.
    - BUILD: address a few cases of "static <type> inline foo()"
    - BUILD: do not specify "const" on functions returning structs or scalars
    - BUILD: htx: fix a used uninitialized warning on is_cookie2
    - MINOR: peers: Add a new command to the CLI for peers.
    - DOC: update for "show peers" CLI command.
    - BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
    - MEDIUM: mworker: store the leaving state of a process
    - MEDIUM: mworker-prog: implements 'option start-on-reload'
    - CLEANUP: mworker: remove the type field in mworker_proc
    - MEDIUM: mworker/cli: export the HAPROXY_MASTER_CLI variable
    - MINOR: cli: don't add a semicolon at the end of HAPROXY_CLI
    - MINOR: mworker: export HAPROXY_MWORKER=1 when running in mworker mode
    - MINOR: init: add a "set-dumpable" global directive to enable core dumps
    - BUG/MINOR: listener/mq: correctly scan all bound threads under low load
    - BUG/MINOR: mworker: mworker_kill should apply on every children
    - BUG/MINOR: mworker: don't exit with an ambiguous value
    - BUG/MINOR: mworker: ensure that we still quits with SIGINT
    - REGTESTS: exclude tests that require ssl, pcre if no such feature is enabled
    - BUG/MINOR: mux-h1: Process input even if the input buffer is empty
    - BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side has done
    - BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf is empty
    - BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
    - MINOR: mux-h1: Handle read0 during TCP splicing
    - BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
    - BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
    - BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
    - BUG/MINOR: http_fetch/htx: Use HTX versions if the proxy enables the HTX mode
    - BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to the rq.
    - BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the rq_lock.
    - MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
    - MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
    - MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet list.
    - BUG/MEDIUM: applets: Don't use task_in_rq().
    - BUG/MAJOR: task: make sure never to delete a queued task
    - MINOR: task/thread: factor out a wake-up condition
    - CLEANUP: task: remain consistent when using the task's handler
    - MEDIUM: tasks: Merge task_delete() and task_free() into task_destroy().
    - MEDIUM: tasks: Don't account a destroyed task as a runned task.
    - BUG/MINOR: contrib/prometheus-exporter: Fix a typo in the run-queue metric type
    - MINOR: contrib/prometheus-exporter: Remove usless rate metrics
    - MINOR: contrib/prometheus-exporter: Rename some metrics to be more usable
    - MINOR: contrib/prometheus-exporter: Follow best practices about metrics type
    - BUG/MINOR: mworker: disable busy polling in the master process
    - MEDIUM: tasks: Use __ha_barrier_store after modifying global_tasks_mask.
    - MEDIUM: ssl: Give ssl_sock its own context.
    - MEDIUM: connections: Move some fields from struct connection to ssl_sock_ctx.
    - MEDIUM: ssl: provide its own subscribe/unsubscribe function.
    - MEDIUM: connections: Provide a xprt_ctx for each xprt method.
    - MEDIUM: ssl: provide our own BIO.
    - BUILD/medium: ssl: Fix build with OpenSSL < 1.1.0
    - MINOR: peers: adds counters on show peers about tasks calls.
    - MEDIUM: enable travis-ci builds
    - MINOR: fd: Add a counter of used fds.
    - MEDIUM: connections: Add a way to control the number of idling connections.
    - BUG/MEDIUM: maps: only try to parse the default value when it's present
    - BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
    - REGTEST: Missing REQUIRE_VERSION declarations.
    - MINOR: proto_tcp: tcp-request content: enable set-dst and set-dst-var
    - BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
    - BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP rules
    - BUG/MEDIUM: stream: Don't request a server connection if a shutw was scheduled
    - BUG/MINOR: 51d: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
    - BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
    - MINOR: gcc: Fix a silly gcc warning in connect_server()
    - MINOR: ssl/cli: async fd io-handlers printable on show fd
    - Revert "CLEANUP: wurfl: remove dead, broken and unmaintained code"
    - BUILD: add USE_WURFL to the list of known build options
    - MINOR: wurfl: indicate in haproxy -vv the wurfl version in use
    - BUILD: wurfl: build fix for 1.9/2.0 code base
    - CLEANUP: wurfl: removed deprecated methods
    - DOC: wurfl: added point of contact in MAINTAINERS file
    - MINOR: wurfl: enabled multithreading mode
    - MINOR: contrib: dummy wurfl library
    - MINOR: dns: dns_requester structures are now in a memory pool
    - MINOR: dns: move callback affection in dns_link_resolution()
    - MINOR: obj_type: new object type for struct stream
    - MINOR: action: new '(http-request|tcp-request content) do-resolve' action
    - MINOR: log: Extract some code to send syslog messages.
    - REGTEST: replace LEVEL option by a more human readable one.
    - REGTEST: rename the reg test files.
    - REGTEST: adapt some reg tests after renaming.
    - REGTEST: make the "run-regtests" script search for tests in reg-tests by default
    - BUG/MAJOR: stream: Missing DNS context initializations.
    - BUG/MEDIUM: stream: Fix the way early aborts on the client side are handled
    - BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet handler
    - BUG/MEDIUM: ssl: Return -1 on recv/send if we got EAGAIN.
    - BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB
    - BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies only
    - BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during startup
    - CLEANUP: task: report calls as unsigned in show sess
    - MINOR: tasks/activity: report the context switch and task wakeup rates
    - MINOR: stream: measure and report a stream's call rate in "show sess"
    - MINOR: applet: measure and report an appctx's call rate in "show sess"
    - BUILD: extend Travis CI config to support more platforms
    - REGTEST: exclude osx and generic targets for 40be_2srv_odd_health_checks
    - REGTEST: relax the IPv6 address format checks in converters_ipmask_concat_strcmp_field_word
    - REGTEST: exclude OSX and generic targets from abns_socket.vtc
    - BUILD: travis: remove the "allow_failures" entry
    - BUG/MINOR: activity: always initialize the profiling variable
    - MINOR: activity: make the profiling status per thread and not global
    - MINOR: activity: enable automatic profiling turn on/off
    - CLEANUP: standard: use proper const to addr_to_str() and port_to_str()
    - BUG/MINOR: proto_http: properly reset the stream's call rate on keep-alive
    - MINOR: connection: make the debugging helper functions safer
    - MINOR: stream/debug: make a stream dump and crash function
    - MEDIUM: appctx/debug: force a crash if an appctx spins over itself forever
    - MEDIUM: stream/debug: force a crash if a stream spins over itself forever
    - MEDIUM: streams: measure processing time and abort when detecting bugs
    - BUILD/MEDIUM: contrib: Dummy DeviceAtlas API.
    - MEDIUM: da: HTX mode support.
    - BUG/MEDIUM: mux-h2: properly deal with too large headers frames
    - BUG/MINOR: http: Call stream_inc_be_http_req_ctr() only one time per request
    - BUG/MEDIUM: spoe: arg len encoded in previous frag frame but len changed
    - MINOR: spoe: Use the sample context to pass frag_ctx info during encoding
    - DOC: contrib/modsecurity: Typos and fix the reject example
    - BUG/MEDIUM: contrib/modsecurity: If host header is NULL, don't try to strdup it
    - MINOR: log: Add "sample" new keyword to "log" lines.
    - MINOR: log: Enable the log sampling and load-balancing feature.
    - DOC: log: Document the sampling and load-balancing logging feature.
    - REGTEST: Add a new reg test for log load-balancing feature.
    - BUG/MAJOR: map/acl: real fix segfault during show map/acl on CLI
    - REGTEST: Make this reg test be Linux specific.
    - CLEANUP: task: move the task_per_thread definition to task.h
    - MINOR: activity: report context switch counts instead of rates
    - MINOR: threads: Implement HA_ATOMIC_LOAD().
    - BUG/MEDIUM: port_range: Make the ring buffer lock-free.
    - BUG/MEDIUM: listener: Fix how unlimited number of consecutive accepts is handled
    - MINOR: config: Test validity of tune.maxaccept during the config parsing
    - CLEANUP: config: Don't alter listener->maxaccept when nbproc is set to 1
    - BUG/MEDIUM: servers: fix typo "src" instead of "srv"
    - BUG/MEDIUM: ssl: Don't pretend we can retry a recv/send if we got a shutr/w.
    - BUG/MINOR: haproxy: fix rule->file memory leak
    - BUG/MINOR: log: properly free memory on logformat parse error and deinit()
    - BUG/MINOR: checks: free memory allocated for tasklets
    - BUG/MEDIUM: pattern: fix memory leak in regex pattern functions
    - BUG/MEDIUM: channels: Don't forget to reset output in channel_erase().
    - BUG/MEDIUM: connections: Make sure we remove CO_FL_SESS_IDLE on disown.
    - MINOR: threads: flatten the per-thread cpu-map
    - MINOR: init/threads: remove the useless tids[] array
    - MINOR: init/threads: make the threads array global
    - BUG/MEDIUM: ssl: Use the early_data API the right way.
    - BUG/MEDIUM: streams: Don't add CF_WRITE_ERROR if early data were rejected.
    - MEDIUM: streams: Add the ability to retry a request on L7 failure.
    - MEDIUM: streams: Add a way to replay failed 0rtt requests.
    - MEDIUM: streams: Add a new keyword for retry-on, "junk-response"
    - BUG/MINOR: stream: also increment the retry stats counter on L7 retries
    - BUG/MEDIUM: checks: make sure the warmup task takes the server lock
    - BUG/MINOR: logs/threads: properly split the log area upon startup
    - BUILD: extend travis-ci matrix
    - CLEANUP: Remove appsession documentation
    - DOC: Fix typo in keyword matrix
    - BUILD: remove "build_libressl" duplicate declaration
    - BUILD: travis-ci: get back to osx without openssl support
    - BUILD: enable several LibreSSL hacks, including
    - BUILD: temporarily mark LibreSSL builds as allowed to fail
    - BUILD: travis: TMPDIR replacement.
    - BUG/MEDIUM: ssl: Don't attempt to use early data with libressl.
    - MINOR: doc: Document allow-0rtt on the server line.
    - MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
    - MEDIUM: proto: Change the prototype of the connect() method.
    - MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
    - MINOR: config: Extract the code of "stick-table" line parsing.
    - BUILD/MINOR: stick-table: Compilation fix.
    - MEDIUM: stick-table: Stop handling stick-tables as proxies.
    - MINOR: stick-tables: Add peers process binding computing.
    - MINOR: stick-table: Add prefixes to stick-table names.
    - MINOR: peers: Do not emit global stick-table names.
    - DOC: Update for "table" lines in "peers" section.
    - REGTEST: Add reg tests for "table" lines in "peers" sections.
    - MEDIUM: regex: modify regex_comp() to atomically allocate/free the my_regex struct
    - REGTEST: make the tls_health_checks test much faster
    - REGTEST: make the "table in peers" test require v2.0
    - BUG/MINOR: mux-h2: rely on trailers output not input to turn them to empty data
    - BUG/MEDIUM: h2/htx: always fail on too large trailers
    - MEDIUM: mux-h2: discard contents that are to be sent after a shutdown
    - BUG/MEDIUM: mux-h2/htx: never wait for EOM when processing trailers
    - BUG/MEDIUM: h2/htx: never leave a trailers block alone with no EOM block
    - REGTEST: Flag some slow reg tests.
    - REGTEST: Reg tests file renaming.
    - REGTEST: Wrong renaming for one reg test.
    - REGTEST: Wrong assumption in IP:port logging test.
    - BUG/MINOR: mworker/ssl: close OpenSSL FDs on reload
    - MINOR: systemd: Use the variables from /etc/default/haproxy
    - MINOR: systemd: Make use of master socket in systemd unit
    - MINOR: systemd: support /etc/sysconfig/ for redhat based distrib
    - BUG/MEDIUM: stick-table: fix regression caused by a change in proxy struct
    - BUG/MEDIUM: tasks: fix possible segfault on task_destroy()
    - CLEANUP: task: remove unneeded tests before task_destroy()
    - MINOR: mworker: support a configurable maximum number of reloads
    - BUG/MINOR: mux-h2: fix the condition to close a cs-less h2s on the backend
    - BUG/MEDIUM: spoe: Be sure the sample is found before setting its context
    - BUG/MINOR: mux-h1: Fix the parsing of trailers
    - BUG/MINOR: htx: Never transfer more than expected in htx_xfer_blks()
    - MINOR: htx: Split on DATA blocks only when blocks are moved to an HTX message
    - MINOR: htx: Don't try to append a trailer block with the previous one
    - MINOR: htx: Remove support for unused OOB HTX blocks
    - BUILD: travis-ci bugfixes and improvements
    - BUG/MEDIUM: servers: Don't use the same srv flag for cookie-set and TFO.
    - BUG/MEDIUM: h2: Make sure we set send_list to NULL in h2_detach().
    - BUILD: ssl: fix again a libressl build failure after the openssl FD leak fix
    - CLEANUP: ssl-sock: use HA_OPENSSL_VERSION_NUMBER instead of OPENSSL_VERSION_NUMBER
    - BUILD: ssl: make libressl use its own version numbers
    - CLEANUP: ssl: remove 57 occurrences of useless tests on LIBRESSL_VERSION_NUMBER
    - MINOR: ssl: enable aes_gcm_dec on LibreSSL
    - BUILD: ssl: fix libressl build again after aes-gcm-enc
    - REORG: ssl: move openssl-compat from proto to common
    - REORG: ssl: move some OpenSSL defines from ssl_sock to openssl-compat
    - CLEANUP: ssl: never include openssl/*.h outside of openssl-compat.h anymore
    - CLEANUP: ssl: make inclusion of openssl headers safe
    - BUILD: add BoringSSL to travis-ci build matrix
    - BUILD: threads: Add __ha_cas_dw fallback for single threaded builds
    - BUG/MINOR: stream: Attach the read side on the response as soon as possible
    - BUG/MEDIUM: http: Use pointer to the begining of input to parse message headers
    - BUG/MEDIUM: h2: Don't check send_wait to know if we're in the send_list.
    - BUG/MEDIUM: streams: Make sur SI_FL_L7_RETRY is set before attempting a retry.
    - MEDIUM: streams: Add a new http action, disable-l7-retry.
    - MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
    - BUG/MINOR: vars: Fix memory leak in vars_check_arg
    - BUILD: travis-ci: make TMPDIR global variable in travis-ci
    - CLEANUP: ssl: move the SSL_OP_* and SSL_MODE_* definitions to openssl-compat
    - CLEANUP: ssl: remove ifdef around SSL_CTX_get_extra_chain_certs()
    - CLEANUP: ssl: move all BIO_* definitions to openssl-compat
    - BUILD: threads: fix again the __ha_cas_dw() definition
    - BUG/MAJOR: mux-h2: do not add a stream twice to the send list
    - Revert "BUG/MINOR: vars: Fix memory leak in vars_check_arg"
    - BUG/MINOR: peers: Fix memory leak in cfg_parse_peers
    - BUG/MINOR: htx: make sure to always initialize the HTTP method when parsing a buffer
    - REGTEST: fix tls_health_checks random failures on MacOS in Travis-CI
    - MINOR: spoe: Set the argument chunk size to 0 when SPOE variables are checked
    - BUG/MINOR: vars: Fix memory leak in vars_check_arg
    - BUG/MAJOR: ssl: segfault upon an heartbeat request
    - MINOR: spoa-server: Clone the v1.7 spoa-example project
    - MINOR: spoa-server: move some definition from spoa_server.c to spoa_server.h
    - MINOR: spoa-server: Externalise debug functions
    - MINOR: spoe-server: rename "worker" functions
    - MINOR: spoa-server: Replace the thread init system by processes
    - MINOR: spoa-server: With debug mode, start only one process
    - MINOR: spoa-server: Allow registering external processes
    - MINOR: spoa-server: Allow registering message processors
    - MINOR: spoa-server: Load files
    - MINOR: spoa-server: Prepare responses
    - MINOR: spoa-server: Execute registered callbacks
    - MINOR: spoa-server: Add Lua processing
    - MINOR: spoa-server: Add python
    - MINOR/DOC: spoe-server: Add documentation
    - BUG/MEDIUM: connections: Don't forget to set xprt_ctx to NULL on close.
    - MINOR: lists: add LIST_ADDED() to check if an element belongs to a list
    - CLEANUP: mux-h2: use LIST_ADDED() instead of LIST_ISEMPTY() where relevant
    - MINOR: mux-h2: add two H2S flags to report the need for shutr/shutw
    - CLEANUP: mux-h2: simply use h2s->flags instead of ret in h2_deferred_shut()
    - CLEANUP: connection: remove the handle field from the wait_event struct
    - BUG/MINOR: log: Wrong log format initialization.
    - BUG/MINOR: mux-h2: make the do_shut{r,w} functions more robust against retries
    - BUG/MINOR: mworker: use after free when the PID not assigned
    - MINOR: mux-h2: remove useless test on stream ID vs last in wake function
    - MINOR: mux-h2: make h2_wake_some_streams() not depend on the CS flags
    - MINOR: mux-h2: make h2s_wake_one_stream() the only function to deal with CS
    - MINOR: mux-h2: make h2s_wake_one_stream() not depend on temporary CS flags
    - BUG/MINOR: mux-h2: make sure to honor KILL_CONN in do_shut{r,w}
    - CLEANUP: mux-h2: don't test for impossible CS_FL_REOS conditions
    - MINOR: mux-h2: add macros to check multiple stream states at once
    - MINOR: mux-h2: stop relying on CS_FL_REOS
    - BUG/MEDIUM: mux-h2: Set EOI on the conn_stream during h2_rcv_buf()
    - BUILD: debug: make gcc not complain on the ABORT_NOW() macro
    - MINOR: debug: add a new BUG_ON macro
    - MINOR: h2: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - MINOR: h1: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - MINOR: connections: Use BUG_ON() to enforce rules in subscribe/unsubscribe.
    - BUILD: ist: turn the lower/upper case tables to literal on obsolete linkers
2019-05-15 16:51:48 +02:00
Olivier Houchard
ddf0e03585 MINOR: streams: Introduce a new retry-on keyword, all-retryable-errors.
Add a new retry-on keyword, "all-retryable-errors", that activates retry
for all errors that are considered retryable.
This currently activates retry for "conn-failure", "empty-response",
"junk-respones", "response-timeout", "0rtt-rejected", "500", "502", "503" and
"504".
2019-05-10 18:05:35 +02:00
Olivier Houchard
602bf7d2ea MEDIUM: streams: Add a new http action, disable-l7-retry.
Add a new action for http-request, disable-l7-retry, that can be used to
disable any attempt at retry requests (see retry-on) if it fails for any
reason other than a connection failure.
This is useful for example to make sure POST requests aren't retried.
2019-05-10 17:49:09 +02:00
William Lallemand
27edc4b915 MINOR: mworker: support a configurable maximum number of reloads
This patch implements a new global parameter for the master-worker mode.
When setting the mworker-max-reloads value, a worker receive a SIGTERM
if its number of reloads is greater than this value.
2019-05-07 19:09:01 +02:00
Frdric Lcaille
4f5b77c57c DOC: Update for "table" lines in "peers" section. 2019-05-07 06:54:07 +02:00
Willy Tarreau
034c88cf03 MEDIUM: tcp: add the "tfo" option to support TCP fastopen on the server
This implements support for the new API which relies on a call to
setsockopt().
On systems that support it (currently, only Linux >= 4.11), this enables
using TCP fast open when connecting to server.
Please note that you should use the retry-on "conn-failure", "empty-response"
and "response-timeout" keywords, or the request won't be able to be retried
on failure.

Co-authored-by: Olivier Houchard <ohouchard@haproxy.com>
2019-05-06 22:29:39 +02:00
Olivier Houchard
22c9b440dc MINOR: doc: Document the interaction of allow-0rtt and retry-on 0rtt-rejected.
Document that haproxy will try to use early data when connecting to the
server if we have retry-on 0rtt-rejected, as well.
2019-05-06 19:01:19 +02:00
Olivier Houchard
8cb2d2e941 MINOR: doc: Document allow-0rtt on the server line.
Briefly document allow-0rtt on the server line, and only the part that apply
to 1.8 and 1.9.

This should be backported to 1.8 and 1.9.
2019-05-06 19:01:14 +02:00