516 Commits

Author SHA1 Message Date
Willy Tarreau
44f6539e47 DOC: remove the comment saying that SSL certs are not checked on the server side
Server certificate check was added in 1.5-dev13 but the comment on the "ssl"
keyword was not removed, leading to confusion about when to use it.
2013-06-25 07:56:20 +02:00
Lukas Tribus
2dd1d1a93f BUG/MINOR: http: fix "set-tos" not working in certain configurations
s->req->prod->conn->addr.to.ss_family contains only useful data if
conn_get_to_addr() is called early. If thats not the case (nothing in the
configuration needs the destination address like logs, transparent, ...)
then "set-tos" doesn't work.

Fix this by checking s->req->prod->conn->addr.from.ss_family instead.
Also fix a minor doc issue about set-tos in http-response.
2013-06-23 18:01:31 +02:00
Willy Tarreau
eab1dc6234 [RELEASE] Released version 1.5-dev19
Released version 1.5-dev19 with the following main changes :
    - MINOR: stats: remove the autofocus on the scope input field
    - BUG/MEDIUM: Fix crt-list file parsing error: filtered name was ignored.
    - BUG/MEDIUM: ssl: EDH ciphers are not usable if no DH parameters present in pem file.
    - BUG/MEDIUM: shctx: makes the code independent on SSL runtime version.
    - MEDIUM: ssl: improve crt-list format to support negation
    - BUG: ssl: fix crt-list for clients not supporting SNI
    - MINOR: stats: show soft-stopped servers in different color
    - BUG/MINOR: config: "source" does not work in defaults section
    - BUG: regex: fix pcre compile error when using JIT
    - MINOR: ssl: add pattern fetch 'ssl_c_sha1'
    - BUG: ssl: send payload gets corrupted if tune.ssl.maxrecord is used
    - MINOR: show PCRE version and JIT status in -vv
    - BUG/MINOR: jit: don't rely on USE flag to detect support
    - DOC: readme: add suggestion to link against static openssl
    - DOC: examples: provide simplified ssl configuration
    - REORG: tproxy: prepare the transparent proxy defines for accepting other OSes
    - MINOR: tproxy: add support for FreeBSD
    - MINOR: tproxy: add support for OpenBSD
    - DOC: examples: provide an example of transparent proxy configuration for FreeBSD 8
    - CLEANUP: fix minor typo in error message.
    - CLEANUP: fix missing include <string.h> in proto/listener.h
    - CLEANUP: protect checks.h from multiple inclusions
    - MINOR: compression: acl "res.comp" and fetch "res.comp_algo"
    - BUG/MINOR: http: add-header/set-header did not accept the ACL condition
    - BUILD: mention in the Makefile that USE_PCRE_JIT is for libpcre >= 8.32
    - BUG/MEDIUM: splicing is broken since 1.5-dev12
    - BUG/MAJOR: acl: add implicit arguments to the resolve list
    - BUG/MINOR: tcp: fix error reporting for TCP rules
    - CLEANUP: peers: remove a bit of spaghetti to prepare for the next bugfix
    - MINOR: stick-table: allow to allocate an entry without filling it
    - BUG/MAJOR: peers: fix an overflow when syncing strings larger than 16 bytes
    - MINOR: session: only call http_send_name_header() when changing the server
    - MINOR: tcp: report the erroneous word in tcp-request track*
    - BUG/MAJOR: backend: consistent hash can loop forever in certain circumstances
    - BUG/MEDIUM: log: fix regression on log-format handling
    - MEDIUM: log: report file name, line number, and directive name with log-format errors
    - BUG/MINOR: cli: "clear table" did not work anymore without a key
    - BUG/MINOR: cli: "clear table xx data.xx" does not work anymore
    - BUG/MAJOR: http: compression still has defects on chunked responses
    - BUG/MINOR: stats: fix confirmation links on the stats interface
    - BUG/MINOR: stats: the status bar does not appear anymore after a change
    - BUG/MEDIUM: stats: allocate the stats frontend also on "stats bind-process"
    - BUG/MEDIUM: stats: fix a regression when dealing with POST requests
    - BUG/MINOR: fix unterminated ACL array in compression
    - BUILD: last fix broke non-linux platforms
    - MINOR: init: indicate the SSL runtime version on -vv.
    - BUG/MEDIUM: compression: the deflate algorithm must use global settings as well
    - BUILD: stdbool is not portable (again)
    - DOC: readme: add a small reminder about restrictions to respect in the code
    - MINOR: ebtree: add new eb_next_dup/eb_prev_dup() functions to visit duplicates
    - BUG/MINOR: acl: fix a double free during exit when using PCRE_JIT
    - DOC: fix wrong copy-paste in the rspdel example
    - MINOR: counters: make it easier to extend the amount of tracked counters
    - MEDIUM: counters: add support for tracking a third counter
    - MEDIUM: counters: add a new "gpc0_rate" counter in stick-tables
    - BUG/MAJOR: http: always ensure response buffer has some room for a response
    - MINOR: counters: add fetch/acl sc*_tracked to indicate whether a counter is tracked
    - MINOR: defaults: allow REQURI_LEN and CAPTURE_LEN to be redefined
    - MINOR: log: add a new flag 'L' for locally processed requests
    - MINOR: http: add full-length header fetch methods
    - MEDIUM: protocol: implement a "drain" function in protocol layers
    - MEDIUM: http: add a new "http-response" ruleset
    - MEDIUM: http: add the "set-nice" action to http-request and http-response
    - MEDIUM: log: add a log level override value in struct session
    - MEDIUM: http: add support for action "set-log-level" in http-request/http-response
    - MEDIUM: http: add support for "set-tos" in http-request/http-response
    - MEDIUM: http: add the "set-mark" action on http-request/http-response rules
    - MEDIUM: tcp: add "tcp-request connection expect-proxy layer4"
    - MEDIUM: acl: automatically detect the type of certain fetches
    - MEDIUM: acl: remove a lot of useless ACLs that are equivalent to their fetches
    - MEDIUM: acl: remove 15 additional useless ACLs that are equivalent to their fetches
    - DOC: major reorg of ACL + sample fetch
    - CLEANUP: http: remove the bogus urlp_ip ACL match
    - MINOR: acl: add the new "env()" fetch method to retrieve an environment variable
    - BUG/MINOR: acl: correctly consider boolean fetches when doing casts
    - BUG/CRITICAL: fix a possible crash when using negative header occurrences
    - DOC: update ROADMAP file
    - MEDIUM: counters: use sc0/sc1/sc2 instead of sc1/sc2/sc3
    - MEDIUM: stats: add proxy name filtering on the statistic page
2013-06-17 15:10:25 +02:00
Willy Tarreau
be4a3eff34 MEDIUM: counters: use sc0/sc1/sc2 instead of sc1/sc2/sc3
It was a bit inconsistent to have gpc start at 0 and sc start at 1,
so make sc start at zero like gpc. No previous release was issued
with sc3 anyway, so no existing setup should be affected.
2013-06-17 15:04:07 +02:00
Willy Tarreau
595ec54d38 MINOR: acl: add the new "env()" fetch method to retrieve an environment variable
This is useful in order to take different actions across restarts without
touching the configuration (eg: soft-stop), or to pass some information
such as the local host name to the next hop.
2013-06-12 22:26:05 +02:00
Willy Tarreau
74ca5041bf DOC: major reorg of ACL + sample fetch
The split between ACL and sample fetch was a terrible mess in the doc,
as it caused all entries to be duplicated with most of them not easy to
find, some missing and some wrong.

The new approach consists in describing the sample fetch methods and
indicating the ACLs that are derived from these fetches. The doc is
much smaller (1500 lines added, 2200 removed, net gain = 700 lines)
and much clearer.

The description of the ACL mechanics was revamped to take account of
the latest evolutions and clearly describe the compatibility between
types of fetches and ACL patterns.

The deprecated keywords have been marked as such, though they still
appear in the examples given for various other keywords.
2013-06-12 22:26:04 +02:00
Willy Tarreau
4f0d919bd4 MEDIUM: tcp: add "tcp-request connection expect-proxy layer4"
This configures the client-facing connection to receive a PROXY protocol
header before any byte is read from the socket. This is equivalent to
having the "accept-proxy" keyword on the "bind" line, except that using
the TCP rule allows the PROXY protocol to be accepted only for certain
IP address ranges using an ACL. This is convenient when multiple layers
of load balancers are passed through by traffic coming from public
hosts.
2013-06-11 20:40:55 +02:00
Willy Tarreau
51347ed94c MEDIUM: http: add the "set-mark" action on http-request/http-response rules
"set-mark" is used to set the Netfilter MARK on all packets sent to the
client to the value passed in <mark> on platforms which support it. This
value is an unsigned 32 bit value which can be matched by netfilter and
by the routing table. It can be expressed both in decimal or hexadecimal
format (prefixed by "0x"). This can be useful to force certain packets to
take a different route (for example a cheaper network path for bulk
downloads). This works on Linux kernels 2.6.32 and above and requires
admin privileges.
2013-06-11 19:34:13 +02:00
Willy Tarreau
42cf39e3b9 MEDIUM: http: add support for "set-tos" in http-request/http-response
This manipulates the TOS field of the IP header of outgoing packets sent
to the client. This can be used to set a specific DSCP traffic class based
on some request or response information. See RFC2474, 2597, 3260 and 4594
for more information.
2013-06-11 19:04:37 +02:00
Willy Tarreau
9a355ec257 MEDIUM: http: add support for action "set-log-level" in http-request/http-response
Some users want to disable logging for certain non-important requests such as
stats requests or health-checks coming from another equipment. Other users want
to log with a higher importance (eg: notice) some special traffic (POST requests,
authenticated requests, requests coming from suspicious IPs) or some abnormally
large responses.

This patch responds to all these needs at once by adding a "set-log-level" action
to http-request/http-response. The 8 syslog levels are supported, as well as "silent"
to disable logging.
2013-06-11 17:50:26 +02:00
Willy Tarreau
f4c43c13be MEDIUM: http: add the "set-nice" action to http-request and http-response
This new action changes the nice factor of the task processing the current
request.
2013-06-11 17:50:26 +02:00
Willy Tarreau
e365c0b92b MEDIUM: http: add a new "http-response" ruleset
Some actions were clearly missing to process response headers. This
patch adds a new "http-response" ruleset which provides the following
actions :
  - allow : stop evaluating http-response rules
  - deny : stop and reject the response with a 502
  - add-header : add a header in log-format mode
  - set-header : set a header in log-format mode
2013-06-11 16:06:12 +02:00
Willy Tarreau
04ff9f105f MINOR: http: add full-length header fetch methods
The req.hdr and res.hdr fetch methods do not work well on headers which
are allowed to contain commas, such as User-Agent, Date or Expires.
More specifically, full-length matching is impossible if a comma is
present.

This patch introduces 4 new fetch functions which are designed to work
with these full-length headers :
  - req.fhdr, req.fhdr_cnt
  - res.fhdr, res.fhdr_cnt

These ones do not stop at commas and permit to return full-length header
values.
2013-06-10 18:39:42 +02:00
Willy Tarreau
570f221cbb MINOR: log: add a new flag 'L' for locally processed requests
People who use "option dontlog-normal" are bothered with redirects and
stats being logged and reported as errors in the logs ("PR" = proxy
blocked the request).

This patch introduces a new flag 'L' for when a request is locally
processed, that is not considered as an error by the log filters. That
way we know a request was intercepted and processed by haproxy without
logging the line when "option dontlog-normal" is in effect.
2013-06-10 16:42:09 +02:00
Willy Tarreau
6f1615f596 MINOR: counters: add fetch/acl sc*_tracked to indicate whether a counter is tracked
Sometimes we'd like to know if a counter is being tracked before adding a header to
an outgoing request. These ones do that.
2013-06-10 10:30:09 +02:00
Willy Tarreau
ba2ffd18b5 MEDIUM: counters: add a new "gpc0_rate" counter in stick-tables
This counter is special in that instead of reporting the gpc0 cumulative
count, it returns its increase rate over the configured period.
2013-05-29 15:54:14 +02:00
Willy Tarreau
e25c917af8 MEDIUM: counters: add support for tracking a third counter
We're often missin a third counter to track base, src and base+src at
the same time. Here we introduce track_sc3 to have this third counter.
It would be wise not to add much more counters because that slightly
increases the session size and processing time though the real issue
is more the declaration of the keywords in the code and in the doc.
2013-05-29 00:37:16 +02:00
Willy Tarreau
5e80e02c2b DOC: fix wrong copy-paste in the rspdel example
As reported by Cristian Ditoiu, the rspdel example is based on the reqrep
action, which is quite confusing!
2013-05-25 08:31:25 +02:00
James Voth
a051b4aa3a MINOR: ssl: add pattern fetch 'ssl_c_sha1'
This new pattern fetch returns the client certificate's SHA-1 fingerprint
(i.e. SHA-1 hash of DER-encoded certificate) in a binary chunk.

This can be useful to pass it to a server in a header or to stick a client
to a server across multiple SSL connections.
2013-05-14 20:55:30 +02:00
Emmanuel Hocdet
7c41a1b59b MEDIUM: ssl: improve crt-list format to support negation
Improve the crt-list file format to allow a rule to negate a certain SNI :

        <crtfile> [[!]<snifilter> ...]

This can be useful when a domain supports a wildcard but you don't want to
deliver the wildcard cert for certain specific domains.
2013-05-07 22:11:54 +02:00
William Lallemand
727db8b4ea MINOR: compression: acl "res.comp" and fetch "res.comp_algo"
Implements the "res.comp" ACL which is a boolean returning 1 when a
response has been compressed by HAProxy or 0 otherwise.

Implements the "res.comp_algo" fetch which contains the name of the
algorithm HAProxy used to compress the response.
2013-04-20 23:53:33 +02:00
Willy Tarreau
289dd92a64 [RELEASE] Released version 1.5-dev18
Released version 1.5-dev18 with the following main changes :
    - DOCS: Add explanation of intermediate certs to crt paramater
    - DOC: typo and minor fixes in compression paragraph
    - MINOR: config: http-request configuration error message misses new keywords
    - DOC: minor typo fix in documentation
    - BUG/MEDIUM: ssl: ECDHE ciphers not usable without named curve configured.
    - MEDIUM: ssl: add bind-option "strict-sni"
    - MEDIUM: ssl: add mapping from SNI to cert file using "crt-list"
    - MEDIUM: regex: Use PCRE JIT in acl
    - DOC: simplify bind option "interface" explanation
    - DOC: tfo: bump required kernel to linux-3.7
    - BUILD: add explicit support for TFO with USE_TFO
    - MEDIUM: New cli option -Ds for systemd compatibility
    - MEDIUM: add haproxy-systemd-wrapper
    - MEDIUM: add systemd service
    - BUG/MEDIUM: systemd-wrapper: don't leak zombie processes
    - BUG/MEDIUM: remove supplementary groups when changing gid
    - BUG/MEDIUM: config: fix parser crash with bad bind or server address
    - BUG/MINOR: Correct logic in cut_crlf()
    - CLEANUP: checks: Make desc argument to set_server_check_status const
    - CLEANUP: dumpstats: Make cli_release_handler() static
    - MEDIUM: server: Break out set weight processing code
    - MEDIUM: server: Allow relative weights greater than 100%
    - MEDIUM: server: Tighten up parsing of weight string
    - MEDIUM: checks: Add agent health check
    - BUG/MEDIUM: ssl: openssl 0.9.8 doesn't open /dev/random before chroot
    - BUG/MINOR: time: frequency counters are not totally accurate
    - BUG/MINOR: http: don't process abortonclose when request was sent
    - BUG/MEDIUM: stream_interface: don't close outgoing connections on shutw()
    - BUG/MEDIUM: checks: ignore late resets after valid responses
    - DOC: fix bogus recommendation on usage of gpc0 counter
    - BUG/MINOR: http-compression: lookup Cache-Control in the response, not the request
    - MINOR: signal: don't block SIGPROF by default
    - OPTIM: epoll: make use of EPOLLRDHUP
    - OPTIM: splice: detect shutdowns and avoid splice() == 0
    - OPTIM: splice: assume by default that splice is working correctly
    - BUG/MINOR: log: temporary fix for lost SSL info in some situations
    - BUG/MEDIUM: peers: only the last peers section was used by tables
    - BUG/MEDIUM: config: verbosely reject peers sections with multiple local peers
    - BUG/MINOR: epoll: use a fix maxevents argument in epoll_wait()
    - BUG/MINOR: config: fix improper check for failed memory alloc in ACL parser
    - BUG/MINOR: config: free peer's address when exiting upon parsing error
    - BUG/MINOR: config: check the proper variable when parsing log minlvl
    - BUG/MEDIUM: checks: ensure the health_status is always within bounds
    - BUG/MINOR: cli: show sess should always validate s->listener
    - BUG/MINOR: log: improper NULL return check on utoa_pad()
    - CLEANUP: http: remove a useless null check
    - CLEANUP: tcp/unix: remove useless NULL check in {tcp,unix}_bind_listener()
    - BUG/MEDIUM: signal: signal handler does not properly check for signal bounds
    - BUG/MEDIUM: tools: off-by-one in quote_arg()
    - BUG/MEDIUM: uri_auth: missing NULL check and memory leak on memory shortage
    - BUG/MINOR: unix: remove the 'level' field from the ux struct
    - CLEANUP: http: don't try to deinitialize http compression if it fails before init
    - CLEANUP: config: slowstart is never negative
    - CLEANUP: config: maxcompcpuusage is never negative
    - BUG/MEDIUM: log: emit '-' for empty fields again
    - BUG/MEDIUM: checks: fix a race condition between checks and observe layer7
    - BUILD: fix a warning emitted by isblank() on non-c99 compilers
    - BUILD: improve the makefile's support for libpcre
    - MEDIUM: halog: add support for counting per source address (-ic)
    - MEDIUM: tools: make str2sa_range support all address syntaxes
    - MEDIUM: config: make use of str2sa_range() instead of str2sa()
    - MEDIUM: config: use str2sa_range() to parse server addresses
    - MEDIUM: config: use str2sa_range() to parse peers addresses
    - MINOR: tests: add a config file to ease address parsing tests.
    - MINOR: ssl: add a global tunable for the max SSL/TLS record size
    - BUG/MINOR: syscall: fix NR_accept4 system call on sparc/linux
    - BUILD/MINOR: syscall: add definition of NR_accept4 for ARM
    - MINOR: config: report missing peers section name
    - BUG/MEDIUM: tools: fix bad character handling in str2sa_range()
    - BUG/MEDIUM: stats: never apply "unix-bind prefix" to the global stats socket
    - MINOR: tools: prepare str2sa_range() to return an error message
    - BUG/MEDIUM: checks: don't call connect() on unsupported address families
    - MINOR: tools: prepare str2sa_range() to accept a prefix
    - MEDIUM: tools: make str2sa_range() parse unix addresses too
    - MEDIUM: config: make str2listener() use str2sa_range() to parse unix addresses
    - MEDIUM: config: use a single str2sa_range() call to parse bind addresses
    - MEDIUM: config: use str2sa_range() to parse log addresses
    - CLEANUP: tools: remove str2sun() which is not used anymore.
    - MEDIUM: config: add complete support for str2sa_range() in dispatch
    - MEDIUM: config: add complete support for str2sa_range() in server addr
    - MEDIUM: config: add complete support for str2sa_range() in 'server'
    - MEDIUM: config: add complete support for str2sa_range() in 'peer'
    - MEDIUM: config: add complete support for str2sa_range() in 'source' and 'usesrc'
    - CLEANUP: minor cleanup in str2sa_range() and str2ip()
    - CLEANUP: config: do not use multiple errmsg at once
    - MEDIUM: tools: support specifying explicit address families in str2sa_range()
    - MAJOR: listener: support inheriting a listening fd from the parent
    - MAJOR: tools: support environment variables in addresses
    - BUG/MEDIUM: http: add-header should not emit "-" for empty fields
    - BUG/MEDIUM: config: ACL compatibility check on "redirect" was wrong
    - BUG/MEDIUM: http: fix another issue caused by http-send-name-header
    - DOC: mention the new HTTP 307 and 308 redirect statues
    - MEDIUM: poll: do not use FD_* macros anymore
    - BUG/MAJOR: ev_select: disable the select() poller if maxsock > FD_SETSIZE
    - BUG/MINOR: acl: ssl_fc_{alg,use}_keysize must parse integers, not strings
    - BUG/MINOR: acl: ssl_c_used, ssl_fc{,_has_crt,_has_sni} take no pattern
    - BUILD: fix usual isdigit() warning on solaris
    - BUG/MEDIUM: tools: vsnprintf() is not always reliable on Solaris
    - OPTIM: buffer: remove one jump in buffer_count()
    - OPTIM: http: improve branching in chunk size parser
    - OPTIM: http: optimize the response forward state machine
    - BUILD: enable poll() by default in the makefile
    - BUILD: add explicit support for Mac OS/X
    - BUG/MAJOR: http: use a static storage for sample fetch context
    - BUG/MEDIUM: ssl: improve error processing and reporting in ssl_sock_load_cert_list_file()
    - BUG/MAJOR: http: fix regression introduced by commit a890d072
    - BUG/MAJOR: http: fix regression introduced by commit d655ffe
    - BUG/CRITICAL: using HTTP information in tcp-request content may crash the process
    - MEDIUM: acl: remove flag ACL_MAY_LOOKUP which is improperly used
    - MEDIUM: samples: use new flags to describe compatibility between fetches and their usages
    - MINOR: log: indicate it when some unreliable sample fetches are logged
    - MEDIUM: samples: move payload-based fetches and ACLs to their own file
    - MINOR: backend: rename sample fetch functions and declare the sample keywords
    - MINOR: frontend: rename sample fetch functions and declare the sample keywords
    - MINOR: listener: rename sample fetch functions and declare the sample keywords
    - MEDIUM: http: unify acl and sample fetch functions
    - MINOR: session: rename sample fetch functions and declare the sample keywords
    - MAJOR: acl: make all ACLs reference the fetch function via a sample.
    - MAJOR: acl: remove the arg_mask from the ACL definition and use the sample fetch's
    - MAJOR: acl: remove fetch argument validation from the ACL struct
    - MINOR: http: add new direction-explicit sample fetches for headers and cookies
    - MINOR: payload: add new direction-explicit sample fetches
    - CLEANUP: acl: remove ACL hooks which were never used
    - MEDIUM: proxy: remove acl_requires and just keep a flag "http_needed"
    - MINOR: sample: provide a function to report the name of a sample check point
    - MAJOR: acl: convert all ACL requires to SMP use+val instead of ->requires
    - CLEANUP: acl: remove unused references to ACL_USE_*
    - MINOR: http: replace acl_parse_ver with acl_parse_str
    - MEDIUM: acl: move the ->parse, ->match and ->smp fields to acl_expr
    - MAJOR: acl: add option -m to change the pattern matching method
    - MINOR: acl: remove the use_count in acl keywords
    - MEDIUM: acl: have a pointer to the keyword name in acl_expr
    - MEDIUM: acl: support using sample fetches directly in ACLs
    - MEDIUM: http: remove val_usr() to validate user_lists
    - MAJOR: sample: maintain a per-proxy list of the fetch args to resolve
    - MINOR: ssl: add support for the "alpn" bind keyword
    - MINOR: http: status code 303 is HTTP/1.1 only
    - MEDIUM: http: implement redirect 307 and 308
    - MINOR: http: status 301 should not be marked non-cacheable
2013-04-03 02:26:31 +02:00
Willy Tarreau
ab861d3856 MINOR: ssl: add support for the "alpn" bind keyword
The ALPN extension is meant to replace the now deprecated NPN extension.
This patch implements support for it. It requires a version of openssl
with support for this extension. Patches are available here right now :

   http://html5labs.interopbridges.com/media/167447/alpn_patches.zip
2013-04-03 02:13:02 +02:00
Willy Tarreau
bef91e7144 MEDIUM: acl: support using sample fetches directly in ACLs
Now it becomes possible to directly use sample fetches as the ACL fetch
methods. In this case, the matching method is mandatory. This allows to
form more ACL combinations from existing fetches and will limit the need
for new ACLs when everything is available to form them from sample fetches
and matches.
2013-04-03 02:13:02 +02:00
Willy Tarreau
5adeda1f63 MAJOR: acl: add option -m to change the pattern matching method
ACL expressions now support "-m" in addition to "-i" and "-f". This new
option is followed by the name of the pattern matching method to be used
on the extracted pattern. This makes it possible to reuse existing sample
fetch methods with other matching methods (eg: regex). A "found" matching
method ignores any pattern and only verifies that the required sample was
found (useful for cookies).
2013-04-03 02:13:01 +02:00
Willy Tarreau
fa95734840 MINOR: payload: add new direction-explicit sample fetches
Similarly to previous commit fixing "hdr" and "cookie" in HTTP, we have to deal
with "payload" and "payload_lv" which are request-only for ACLs and req/resp for
sample fetches depending on the context, and to a less extent with other req_*
and rep_*/rep_* fetches. So let's add explicit "req." and "res." variants and
make the ACLs rely on that instead.
2013-04-03 02:12:59 +02:00
Willy Tarreau
18ed2569f5 MINOR: http: add new direction-explicit sample fetches for headers and cookies
Since "hdr" and "cookie" were ambiguously referring to the request or response
depending on the context, we need a way to explicitly specify the direction.
By prefixing the fetches names with "req." and "res.", we can now restrict such
fetches to the appropriate direction. At the moment the fetches are explicitly
declared by later we might think about having an automatic match when "req." or
"res." appears. These explicit fetches are now used by the relevant ACLs.
2013-04-03 02:12:59 +02:00
Willy Tarreau
281c799e25 MINOR: session: rename sample fetch functions and declare the sample keywords
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :

  sc1_bytes_in_rate, sc1_bytes_out_rate, sc1_clr_gpc0, sc1_conn_cnt,
  sc1_conn_cur, sc1_conn_rate, sc1_get_gpc0, sc1_http_err_cnt,
  sc1_http_err_rate, sc1_http_req_cnt, sc1_http_req_rate, sc1_inc_gpc0,
  sc1_kbytes_in, sc1_kbytes_out, sc1_sess_cnt, sc1_sess_rate, sc1_trackers,
  sc2_bytes_in_rate, sc2_bytes_out_rate, sc2_clr_gpc0, sc2_conn_cnt,
  sc2_conn_cur, sc2_conn_rate, sc2_get_gpc0, sc2_http_err_cnt,
  sc2_http_err_rate, sc2_http_req_cnt, sc2_http_req_rate, sc2_inc_gpc0,
  sc2_kbytes_in, sc2_kbytes_out, sc2_sess_cnt, sc2_sess_rate, sc2_trackers,
  src_bytes_in_rate, src_bytes_out_rate, src_clr_gpc0, src_conn_cnt,
  src_conn_cur, src_conn_rate, src_get_gpc0, src_http_err_cnt,
  src_http_err_rate, src_http_req_cnt, src_http_req_rate, src_inc_gpc0,
  src_kbytes_in, src_kbytes_out, src_sess_cnt, src_sess_rate,
  src_updt_conn_cnt, table_avl, table_cnt,

The fetch functions have been renamed "smp_fetch_*".
2013-04-03 02:12:58 +02:00
Willy Tarreau
409bcde176 MEDIUM: http: unify acl and sample fetch functions
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :

    cook, cook_cnt, cook_val, hdr_cnt, hdr_ip, hdr_val, http_auth,
    http_auth_group, http_first_req, method, req_proto_http, req_ver,
    resp_ver, scook, scook_cnt, scook_val, shdr, shdr_cnt, shdr_ip,
    shdr_val, status, urlp, urlp_val,

Most of them won't bring much benefit at the moment, or are even aliases of
existing ones, however they'll be needed for ACL->SMP convergence.

A new val_usr() function was added to resolve userlist names into pointers.

The http_auth_group ACL forgot to make its first argument mandatory, so
there was a check in cfgparse to report a vague error. Now that args are
correctly parsed, let's report something more precise.

All urlp* ACLs now support an optional 3rd argument like their sample
counter-part which is the optional delimiter.

The fetch functions have been renamed "smp_fetch_*".

Some args controls on the sample keywords have been relaxed so that we
can soon use them for ACLs :

  - cookie now accepts to have an optional name ; it will return the
    first matching cookie if the name is not set ;
  - same for set-cookie and hdr
2013-04-03 02:12:57 +02:00
Willy Tarreau
0ccb744ffb MINOR: listener: rename sample fetch functions and declare the sample keywords
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :

          dst_conn, so_id,

The fetch functions have been renamed "smp_fetch_*".
2013-04-03 02:12:57 +02:00
Willy Tarreau
d6896bc72a MINOR: frontend: rename sample fetch functions and declare the sample keywords
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :

      fe_conn, fe_id, fe_sess_rate

The fetch functions have been renamed "smp_fetch_*".
2013-04-03 02:12:57 +02:00
Willy Tarreau
1a7eca19b8 MINOR: backend: rename sample fetch functions and declare the sample keywords
The following sample fetch functions were only usable by ACLs but are now
usable by sample fetches too :

  avg_queue, be_conn, be_id, be_sess_rate, connslots, nbsrv,
  queue, srv_conn, srv_id, srv_is_up, srv_sess_rate

The fetch functions have been renamed "smp_fetch_*".
2013-04-03 02:12:57 +02:00
Willy Tarreau
d4c33c8889 MEDIUM: samples: move payload-based fetches and ACLs to their own file
The file acl.c is a real mess, it both contains functions to parse and
process ACLs, and some sample extraction functions which act on buffers.
Some other payload analysers were arbitrarily dispatched to proto_tcp.c.

So now we're moving all payload-based fetches and ACLs to payload.c
which is capable of extracting data from buffers and rely on everything
that is protocol-independant. That way we can safely inflate this file
and only use the other ones when some fetches are really specific (eg:
HTTP, SSL, ...).

As a result of this cleanup, the following new sample fetches became
available even if they're not really useful :

  always_false, always_true, rep_ssl_hello_type, rdp_cookie_cnt,
  req_len, req_ssl_hello_type, req_ssl_sni, req_ssl_ver, wait_end

The function 'acl_fetch_nothing' was wrong and never used anywhere so it
was removed.

The "rdp_cookie" sample fetch used to have a mandatory argument while it
was optional in ACLs, which are supposed to iterate over RDP cookies. So
we're making it optional as a fetch too, and it will return the first one.
2013-04-03 02:12:57 +02:00
Lukas Tribus
0999f7662c BUILD: add explicit support for TFO with USE_TFO
TCP Fast Open is supported in server mode since Linux 3.7, but current
libc's don't define TCP_FASTOPEN=23. Introduce the new USE flag USE_TFO
to define it manually in compat.h. Also note this in the TFO related
documentation.
2013-04-02 17:40:43 +02:00
Willy Tarreau
ad1731d553 BUG/MEDIUM: ssl: improve error processing and reporting in ssl_sock_load_cert_list_file()
fe61656b added the ability to load a list of certificates from a file,
but error control was incomplete and misleading, as some errors such
as missing files were not reported, and errors reported with Alert()
instead of memprintf() were inappropriate and mixed with upper errors.
Also, the code really supports a single SNI filter right now, so let's
correct it and the doc for that, leaving room for later change if needed.
2013-04-02 17:39:04 +02:00
Emmanuel Hocdet
fe61656bb2 MEDIUM: ssl: add mapping from SNI to cert file using "crt-list"
It designates a list of PEM file with an optional list of SNI filter
per certificate, with the following format for each line :

    <crtfile>[ <snifilter>]*

Wildcards are supported in the SNI filter. The certificates will be
presented to clients who provide a valid TLS Server Name Indication
field matching one of SNI filter. If no SNI filter is specified the
CN and alt subjects are used.
2013-04-02 16:59:19 +02:00
Alex Davies
0fbf016508 DOCS: Add explanation of intermediate certs to crt paramater
This change makes the "crt" block of the documentation easier to use
for those not clear on what needs to go in what file, specifically for
those using CAs that require intermediate certificates.
2013-04-01 23:38:46 +02:00
Willy Tarreau
b67fdc4cd8 DOC: mention the new HTTP 307 and 308 redirect statues 2013-03-30 11:22:59 +01:00
Willy Tarreau
dad36a3ee3 MAJOR: tools: support environment variables in addresses
Now that all addresses are parsed using str2sa_range(), it becomes easy
to add support for environment variables and use them everywhere an address
is needed. Environment variables are used as $VAR or ${VAR} as in shell.
Any number of variables may compose an address, allowing various fantasies
such as "fd@${FD_HTTP}" or "${LAN_DC1}.1:80".

These ones are usable in logs, bind, servers, peers, stats socket, source,
dispatch, and check address.
2013-03-11 01:30:02 +01:00
Willy Tarreau
40aa070c51 MAJOR: listener: support inheriting a listening fd from the parent
Using the address syntax "fd@<num>", a listener may inherit a file
descriptor that the caller process has already bound and passed as
this number. The fd's socket family is detected using getsockname(),
and the usual initialization is performed through the existing code
for that family, but the socket creation is skipped.

Whether the parent has performed the listen() call or not is not
important as this is detected.

For UNIX sockets, we immediately clear the path after preparing a
socket so that we never remove it in case an abort would happen due
to a late error during startup.
2013-03-11 01:30:01 +01:00
Willy Tarreau
24709286fe MEDIUM: tools: support specifying explicit address families in str2sa_range()
This change allows one to force the address family in any address parsed
by str2sa_range() by specifying it as a prefix followed by '@' then the
address. Currently supported address prefixes are 'ipv4@', 'ipv6@', 'unix@'.
This also helps forcing resolving for host names (when getaddrinfo is used),
and force the family of the empty address (eg: 'ipv4@' = 0.0.0.0 while
'ipv6@' = ::).

The main benefits is that unix sockets can now get a local name without
being forced to begin with a slash. This is useful during development as
it is no longer necessary to have stats socket sent to /tmp.
2013-03-10 22:46:55 +01:00
Emeric Brun
6924ef8b12 BUG/MEDIUM: ssl: ECDHE ciphers not usable without named curve configured.
Fix consists to use prime256v1 as default named curve to init ECDHE ciphers if none configured.
2013-03-06 19:08:26 +01:00
Willy Tarreau
bfd5946aa1 MINOR: ssl: add a global tunable for the max SSL/TLS record size
Add new tunable "tune.ssl.maxrecord".

Over SSL/TLS, the client can decipher the data only once it has received
a full record. With large records, it means that clients might have to
download up to 16kB of data before starting to process them. Limiting the
record size can improve page load times on browsers located over high
latency or low bandwidth networks. It is suggested to find optimal values
which fit into 1 or 2 TCP segments (generally 1448 bytes over Ethernet
with TCP timestamps enabled, or 1460 when timestamps are disabled), keeping
in mind that SSL/TLS add some overhead. Typical values of 1419 and 2859
gave good results during tests. Use "strace -e trace=write" to find the
best value.

This trick was first suggested by Mike Belshe :

   http://www.belshe.com/2010/12/17/performance-and-the-tls-record-size/

Then requested again by Ilya Grigorik who provides some hints here :

   http://ofps.oreilly.com/titles/9781449344764/_transport_layer_security_tls.html#ch04_00000101
2013-02-21 07:53:13 +01:00
Lukas Tribus
0defb90784 DOC: tfo: bump required kernel to linux-3.7
Support for server side TFO was actually introduced in linux-3.7,
linux-3.6 just has client support.

This patch fixes documentation and a code comment about the
kernel requirement. It also fixes a wrong tfo related code
comment in src/proto_tcp.c.
2013-02-14 00:03:04 +01:00
Baptiste Assmann
ea3e73b931 DOC: minor typo fix in documentation 2013-02-13 12:49:46 +01:00
Simon Horman
a2b9dadedd MEDIUM: checks: Add agent health check
Support a agent health check performed by opening a TCP socket to a
pre-defined port and reading an ASCII string. The string should have one of
the following forms:

* An ASCII representation of an positive integer percentage.
  e.g. "75%"

  Values in this format will set the weight proportional to the initial
  weight of a server as configured when haproxy starts.

* The string "drain".

  This will cause the weight of a server to be set to 0, and thus it will
  not accept any new connections other than those that are accepted via
  persistence.

* The string "down", optionally followed by a description string.

  Mark the server as down and log the description string as the reason.

* The string "stopped", optionally followed by a description string.

  This currently has the same behaviour as down (iii).

* The string "fail", optionally followed by a description string.

  This currently has the same behaviour as down (iii).

A agent health check may be configured using "option lb-agent-chk".
The use of an alternate check-port, used to obtain agent heath check
information described above as opposed to the port of the service,
may be useful in conjunction with this option.

e.g.

    option  lb-agent-chk
    server  http1_1 10.0.0.10:80 check port 10000 weight 100

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-02-13 11:03:28 +01:00
Simon Horman
58b5d292b3 MEDIUM: server: Allow relative weights greater than 100%
Allow relative weights greater than 100%,
capping the absolute value to 256 which is
the largest supported absolute weight.

Signed-off-by: Simon Horman <horms@verge.net.au>
2013-02-13 10:56:28 +01:00
Lukas Tribus
fce2e96b9d DOC: simplify bind option "interface" explanation
The current documentation of the bind option "interface" can be misleading
(as seen on the ML recently).

This patch tries to address misunderstandings by :

 - avoiding the words listen or bind in the behavior description, using
 "restrict to interface" instead

 - using a different sentence construction (partially stolen from
 "man 7 socket": SO_BINDTODEVICE)

 - "defragmentation": moving behavior related explanations to the beginning
 and restrictions, use-cases and requirements to the end.
2013-02-12 23:42:11 +01:00
Emmanuel Hocdet
656233715e MEDIUM: ssl: add bind-option "strict-sni"
This new option ensures that there is no possible fallback to a default
certificate if the client does not provide an SNI which is explicitly
handled by a certificate.
2013-01-24 17:23:33 +01:00
Michael Scherer
ab012dd394 BUG/MEDIUM: remove supplementary groups when changing gid
Without it, haproxy will retain the group membership of root, which may
give more access than intended to the process. For example, haproxy would
still be in the wheel group on Fedora 18, as seen with :

  # haproxy -f /etc/haproxy/haproxy.cfg

  # ps a -o pid,user,group,command | grep hapr
  3545 haproxy  haproxy  haproxy -f /etc/haproxy/haproxy.cfg
  4356 root     root     grep --color=auto hapr
  # grep Group /proc/3545/status
  Groups: 0 1 2 3 4 6 10
  # getent group wheel
  wheel10:root,misc

[WT: The issue has been investigated by independent security research team
     and realized by itself not being able to allow security exploitation.
     Additionally, dropping groups is not allowed to unprivileged users,
     though this mode of deployment is quite common. Thus a warning is
     emitted in this case to inform the user. The fix could be backported
     into all supported versions as the issue has always been there. ]
2013-01-18 10:25:25 +01:00