haproxy/README
Willy Tarreau 5c0e41b7cb [RELEASE] Released version 1.9-dev7
Released version 1.9-dev7 with the following main changes :
    - BUILD: cache: fix a build warning regarding too large an integer for the age
    - CLEANUP: fix typos in the comments of the Makefile
    - CLEANUP: fix a typo in a comment for the contrib/halog subsystem
    - CLEANUP: fix typos in comments for the contrib/modsecurity subsystem
    - CLEANUP: fix typos in comments for contrib/spoa_example
    - CLEANUP: fix typos in comments for contrib/wireshark-dissectors
    - DOC: Fix typos in README and CONTRIBUTING
    - MINOR: log: slightly improve error message syntax on log failure
    - DOC: logs: the format directive was missing from the second log part
    - MINOR: log: report the number of dropped logs in the stats
    - MEDIUM: log: add support for logging to existing file descriptors
    - MEDIUM: log: support a new "short" format
    - MEDIUM: log: add a new "raw" format
    - BUG/MEDIUM: stream-int: change the way buffer room is requested by a stream-int
    - BUG/MEDIUM: stream-int: convert some co_data() checks to channel_is_empty()
    - MINOR: namespaces: don't build namespace.c if disabled
    - BUILD/MEDIUM: threads/affinity: DragonFly build fix
    - MINOR: http: Add new "early-hint" http-request action.
    - MINOR: http: Make new "early-hint" http-request action really be parsed.
    - MINOR: http: Implement "early-hint" http request rules.
    - MINOR: doc: Add information about "early-hint" http-request action.
    - DOC: early-hints: fix truncated line.
    - MINOR: mworker: only close std{in,out,err} in daemon mode
    - BUG/MEDIUM: log: don't CLOEXEC the inherited FDs
    - BUG/MEDIUM: Make sure stksess is properly aligned.
    - BUG/MEDIUM: stream-int: make failed splice_in always subscribe to recv
    - BUG/MEDIUM: stream-int: clear CO_FL_WAIT_ROOM after splicing data in
    - BUG/MINOR: stream-int: make sure not to go through the rcv_buf path after splice()
    - CONTRIB: debug: fix build related to conn_stream flags change
    - REGTEST: fix scripts 1 and 3 to accept development version
    - BUG/MINOR: http_fetch: Remove the version part when capturing the request uri
    - MINOR: http: Regroup return statements of http_req_get_intercept_rule at the end
    - MINOR: http: Regroup return statements of http_res_get_intercept_rule at the end
    - BUG/MINOR: http: Be sure to sent fully formed HTTP 103 responses
    - MEDIUM: jobs: support unstoppable jobs for soft stop
    - MEDIUM: listeners: support unstoppable listener
    - MEDIUM: cli: worker socketpair is unstoppable
    - BUG/MINOR: stream-int: set SI_FL_WANT_PUT in sess_establish()
    - MINOR: stream: move the conn_stream specific calls to the stream-int
    - BUG/MINOR: config: Copy default error messages when parsing of a backend starts
    - CLEANUP: h2: minimum documentation for recent API changes
    - MINOR: mux: implement a get_first_cs() method
    - MINOR: stream-int: make conn_si_send_proxy() use cs_get_first()
    - MINOR: stream-int: relax the forwarding rules in stream_int_notify()
    - MINOR: stream-int: expand the flags to 32-bit
    - MINOR: stream-int: rename SI_FL_WAIT_ROOM to SI_FL_RXBLK_ROOM
    - MINOR: stream-int: introduce new SI_FL_RXBLK flags
    - MINOR: stream-int: add new functions si_{rx,tx}_{blocked,endp_ready}()
    - MINOR: stream-int: replace SI_FL_WANT_PUT with !SI_FL_RX_WAIT_EP
    - MINOR: stream-int: use si_rx_blocked()/si_tx_blocked() to check readiness
    - MEDIUM: stream-int: use si_rx_buff_{rdy,blk} to report buffer readiness
    - MINOR: stream-int: replace si_{want,stop}_put() with si_rx_endp_{more,done}()
    - MEDIUM: stream-int: update the endp polling status only at the end of si_cs_recv()
    - MINOR: stream-int: make si_sync_recv() simply check ENDP before si_cs_recv()
    - MINOR: stream-int: automatically mark applets as ready if they block on the channel
    - MEDIUM: stream-int: fix the si_cant_put() calls used for end point readiness
    - MEDIUM: stream-int: fix the si_cant_put() calls used for buffer readiness
    - MEDIUM: stream-int: use si_rx_shut_blk() to indicate the SI is closed
    - MEDIUM: stream-int: unconditionally call si_chk_rcv() in update and notify
    - MEDIUM: stream-int: make use of si_rx_chan_{rdy,blk} to control the stream-int from the channel
    - MINOR: stream-int: replace si_cant_put() with si_rx_room_{blk,rdy}()
    - MEDIUM: connections: Wait until the connection is established to try to recv.
    - MEDIUM: mux: Teach the mux_pt how to deal with idle connections.
    - MINOR: mux: Add a new "avail_streams" method.
    - MINOR: mux: Add a destroy() method.
    - MINOR: sessions: Start to store the outgoing connection in sessions.
    - MAJOR: connections: Detach connections from streams.
    - MINOR: conn_stream: Add a flag to notify the mux it should flush its buffers
    - MINOR: htx: Add proto_htx.c file
    - MINOR: conn_stream: Add a flag to notify the mux it must respect the reserve
    - MINOR: http: Add standalone functions to parse a start-line or a header
    - MINOR: http: Call http_send_name_header with the stream instead of the txn
    - MINOR: conn_stream: Add a flag to notify the SI some data were received
    - MINOR: http: Add macros to check if a stream uses the HTX representation
    - MEDIUM: proto_htx: Add HTX analyzers and use it when the mux H1 is used
    - MEDIUM: mux-h1: Add dummy mux to handle HTTP/1.1 connections
    - MEDIUM: mux-h1: Add parsing of incoming and ougoing HTTP messages
    - MAJOR: mux-h1/proto_htx: Handle keep-alive connections in the mux
    - MEDIUM: mux-h1: Add support of the kernel TCP splicing to forward data
    - MEDIUM: htx: Add API to deal with the internal representation of HTTP messages
    - MINOR: http_htx: Add functions to manipulate HTX messages in http_htx.c
    - MINOR: proto_htx: Add some functions to handle HTX messages
    - MAJOR: mux-h1/proto_htx: Switch mux-h1 and HTX analyzers on the HTX representation
    - MINOR: http_htx: Add functions to replace part of the start-line
    - MINOR: http_htx: Add functions to retrieve a specific occurrence of a header
    - MINOR: proto_htx: Rewrite htx_apply_redirect_rule to handle HTX messages
    - MINOR: proto_htx: Add the internal function htx_del_hdr_value
    - MINOR: proto_htx: Add the internal function htx_fmt_res_line
    - MINOR: proto_htx: Add functions htx_transform_header and htx_transform_header_str
    - MINOR: proto_htx: Add functions htx_req_replace_stline and htx_res_set_status
    - MINOR: proto_htx: Add function to build and send HTTP 103 responses
    - MINOR: proto_htx: Add functions htx_req_get_intercept_rule and htx_res_get_intercept_rule
    - MINOR: proto_htx: Add functions to apply req* and rsp* rules on HTX messages
    - MINOR: proto_htx: Add functions to manage cookies on HTX messages
    - MINOR: proto_htx: Add functions to check the cacheability of HTX messages
    - MINOR: proto_htx: Add functions htx_send_name_header
    - MINOR: proto_htx: Add functions htx_perform_server_redirect
    - MINOR: proto_htx: Add functions to handle the stats applet
    - MEDIUM: proto_htx: Adapt htx_process_req_common to handle HTX messages
    - MEDIUM: proto_htx: Adapt htx_process_request to handle HTX messages
    - MINOR: proto_htx: Adapt htx_process_tarpit to handle HTX messages
    - MEDIUM: proto_htx: Adapt htx_wait_for_request_body to handle HTX messages
    - MEDIUM: proto_htx: Adapt htx_process_res_common to handle HTX messages
    - MINOR: http_fetch: Add smp_prefetch_htx
    - MEDIUM: http_fetch: Adapt all fetches to handle HTX messages
    - MEDIUM: mux-h1: Wait for connection establishment before consuming channel's data
    - MINOR: stats/htx: Adapt the stats applet to handle HTX messages
    - MINOR: stream: Don't reset sov value with HTX messages
    - MEDIUM: mux-h1: Handle errors and timeouts in the stream
    - MINOR: filters/htx: Forbid filters when the HTX is enabled on a proxy
    - MINOR: lua/htx: Forbid lua usage when the HTX is enabled on a proxy
    - CLEANUP: Fix some typos in the haproxy subsystem
    - CLEANUP: Fix typos in the dns subsystem
    - CLEANUP: Fix typos in the pattern subsystem
    - CLEANUP: fix 2 typos in the xxhash subsystem
    - CLEANUP: fix a few typos in the comments of the server subsystem
    - CLEANUP: fix a misspell in tests/filltab25.c
    - CLEANUP: fix a typo found in the stream subsystem
    - CLEANUP: fix typos in comments in ebtree
    - CLEANUP: fix typos in reg-tests
    - CLEANUP: fix typos in the comments of the vars subsystem
    - CLEANUP: fix typos in the hlua_fcn subsystem
    - CLEANUP: fix typos in the proto_http subsystem
    - CLEANUP: fix typos in the proxy subsystem
    - CLEANUP: fix typos in the ssl_sock subsystem
    - DOC: Fix typos in different subsections of the documentation
    - DOC: fix a few typos in the documentation
    - MINOR: Fix an error message thrown when we run out of memory
    - MINOR: Fix typos in error messages in the proxy subsystem
    - MINOR: fix typos in the examples files
    - CLEANUP: Fix a typo in the stats subsystem
    - CLEANUP: Fix typos in the acl subsystem
    - CLEANUP: Fix typos in the cache subsystem
    - CLEANUP: Fix typos in the cfgparse subsystem
    - CLEANUP: Fix typos in the filters subsystem
    - CLEANUP: Fix typos in the http subsystem
    - CLEANUP: Fix typos in the log subsystem
    - CLEANUP: Fix typos in the peers subsystem
    - CLEANUP: Fix typos in the regex subsystem
    - CLEANUP: Fix typos in the sample subsystem
    - CLEANUP: Fix typos in the spoe subsystem
    - CLEANUP: Fix typos in the standard subsystem
    - CLEANUP: Fix typos in the stick_table subsystem
    - CLEANUP: Fix typos in the task subsystem
    - MINOR: Fix typo in error message in the standard subsystem
    - CLEANUP: fix typos in the comments of hlua
    - MINOR: Fix typo in the error 500 output of hlua
    - MINOR: Fix a typo in a warning message in the spoe subsystem
2018-11-18 22:33:00 +01:00

329 lines
15 KiB
Plaintext
Raw Blame History

----------------------
HAProxy how-to
----------------------
version 1.9
willy tarreau
2018/11/18
1) How to build it
------------------
This is a development version, so it is expected to break from time to time,
to add and remove features without prior notification and it should not be used
in production. If you are not used to build from sources or if you are not used
to follow updates then it is recommended that instead you use the packages provided
by your software vendor or Linux distribution. Most of them are taking this task
seriously and are doing a good job at backporting important fixes. If for any
reason you'd prefer a different version than the one packaged for your system,
you want to be certain to have all the fixes or to get some commercial support,
other choices are available at :
http://www.haproxy.com/
To build haproxy, you will need :
- GNU make. Neither Solaris nor OpenBSD's make work with the GNU Makefile.
If you get many syntax errors when running "make", you may want to retry
with "gmake" which is the name commonly used for GNU make on BSD systems.
- GCC between 2.95 and 4.8. Others may work, but not tested.
- GNU ld
Also, you might want to build with libpcre support, which will provide a very
efficient regex implementation and will also fix some badness on Solaris' one.
To build haproxy, you have to choose your target OS amongst the following ones
and assign it to the TARGET variable :
- linux22 for Linux 2.2
- linux24 for Linux 2.4 and above (default)
- linux24e for Linux 2.4 with support for a working epoll (> 0.21)
- linux26 for Linux 2.6 and above
- linux2628 for Linux 2.6.28, 3.x, and above (enables splice and tproxy)
- solaris for Solaris 8 or 10 (others untested)
- freebsd for FreeBSD 5 to 10 (others untested)
- netbsd for NetBSD
- osx for Mac OS/X
- openbsd for OpenBSD 5.7 and above
- aix51 for AIX 5.1
- aix52 for AIX 5.2
- cygwin for Cygwin
- haiku for Haiku
- generic for any other OS or version.
- custom to manually adjust every setting
You may also choose your CPU to benefit from some optimizations. This is
particularly important on UltraSparc machines. For this, you can assign
one of the following choices to the CPU variable :
- i686 for intel PentiumPro, Pentium 2 and above, AMD Athlon
- i586 for intel Pentium, AMD K6, VIA C3.
- ultrasparc : Sun UltraSparc I/II/III/IV processor
- native : use the build machine's specific processor optimizations. Use with
extreme care, and never in virtualized environments (known to break).
- generic : any other processor or no CPU-specific optimization. (default)
Alternatively, you may just set the CPU_CFLAGS value to the optimal GCC options
for your platform.
You may want to build specific target binaries which do not match your native
compiler's target. This is particularly true on 64-bit systems when you want
to build a 32-bit binary. Use the ARCH variable for this purpose. Right now
it only knows about a few x86 variants (i386,i486,i586,i686,x86_64), two
generic ones (32,64) and sets -m32/-m64 as well as -march=<arch> accordingly.
If your system supports PCRE (Perl Compatible Regular Expressions), then you
really should build with libpcre which is between 2 and 10 times faster than
other libc implementations. Regex are used for header processing (deletion,
rewriting, allow, deny). The only inconvenient of libpcre is that it is not
yet widely spread, so if you build for other systems, you might get into
trouble if they don't have the dynamic library. In this situation, you should
statically link libpcre into haproxy so that it will not be necessary to
install it on target systems. Available build options for PCRE are :
- USE_PCRE=1 to use libpcre, in whatever form is available on your system
(shared or static)
- USE_STATIC_PCRE=1 to use a static version of libpcre even if the dynamic
one is available. This will enhance portability.
- with no option, use your OS libc's standard regex implementation (default).
Warning! group references on Solaris seem broken. Use static-pcre whenever
possible.
If your system doesn't provide PCRE, you are encouraged to download it from
http://www.pcre.org/ and build it yourself, it's fast and easy.
Recent systems can resolve IPv6 host names using getaddrinfo(). This primitive
is not present in all libcs and does not work in all of them either. Support in
glibc was broken before 2.3. Some embedded libs may not properly work either,
thus, support is disabled by default, meaning that some host names which only
resolve as IPv6 addresses will not resolve and configs might emit an error
during parsing. If you know that your OS libc has reliable support for
getaddrinfo(), you can add USE_GETADDRINFO=1 on the make command line to enable
it. This is the recommended option for most Linux distro packagers since it's
working fine on all recent mainstream distros. It is automatically enabled on
Solaris 8 and above, as it's known to work.
It is possible to add native support for SSL using the GNU makefile, by passing
"USE_OPENSSL=1" on the make command line. The libssl and libcrypto will
automatically be linked with haproxy. Some systems also require libz, so if the
build fails due to missing symbols such as deflateInit(), then try again with
"ADDLIB=-lz".
Your are strongly encouraged to always use an up-to-date version of OpenSSL, as
found on https://www.openssl.org/ as vulnerabilities are occasionally found and
you don't want them on your systems. HAProxy is known to build correctly on all
currently supported branches (0.9.8, 1.0.0, 1.0.1, 1.0.2 and 1.1.0 at the time
of writing). Branch 1.0.2 is currently recommended for the best combination of
features and stability. Asynchronous engines require OpenSSL 1.1.0 though. It's
worth mentioning that some OpenSSL derivatives are also reported to work but
may occasionally break. Patches to fix them are welcome but please read the
CONTRIBUTING file first.
To link OpenSSL statically against haproxy, build OpenSSL with the no-shared
keyword and install it to a local directory, so your system is not affected :
$ export STATICLIBSSL=/tmp/staticlibssl
$ ./config --prefix=$STATICLIBSSL no-shared
$ make && make install_sw
When building haproxy, pass that path via SSL_INC and SSL_LIB to make and
include additional libs with ADDLIB if needed (in this case for example libdl):
$ make TARGET=linux26 USE_OPENSSL=1 SSL_INC=$STATICLIBSSL/include SSL_LIB=$STATICLIBSSL/lib ADDLIB=-ldl
It is also possible to include native support for zlib to benefit from HTTP
compression. For this, pass "USE_ZLIB=1" on the "make" command line and ensure
that zlib is present on the system. Alternatively it is possible to use libslz
for a faster, memory less, but slightly less efficient compression, by passing
"USE_SLZ=1".
Zlib is commonly found on most systems, otherwise updates can be retrieved from
http://www.zlib.net/. It is easy and fast to build. Libslz can be downloaded
from http://1wt.eu/projects/libslz/ and is even easier to build.
By default, the DEBUG variable is set to '-g' to enable debug symbols. It is
not wise to disable it on uncommon systems, because it's often the only way to
get a complete core when you need one. Otherwise, you can set DEBUG to '-s' to
strip the binary.
For example, I use this to build for Solaris 8 :
$ make TARGET=solaris CPU=ultrasparc USE_STATIC_PCRE=1
And I build it this way on OpenBSD or FreeBSD :
$ gmake TARGET=freebsd USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
And on a classic Linux with SSL and ZLIB support (eg: Red Hat 5.x) :
$ make TARGET=linux26 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
And on a recent Linux >= 2.6.28 with SSL and ZLIB support :
$ make TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1
In order to build a 32-bit binary on an x86_64 Linux system with SSL support
without support for compression but when OpenSSL requires ZLIB anyway :
$ make TARGET=linux26 ARCH=i386 USE_OPENSSL=1 ADDLIB=-lz
The SSL stack supports session cache synchronization between all running
processes. This involves some atomic operations and synchronization operations
which come in multiple flavors depending on the system and architecture :
Atomic operations :
- internal assembler versions for x86/x86_64 architectures
- gcc builtins for other architectures. Some architectures might not
be fully supported or might require a more recent version of gcc.
If your architecture is not supported, you willy have to either use
pthread if supported, or to disable the shared cache.
- pthread (posix threads). Pthreads are very common but inter-process
support is not that common, and some older operating systems did not
report an error when enabling multi-process mode, so they used to
silently fail, possibly causing crashes. Linux's implementation is
fine. OpenBSD doesn't support them and doesn't build. FreeBSD 9 builds
and reports an error at runtime, while certain older versions might
silently fail. Pthreads are enabled using USE_PTHREAD_PSHARED=1.
Synchronization operations :
- internal spinlock : this mode is OS-independent, light but will not
scale well to many processes. However, accesses to the session cache
are rare enough that this mode could certainly always be used. This
is the default mode.
- Futexes, which are Linux-specific highly scalable light weight mutexes
implemented in user-space with some limited assistance from the kernel.
This is the default on Linux 2.6 and above and is enabled by passing
USE_FUTEX=1
- pthread (posix threads). See above.
If none of these mechanisms is supported by your platform, you may need to
build with USE_PRIVATE_CACHE=1 to totally disable SSL cache sharing. Then
it is better not to run SSL on multiple processes.
If you need to pass other defines, includes, libraries, etc... then please
check the Makefile to see which ones will be available in your case, and
use the USE_* variables in the Makefile.
AIX 5.3 is known to work with the generic target. However, for the binary to
also run on 5.2 or earlier, you need to build with DEFINE="-D_MSGQSUPPORT",
otherwise __fd_select() will be used while not being present in the libc, but
this is easily addressed using the "aix52" target. If you get build errors
because of strange symbols or section mismatches, simply remove -g from
DEBUG_CFLAGS.
You can easily define your own target with the GNU Makefile. Unknown targets
are processed with no default option except USE_POLL=default. So you can very
well use that property to define your own set of options. USE_POLL can even be
disabled by setting USE_POLL="". For example :
$ gmake TARGET=tiny USE_POLL="" TARGET_CFLAGS=-fomit-frame-pointer
1.1) Device Detection
---------------------
HAProxy supports several device detection modules relying on third party
products. Some of them may provide free code, others free libs, others free
evaluation licenses. Please read about their respective details in the
following files :
doc/DeviceAtlas-device-detection.txt for DeviceAtlas
doc/51Degrees-device-detection.txt for 51Degrees
doc/WURFL-device-detection.txt for Scientiamobile WURFL
2) How to install it
--------------------
To install haproxy, you can either copy the single resulting binary to the
place you want, or run :
$ sudo make install
If you're packaging it for another system, you can specify its root directory
in the usual DESTDIR variable.
3) How to set it up
-------------------
There is some documentation in the doc/ directory :
- intro.txt : this is an introduction to haproxy, it explains what it is
what it is not. Useful for beginners or to re-discover it when planning
for an upgrade.
- architecture.txt : this is the architecture manual. It is quite old and
does not tell about the nice new features, but it's still a good starting
point when you know what you want but don't know how to do it.
- configuration.txt : this is the configuration manual. It recalls a few
essential HTTP basic concepts, and details all the configuration file
syntax (keywords, units). It also describes the log and stats format. It
is normally always up to date. If you see that something is missing from
it, please report it as this is a bug. Please note that this file is
huge and that it's generally more convenient to review Cyril Bont<6E>'s
HTML translation online here :
http://cbonte.github.io/haproxy-dconv/configuration-1.6.html
- management.txt : it explains how to start haproxy, how to manage it at
runtime, how to manage it on multiple nodes, how to proceed with seamless
upgrades.
- gpl.txt / lgpl.txt : the copy of the licenses covering the software. See
the 'LICENSE' file at the top for more information.
- the rest is mainly for developers.
There are also a number of nice configuration examples in the "examples"
directory as well as on several sites and articles on the net which are linked
to from the haproxy web site.
4) How to report a bug
----------------------
It is possible that from time to time you'll find a bug. A bug is a case where
what you see is not what is documented. Otherwise it can be a misdesign. If you
find that something is stupidly design, please discuss it on the list (see the
"how to contribute" section below). If you feel like you're proceeding right
and haproxy doesn't obey, then first ask yourself if it is possible that nobody
before you has even encountered this issue. If it's unlikely, the you probably
have an issue in your setup. Just in case of doubt, please consult the mailing
list archives :
http://marc.info/?l=haproxy
Otherwise, please try to gather the maximum amount of information to help
reproduce the issue and send that to the mailing list :
haproxy@formilux.org
Please include your configuration and logs. You can mask your IP addresses and
passwords, we don't need them. But it's essential that you post your config if
you want people to guess what is happening.
Also, keep in mind that haproxy is designed to NEVER CRASH. If you see it die
without any reason, then it definitely is a critical bug that must be reported
and urgently fixed. It has happened a couple of times in the past, essentially
on development versions running on new architectures. If you think your setup
is fairly common, then it is possible that the issue is totally unrelated.
Anyway, if that happens, feel free to contact me directly, as I will give you
instructions on how to collect a usable core file, and will probably ask for
other captures that you'll not want to share with the list.
5) How to contribute
--------------------
Please carefully read the CONTRIBUTING file that comes with the sources. It is
mandatory.
-- end