4088 Commits

Author SHA1 Message Date
Willy Tarreau
6457d0fac3 CLEANUP: cli: ensure we can never double-free error messages
The release handler used to be called twice for some time and just by
pure luck we never ended up double-freeing the data there. Add a NULL
to ensure this can never happen should a future change permit this
situation again.
2015-10-07 20:00:24 +02:00
Andrew Hayworth
68d0534885 MINOR: cli: Dump all resolvers stats if no resolver section is given
This commit adds support for dumping all resolver stats. Specifically
if a command 'show stats resolvers' is issued withOUT a resolver section
id, we dump all known resolver sections. If none are configured, a
message is displayed indicating that.
2015-10-06 07:08:09 +02:00
Ben Cabot
49795eb00c BUG: config: external-check command validation is checking for incorrect arguments.
When using the external-check command option HAProxy was failing to
start with a fatal error "'external-check' cannot handle unexpected
argument". When looking at the code it was looking for an incorrect
argument. Also correcting an Alert message text as spotted by by
PiBa-NL.
2015-10-02 23:11:49 +02:00
Thierry FOURNIER
ab95e656ea MINOR: http/tcp: fill the avalaible actions
This patch adds a function that generates the list of avalaible actions
for the error message.
2015-10-02 22:56:11 +02:00
Thierry FOURNIER
56da1012d2 MINOR: lua: rename the tune.lua.applet-timeout
The name of applet is "service", so this patch renames the
tune.lua.applet-timeout to tune.lua.service-timeout
2015-10-02 22:56:10 +02:00
Dmitry Sivachenko
eab7f3996f BUG/MEDIUM: str2ip: make getaddrinfo() consider local address selection policy
When first parameter to getaddrinfo() is not NULL (it is always not NULL
in str2ip()), on Linux AI_PASSIVE value for ai_flags is ignored. On
FreeBSD, when AI_PASSIVE is specified and hostname parameter is not NULL,
getaddrinfo() ignores local address selection policy, always returning
AAAA record. Pass zero ai_flags to behave correctly on FreeBSD, this
change should be no-op for Linux.

This fix should be backported to 1.5 as well, after some observation
period.
2015-10-02 01:01:58 +02:00
Dragan Dosen
43885c728e BUG/MEDIUM: logs: segfault writing to log from Lua
Michael Ezzell reported a bug causing haproxy to segfault during startup
when trying to send syslog message from Lua. The function __send_log() can
be called with *p that is NULL and/or when the configuration is not fully
parsed, as is the case with Lua.

This patch fixes this problem by using individual vectors instead of the
pre-generated strings log_htp and log_htp_rfc5424.

Also, this patch fixes a problem causing haproxy to write the wrong pid in
the logs -- the log_htp(_rfc5424) strings were generated at the haproxy
start, but "pid" value would be changed after haproxy is started in
daemon/systemd mode.
2015-10-02 00:57:45 +02:00
Thierry FOURNIER
10770faf8e MEDIUM: lua: change the timeout execution
Now, the Lua timeout is relative to the effective run time.
When the Lua is waiting for I/O, this time is not took in
lua run time account.
2015-09-29 19:13:49 +02:00
Thierry FOURNIER
bee90aeda1 MINOR: lua: remove the run flag
Only the main execution function can set the run flag, because it is
the last function before the execution time.

This patch removes the flag set by another function. It will be used
by the new lua timeout counter.
2015-09-29 18:57:03 +02:00
Willy Tarreau
31138fae9f BUG/MEDIUM: server: fix misuse of format string in load-server-state's warnings
Commit e11cfcd ("MINOR: config: new backend directives:
load-server-state-from-file and server-state-file-name") introduced a bug
which can cause haproxy to crash upon startup by sending user-controlled
data in a format string when emitting a warning. Fix the way the warning
message is built to avoid this.

No backport is needed, this was introduced in 1.6-dev6 only.
2015-09-29 18:51:40 +02:00
Willy Tarreau
e1aebb2994 BUILD: server: fix build warnings introduced by load-server-state
Commit e11cfcd ("MINOR: config: new backend directives:
load-server-state-from-file and server-state-file-name") caused these
warnings when building with Clang :

src/server.c:1972:21: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare]
                            (srv_uweight < 0) || (srv_uweight > SRV_UWGHT_MAX))
                             ~~~~~~~~~~~ ^ ~
src/server.c:1980:21: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare]
                            (srv_iweight < 0) || (srv_iweight > SRV_UWGHT_MAX))
                             ~~~~~~~~~~~ ^ ~

Indeed, srv_iweight and srv_uweight are unsigned. Just drop the offending test.
2015-09-29 18:32:57 +02:00
Willy Tarreau
fc2a2d97d6 CLEANUP: tcp: silent-drop: only drain the connection when quick-ack is disabled
The conn_sock_drain() call is only there to force the system to ACK
pending data in case of TCP_QUICKACK so that the client doesn't retransmit,
otherwise it leads to a real RST making the feature useless. There's no
point in draining the connection when quick ack cannot be disabled, so
let's move the call inside the ifdef part.
2015-09-29 18:15:01 +02:00
Willy Tarreau
f50ec0fdbc BUG/MINOR: tcp: make silent-drop always force a TCP reset
The silent-drop action is supposed to close with a TCP reset that is
either not sent or not too far. But since it's on the client-facing
side, the socket's lingering is enabled by default and the RST only
occurs if some pending unread data remain in the queue when closing.
This causes some clean shutdowns to occur with retransmits, which is
not good at all. Force linger_risk on the socket to flush all data
and destroy the socket.

No backport is needed, this was introduced in 1.6-dev6.
2015-09-29 18:11:32 +02:00
Pradeep Jindal
bb2acf589f MINOR: payload: add support for tls session ticket ext
req.ssl_st_ext : integer
  Returns 0 if the client didn't send a SessionTicket TLS Extension (RFC5077)
  Returns 1 if the client sent SessionTicket TLS Extension
  Returns 2 if the client also sent non-zero length TLS SessionTicket
2015-09-29 14:07:32 +02:00
Willy Tarreau
2d392c2c2f MEDIUM: tcp: add new tcp action "silent-drop"
This stops the evaluation of the rules and makes the client-facing
connection suddenly disappear using a system-dependant way that tries
to prevent the client from being notified. The effect it then that the
client still sees an established connection while there's none on
HAProxy. The purpose is to achieve a comparable effect to "tarpit"
except that it doesn't use any local resource at all on the machine
running HAProxy. It can resist much higher loads than "tarpit", and
slow down stronger attackers. It is important to undestand the impact
of using this mechanism. All stateful equipments placed between the
client and HAProxy (firewalls, proxies, load balancers) will also keep
the established connection for a long time and may suffer from this
action. On modern Linux systems running with enough privileges, the
TCP_REPAIR socket option is used to block the emission of a TCP
reset. On other systems, the socket's TTL is reduced to 1 so that the
TCP reset doesn't pass the first router, though it's still delivered to
local networks.
2015-09-28 22:14:57 +02:00
Willy Tarreau
45059e9b98 BUG/MEDIUM: tcp: fix inverted condition to call custom actions
tcp-request connection had an inverted condition on action_ptr, resulting
in no registered actions to be usable since commit 4214873 ("MEDIUM: actions:
remove ACTION_STOP") merged in 1.6-dev5. Very few new actions were impacted.
No backport is needed.
2015-09-28 18:32:41 +02:00
Dragan Dosen
5b78d9b437 MEDIUM: logs: pass the trailing "\n" as an iovec
This patch passes the trailing "\n" as an iovec in the function
__send_log(), so that we don't need to modify the original log message.
2015-09-28 18:31:09 +02:00
Dragan Dosen
c8cfa7b4f3 MEDIUM: logs: have global.log_send_hostname not contain the trailing space
This patch unifies global.log_send_hostname addition in the log header
processing.
2015-09-28 18:27:45 +02:00
Willy Tarreau
47c8c029db MEDIUM: init: completely deallocate unused peers
When peers are stopped due to not being running on the appropriate
process, we want to completely release them and unregister their signals
and task in order to ensure there's no way they may be called in the
future.

Note: ideally we should have a list of all tables attached to a peers
section being disabled in order to unregister them and void their
sync_task. It doesn't appear to be *that* easy for now.
2015-09-28 16:43:48 +02:00
Willy Tarreau
20b7afbc14 BUG/MEDIUM: proxy: do not wake stopped proxies' tasks during soft_stop()
When performing a soft stop, we used to wake up every proxy's task and
each of their table's task. The problem is that since we're able to stop
proxies and peers not bound to a specific process, we may end up calling
random junk by doing so of the proxy we're waking up is already stopped.
This causes a segfault to appear during soft reloads for old processes
not bound to a peers section if such a section exists in other processes.

Let's only consider proxies that are not stopped when doing this.

This fix must be backported to 1.5 which also has the same issue.
2015-09-28 16:35:04 +02:00
Willy Tarreau
337a666572 BUG/MEDIUM: proxy: ignore stopped peers
Since commit f83d3fe ("MEDIUM: init: stop any peers section not bound
to the correct process"), it is possible to stop unused peers on certain
processes. The problem is that the pause/resume/stop functions are not
aware of this and will pass a NULL proxy pointer to the respective
functions, resulting in segfaults in unbound processes during soft
restarts.

Properly check that the peers' frontend is still valid before calling
them.

This bug also affects 1.5 so the fix must be backported. Note that this
fix is not enough to completely get rid of the segfault, the next one
is needed as well.
2015-09-28 16:27:44 +02:00
David Carlier
608c65ac6a MAJOR: da: Update of the DeviceAtlas API module
Introduction of the new keyword about the client's cookie name
via a new config entry.

Splits the work between the original convertor and the new fetch type.
The latter iterates through the whole list of HTTP headers but first,
make sure the request data are ready to process and set the input
to the string type rightly.

The API is then able to filter itself which headers are gueninely useful,
with a special care of the Client's cookie.

[WT: the immediate impact for the end user is that configs making use of
    "da-csv" won't load anymore and will need to be modified to use either
    "da-csv-conv" or "da-csv-fetch"]
2015-09-28 14:01:27 +02:00
David Carlier
5801a8247a MINOR: global: Few new struct fields for da module
The name and length of the client cookie, useful for extracting
cookie value's function and a simple bitfield one to define if
set or not.
2015-09-28 14:01:27 +02:00
David Carlier
4686f792b4 MINOR: proto_http: Externalisation of previously internal functions
Needs to expose the HTTP headers 'iterator' and the client's cookie
value extraction functions.
2015-09-28 14:01:27 +02:00
Dragan Dosen
0b85ecee53 MEDIUM: logs: add a new RFC5424 log-format for the structured-data
This patch adds a new RFC5424-specific log-format for the structured-data
that is automatically send by __send_log() when the sender is in RFC5424
mode.

A new statement "log-format-sd" should be used in order to set log-format
for the structured-data part in RFC5424 formatted syslog messages.
Example:

    log-format-sd [exampleSDID@1234\ bytes=\"%B\"\ status=\"%ST\"]
2015-09-28 14:01:27 +02:00
Dragan Dosen
1322d09a6f MEDIUM: logs: add support for RFC5424 header format per logger
The function __send_log() iterates over senders and passes the header as
the first vector to sendmsg(), thus it can send a logger-specific header
in each message.

A new logger arguments "format rfc5424" should be used in order to enable
RFC5424 header format. For example:

    log 10.2.3.4:1234 len 2048 format rfc5424 local2 info
2015-09-28 14:01:27 +02:00
Dragan Dosen
68d2e3a742 MEDIUM: logs: remove the hostname, tag and pid part from the logheader
At the moment we have to call snprintf() for every log line just to
rebuild a constant. Thanks to sendmsg(), we send the message in 3 parts:
time-based header, proxy-specific hostname+log-tag+pid, session-specific
message.
2015-09-28 14:01:27 +02:00
Dragan Dosen
59cee973cd MEDIUM: log: use a separate buffer for the header and for the message
Make sendmsg() use two vectors, one for the message header that is updated
by update_log_hdr() and one for the message buffer.
2015-09-28 14:01:27 +02:00
Dragan Dosen
609ac2ab6c MEDIUM: log: replace sendto() with sendmsg() in __send_log()
This patch replaces sendto() with sendmsg() in __send_log() and makes use
of an iovec to send the log message.
2015-09-28 14:01:27 +02:00
David Carlier
834cb2e445 BUG/MEDIUM: main: Freeing a bunch of static pointers
static_table_key, get_http_auth_buff and swap_buffer static variables
are now freed during deinit and the two previously new functions are
called as well. In addition, the 'trash' string buffer is cleared.
2015-09-28 14:00:00 +02:00
David Carlier
60deeba090 MINOR: chunk: New function free_trash_buffers()
This new function is meant to be called in the general deinit phase,
to free those two internal chunks.
2015-09-28 14:00:00 +02:00
David Carlier
845efb53c7 MINOR: cfgparse: New function cfg_unregister_sections()
A new function introduced meant to be called during general deinit phase.
During the configuration parsing, the section entries are all allocated.
This new function free them.
2015-09-28 14:00:00 +02:00
Willy Tarreau
270978492c MEDIUM: config: set tune.maxrewrite to 1024 by default
The tune.maxrewrite parameter used to be pre-initialized to half of
the buffer size since the very early days when buffers were very small.
It has grown to absurdly large values over the years to reach 8kB for a
16kB buffer. This prevents large requests from being accepted, which is
the opposite of the initial goal.

Many users fix it to 1024 which is already quite large for header
addition.

So let's change the default setting policy :
  - pre-initialize it to 1024
  - let the user tweak it
  - in any case, limit it to tune.bufsize / 2

This results in 15kB usable to buffer HTTP messages instead of 8kB, and
doesn't affect existing configurations which already force it.
2015-09-28 13:59:41 +02:00
Willy Tarreau
9b69454570 BUG/MINOR: config: check that tune.bufsize is always positive
We must not accept negative values for tune.bufsize as they will only
result in crashing the process during the parsing.
2015-09-28 13:59:38 +02:00
Thierry FOURNIER
a30b5dbf85 MINOR: lua: add AppletHTTP class and service
This class is used by Lua code for running as an applet called in HTTP mode
It defines also the associated lua service
2015-09-28 01:03:48 +02:00
Thierry FOURNIER
f0a64b676f MINOR: lua: add AppletTCP class and service
This class is used by Lua code for running as an applet called in TCP mode.
It defines also the lua service.
2015-09-28 01:03:48 +02:00
Thierry FOURNIER
5a363e71b2 MINOR: stream/applet: add use-service action
This new target can be called from the frontend or the backend. It
is evaluated just before the backend choice and just before the server
choice. So, the input stream or HTTP request can be forwarded to a
server or to an internal service.
2015-09-28 01:03:48 +02:00
Thierry FOURNIER
2f3867f429 BUG/MEDIUM: lua: don't reset undesired flags in hlua_ctx_resume
Some flags like HLUA_MUST_GC must not be cleared otherwise sessions are
not properly cleaned.
2015-09-28 01:03:43 +02:00
Willy Tarreau
acc980036f MEDIUM: action: add a new flag ACT_FLAG_FIRST
This flag is used by custom actions to know that they're called for the
first time. The only case where it's not set is when they're resuming
from a yield. It will be needed to let them know when they have to
allocate some resources.
2015-09-27 23:34:39 +02:00
Thierry FOURNIER
7c39ab4ac2 OPTIM/MEDIUM: lua: executes the garbage collector only when using cosocket
The garbage collector is a little bit heavy to run, and it was added
only for cosockets. This patch prevent useless executions when no
cosockets are used.
2015-09-27 22:56:40 +02:00
Thierry FOURNIER
6ab4d8ec0e MEDIUM: lua: change the GC policy
The GC is run each times that an Lua function exit with an error or with a success.
2015-09-27 22:22:37 +02:00
Thierry FOURNIER
6d695e6314 BUG/MEDIUM: lua: socket destroy before reading pending data
When the channel is down, the applet is waked up. Before this patch,
the applet closes the stream and the unread data are discarded.

After this patch, the stream is not closed except if the buffers are
empty.

It will be closed later by the close function, or by the garbage collector.
2015-09-27 21:50:26 +02:00
Thierry FOURNIER
8c8fbbe103 MINOR: lua/applet: the cosocket applet should use appctx_wakeup in place of task_wakeup
It seems that in the new applet system, the applet me be wakeup with the
function appctx_wakeup() in place of the function task_wakeup().
2015-09-27 19:30:26 +02:00
Willy Tarreau
bb3c09ab2b CLEANUP: config: make the errorloc/errorfile messages less confusing
Some users believe that "status code XXX not handled" means "not handled
by haproxy". Let's be clear that's only about the option.
2015-09-27 15:13:30 +02:00
Thierry FOURNIER
c2f5653452 MINOR: lua: extend socket address to support non-IP families
The HAProxy Lua socket respects the Lua Socket tcp specs. these specs
are a little bit limited, it not permits to connect to unix socket.

this patch extends a little it the specs. It permit to accept a
syntax that begin with "unix@", "ipv4@", "ipv6@", "fd@" or "abns@".
2015-09-27 15:04:32 +02:00
Thierry FOURNIER
7fe3be7281 MINOR: standard: avoid DNS resolution from the function str2sa_range()
This patch blocks the DNS resolution in the function str2sa_range(),
this is useful if the function is used during the HAProxy runtime.
2015-09-27 15:04:32 +02:00
Willy Tarreau
528192d310 MEDIUM: lua: only allow actions to yield if not in a final call
Actions may yield but must not do it during the final call from a ruleset
because it indicates there will be no more opportunity to complete or
clean up. This is indicated by ACT_FLAG_FINAL in the action's flags,
which must be passed to hlua_resume().

Thanks to this, an action called from a TCP ruleset is properly woken
up and possibly finished when the client disconnects.
2015-09-27 11:04:19 +02:00
Willy Tarreau
394586836f MEDIUM: http: pass ACT_FLAG_FINAL to custom actions
In HTTP it's more difficult to know when to pass the flag or not
because all actions are supposed to be final and there's no inspection
delay. Also, the input channel may very well be closed without this
being an error. So we only set the flag when option abortonclose is
set and the input channel is closed, which is the only case where the
user explicitly wants to forward a close down the chain.
2015-09-27 11:04:06 +02:00
Willy Tarreau
c1b10d38d7 MEDIUM: actions: add new flag ACT_FLAG_FINAL to notify about last call
This new flag indicates to a custom action that it must not yield because
it will not be called anymore. This addresses an issue introduced by commit
bc4c1ac ("MEDIUM: http/tcp: permit to resume http and tcp custom actions"),
which made it possible to yield even after the last call and causes Lua
actions not to be stopped when the session closes. Note that the Lua issue
is not fixed yet at this point. Also only TCP rules were handled, for now
HTTP rules continue to let the action yield since we don't know whether or
not it is a final call.
2015-09-27 11:04:06 +02:00
Willy Tarreau
658b85b68d MEDIUM: actions: pass a new "flags" argument to custom actions
Since commit bc4c1ac ("MEDIUM: http/tcp: permit to resume http and tcp
custom actions"), some actions may yield and be called back when new
information are available. Unfortunately some of them may continue to
yield because they simply don't know that it's the last call from the
rule set. For this reason we'll need to pass a flag to the custom
action to pass such information and possibly other at the same time.
2015-09-27 11:04:06 +02:00