[MEDIUM] checks: add support for HTTP contents lookup
This patch adds the "http-check expect [r]{string,status}" statements
which enable health checks based on whether the response status or body
to an HTTP request contains a string or matches a regex.
This probably is one of the oldest patches that remained unmerged. Over
the time, several people have contributed to it, among which FinalBSD
(first and second implementations), Nick Chalk (port to 1.4), Anze
Skerlavaj (tests and fixes), Cyril Bont (general fixes), and of course
myself for the final fixes and doc during integration.
Some people already use an old version of this patch which has several
issues, among which the inability to search for a plain string that is
not at the beginning of the data, and the inability to look for response
contents that are provided in a second and subsequent recv() calls. But
since some configs are already deployed, it was quite important to ensure
a 100% compatible behaviour on the working cases.
Thus, that patch fixes the issues while maintaining config compatibility
with already deployed versions.
(cherry picked from commit b507c43a3ce9a8e8e4b770e52e4edc20cba4c37f)
This commit is contained in:
Willy Tarreau
parent
b4c81e4c81
commit
bd741540d2
@ -868,6 +868,7 @@ fullconn X - X X
|
||||
grace X X X X
|
||||
hash-type X - X X
|
||||
http-check disable-on-404 X - X X
|
||||
http-check expect - - X X
|
||||
http-check send-state X - X X
|
||||
http-request - X X X
|
||||
id - X X X
|
||||
@ -2083,9 +2084,93 @@ http-check disable-on-404
|
||||
generate an alert, just a notice. If the server responds 2xx or 3xx again, it
|
||||
will immediately be reinserted into the farm. The status on the stats page
|
||||
reports "NOLB" for a server in this mode. It is important to note that this
|
||||
option only works in conjunction with the "httpchk" option.
|
||||
option only works in conjunction with the "httpchk" option. If this option
|
||||
is used with "http-check expect", then it has precedence over it so that 404
|
||||
responses will still be considered as soft-stop.
|
||||
|
||||
See also : "option httpchk"
|
||||
See also : "option httpchk", "http-check expect"
|
||||
|
||||
|
||||
http-check expect [!] <match> <pattern>
|
||||
Make HTTP health checks consider reponse contents or specific status codes
|
||||
May be used in sections : defaults | frontend | listen | backend
|
||||
no | no | yes | yes
|
||||
Arguments :
|
||||
<match> is a keyword indicating how to look for a specific pattern in the
|
||||
response. The keyword may be one of "status", "rstatus",
|
||||
"string", or "rstring". The keyword may be preceeded by an
|
||||
exclamation mark ("!") to negate the match. Spaces are allowed
|
||||
between the exclamation mark and the keyword. See below for more
|
||||
details on the supported keywords.
|
||||
|
||||
<pattern> is the pattern to look for. It may be a string or a regular
|
||||
expression. If the pattern contains spaces, they must be escaped
|
||||
with the usual backslash ('\').
|
||||
|
||||
By default, "option httpchk" considers that response statuses 2xx and 3xx
|
||||
are valid, and that others are invalid. When "http-check expect" is used,
|
||||
it defines what is considered valid or invalid. Only one "http-check"
|
||||
statement is supported in a backend. If a server fails to respond or times
|
||||
out, the check obviously fails. The available matches are :
|
||||
|
||||
status <string> : test the exact string match for the HTTP status code.
|
||||
A health check respose will be considered valid if the
|
||||
response's status code is exactly this string. If the
|
||||
"status" keyword is prefixed with "!", then the response
|
||||
will be considered invalid if the status code matches.
|
||||
|
||||
rstatus <regex> : test a regular expression for the HTTP status code.
|
||||
A health check respose will be considered valid if the
|
||||
response's status code matches the expression. If the
|
||||
"rstatus" keyword is prefixed with "!", then the response
|
||||
will be considered invalid if the status code matches.
|
||||
This is mostly used to check for multiple codes.
|
||||
|
||||
string <string> : test the exact string match in the HTTP response body.
|
||||
A health check respose will be considered valid if the
|
||||
response's body contains this exact string. If the
|
||||
"string" keyword is prefixed with "!", then the response
|
||||
will be considered invalid if the body contains this
|
||||
string. This can be used to look for a mandatory word at
|
||||
the end of a dynamic page, or to detect a failure when a
|
||||
specific error appears on the check page (eg: a stack
|
||||
trace).
|
||||
|
||||
rstring <regex> : test a regular expression on the HTTP response body.
|
||||
A health check respose will be considered valid if the
|
||||
response's body matches this expression. If the "rstring"
|
||||
keyword is prefixed with "!", then the response will be
|
||||
considered invalid if the body matches the expression.
|
||||
This can be used to look for a mandatory word at the end
|
||||
of a dynamic page, or to detect a failure when a specific
|
||||
error appears on the check page (eg: a stack trace).
|
||||
|
||||
It is important to note that the responses will be limited to a certain size
|
||||
defined by the global "tune.chksize" option, which defaults to 16384 bytes.
|
||||
Thus, too large responses may not contain the mandatory pattern when using
|
||||
"string" or "rstring". If a large response is absolutely required, it is
|
||||
possible to change the default max size by setting the global variable.
|
||||
However, it is worth keeping in mind that parsing very large responses can
|
||||
waste some CPU cycles, especially when regular expressions are used, and that
|
||||
it is always better to focus the checks on smaller resources.
|
||||
|
||||
Last, if "http-check expect" is combined with "http-check disable-on-404",
|
||||
then this last one has precedence when the server responds with 404.
|
||||
|
||||
Examples :
|
||||
# only accept status 200 as valid
|
||||
http-request expect status 200
|
||||
|
||||
# consider SQL errors as errors
|
||||
http-request expect ! string SQL\ Error
|
||||
|
||||
# consider status 5xx only as errors
|
||||
http-request expect ! rstatus ^5
|
||||
|
||||
# check that we have a correct hexadecimal tag before /html
|
||||
http-request expect rstring <!--tag:[0-9a-f]*</html>
|
||||
|
||||
See also : "option httpchk", "http-check disable-on-404"
|
||||
|
||||
|
||||
http-check send-state
|
||||
|
||||
@ -140,6 +140,14 @@
|
||||
#define PR_O2_SSL3_CHK 0x00100000 /* use SSLv3 CLIENT_HELLO packets for server health */
|
||||
#define PR_O2_FAKE_KA 0x00200000 /* pretend we do keep-alive with server eventhough we close */
|
||||
#define PR_O2_LDAP_CHK 0x00400000 /* use LDAP check for server health */
|
||||
|
||||
#define PR_O2_EXP_NONE 0x00000000 /* http-check : no expect rule */
|
||||
#define PR_O2_EXP_STS 0x00800000 /* http-check expect status */
|
||||
#define PR_O2_EXP_RSTS 0x01000000 /* http-check expect rstatus */
|
||||
#define PR_O2_EXP_STR 0x01800000 /* http-check expect string */
|
||||
#define PR_O2_EXP_RSTR 0x02000000 /* http-check expect rstring */
|
||||
#define PR_O2_EXP_TYPE 0x03800000 /* mask for http-check expect type */
|
||||
#define PR_O2_EXP_INV 0x04000000 /* http-check expect !<rule> */
|
||||
/* end of proxy->options2 */
|
||||
|
||||
/* bits for sticking rules */
|
||||
@ -283,6 +291,8 @@ struct proxy {
|
||||
int grace; /* grace time after stop request */
|
||||
char *check_req; /* HTTP or SSL request to use for PR_O_HTTP_CHK|PR_O_SSL3_CHK */
|
||||
int check_len; /* Length of the HTTP or SSL3 request */
|
||||
char *expect_str; /* http-check expected content */
|
||||
regex_t *expect_regex; /* http-check expected content */
|
||||
struct chunk errmsg[HTTP_ERR_SIZE]; /* default or customized error messages for known errors */
|
||||
int uuid; /* universally unique proxy ID, used for SNMP */
|
||||
unsigned int backlog; /* force the frontend's listen backlog */
|
||||
|
||||
@ -2971,8 +2971,91 @@ stats_error_parsing:
|
||||
/* enable emission of the apparent state of a server in HTTP checks */
|
||||
curproxy->options2 |= PR_O2_CHK_SNDST;
|
||||
}
|
||||
else if (strcmp(args[1], "expect") == 0) {
|
||||
const char *ptr_arg;
|
||||
int cur_arg;
|
||||
|
||||
if (curproxy->options2 & PR_O2_EXP_TYPE) {
|
||||
Alert("parsing [%s:%d] : '%s %s' already specified.\n", file, linenum, args[0], args[1]);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
|
||||
cur_arg = 2;
|
||||
/* consider exclamation marks, sole or at the beginning of a word */
|
||||
while (*(ptr_arg = args[cur_arg])) {
|
||||
while (*ptr_arg == '!') {
|
||||
curproxy->options2 ^= PR_O2_EXP_INV;
|
||||
ptr_arg++;
|
||||
}
|
||||
if (*ptr_arg)
|
||||
break;
|
||||
cur_arg++;
|
||||
}
|
||||
/* now ptr_arg points to the beginning of a word past any possible
|
||||
* exclamation mark, and cur_arg is the argument which holds this word.
|
||||
*/
|
||||
if (strcmp(ptr_arg, "status") == 0) {
|
||||
if (!*(args[cur_arg + 1])) {
|
||||
Alert("parsing [%s:%d] : '%s %s %s' expects <string> as an argument.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
curproxy->options2 |= PR_O2_EXP_STS;
|
||||
curproxy->expect_str = strdup(args[cur_arg + 1]);
|
||||
}
|
||||
else if (strcmp(ptr_arg, "string") == 0) {
|
||||
if (!*(args[cur_arg + 1])) {
|
||||
Alert("parsing [%s:%d] : '%s %s %s' expects <string> as an argument.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
curproxy->options2 |= PR_O2_EXP_STR;
|
||||
curproxy->expect_str = strdup(args[cur_arg + 1]);
|
||||
}
|
||||
else if (strcmp(ptr_arg, "rstatus") == 0) {
|
||||
if (!*(args[cur_arg + 1])) {
|
||||
Alert("parsing [%s:%d] : '%s %s %s' expects <regex> as an argument.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
curproxy->options2 |= PR_O2_EXP_RSTS;
|
||||
curproxy->expect_regex = calloc(1, sizeof(regex_t));
|
||||
if (regcomp(curproxy->expect_regex, args[cur_arg + 1], REG_EXTENDED) != 0) {
|
||||
Alert("parsing [%s:%d] : '%s %s %s' : bad regular expression '%s'.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg, args[cur_arg + 1]);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
}
|
||||
else if (strcmp(ptr_arg, "rstring") == 0) {
|
||||
if (!*(args[cur_arg + 1])) {
|
||||
Alert("parsing [%s:%d] : '%s %s %s' expects <regex> as an argument.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
curproxy->options2 |= PR_O2_EXP_RSTR;
|
||||
curproxy->expect_regex = calloc(1, sizeof(regex_t));
|
||||
if (regcomp(curproxy->expect_regex, args[cur_arg + 1], REG_EXTENDED) != 0) {
|
||||
Alert("parsing [%s:%d] : '%s %s %s' : bad regular expression '%s'.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg, args[cur_arg + 1]);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
}
|
||||
else {
|
||||
Alert("parsing [%s:%d] : '%s %s' only supports [!] 'status', 'string', 'rstatus', 'rstring', found '%s'.\n",
|
||||
file, linenum, args[0], args[1], ptr_arg);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
}
|
||||
else {
|
||||
Alert("parsing [%s:%d] : '%s' only supports 'disable-on-404'.\n", file, linenum, args[0]);
|
||||
Alert("parsing [%s:%d] : '%s' only supports 'disable-on-404', 'expect' .\n", file, linenum, args[0]);
|
||||
err_code |= ERR_ALERT | ERR_FATAL;
|
||||
goto out;
|
||||
}
|
||||
|
||||
124
src/checks.c
124
src/checks.c
@ -47,6 +47,8 @@
|
||||
#include <proto/server.h>
|
||||
#include <proto/task.h>
|
||||
|
||||
static int httpchk_expect(struct server *s, int done);
|
||||
|
||||
const struct check_status check_statuses[HCHK_STATUS_SIZE] = {
|
||||
[HCHK_STATUS_UNKNOWN] = { SRV_CHK_UNKNOWN, "UNK", "Unknown" },
|
||||
[HCHK_STATUS_INI] = { SRV_CHK_UNKNOWN, "INI", "Initializing" },
|
||||
@ -938,21 +940,24 @@ static int event_srv_chk_r(int fd)
|
||||
}
|
||||
|
||||
s->check_code = str2uic(s->check_data + 9);
|
||||
|
||||
desc = ltrim(s->check_data + 12, ' ');
|
||||
|
||||
/* check the reply : HTTP/1.X 2xx and 3xx are OK */
|
||||
if (*(s->check_data + 9) == '2' || *(s->check_data + 9) == '3') {
|
||||
cut_crlf(desc);
|
||||
set_server_check_status(s, HCHK_STATUS_L7OKD, desc);
|
||||
}
|
||||
else if ((s->proxy->options & PR_O_DISABLE404) &&
|
||||
(s->state & SRV_RUNNING) &&
|
||||
(s->check_code == 404)) {
|
||||
if ((s->proxy->options & PR_O_DISABLE404) &&
|
||||
(s->state & SRV_RUNNING) && (s->check_code == 404)) {
|
||||
/* 404 may be accepted as "stopping" only if the server was up */
|
||||
cut_crlf(desc);
|
||||
set_server_check_status(s, HCHK_STATUS_L7OKCD, desc);
|
||||
}
|
||||
else if (s->proxy->options2 & PR_O2_EXP_TYPE) {
|
||||
/* Run content verification check... We know we have at least 13 chars */
|
||||
if (!httpchk_expect(s, done))
|
||||
goto wait_more_data;
|
||||
}
|
||||
/* check the reply : HTTP/1.X 2xx and 3xx are OK */
|
||||
else if (*(s->check_data + 9) == '2' || *(s->check_data + 9) == '3') {
|
||||
cut_crlf(desc);
|
||||
set_server_check_status(s, HCHK_STATUS_L7OKD, desc);
|
||||
}
|
||||
else {
|
||||
cut_crlf(desc);
|
||||
set_server_check_status(s, HCHK_STATUS_L7STS, desc);
|
||||
@ -1517,6 +1522,107 @@ int start_checks() {
|
||||
return 0;
|
||||
}
|
||||
|
||||
/*
|
||||
* Perform content verification check on data in s->check_data buffer.
|
||||
* The buffer MUST be terminated by a null byte before calling this function.
|
||||
* Sets server status appropriately. The caller is responsible for ensuring
|
||||
* that the buffer contains at least 13 characters. If <done> is zero, we may
|
||||
* return 0 to indicate that data is required to decide of a match.
|
||||
*/
|
||||
static int httpchk_expect(struct server *s, int done)
|
||||
{
|
||||
static char status_msg[] = "HTTP status check returned code <000>";
|
||||
char status_code[] = "000";
|
||||
char *contentptr;
|
||||
int crlf;
|
||||
int ret;
|
||||
|
||||
switch (s->proxy->options2 & PR_O2_EXP_TYPE) {
|
||||
case PR_O2_EXP_STS:
|
||||
case PR_O2_EXP_RSTS:
|
||||
memcpy(status_code, s->check_data + 9, 3);
|
||||
memcpy(status_msg + strlen(status_msg) - 4, s->check_data + 9, 3);
|
||||
|
||||
if ((s->proxy->options2 & PR_O2_EXP_TYPE) == PR_O2_EXP_STS)
|
||||
ret = strncmp(s->proxy->expect_str, status_code, 3) == 0;
|
||||
else
|
||||
ret = regexec(s->proxy->expect_regex, status_code, MAX_MATCH, pmatch, 0) == 0;
|
||||
|
||||
/* we necessarily have the response, so there are no partial failures */
|
||||
if (s->proxy->options2 & PR_O2_EXP_INV)
|
||||
ret = !ret;
|
||||
|
||||
set_server_check_status(s, ret ? HCHK_STATUS_L7OKD : HCHK_STATUS_L7STS, status_msg);
|
||||
break;
|
||||
|
||||
case PR_O2_EXP_STR:
|
||||
case PR_O2_EXP_RSTR:
|
||||
/* very simple response parser: ignore CR and only count consecutive LFs,
|
||||
* stop with contentptr pointing to first char after the double CRLF or
|
||||
* to '\0' if crlf < 2.
|
||||
*/
|
||||
crlf = 0;
|
||||
for (contentptr = s->check_data; *contentptr; contentptr++) {
|
||||
if (crlf >= 2)
|
||||
break;
|
||||
if (*contentptr == '\r')
|
||||
continue;
|
||||
else if (*contentptr == '\n')
|
||||
crlf++;
|
||||
else
|
||||
crlf = 0;
|
||||
}
|
||||
|
||||
/* Check that response contains a body... */
|
||||
if (crlf < 2) {
|
||||
if (!done)
|
||||
return 0;
|
||||
|
||||
set_server_check_status(s, HCHK_STATUS_L7RSP,
|
||||
"HTTP content check could not find a response body");
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* Check that response body is not empty... */
|
||||
if (*contentptr == '\0') {
|
||||
set_server_check_status(s, HCHK_STATUS_L7RSP,
|
||||
"HTTP content check found empty response body");
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* Check the response content against the supplied string
|
||||
* or regex... */
|
||||
if ((s->proxy->options2 & PR_O2_EXP_TYPE) == PR_O2_EXP_STR)
|
||||
ret = strstr(contentptr, s->proxy->expect_str) != NULL;
|
||||
else
|
||||
ret = regexec(s->proxy->expect_regex, contentptr, MAX_MATCH, pmatch, 0) == 0;
|
||||
|
||||
/* if we don't match, we may need to wait more */
|
||||
if (!ret && !done)
|
||||
return 0;
|
||||
|
||||
if (ret) {
|
||||
/* content matched */
|
||||
if (s->proxy->options2 & PR_O2_EXP_INV)
|
||||
set_server_check_status(s, HCHK_STATUS_L7RSP,
|
||||
"HTTP check matched unwanted content");
|
||||
else
|
||||
set_server_check_status(s, HCHK_STATUS_L7OKD,
|
||||
"HTTP content check matched");
|
||||
}
|
||||
else {
|
||||
if (s->proxy->options2 & PR_O2_EXP_INV)
|
||||
set_server_check_status(s, HCHK_STATUS_L7OKD,
|
||||
"HTTP check did not match unwanted content");
|
||||
else
|
||||
set_server_check_status(s, HCHK_STATUS_L7RSP,
|
||||
"HTTP content check did not match");
|
||||
}
|
||||
break;
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
|
||||
/*
|
||||
* Local variables:
|
||||
* c-indent-level: 8
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user