24 Commits

Author SHA1 Message Date
William Dauchy
04e90df7cb DOC: contrib/prometheus-exporter: remove htx reference
now that htx is the default everywhere, we can remove the need to put
htx as a mandatory option to setup prometheus.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-19 18:03:59 +01:00
William Dauchy
e3f7bd5ae9 MEDIUM: contrib/prometheus-exporter: add listen stats
this was a missing piece for a while now even though it was planned. This
patch adds listen stats.
Nothing in particular but we make use of the status helper previously
added.  `promex_st_metrics` diff also looks scary, but I had to realign
all lines.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-15 14:14:59 +01:00
William Dauchy
69164228b8 MEDIUM: contrib/prometheus-exporter: export base stick table stats
I saw some people falling back to unix socket to collect some data they
could not find in prometheus exporter. One of them is base info from
stick tables (used/size).
I do not plan to extend it more for now; keys are quite a mess to
handle.

This should resolve github issue #1008.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-08 10:49:08 +01:00
Christopher Faulet
1a68cd0689 DOC: contrib/prometheus-exporter: Add missing metrics in README
Some metrics were missing (haproxy_process_uptime_seconds and
haproxy_process_build_info). To ease the review against the service output,
the same order is used in the README.
2021-02-01 15:16:33 +01:00
William Dauchy
4b7bf7eccd CLEANUP: contrib/prometheus-exporter: remove description in README
Now that we got ride of description in prometheus code, let's assume we
no longer need to maintain it in README, and diret user to the output of
prometheus to get more info.

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 15:16:33 +01:00
William Dauchy
7741c33779 MINOR: contrib/prometheus-exporter: add recv logs_logs_total field
this field was added by commit 45c457a62941a7c4a86ce4327d7755edcd4b230e
("MINOR: log: adds counters on received syslog messages.")

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 15:16:33 +01:00
William Dauchy
e5a26a250d MINOR: contrib/prometheus-exporter: add uweight field
this field was added in commit bd7151002437af1a034a9fdbb582b3cbef5a78d1
("MINOR: stats: report server's user-configured weight next to effective
weight")

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 15:16:33 +01:00
William Dauchy
de3c326389 MAJOR: contrib/prometheus-exporter: move health check status to labels
this patch is a breaking change between v2.3 and v2.4: we move from
using gauge value for health check states to labels values. The diff is
quite small thanks to the preparation work from Christopher to allow
more flexibility in labels, see commit
5a2f938732126f43bbf4cea5482c01552b0d0314 ("MEDIUM:
contrib/prometheus-exporter: Use dynamic labels instead of static ones")

this is a follow up of commit c6464591a365bfcf509b322bdaa4d608c9395d75
("MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to
labels"). The main goal being to be better aligned with prometheus use
cases in terms of queries. More specifically to health checks, Pierre C.
mentioned the possible quirks he had to put in place in order to make
use of those metrics through prometheus:

  <aggregator_function> by(proxy, check_status) (count_values by(proxy,
  instance) ("check_status", haproxy_server_check_status))

I am perfectly aware this introduces a lot more metrics but I don't see
how we can improve the usability without it. The main issue remains in
the cardinality of the states which are > 20. Prometheus recommends to
stay below a cardinality of 10 for a given metric but I consider our
case very specific, because highly linked to the level of precision
haproxy exposes.

Even before this patch I saw several large production setup (a few
hundreds of MB in output) which are making use of the scope parameter to
simply ignore the server metrics, so that the scrapping can be faster,
and memory consumed on client side not too high. So I believe we should
eventually continue in that direction and offer more granularity of
filtering of the output. That being said it is already possible to
filter out the data on prometheus client side.

this is related to github issue #1029

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-02-01 15:16:33 +01:00
Christopher Faulet
c55a626217 MINOR: contrib/prometheus-exporter: Add missing global and per-server metrics
Following metrics are now exported by the prometheus exporter to reflect recent
changes on HAProxy :

  * haproxy_process_failed_resolutions
  * haproxy_process_bytes_out_total
  * haproxy_process_spliced_bytes_out_total
  * haproxy_process_bytes_out_rate

and

 * haproxy_server_unsafe_idle_connections_current
 * haproxy_server_safe_idle_connections_current
 * haproxy_server_used_connections_current
 * haproxy_server_need_connections_current
2020-07-15 14:08:14 +02:00
Pierre Cheynier
1e36976352 CLEANUP: contrib/prometheus-exporter: typo fixes for ssl reuse metric
A typo I identified while having a look to our metric inventory.
(s/frontent/frontend)
2020-07-07 23:15:03 +02:00
Ilya Shipitsin
1fae8db7b7 DOC: assorted typo fixes in the documentation
This is the fourth round of cleanups in various docs
2020-03-18 11:34:33 +01:00
Christopher Faulet
2711e51016 MINOR: contrib/prometheus-exporter: Add the last heathcheck duration metric
ST_F_CHECK_DURATION is now part of exported server metrics, named
haproxy_server_check_duration_seconds and expressed in seconds. For a given
server, this value is exported only if the healthcheck is finished (the status
is greater or equal to HCHK_STATUS_CHECKED).

This patch fixes the issue #519. It may be backported as fat as 2.0.
2020-02-28 10:49:09 +01:00
Christopher Faulet
e4a2c8d7e7 MINOR: contrib/prometheus-exporter: Export internal errors per proxy/server
The new ST_F_EINT stats field is now exported for each proxy/server.
2020-01-20 15:18:45 +01:00
Christopher Faulet
cf403f32e4 MINOR: contrib/prometheus-exporter: Add heathcheck status/code in server metrics
ST_F_CHECK_STATUS and ST_F_CHECK_CODE are now part of exported server metrics:

  * haproxy_server_check_status
  * haproxy_server_check_code

The heathcheck status is an integer corresponding to HCHK_STATUS value.
2020-01-20 15:18:45 +01:00
Christopher Faulet
eba2294e5b MINOR: contrib/prometheus-exporter: Add a param to ignore servers in maintenance
By passing the parameter "no-maint" in the query-string, it is now possible to
ignore servers in maintenance. It means that the metrics for servers in this
state will not be exported.
2019-11-20 14:11:47 +01:00
Christopher Faulet
78407ce156 MINOR: contrib/prometheus-exporter: filter exported metrics by scope
Now, the prometheus exporter parses the HTTP query-string to filter or to adapt
the exported metrics. In this first version, it is only possible select the
scopes of metrics to export. To do so, one or more parameters with "scope" as
name must be passed in the query-string, with one of those values: global,
frontend, backend, server or '*' (means all). A scope parameter with no value
means to filter out all scopes (nothing is returned). The scope parameters are
parsed in their appearance order in the query-string. So an empty scope will
reset all scopes already parsed. But it can be overridden by following scope
parameters in the query-string. By default everything is exported.

The filtering can also be done on prometheus scraping configuration, but general
aim is to optimise the source of data to improve load and scraping time. This is
particularly true for huge configuration with thousands of backends and servers.
Also note that this configuration was possible on the previous official haproxy
exporter but with even more parameters to select the needed metrics. Here we
thought it was sufficient to simply avoid a given type of metric. However, more
filters are still possible.

Thanks to William Dauchy. This patch is based on his work.
2019-11-20 14:11:47 +01:00
Christopher Faulet
20ab80c0c0 MINOR: contrib/prometheus-exporter: report the number of idle conns per server
This adds two extra metrics per server, one for the current number of idle
connections and one for the configured limit :

 * haproxy_server_idle_connections_current
 * haproxy_server_idle_connections_limit
2019-11-15 14:24:06 +01:00
Christopher Faulet
68b6968ecd BUG/MINOR: contrib/prometheus-exporter: Rename some metrics
The following metrics have been renamed without the "_http" part :

 * http_queue_time_average_seconds     => queue_time_average_seconds
 * http_connect_time_average_seconds   => connect_time_average_seconds
 * http_response_time_average_seconds  => response_time_average_seconds
 * http_total_time_average_seconds     => total_time_average_seconds

These metrics are reported per backend and per server and are not specific to
HTTP sessions.
2019-11-15 14:24:06 +01:00
Christopher Faulet
8fc027d468 MINOR: contrib/prometheus-exporter: Report metrics about max times for sessions
Now, for the sessions, the maximum times (queue, connect, response, total) are
reported in addition of the averages over the last 1024 connections. These
metrics are reported per backend and per server. Here are the metrics name :

  * haproxy_backend_max_queue_time_seconds
  * haproxy_backend_max_connect_time_seconds
  * haproxy_backend_max_response_time_seconds
  * haproxy_backend_max_total_time_seconds

and

  * haproxy_server_max_queue_time_seconds
  * haproxy_server_max_connect_time_seconds
  * haproxy_server_max_response_time_seconds
  * haproxy_server_max_total_time_seconds

This patch is related to #272.
2019-11-15 14:24:01 +01:00
Rick Rackow
35efbe2cda DOC: fix typo in Prometheus exporter doc
It's just a minor typo in a section title.
2019-10-09 04:38:15 +02:00
Willy Tarreau
d254aa8139 DOC: update few references to the linux* targets and change them to linux-glibc
The INSTALL guide, the Lua doc and the Prometheus exporter's README all
used to reference "linux2628", "linux26" or even "linux". These were all
updated to consistently reflect "linux-glibc" instead. The default options
were updated there as well so that it should build cleanly on most distros.
2019-06-15 18:03:48 +02:00
Christopher Faulet
8c8e4b1263 MINOR: contrib/prometheus-exporter: Rename some metrics to be more usable
Some metrics have been renamed and their type adapted to be more usable in
Prometheus:

  * haproxy_process_uptime_seconds -> haproxy_process_start_time_seconds
  * haproxy_process_max_memory -> haproxy_process_max_memory_bytes
  * haproxy_process_pool_allocated_total -> haproxy_process_pool_allocated_bytes
  * haproxy_process_pool_used_total -> haproxy_process_pool_used_bytes
  * haproxy_process_ssl_cache_lookups -> haproxy_process_ssl_cache_lookups_total
  * haproxy_process_ssl_cache_misses -> haproxy_process_ssl_cache_misses_total

No backport needed. See issue #81 on github.
2019-04-18 10:27:16 +02:00
Christopher Faulet
c58fc0dec9 MINOR: contrib/prometheus-exporter: Remove usless rate metrics
Following metrics have been removed:

  * haproxy_frontend_connections_rate_current (ST_F_CONN_RATE)
  * haproxy_frontend_http_requests_rate_current (ST_F_REQ_RATE)
  * haproxy_*_current_session_rate (ST_F_RATE)

These rates can be deduced using the total value with this kind of formula:

  rate(haproxy_frontend_connections_total[1m])

No backport needed. See issue #81 on github.
2019-04-18 10:27:16 +02:00
Christopher Faulet
f959d0809e CONTRIB: contrib/prometheus-exporter: Add a Prometheus exporter for HAProxy
It has been developped as a service applet. Internally, it is called
"promex". To build HAProxy with the promex service, you should use the Makefile
variable "EXTRA_OBJS". To be used, it must be enabled in the configuration with
an "http-request" rule and the corresponding HTTP proxy must enable the HTX
support. For instance:

    frontend test
        mode http
        ...
	option http-use-htx
	http-request use-service prometheus-exporter if { path /metrics }
        ...

See contrib/prometheus-exporter/README for details.
2019-02-08 13:55:21 +01:00