18458 Commits

Author SHA1 Message Date
Amaury Denoyelle
f8aaf8bdfa BUG/MEDIUM: mux-quic: fix crash on early app-ops release
H3 SETTINGS emission has recently been delayed. The idea is to send it
with the first STREAM to reduce sendto syscall invocation. This was
implemented in the following patch :
  3dd79d378c86b3ebf60e029f518add5f1ed54815
  MINOR: h3: Send the h3 settings with others streams (requests)

This patch works fine under nominal conditions. However, it will cause a
crash if a HTTP/3 connection is released before having sent any data,
for example when receiving an invalid first request. In this case,
qc_release will first free qcc.app_ops HTTP/3 application protocol layer
via release callback. Then qc_send is called to emit any closing frames
built by app_ops release invocation. However, in qc_send, as no data has
been sent, it will try to complete application layer protocol
intialization, with a SETTINGS emission for HTTP/3. Thus, qcc.app_ops is
reused, which is invalid as it has been just freed. This will cause a
crash with h3_finalize in the call stack.

This bug can be reproduced artificially by generating incomplete HTTP/3
requests. This will in time trigger http-request timeout without any
data send. This is done by editing qc_handle_strm_frm function.

-       ret = qcc_recv(qc->qcc, strm_frm->id, strm_frm->len,
+       ret = qcc_recv(qc->qcc, strm_frm->id, strm_frm->len - 1,
                       strm_frm->offset.key, strm_frm->fin,
                       (char *)strm_frm->data);

To fix this, application layer closing API has been adjusted to be done
in two-steps. A new shutdown callback is implemented : it is used by the
HTTP/3 layer to generate GOAWAY frame in qc_release prologue.
Application layer context qcc.app_ops is then freed later in qc_release
via the release operation which is now only used to liberate app layer
ressources. This fixes the problem as the intermediary qc_send
invocation will be able to reuse app_ops before it is freed.

This patch fixes the crash, but it would be better to adjust H3 SETTINGS
emission in case of early connection closing : in this case, there is no
need to send it. This should be implemented in a future patch.

This should fix the crash recently experienced by Tristan in github
issue #1801.

This must be backported up to 2.6.
2022-09-15 10:41:44 +02:00
William Lallemand
95fc737fc6 MEDIUM: quic: separate path for rx and tx with set_encryption_secrets
With quicTLS the set_encruption_secrets callback is always called with
the read_secret and the write_secret.

However this is not the case with libreSSL, which uses the
set_read_secret()/set_write_secret() mecanism. It still provides the
set_encryption_secrets() callback, which is called with a NULL
parameter for the write_secret during the read, and for the read_secret
during the write.

The exchange key was not designed in haproxy to be called separately for
read and write, so this patch allow calls with read or write key to
NULL.
2022-09-14 18:16:37 +02:00
William Lallemand
1c8f3b386d MINOR: httpclient: export httpclient_create_proxy()
Export httpclient_create_proxy() in http_client.h
2022-09-14 14:34:39 +02:00
Mathias Weiersmueller
243c2d1822 DOC: fix TOC in starter guide for subsection 3.3.8. Statistics
This subsection has been moved from 3.4.9 to 3.3.8 somewhere along
2.4, but the TOC has not been updated - resulting in a invalid
anchor in the HTML version.

Needs to be backported to 2.4+
2022-09-14 06:01:55 +02:00
William Lallemand
992ad62e3c MEDIUM: httpclient: allow to use another proxy
httpclient_new_from_proxy() is a variant of httpclient_new() which
allows to create the requests from a different proxy.

The proxy and its 2 servers are now stored in the httpclient structure.

The proxy must have been created with httpclient_create_proxy() to be
used.

The httpclient_postcheck() callback will finish the initialization of
all proxies created with PR_CAP_HTTPCLIENT.
2022-09-13 17:12:38 +02:00
William Lallemand
54aec5f678 MEDIUM: httpclient: httpclient_create_proxy() creates a proxy for httpclient
httpclient_create_proxy() is a function which creates a proxy that could
be used for the httpclient. It will allocate a proxy, a raw server and
an ssl server.

This patch moves most of the code from httpclient_precheck() into a
generic function httpclient_create_proxy().

The proxy will have the PR_CAP_HTTPCLIENT capability.

This could be used for specifics httpclient instances that needs
different proxy settings.
2022-09-13 17:12:38 +02:00
William Lallemand
23bc0b20bd REGTESTS: ssl/log: test the log-forward with SSL
Test the log-forward section with an SSL server and an SSL bind.

Must be backported as far as 2.3.
2022-09-13 17:03:30 +02:00
Emeric Brun
d6e581de4b BUG/MEDIUM: sink: bad init sequence on tcp sink from a ring.
The init of tcp sink, particularly for SSL, was done
too early in the code, during parsing, and this can cause
a crash specially if nbthread was not configured.

This was detected by William using ASAN on a new regtest
on log forward.

This patch adds the 'struct proxy' created for a sink
to a list and this list is now submitted to the same init
code than the main proxies list or the log_forward's proxies
list. Doing this, we are assured to use the right init sequence.
It also removes the ini code for ssl from post section parsing.

This patch should be backported as far as v2.2

Note: this fix uses 'goto' labels created by commit
'BUG/MAJOR: log-forward: Fix log-forward proxies not fully initialized'
but this code didn't exist before v2.3 so this patch needs to be
adapted for v2.2.
2022-09-13 17:03:30 +02:00
William Lallemand
ebf600a838 REGTESTS: log: test the log-forward feature
This reg-test test the log-forward feature by chaining a UDP and a TCP
log-forwarder.

It could be backported as far as 2.3.
2022-09-13 11:04:24 +02:00
Willy Tarreau
439be5838d MINOR: flags/mux-h2: decode H2C and H2S flags
The new functions h2c_show_flags() and h2s_show_flags() decode the flags
state into a string, and are used by dev/flags:

  $ ./dev/flags/flags h2c 0x0600
  h2c->flags = H2_CF_DEM_IN_PROGRESS | H2_CF_DEM_SHORT_READ

  $ ./dev/flags/flags h2s 0x7003
  h2s->flags = H2_SF_HEADERS_RCVD | H2_SF_OUTGOING_DATA | H2_SF_HEADERS_SENT \
             | H2_SF_ES_SENT | H2_SF_ES_RCVD
2022-09-12 19:33:07 +02:00
Willy Tarreau
6c0fadfb7d REORG: mux-h2: extract flags and enums into mux_h2-t.h
Originally in 1.8 we wanted to have an independent mux that could possibly
be disabled and would not impose dependencies on the outside. Everything
would fit into a single C file and that was fine.

Nowadays muxes are unavoidable, and not being able to easily inspect them
from outside is sometimes a bit of a pain. In particular, the flags utility
still cannot be used to decode their flags.

As a first step towards this, this patch moves the flags and enums to
mux_h2-t.h, as well as the two state decoding inline functions. It also
dropped the H2_SS_*_BIT defines that nobody uses. The mux_h2.c file remains
the only one to include that for now.
2022-09-12 19:33:07 +02:00
Willy Tarreau
799e5410b4 MINOR: flags/fd: decode FD flags states
The new function is fd_show_flags() and it reports known FD flags:

  $ ./dev/flags/flags fd 0x000121
  fd->flags = FD_POLL_IN | FD_EV_READY_W | FD_EV_ACTIVE_R
2022-09-12 19:33:07 +02:00
Aurelien DARRAGON
a57786e87d BUG/MINOR: listener: null pointer dereference suspected by coverity
Please refer to GH #1859 for more info.
Coverity suspected improper proxy pointer handling.
Without the fix it is considered safe for the moment, but it might not
be the case in the future as we want to keep the ability to have
isolated listeners.

Making sure stop_listener(), pause_listener(), resume_listener()
and listener_release() functions make proper use
of px pointer in that context.

No need for backport except if multi-connection protocols (ie:FTP)
were to be backported as well.
2022-09-12 10:12:18 +02:00
Aurelien DARRAGON
187396e34e CLEANUP: listener: function comment typo in stop_listener()
A minor typo related to stop_listener() function comment
was introduced in 0013288.

This makes stop_listener() function comment easier to read.
2022-09-12 10:12:13 +02:00
Christopher Faulet
28bc152aa4 REGTESTS: healthcheckmail: Relax matching on the healthcheck log message
Depending on the timing, the conneciton on lisrv listener may be fully
accepted before any reject. Thus, instead of getting a socket error, an
invalid L7 response is reported. There is no reason to be strick on the
error type. Any failure is good here, because we just want to test the
email-alert feature.

This patch should fix issue #1857. It may be backported as far as 2.2.
2022-09-12 10:02:27 +02:00
Christopher Faulet
af5336fd23 BUG/MINOR: mux-h1: Increment open_streams counter when H1 stream is created
Since this counter was added, it was incremented at the wrong place for
client streams. It was incremented when the stream-connector (formely the
conn-stream) was created while it should be done when the H1 stream is
created. Thus, on parsing error, on H1>H2 upgrades or TCP>H1 upgrades, the
counter is not incremented. However, it is always decremented when the H1
stream is destroyed.

On bakcned side, there is no issue.

This patch must be backported to 2.6.
2022-09-12 09:54:11 +02:00
Willy Tarreau
af985e0151 CLEANUP: pollers: remove dead code in the polling loop
As reported by Ilya and Coverity in issue #1858, since recent commit
eea152ee6 ("BUG/MINOR: signals/poller: ensure wakeup from signals")
which removed the test for the global signal flag from the pollers'
loop, the remaining "wake" flag doesn't need to be tested since it
already participates to zeroing the wait_time and will be caught
on the previous line.

Let's just remove that test now.
2022-09-12 09:35:44 +02:00
Willy Tarreau
62bde43779 BUILD: flags: fix the fallback macros for missing stdio
The fallback macros for when stdio is not there didn't have the "..."
and were causing build issues on platforms with stricter dependencies
between includes.
2022-09-09 17:46:45 +02:00
Willy Tarreau
233c0a586d BUILD: flags: fix build warning in some macros used by show_flags
Some gcc versions seem to be upset by the use of enums as booleans,
so OK, let's cast all of them as uint, that's no big deal.
2022-09-09 17:36:27 +02:00
Aurelien DARRAGON
cddec0aef5 BUG/MINOR: stats: fixing stat shows disabled frontend status as 'OPEN'
This patch adresses the issue #1626.

Adding support for PR_FL_PAUSED flag in the function stats_fill_fe_stats().
The command 'show stat' now properly reports a disabled frontend
using "PAUSED" state label.

This patch depends on the following commits:
  - 7d00077fd5 "BUG/MEDIUM: proxy: ensure pause_proxy()
  and resume_proxy() own PROXY_LOCK".
  - 001328873c "MINOR: listener: small API change"
  - d46f437de6 "MINOR: proxy/listener: support for additional PAUSED state"

It should be backported to 2.6, 2.5 and 2.4
2022-09-09 17:24:22 +02:00
Aurelien DARRAGON
d46f437de6 MINOR: proxy/listener: support for additional PAUSED state
This patch is a prerequisite for #1626.
Adding PAUSED state to the list of available proxy states.
The flag is set when the proxy is paused at runtime (pause_listener()).
It is cleared when the proxy is resumed (resume_listener()).

It should be backported to 2.6, 2.5 and 2.4
2022-09-09 17:23:01 +02:00
Aurelien DARRAGON
001328873c MINOR: listener: small API change
A minor API change was performed in listener(.c/.h) to restore consistency
between stop_listener() and (resume/pause)_listener() functions.

LISTENER_LOCK was never locked prior to calling stop_listener():
lli variable hint is thus not useful anymore.

Added PROXY_LOCK locking in (resume/pause)_listener() functions
with related lpx variable hint (prerequisite for #1626).

It should be backported to 2.6, 2.5 and 2.4
2022-09-09 17:23:01 +02:00
Aurelien DARRAGON
7d00077fd5 BUG/MEDIUM: proxy: ensure pause_proxy() and resume_proxy() own PROXY_LOCK
There was a race involving hlua_proxy_* functions
and some proxy management functions.

pause_proxy() and resume_proxy() can be used directly from lua code,
but that could lead to some race as lua code didn't make sure PROXY_LOCK
was owned before calling the proxy functions.

This patch makes sure it won't happen again elsewhere in the code
by locking PROXY_LOCK directly in resume and pause proxy functions
so that it's not the caller's responsibility anymore.
(based on stop_proxy() behavior that was already safe prior to the patch)

This should be backported to stable series.
Note that the API will likely differ < 2.4
2022-09-09 17:23:01 +02:00
Willy Tarreau
6edae6ff48 MINOR: flags/http_ana: use flag dumping to show http msg states
The function is hmsg_show_flags(). It shows the HTTP_MSGF_* flags.
2022-09-09 17:18:57 +02:00
Willy Tarreau
5349779e40 MINOR: flags/htx: use flag dumping to show htx and start-line flags
The function are respectively htx_show_flags() and hsl_show_flags().
2022-09-09 16:59:29 +02:00
Willy Tarreau
cdefa80e6c DEV: flags: remove the now useless intermediary functions
There's no more point keeping functions that are just wrappers around
other ones, let's directly call them from the main entry point. It helps
visually control the mapping between output formats and their definition
and doesn't require to invent long names. For a bit more readability, the
tmpbuf and its size adopted slightly shorter names.
2022-09-09 16:52:09 +02:00
Willy Tarreau
a8db91ffa2 DEV: flags: remove the now unused SHOW_FLAG() definition
All dump functions were converted, no need to use this macro anymore.
2022-09-09 16:52:09 +02:00
Willy Tarreau
e2afad0af4 MINOR: flags/http_ana: use flag dumping for txn flags
The new function is txn_show_flags(). It dumps the TXN flags
as well as the client and server cookie types.
2022-09-09 16:52:09 +02:00
Willy Tarreau
92a2d3c02b MINOR: flags/task: use flag dumping for task state
The new function is task_show_state().
2022-09-09 16:52:09 +02:00
Willy Tarreau
e9d1283cc5 MINOR: flags/stream: use flag dumping for stream flags
The new function is strm_show_flags(). It dumps the stream flags
as well as the err type under SF_ERR_MASK and the final state under
SF_FINST_MASK.
2022-09-09 16:52:09 +02:00
Willy Tarreau
f4cb98ce56 MINOR: flags/stream: use flag dumping for stream error type
The new function is strm_et_show_flags(). Only the error type is
handled at the moment, as a bit more complex logic is needed to
mix the values and enums present in some fields.
2022-09-09 16:52:09 +02:00
Willy Tarreau
4bab7d81b6 MINOR: flags/stconn: use flag dumping for stconn and sedesc flags
The two new functions are se_show_flags() and sc_show_flags().
Maybe something could be done for SC_ST_* values but as it's a
small enum, a simple switch/case should work fine.
2022-09-09 16:52:08 +02:00
Willy Tarreau
9d9e101689 MINOR: flags/connection: use flag dumping for connection flags
The new function is conn_show_flags(), it only deals with flags. Nothing
is planned for connection error types at the moment.
2022-09-09 16:15:10 +02:00
Willy Tarreau
cdc9ddc8cf MINOR: flags/channel: use flag dumping for channel flags and analysers
The two new functions are chn_show_analysers() and chn_show_flags().
They work on an existing buffer so one was declared in flags.c for this
purpose. File flags.c does not have to know about channel flags anymore.
2022-09-09 16:15:10 +02:00
Willy Tarreau
7a955b5d73 MINOR: flags: implement a macro used to dump enums inside masks
Some of our flags have enums inside a mask. The new macro __APPEND_ENUM
is able to deal with that by comparing the flag's value against an exact
one under the mask. One needs to take care of eliminating the zero value
though, otherwise delimiters will not always be properly placed (e.g. if
some flags were dumped before and what remains is exactly zero). The
bits of the mask are cleared only upon exact matches.
2022-09-09 16:15:10 +02:00
Willy Tarreau
77acaf5af5 MINOR: flags: add a new file to host flag dumping macros
The "flags" utility is useful but painful to maintain up to date. This
commit aims at providing a low-maintenance solution to keep flags up to
date, by proposing some macros that build a string from a set of flags
in a way that requires the least possible verbosity.

The idea will be to add an inline function dedicated to this just after
the flags declaration, and enumerate the flags one is interested in, and
that function will fill a string based on them.

Placing this inside the type files allows both haproxy and external tools
like "flags" to use it, but comes with a few constraints. First, the
files will be slightly less readable if these functions are huge, so they
need to stay as compact as possible. Second, the function will need
anprintf() and we don't want to include stdio.h in type files as it
proved to be particularly heavy and to cause definition headaches in
the past.

As such the file here only contains a macro enclosed in #ifdef EOF (that
is defined in stdio), and provides an alternate empty one when no stdio
is defined. This way it's the caller that has to include stdio first or
it won't get anything back, and in practice the locations relying on
this always have it.

The macro has to be used in 3 steps:
  - prologue: dumps 0 and exits if the value is zero
  - flags: the macro can be recursively called and it will push the
    flag from bottom to top so that they appear in the same order as
    today without requiring to be declared the other way around
  - epilogue: dump remaining flags that were not identified

The macro was arranged so that a single character can be used with no
other argument to declare all flags at once. Example:

  #define _(n, ...) __APPEND_FLAG(buf, len, del, flg, n, #n, __VA_ARGS__)
     _(0);
     _(X_FLAG1, _(X_FLAG2, _(X_FLAG3, _(X_FLAG4))));
     _(~0);
  #undef _

Existing files will have to be updated to rely on it, and more files
could come soon.
2022-09-09 14:47:31 +02:00
Willy Tarreau
20273ceec0 DEV: flags: add missing CO_FL_FDLESS connection flag
This was added in 2.6 by commit c78a9698e ("MINOR: connection: add a new
flag CO_FL_FDLESS on fd-less connections") but forgotten in flags.c.
This must be backported to 2.6.
2022-09-09 14:46:15 +02:00
Willy Tarreau
c7ac17412b DEV: flags: fix usage message to reflect available options
The proposed decoding options were not updated after the changes in 2.6,
let's fix that by taking the names from the existing declaration. This
should be backported to 2.6.
2022-09-09 14:26:29 +02:00
Ilya Shipitsin
b0ab121da1 CI: cirrus-ci: bump FreeBSD image to 13-1
we use FreeBSD binary packages that we rebuilt for FreeBSD-13.1

Newer FreeBSD version for package zstd:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1301000
- running kernel: 1300139
2022-09-09 13:30:17 +02:00
Matthias Wirth
eea152ee68 BUG/MINOR: signals/poller: ensure wakeup from signals
Add self-wake in signal_handler() to fix a race condition with a signal
coming in between checking signal_queue_len and entering polling sleep.

The changes in commit 43c891dda ("BUG/MINOR: signals/poller: set the
poller timeout to 0 when there are signals") were insufficient.

Move the signal_queue_len check from the poll implementations to
run_poll_loop() to keep that logic in one place.

The poll loops are terminated either by the parameter wake being set or
wake up due to a write to their poller_wr_pipe by wake_thread() in
signal_handler().

This fixes issue #1841.

Must be backported in every stable version.
2022-09-09 11:15:22 +02:00
Frédéric Lécaille
ef2d2340e6 BUILD: udp-perturb: Add a make target for udp-perturb tool
This is only to rely on make to build this tool.
2022-09-08 20:47:28 +02:00
Frédéric Lécaille
192093b581 MINOR: dev/udp: Apply the corruption to both directions
Harden the UDP datagram corruption applying it on both sides. This approaches
the conditions of some tests run by the QUIC interop runner developed by
Marten Seeman.
2022-09-08 20:38:59 +02:00
Frédéric Lécaille
3dd79d378c MINOR: h3: Send the h3 settings with others streams (requests)
This is the ->finalize application callback which prepares the unidirectional STREAM
frames for h3 settings and wakeup the mux I/O handler to send them. As haproxy is
at the same time always waiting for the client request, this makes haproxy
call sendto() to send only about 20 bytes of stream data. Furthermore in case
of heavy loss, this give less chances to short h3 requests to succeed.

Drawback: as at this time the mux sends its streams by their IDs ascending order
the stream 0 is always embedded before the unidirectional stream 3 for h3 settings.
Nevertheless, as these settings may be lost and received after other h3 request
streams, this is permitted by the RFC.

Perhaps there is a better way to do. This will have to be checked with Amaury.

Must be backported to 2.6.
2022-09-08 18:04:58 +02:00
Frédéric Lécaille
befcf7031d MINOR: h3: Missing connection argument for a TRACE_LEAVE() argument
This should help in debbuging issues to be able to associate this trace to a
QUIC connection.

Must be backported to 2.6.
2022-09-08 18:04:58 +02:00
Frédéric Lécaille
2eb5faa2ad MINOR: h3: Add the quic_conn object to h3 traces
This is very useful to associate h3 traces to a QUIC connection when debugging.

Must be backported to 2.6.
2022-09-08 18:04:58 +02:00
Frédéric Lécaille
1c725aa9cd BUG/MINOR: h3: Crash when h3 trace verbosity is "minimal"
This was due to a missing check in h3_trace() about the first argument
presence (connection) and h3_parse_settings_frm() which calls TRACE_LEAVE()
without any argument. Then this argument was dereferenced.

Must be backported to 2.6
2022-09-08 18:04:58 +02:00
Frédéric Lécaille
3c1b81fdd7 BUG/MINOR: quic: Trace fix about packet number space information.
<qc> variable was confused with <qel>. The consequence was that it was
always the same packet number space which was displayed: the first one (or
the Initial packet number space).

Must be backported to 2.6.
2022-09-08 18:04:58 +02:00
Frédéric Lécaille
bb995eafc7 BUG/MINOR: quic: Speed up the handshake completion only one time
It is possible to speed up the handshake completion but only one time
by connection as mentionned in RFC 9002 "6.2.3. Speeding up Handshake Completion".
Add a flag to prevent this process to be run several times
(see https://www.rfc-editor.org/rfc/rfc9002#name-speeding-up-handshake-compl).

Must be backported to 2.6.
2022-09-08 18:04:58 +02:00
William Lallemand
43c891dda0 BUG/MINOR: signals/poller: set the poller timeout to 0 when there are signals
When receiving a signal before entering the poller, and without any
activity in the process, the poller will be entered with a timeout
calculated without checking the signals.

Since commit 4f59d3 ("MINOR: time: increase the minimum wakeup interval
to 60s") the issue is much more visible because it could be stuck for
60s.

When in mworker mode, if a worker quits and the SIGCHLD signal deliver
at the right time to the master, this one could be stuck for the time of
the timeout.

This should fix issue #1841

Must be backported in every stable version.
2022-09-08 17:46:31 +02:00
Willy Tarreau
e86bc35672 MINOR: activity/cli: support sorting task profiling by total CPU time
The new "bytime" sorting criterion uses the reported CPU time instead of
the usage. This is convenient to spot tasks that are mostly reponsible
for the CPU usage in a running process. It supports both the detailed
and the aggregated format. The output looks like this:

> show profiling tasks bytime
Tasks activity:
  function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
  qc_io_cb                     117739   1.961m    999.1us   37.45s    318.1us <- h3_snd_buf@src/h3.c:1084 tasklet_wakeup
  process_stream              7376273   1.384m    11.26us   1.013h    494.2us <- stream_new@src/stream.c:563 task_wakeup
  process_stream              8104400   1.133m    8.389us   1.130h    502.0us <- sc_notify@src/stconn.c:1209 task_wakeup
  qc_io_cb                      43280   45.76s    1.057ms   13.95s    322.3us <- qc_stream_desc_ack@src/quic_stream.c:128 tasklet_wakeup
  h1_io_cb                   11025715   24.82s    2.251us   5.406m    29.42us <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  quic_conn_app_io_cb          312861   23.86s    76.27us   2.373s    7.584us <- qc_lstnr_pkt_rcv@src/xprt_quic.c:6184 tasklet_wakeup_after
  qc_io_cb                      37063   12.65s    341.4us   6.409s    172.9us <- qc_treat_acked_tx_frm@src/xprt_quic.c:1695 tasklet_wakeup
  h1_io_cb                    4783520   11.79s    2.463us   1.419h    1.068ms <- conn_subscribe@src/connection.c:732 tasklet_wakeup
  sc_conn_io_cb              12269693   11.51s    938.0ns   2.117h    621.2us <- sc_app_chk_rcv_conn@src/stconn.c:762 tasklet_wakeup
  sc_conn_io_cb               6479006   10.94s    1.689us   7.984m    73.93us <- h1_wake_stream_for_recv@src/mux_h1.c:2600 tasklet_wakeup
  qc_io_cb                      12011   10.72s    892.5us   2.120s    176.5us <- qcc_release_remote_stream@src/mux_quic.c:1200 tasklet_wakeup
  h2_io_cb                     246423   6.225s    25.26us   56.52s    229.4us <- h2_snd_buf@src/mux_h2.c:6712 tasklet_wakeup
  h2_io_cb                     137744   6.076s    44.11us   16.59s    120.4us <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  quic_lstnr_dghdlr            323575   3.062s    9.462us   3.424m    634.9us <- quic_lstnr_dgram_dispatch@src/quic_sock.c:255 tasklet_wakeup
  sc_conn_io_cb               1206939   1.616s    1.338us   27.62m    1.373ms <- qcs_notify_send@src/mux_quic.c:529 tasklet_wakeup
  h2_io_cb                     212370   251.2ms   1.182us   6.476s    30.49us <- h2c_restart_reading@src/mux_h2.c:856 tasklet_wakeup
  h1_io_cb                      44109   197.0ms   4.466us   31.89s    723.0us <- h1_takeover@src/mux_h1.c:4085 tasklet_wakeup
  quic_conn_app_io_cb            3029   87.59ms   28.92us   999.0ms   329.8us <- qc_process_timer@src/xprt_quic.c:4635 tasklet_wakeup
  task_run_applet                  40   35.77ms   894.3us   4.407ms   110.2us <- sc_applet_create@src/stconn.c:489 appctx_wakeup
  task_run_applet                  18   27.36ms   1.520ms   19.56us   1.086us <- sc_app_chk_snd_applet@src/stconn.c:996 appctx_wakeup
  sc_conn_io_cb                  2186   11.76ms   5.377us   963.0ms   440.5us <- h1_wake_stream_for_send@src/mux_h1.c:2610 tasklet_wakeup
  qc_io_cb                          8   9.880ms   1.235ms   5.871ms   733.9us <- qcs_consume@src/mux_quic.c:800 tasklet_wakeup
  quic_conn_io_cb                   4   5.951ms   1.488ms   38.85us   9.713us <- qc_lstnr_pkt_rcv@src/xprt_quic.c:6184 tasklet_wakeup_after
  qc_io_cb                        101   4.975ms   49.26us   13.91ms   137.8us <- qc_process_timer@src/xprt_quic.c:4602 tasklet_wakeup
  h1_io_cb                       2186   1.809ms   827.0ns   720.2ms   329.5us <- sock_conn_iocb@src/sock.c:849 tasklet_wakeup
  qc_process_timer               3031   1.735ms   572.0ns   1.153s    380.3us <- wake_expired_tasks@src/task.c:344 task_wakeup
  accept_queue_process            359   1.362ms   3.793us   80.32ms   223.7us <- listener_accept@src/listener.c:1099 tasklet_wakeup
  quic_conn_app_io_cb               2   921.1us   460.6us   203.1us   101.5us <- qc_xprt_start@src/xprt_quic.c:7122 tasklet_wakeup
  h1_timeout_task                2618   526.8us   201.0ns   1.121s    428.4us <- h1_release@src/mux_h1.c:1087 task_wakeup
  process_resolvers               316   283.3us   896.0ns   14.96ms   47.33us <- wake_expired_tasks@src/task.c:429 task_drop_running
  sc_conn_io_cb                   420   235.6us   560.0ns   116.7ms   277.8us <- h2s_notify_recv@src/mux_h2.c:1298 tasklet_wakeup
  qc_idle_timer_task                1   225.5us   225.5us   506.0ns   506.0ns <- wake_expired_tasks@src/task.c:344 task_wakeup
  accept_queue_process             36   153.0us   4.250us   5.834ms   162.1us <- accept_queue_process@src/listener.c:165 tasklet_wakeup
  sc_conn_io_cb                    18   54.05us   3.003us   11.50us   638.0ns <- sock_conn_iocb@src/sock.c:869 tasklet_wakeup
  h2_io_cb                          6   38.88us   6.480us   2.089ms   348.2us <- h2_do_shutw@src/mux_h2.c:4656 tasklet_wakeup
  srv_cleanup_idle_conns           54   37.72us   698.0ns   14.21ms   263.1us <- wake_expired_tasks@src/task.c:429 task_drop_running
  sc_conn_io_cb                    50   32.86us   657.0ns   28.83ms   576.5us <- qcs_notify_recv@src/mux_quic.c:519 tasklet_wakeup
  qc_io_cb                          2   30.25us   15.12us   6.093us   3.046us <- qc_init@src/mux_quic.c:2057 tasklet_wakeup
  srv_cleanup_toremove_conns        1   27.16us   27.16us   905.6us   905.6us <- srv_cleanup_idle_conns@src/server.c:5948 task_wakeup
  task_run_applet                  39   19.61us   502.0ns   818.7us   20.99us <- run_tasks_from_lists@src/task.c:652 task_drop_running
  quic_accept_run                   2   15.46us   7.727us   305.5us   152.8us <- quic_accept_push_qc@src/quic_sock.c:458 tasklet_wakeup
  h2_timeout_task                  32   12.91us   403.0ns   4.207ms   131.5us <- h2_release@src/mux_h2.c:1191 task_wakeup
  quic_conn_app_io_cb               1   9.645us   9.645us   1.445us   1.445us <- qc_process_timer@src/xprt_quic.c:4589 tasklet_wakeup

> show profiling tasks bytime aggr
Tasks activity:
  function                      calls   cpu_tot   cpu_avg   lat_tot   lat_avg
  qc_io_cb                     212301   3.147m    889.5us   1.009m    285.2us
  process_stream             15503573   2.519m    9.747us   2.148h    498.7us
  h1_io_cb                   15916733   36.95s    2.321us   1.535h    347.1us
  quic_conn_app_io_cb          318845   24.21s    75.92us   3.410s    10.70us
  sc_conn_io_cb              20037058   24.19s    1.207us   2.737h    491.8us
  h2_io_cb                     596543   12.55s    21.04us   1.326m    133.4us
  quic_lstnr_dghdlr            326624   3.094s    9.473us   3.462m    635.9us
  task_run_applet                 100   64.43ms   644.3us   5.285ms   52.85us
  quic_conn_io_cb                   4   5.951ms   1.488ms   38.85us   9.713us
  qc_process_timer               3061   1.750ms   571.0ns   1.162s    379.5us
  accept_queue_process            396   1.521ms   3.840us   86.16ms   217.6us
  h1_timeout_task                2618   526.8us   201.0ns   1.121s    428.4us
  process_resolvers               319   286.0us   896.0ns   16.82ms   52.73us
  qc_idle_timer_task                1   225.5us   225.5us   506.0ns   506.0ns
  srv_cleanup_idle_conns           54   37.72us   698.0ns   14.21ms   263.1us
  srv_cleanup_toremove_conns        1   27.16us   27.16us   905.6us   905.6us
  quic_accept_run                   2   15.46us   7.727us   305.5us   152.8us
  h2_timeout_task                  32   12.91us   403.0ns   4.207ms   131.5us
2022-09-08 16:38:10 +02:00