17096 Commits

Author SHA1 Message Date
Christopher Faulet
64b8d33577 MINOR: connection: unconst mux's get_fist_cs() callback function
This change is mandatory for next commits.
2022-04-13 15:10:15 +02:00
Christopher Faulet
3704663e5f MINOR: applet: Use the CS to register and release applets instead of SI
si_register_applet() and si_applet_release() are renamed
cs_register_applet() and cs_applet_release() and now manipulate a
conn-stream instead of a stream-inteface.
2022-04-13 15:10:15 +02:00
Christopher Faulet
0c6a64cd5f MEDIUM: stream-int/conn-stream: Move si_ops in the conn-stream scope
The si_ops structure is renamed to cs_app_ops and the callback functions are
changed to manipulate a conn-stream instead of a stream-interface..
2022-04-13 15:10:15 +02:00
Christopher Faulet
da098e6c17 MINOR: stream-int/conn-stream: Move si_shut* and si_chk* in conn-stream scope
si_shutr(), si_shutw(), si_chk_rcv() and si_chk_snd() are moved in the
conn-stream scope and renamed, respectively, cs_shutr(), cs_shutw(),
cs_chk_rcv(), cs_chk_snd() and manipulate a conn-stream instead of a
stream-interface.
2022-04-13 15:10:15 +02:00
Christopher Faulet
69ef6c9ef4 MINOR: conn-stream: Rename CS functions dedicated to connections
Some conn-stream functions are only used when there is a connection. Thus,
they was renamed with "cs_conn_" prefix. In addition, we expect to have a
connection, so a BUG_ON is added to be sure the functions are never called
in another context.
2022-04-13 15:10:15 +02:00
Christopher Faulet
2f35e7b6ab MEDIUM: stream-int/conn-stream: Handle I/O subscriptions in the conn-stream
wait_event structure is moved in the conn-stream. The tasklet is only
created if the conn-stream is attached to a mux and released when the mux is
detached. This implies a subtle change. In stream_int_chk_rcv() function,
the wakeup of the tasklet was removed because there is no longer tasklet at
this stage (stream_int_chk_rcv() is a callback function of si_embedded_ops).
2022-04-13 15:10:15 +02:00
Christopher Faulet
070b91bc11 MEDIUM: conn-stream: Be prepared to fail to attach a cs to a mux
To be able to move wait_event from the stream-interface to the conn-stream,
we must be prepare to handle errors when a mux is attached to a conn-stream.
Indeed, the wait_event's tasklet will be allocated when both a mux and a
stream will be both attached to a stream. So, we must be prepared to handle
allocation errors.
2022-04-13 15:10:15 +02:00
Christopher Faulet
0797656ead MINOR: conn-stream/connection: Move SHR/SHW modes in the connection scope
These flags only concerns the connection part. In addition, it is required
for a next commit, to avoid circular deps. Thus CS_SHR_* and CS_SHW_* were
renamed with the "CO_" prefix.
2022-04-13 15:10:15 +02:00
Christopher Faulet
e39a4dfdf0 MINOR: stream-int/conn-stream: Move si_conn_ready() in the conn-stream scope
si_conn_ready() is renamed cs_conn_ready() and handle a conn-stream insted
of a stream-interface. The function is now in cs_utils.h.
2022-04-13 15:10:15 +02:00
Christopher Faulet
0a4dcb65ff MINOR: stream-int/backend: Move si_connect() in the backend scope
si_connect() is moved in backend.c and renamed as do_connect_server(). In
addition, the function now manipulate a stream instead of a
stream-interface.
2022-04-13 15:10:15 +02:00
Christopher Faulet
9125f3cc77 MINOR: stream-int/stream: Move si_retnclose() in the stream scope
si_retnclose() is used to send a reply to a client before closing. There is
no use on the server side, in spite of the function is generic. Thus, it is
renamed stream_retnclose() and moved into the stream scope. The function now
handle a stream and explicitly send a message to the client.
2022-04-13 15:10:15 +02:00
Christopher Faulet
62e757470a MEDIUM: stream-int/conn-stream: Move stream-interface state in the conn-stream
The stream-interface state (SI_ST_*) is now in the conn-stream. It is a
mechanical replacement for now. Nothing special. SI_ST_* and SI_SB_* were
renamed accordingly. Utils functions to manipulate these infos were moved
under the conn-stream scope.

But it could be good to keep in mind that this part should be
reworked. Indeed, at the CS level, we only need to know if it is ready to
receive or to send. The state of conn-stream from INI to EST is only used on
the server side. The client CS is immediately set to EST. Thus current
SI_ST_* states should probably be moved to the stream to reflect the server
connection state during the establishment stage.
2022-04-13 15:10:15 +02:00
Christopher Faulet
50264b41c8 MEDIUM: stream-int: Move SI err_type in the stream
Only the server side is concerned by the stream-interface error type. It is
useless to have an err_type field on the client side. So, it is now move to
the stream. SI_ET_* are renames STRM_ET_* and moved in stream-t.h header
file.
2022-04-13 15:10:14 +02:00
Christopher Faulet
a70a3548bc MINOR: stream: Only save previous connection state for the server side
The previous connection state on the client side was only used for debugging
purpose to report client close. But this may be handled when the client
stream-interface is switched from SI_ST_DIS to SI_ST_CLO.

So, there only remains the previous connection state on the server side that
is used by the stream, in process_stream(), to be able to set the correct
termination flags. Thus, instead of keeping this info in the
stream-interface for only one side, the info is now stored in the stream
itself.
2022-04-13 15:10:14 +02:00
Christopher Faulet
78ed7f247b CLEANUP: stream-int: Remove unused SI_FL_CLEAN_ABRT flag
This flag is unused. So remove it to be able to remove the stream-interface.
2022-04-13 15:10:14 +02:00
Christopher Faulet
d139138bbc MINOR: stream-int: Remove SI_FL_SRC_ADDR to rely on stream flags instead
Flag to get the source ip/port with getsockname is now handled at the stream
level. Thus SI_FL_SRC_ADDR stream-int flag is replaced by SF_SRC_ADDR stream
flag.
2022-04-13 15:10:14 +02:00
Christopher Faulet
a728518c15 MINOR: stream-int: Remove SI_FL_INDEP_STR to rely on CS flags instead
Flag to consider a stream as indepenent is now handled at the conn-stream
level. Thus SI_FL_INDEP_STR stream-int flag is replaced by CS_FL_INDEP_STR
conn-stream flags.
2022-04-13 15:10:14 +02:00
Christopher Faulet
974da9f8a4 MINOR: stream-int: Remove SI_FL_DONT_WAKE to rely on CS flags instead
Flag to not wake the stream up on I/O is now handled at the conn-stream
level. Thus SI_FL_DONT_WAKE stream-int flag is replaced by CS_FL_DONT_WAKE
conn-stream flags.
2022-04-13 15:10:14 +02:00
Christopher Faulet
8abe712749 MINOR: stream-int: Remove SI_FL_NOLINGER/NOHALF to rely on CS flags instead
Flags to disable lingering and half-close are now handled at the conn-stream
level. Thus SI_FL_NOLINGER and SI_FL_NOHALF stream-int flags are replaced by
CS_FL_NOLINGER and CS_FL_NOHALF conn-stream flags.
2022-04-13 15:10:14 +02:00
Christopher Faulet
ca2b5274b5 MINOR: mux-h2/mux-fcgi: Fully rely on CS_EP_KILL_CONN
Instead of using a internal flag to kill the connection with the stream, we
now fully rely on CS_EP_KILL_CONN flag.
2022-04-13 15:10:14 +02:00
Christopher Faulet
9a52123800 MINOR: stream-int: Remove SI_FL_KILL_CON to rely on conn-stream endpoint only
Instead of setting a stream-interface flag to then set the corresponding
conn-stream endpoint flag, we now only rely the conn-stream endoint. Thus
SI_FL_KILL_CON is replaced by CS_EP_KILL_CONN.

In addition si_must_kill_conn() is replaced by cs_must_kill_conn().
2022-04-13 15:10:14 +02:00
Christopher Faulet
7b5ca8f457 MINOR: channel: Use conn-streams as channel producer and consumer
chn_prod() and chn_cons() now return a conn-stream instead of a
stream-interface.
2022-04-13 15:10:14 +02:00
Christopher Faulet
6cd56d5a69 MEDIUM: conn-stream: Use endpoint error instead of conn-stream error
Instead of relying on the conn-stream error, via CS_FL_ERR flags, we now
directly use the error at the endpoint level with the flag CS_EP_ERROR. It
should be safe to do so. But we must be careful because it is still possible
that an error is processed too early. Anyway, a conn-stream has always a
valid endpoint, maybe detached from any endpoint, but valid.
2022-04-13 15:10:14 +02:00
Christopher Faulet
af642df3b8 MINOR: stream-int/conn-stream: Report error to the CS instead of the SI
SI_FL_ERR is removed and replaced by CS_FL_ERROR. It is a transient patch
because the idea is to rely on the endpoint to handle errors at this
level. But if for any reason it is not possible, the stream-interface flags
will still be replaced.
2022-04-13 15:10:14 +02:00
Christopher Faulet
ae024ced03 MEDIUM: stream-int/stream: Use connect expiration instead of SI expiration
The expiration date in the stream-interface was only used on the server side
to set the connect, queue or turn-around timeout. It was checked on the
frontend stream-interface, but never used concretely. So it was removed and
replaced by a connect expiration date in the stream itself. Thus, SI_FL_EXP
flag in stream-interfaces is replaced by a stream flag, SF_CONN_EXP.
2022-04-13 15:10:14 +02:00
Christopher Faulet
1d9877700e MINOR: stream-int/conn-stream: Move half-close timeout in the conn-stream
The half-close timeout (hcto) is now part of the conn-stream. It is a step
closer to the stream-interface removal.
2022-04-13 15:10:14 +02:00
Christopher Faulet
8da67aae3e MEDIUM: stream-int/conn-stream: Move src/dst addresses in the conn-stream
The source and destination addresses at the applicative layer are moved from
the stream-interface to the conn-stream. This simplifies a bit the code and
it is a logicial step to remove the stream-interface.
2022-04-13 15:10:14 +02:00
Christopher Faulet
731c8e6cf9 MINOR: stream: Simplify retries counter calculation
The conn_retries counter was set to the max value and decremented at each
connection retry. Thus the counter reflected the number of retries left and
not the real number of retries. All calculations of redispatch or reporting
of number of retries experienced were made using subtracts from the
configured retries, which was complicated and didn't bring any benefit.

Now, this counter is set to 0 and incremented at each retry. We know we've
reached the maximum allowed connection retries by comparing it to the
configured value. In all other cases, we directly use the counter.

This patch should address the feature request #1608.
2022-04-13 15:10:14 +02:00
Christopher Faulet
909f318259 MINOR: stream-int/stream: Move conn_retries counter in the stream
The conn_retries counter may be moved into the stream structure. It only
concerns the connection establishment. The frontend stream-interface does not
use it. So it is a logical change.
2022-04-13 15:10:14 +02:00
Christopher Faulet
c216185c71 CLEANUP: http-ana: Remove http_alloc_txn() function
Since the 2.4, this function is no longer used. Thus we can remove it.
2022-04-13 15:10:14 +02:00
Christopher Faulet
e05bf9e413 MINOR: stream-int/txn: Move buffer for L7 retries in the HTTP transaction
The L7 retries only concerns the stream when a server connection is
established. Thus instead of storing the L7 buffer into the
stream-interface, it may be moved to the stream. And because it is only
available for HTTP streams, it may be moved in the HTTP transaction.

Associated flags are also moved into the HTTP transaction.
2022-04-13 15:10:14 +02:00
Christopher Faulet
908628c4c0 MEDIUM: tree-wide: Use CS util functions instead of SI ones
At many places, we now use the new CS functions to get a stream or a channel
from a conn-stream instead of using the stream-interface API. It is the
first step to reduce the scope of the stream-interfaces. The main change
here is about the applet I/O callback functions. Before the refactoring, the
stream-interface was the appctx owner. Thus, it was heavily used. Now, as
far as possible,the conn-stream is used. Of course, it remains many calls to
the stream-interface API.
2022-04-13 15:10:14 +02:00
Christopher Faulet
1cf9ef25a9 MINOR: conn-stream: Add header file with util functions related to conn-streams
cs_utils.h header file will contain all util functions related to the
conn_streams. For now, few functions were added, all are equivalent to SI
functions. Idea is to progressively replace SI functions by CS ones.
2022-04-13 15:10:14 +02:00
Christopher Faulet
3099511571 MINOR: conn-stream: Add ISBACK conn-stream flag
CS_FL_ISBACK is a new flag, set on backend conn-streams. We must just be
careful to preserve this flag when the endpoint is detached from the
conn-stream.
2022-04-13 15:10:14 +02:00
Christopher Faulet
1bceee21e3 MINOR: mux-pt: Rely on the endpoint instead of the conn-stream when possible
Instead of testing if a conn-stream exists or not, we rely on CS_EP_ORPHAN
endpoint flag. In addition, if possible, we access the endpoint from the
mux_pt context. Finally, the endpoint flags are now reported in trace
messages.
2022-04-13 15:10:14 +02:00
Christopher Faulet
b041b23ae4 MEDIUM: conn-stream: Move remaning flags from CS to endpoint
All old flags CS_FL_* are now moved in the endpoint scope and renamed
CS_EP_* accordingly. It is a systematic replacement. There is no true change
except for the health-check and the endpoint reset. Here it is a bit special
because the same conn-stream is reused. Thus, we must handle endpoint
allocation errors. To do so, cs_reset_endp() has been adapted.

Thanks to this last change, it will now be possible to simplify the
multiplexer and probably the applets too. A review must also be performed to
remove some flags in the channel or the stream-interface. The HTX will
probably be simplified too. Finally, there is now some place in the
conn-stream to move info from the stream-interface.
2022-04-13 15:10:14 +02:00
Christopher Faulet
9ec2f4dc7c MAJOR: conn-stream: Share endpoint struct between the CS and the mux/applet
The conn-stream endpoint is now shared between the conn-stream and the
applet or the multiplexer. If the mux or the applet is created first, it is
responsible to also create the endpoint and share it with the conn-stream.
If the conn-stream is created first, it is the opposite.

When the endpoint is only owned by an applet or a mux, it is called an
orphan endpoint (there is no conn-stream). When it is only owned by a
conn-stream, it is called a detached endpoint (there is no mux/applet).

The last entity that owns an endpoint is responsible to release it. When a
mux or an applet is detached from a conn-stream, the conn-stream
relinquishes the endpoint to recreate a new one. This way, the endpoint
state is never lost for the mux or the applet.
2022-04-13 15:10:14 +02:00
Christopher Faulet
cb2fa368e9 REORG: applet: Uninline appctx_new function
appctx_new() is moved in the C file and appctx_init() is now private.
2022-04-13 15:10:14 +02:00
Christopher Faulet
a9e8b3979d MEDIUM: conn-stream: Pre-allocate endpoint to create CS from muxes and applets
It is a transient commit to prepare next changes. Now, when a conn-stream is
created from an applet or a multiplexer, an endpoint is always provided. In
addition, the API to create a conn-stream was specialized to have one
function per type.

The next step will be to share the endpoint structure.
2022-04-13 15:10:14 +02:00
Christopher Faulet
b669d684c0 MEDIUM: conn-stream: Be able to pass endpoint to create a conn-stream
It is a transient commit to prepare next changes. It is possible to pass a
pre-allocated endpoint to create a new conn-stream. If it is NULL, a new
endpoint is created, otherwise the existing one is used. There no more
change at the conn-stream level.

In the applets, all conn-stream are created with no pre-allocated
endpoint. But for multiplexers, an endpoint is systematically created before
creating the conn-stream.
2022-04-13 15:10:14 +02:00
Christopher Faulet
e9e4820288 MINOR: conn-stream: Move some CS flags to the endpoint
Some CS flags, only related to the endpoint, are moved into the endpoint
struct. More will probably moved later. Those ones are not critical. So it
is pretty safe to move them now and this will ease next changes.
2022-04-13 15:10:14 +02:00
Christopher Faulet
db90f2aa9f MEDIUM: conn-stream: Add an endpoint structure in the conn-stream
Group the endpoint target of a conn-stream, its context and the associated
flags in a dedicated structure in the conn-stream. It is not inlined in the
conn-stream structure. There is a dedicated pool.

For now, there is no complexity. It is just an indirection to get the
endpoint or its context. But the purpose of this structure is to be able to
share a refcounted context between the mux and the conn-stream. This way, it
will be possible to preserve it when the mux is detached from the
conn-stream.
2022-04-13 15:10:14 +02:00
Christopher Faulet
bb772d09f5 REORG: Initialize the conn-stream by hand in cs_init()
The function cs_init() is only called by cs_new(). The conn-stream
initialization will be reviewed. It is easier to do it in cs_new() instead
of using a dedicated function. cs_new() is pretty simple, there is no reason
to split the code in this case.
2022-04-13 15:10:14 +02:00
Christopher Faulet
9388204db1 MAJOR: conn-stream: Invert conn-stream endpoint and its context
This change is only significant for the multiplexer part. For the applets,
the context and the endpoint are the same. Thus, there is no much change. For
the multiplexer part, the connection was used to set the conn-stream
endpoint and the mux's stream was the context. But it is a bit strange
because once a mux is installed, it takes over the connection. In a
wonderful world, the connection should be totally hidden behind the mux. The
stream-interface and, in a lesser extent, the stream, still access the
connection because that was inherited from the pre-multiplexer era.

Now, the conn-stream endpoint is the mux's stream (an opaque entity for the
conn-stream) and the connection is the context. Dedicated functions have
been added to attached an applet or a mux to a conn-stream.
2022-04-13 15:10:14 +02:00
Christopher Faulet
2479e5f775 MEDIUM: applet: Set the appctx owner during allocation
The appctx owner is now always a conn-stream. Thus, it can be set during the
appctx allocation. But, to do so, the conn-stream must be created first. It
is not a problem on the server side because the conn-stream is created with
the stream. On the client side, we must take care to create the conn-stream
first.

This change should ease other changes about the applets bootstrapping.
2022-04-13 15:10:13 +02:00
Christopher Faulet
81a40f630e MINOR: conn-stream: Add flags to set the type of the endpoint
This patch is mandatory to invert the endpoint and the context in the
conn-stream. There is no common type (at least for now) for the entity
representing a mux (h1s, h2s...), thus we must set its type when the
endpoint is attached to a conn-stream. There is 2 types for the conn-stream
endpoints: the mux (CS_FL_ENDP_MUX) and the applet (CS_FL_ENDP_APP).
2022-04-13 15:10:13 +02:00
Christopher Faulet
4aa1d2838c MINOR: applet: Make .init callback more generic
For now there is no much change. Only the appctx is passed as argument when
the .init callback function is called. And it is not possible to yield at
this stage. It is not a problem because the feature is not used. Only the
lua defines this callback function for the lua TCP/HTTP services. The idea
is to be able to use it for all applets to initialize the appctx context.
2022-04-13 15:10:13 +02:00
Christopher Faulet
91449b0351 BUG/MINOR: mux-h1: Don't release unallocated CS on error path
cs_free() must not be called when we fail to allocate the conn-stream in
h1s_new_cs() function. This bug was introduced by the commit cda94accb
("MAJOR: stream/conn_stream: Move the stream-interface into the
conn-stream").

It is 2.6-specific, no backport is needed.
2022-04-13 15:10:13 +02:00
Willy Tarreau
f1de1b51ca BUG/MINOR: cache: do not display expired entries in "show cache"
It was mentioned in issue #12 that expired entries would appear with a
negative expire delay in "show cache". Instead of listing them, let's
just evict them.

This could be backported to all versions since this was reported on
1.8 already.
2022-04-13 11:21:39 +02:00
Willy Tarreau
15dbedd63d BUG/MINOR: mux-h2: do not send GOAWAY if SETTINGS were not sent
It was reported in issue #13 that a GOAWAY frame was sent on timeout even
if no SETTINGS frame was sent. The approach imagined by then was to track
the fact that a SETTINGS frame was already sent to avoid this, but that's
already what is done through the state, though it doesn't stand due to the
fact that we switch the frame to the error state. Thus instead what we're
doing here is to instead set the GOAWAY_FAILED flag in h2c_error() before
switching to the ERROR state when the state indicates we've not yet sent
settings, and refrain from sending anything from the h2c_send_goaway_error()
function for such states.

This could be backported to all versions where it applies well.
2022-04-13 09:40:52 +02:00