* Documentation about the hot-reconfiguration mechanism.

This commit is contained in:
willy tarreau 2006-01-20 20:43:32 +01:00
parent 808b4e6f61
commit 22739efb4a
3 changed files with 102 additions and 9 deletions

View File

@ -502,7 +502,7 @@ Flows :
4. Soft-stop for application maintenance 4. Soft-stop for application maintenance
======================================== ========================================
When an application is spread across several severs, the time to update all When an application is spread across several servers, the time to update all
instances increases, so the application seems jerky for a longer period. instances increases, so the application seems jerky for a longer period.
HAproxy offers several solutions for this. Although it cannot be reconfigured HAproxy offers several solutions for this. Although it cannot be reconfigured
@ -704,10 +704,73 @@ which is available for download here :
runs. You must really forward the check port to the real application. runs. You must really forward the check port to the real application.
- health-checks will be sent twice as often, once for each standard server, - health-checks will be sent twice as often, once for each standard server,
and once for reach backup server. All this will be multiplicated by the and once for each backup server. All this will be multiplicated by the
number of processes if you use multi-process mode. You will have to ensure number of processes if you use multi-process mode. You will have to ensure
that all the checks sent to the server do not overload it. that all the checks sent to the server do not overload it.
=======================
4.3 Hot reconfiguration
=======================
There are two types of haproxy users :
- those who can never do anything in production out of maintenance periods ;
- those who can do anything at any time provided that the consequences are
limited.
The first ones have no problem stopping the server to change configuration
because they got some maintenance periods during which they can break anything.
So they will even prefer doing a clean stop/start sequence to ensure everything
will work fine upon next reload. Since those have represented the majority of
haproxy uses, there has been little effort trying to improve this.
However, the second category is a bit different. They like to be able to fix an
error in a configuration file without anyone noticing. This can sometimes also
be the case for the first category because humans are not failsafe.
For this reason, a new hot reconfiguration mechanism has been introduced in
version 1.1.34. Its usage is very simple and works even in chrooted
environments with lowered privileges. The principle is very simple : upon
reception of a SIGTTOU signal, the proxy will stop listening to all the ports.
This will release the ports so that a new instance can be started. Existing
connections will not be broken at all. If the new instance fails to start,
then sending a SIGTTIN signal back to the original processes will restore
the listening ports. This is possible without any special privileges because
the sockets will not have been closed, so the bind() is still valid. Otherwise,
if the new process starts successfully, then sending a SIGUSR1 signal to the
old one ensures that it will exit as soon as its last session ends.
A hot reconfiguration script would look like this :
# save previous state
mv /etc/haproxy/config /etc/haproxy/config.old
mv /var/run/haproxy.pid /var/run/haproxy.pid.old
mv /etc/haproxy/config.new /etc/haproxy/config
kill -TTOU $(cat /var/run/haproxy.pid.old)
if haproxy -p /var/run/haproxy.pid -f /etc/haproxy/config; then
echo "New instance successfully loaded, stopping previous one."
kill -USR1 $(cat /var/run/haproxy.pid.old)
rm -f /var/run/haproxy.pid.old
exit 1
else
echo "New instance failed to start, resuming previous one."
kill -TTIN $(cat /var/run/haproxy.pid.old)
rm -f /var/run/haproxy.pid
mv /var/run/haproxy.pid.old /var/run/haproxy.pid
mv /etc/haproxy/config /etc/haproxy/config.new
mv /etc/haproxy/config.old /etc/haproxy/config
exit 0
fi
After this, you can still force old connections to end by sending
a SIGTERM to the old process if it still exists :
kill $(cat /var/run/haproxy.pid.old)
rm -f /var/run/haproxy.pid.old
Be careful with this as in multi-process mode, some pids might already
have been reallocated to completely different processes.
================================================== ==================================================
5. Multi-site load-balancing with local preference 5. Multi-site load-balancing with local preference

View File

@ -490,16 +490,16 @@ Example :
2.4) Soft stop 2.4) Soft stop
-------------- --------------
It is possible to stop services without breaking existing connections by the It is possible to stop services without breaking existing connections by the
sending of the SIG_USR1 signal to the process. All services are then put into sending of the SIGUSR1 signal to the process. All services are then put into
soft-stop state, which means that they will refuse to accept new connections, soft-stop state, which means that they will refuse to accept new connections,
except for those which have a non-zero value in the 'grace' parameter, in which except for those which have a non-zero value in the 'grace' parameter, in which
case they will still accept connections for the specified amount of time, in case they will still accept connections for the specified amount of time, in
milliseconds. This allows to tell a load-balancer that the service is failing, milliseconds. This makes it possible to tell a load-balancer that the service
while still doing the job during the time it needs to detect it. is failing, while still doing the job during the time it needs to detect it.
Note: active connections are never killed. In the worst case, the user will have Note: active connections are never killed. In the worst case, the user will have
to wait for all of them to close or to time-out, or simply kill the process to wait for all of them to close or to time-out, or simply kill the process
normally (SIG_TERM). The default 'grace' value is '0'. normally (SIGTERM). The default 'grace' value is '0'.
Example : Example :
--------- ---------
@ -515,6 +515,20 @@ Example :
grace 0 grace 0
As of version 1.1.34, a new soft-reconfiguration mechanism has been introduced.
It is now possible to "pause" all the proxies by sending a SIGTTOU signal to
the processes. This will disable the listening socket without breaking existing
connections. After that, sending a SIGTTIN signal to those processes enables
the listening sockets again. This is very useful to try to load a new
configuration or even a new version of haproxy without breaking existing
connections. If the load succeeds, then simply send a SIGUSR1 which will make
the previous proxies exit immediately once their sessions are closed ; and if
the load fails, then simply send a SIGTTIN to restore the service immediately.
Please note that the 'grace' parameter is ignored for SIGTTOU, as well as for
SIGUSR1 when the process was in the pause mode. Please also note that it would
be useful to save the pidfile before starting a new instance.
2.5) Connections expiration time 2.5) Connections expiration time
-------------------------------- --------------------------------
It is possible (and recommended) to configure several time-outs on TCP It is possible (and recommended) to configure several time-outs on TCP

View File

@ -511,7 +511,7 @@ Exemple :
2.4) Arrêt en douceur 2.4) Arrêt en douceur
--------------------- ---------------------
Il est possible d'arrêter les services en douceur en envoyant un signal Il est possible d'arrêter les services en douceur en envoyant un signal
SIG_USR1 au processus relais. Tous les services seront alors mis en phase SIGUSR1 au processus relais. Tous les services seront alors mis en phase
d'arrêt, mais pourront continuer d'accepter des connexions pendant un temps d'arrêt, mais pourront continuer d'accepter des connexions pendant un temps
défini par le paramètre 'grace' (en millisecondes). Cela permet par exemple, défini par le paramètre 'grace' (en millisecondes). Cela permet par exemple,
de faire savoir rapidement à un répartiteur de charge qu'il ne doit plus de faire savoir rapidement à un répartiteur de charge qu'il ne doit plus
@ -521,8 +521,9 @@ s'en rende compte.
Remarque : Remarque :
---------- ----------
Les connexions actives ne sont jamais cassées. Dans le pire des cas, il faudra Les connexions actives ne sont jamais cassées. Dans le pire des cas, il faudra
attendre en plus leur expiration avant l'arrêt total du processus. La valeur attendre en plus leur expiration avant l'arrêt total du processus, ou bien tuer
par défaut est 0 (pas de grâce, arrêt immédiat de l'écoute). manuellement le processus par l'envoi d'un signal SIGTERM. La valeur par défaut
du paramètre 'grace' est 0 (pas de grâce, arrêt immédiat de l'écoute).
Exemple : Exemple :
--------- ---------
@ -537,6 +538,21 @@ Exemple :
mode health mode health
grace 0 grace 0
A partir de la version 1.1.34, un nouveau mécanisme de reconfiguration à chaud
a été introduit. Il est désormais possible de mettre les proxies en "pause" en
envoyant un signal SIGTTOU aux processus. Cela désactivera les sockets d'écoute
sans casser les sessions existantes. Suite à cela, l'envoi d'un signal SIGTTIN
réactivera les sockets d'écoute. Ceci est très pratique pour tenter de charger
une nouvelle configuration ou même une nouvelle version de haproxy sans casser
les connexions existantes. Si le rechargement s'effectue correctement, il ne
reste plus qu'à envoyer un signal SIGUSR1 aux anciens processus, ce qui
provoquera leur arrêt immédiat dès que leurs connexions seront terminées ; en
revanche, si le rechargement échoue, il suffit d'envoyer un signal SIGTTIN pour
remettre les ports en écoute et rétablir le service immédiatement. Veuillez
noter que le paramètre 'grace' est ignoré pour le signal SIGTTOU ainsi que le
signal SIGUSR1 une fois le processus en pause. Aussi, il peut s'avérer très
utile de sauver le fichier de pid avant de démarrer une nouvelle instance.
2.5) Temps d'expiration des connexions 2.5) Temps d'expiration des connexions
-------------------------------------- --------------------------------------