kozorizki/haproxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DOC: LUA: fix some typos and syntax errors

Browse Source 
This fix must be backported to 1.6.

Signed-off-by: Godbach <nylzhaowei@gmail.com>
This commit is contained in:
Godbach 2016-02-16 11:59:17 +08:00 committed by Willy Tarreau
parent 3724da1261
commit 06c8099922
1 changed files with 94 additions and 94 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

188
doc/lua.txt
View File

@ -10,7 +10,7 @@
HAProxy is a powerful load balancer. It embeds many options and many
configuration styles in order to give a solution to many load balancing
problems. However, HAProxy is not universal and some special or specific
problems doesn't have solution with the native software.
problems do not have solution with the native software.
This text is not a full explanation of the Lua syntax.
@ -25,7 +25,7 @@ Why a scripting language in HAProxy
===================================
HAProxy 1.5 makes at possible to do many things using samples, but some people
wants to more combining results of samples fetches, programming conditions and
want to more combining results of samples fetches, programming conditions and
loops which is not possible. Sometimes people implement these functionnalities
in patches which have no meaning outside their network. These people must
maintain these patches, or worse we must integrate them in the HAProxy
@ -44,7 +44,7 @@ between samples and patterns. The samples are extracted using fetch functions
easily extensible, and are used by actions which are also extensible. It seems
natural to allow Lua to give samples, modify them, and to be an action target.
So, Lua uses the same entities as the configuration language. This is the most
natural and reliable way fir the Lua integration. So, the Lua engine allow one
natural and reliable way for the Lua integration. So, the Lua engine allows one
to add new sample fetch functions, new converter functions and new actions.
These new entities can access the existing samples fetches and converters
allowing to extend them without rewriting them.
@ -52,7 +52,7 @@ allowing to extend them without rewriting them.
The writing of the first Lua functions shows that implementing complex concepts
like protocol analysers is easy and can be extended to full services. It appears
that these services are not easy to implement with the HAProxy configuration
model which is base on four steps: fetch, convert, compare and action. HAProxy
model which is based on four steps: fetch, convert, compare and action. HAProxy
is extended with a notion of services which are a formalisation of the existing
services like stats, cli and peers. The service is an autonomous entity with a
behaviour pattern close to that of an external client or server. The Lua engine
@ -63,7 +63,7 @@ This scripting language is useful for testing new features as proof of concept.
Later, if there is general interest, the proof of concept could be integrated
with C language in the HAProxy core.
The HAProxy Lua integration also provides also a simple way for distributing Lua
The HAProxy Lua integration also provides a simple way for distributing Lua
packages. The final user needs only to install the Lua file, load it in HAProxy
and follow the attached documentation.
@ -92,8 +92,8 @@ more about Lua choice
---------------------
Lua language is very simple to extend. It is easy to add new functions written
in C in the core language. It not require to embed very intrusive libraries, and
we do not change compilation processes.
in C in the core language. It is not required to embed very intrusive libraries,
and we do not change compilation processes.
The amount of memory consumed can be controlled, and the issues due to lack of
memory are perfectly caught. The maximum amount of memory allowed for the Lua
@ -172,7 +172,7 @@ Some other functions are prohibited:
- os.execute(), waits for the end of the required execution blocking HAProxy.
- os.exit(), is not really dangerous for the process, but its not the good way
- os.exit(), is not really dangerous for the process, but it's not the good way
for exiting the HAProxy process.
- print(), writes data on stdout. In some cases these writes are blocking, the
@ -188,7 +188,7 @@ there are compatible with the non blocking design. These functions are:
Responsive design
-----------------
HAProxy must process connexions accept, forwarding data and processing timeouts
HAProxy must process connections accept, forwarding data and processing timeouts
as soon as possible. The first thing is to believe that a Lua script with a long
execution time should impact the expected responsive behaviour.
@ -200,12 +200,12 @@ configured with the following "tune" option:
tune.lua.forced-yield <nb>
The default value is 10 000. For determining it, I ran benchmark on my laptop.
I executed a Lua loop between 10 seconds with differents values for the
I executed a Lua loop between 10 seconds with different values for the
"tune.lua.forced-yield" option, and I noted the results:
configured | Number of
instructions | loops executed
between two | in milions
between two | in millions
forced yields |
---------------+---------------
10 | 160
@ -242,7 +242,7 @@ services.
Execution time
--------------
The Lua execution time is measured and limited. Each group of functions have its
The Lua execution time is measured and limited. Each group of functions has its
own timeout configured. The time measured is the real Lua execution time, and
not the difference between the end time and the start time. The groups are:
@ -251,20 +251,20 @@ not the difference between the end time and the start time. The groups are:
- task, by default does not have timeout,
- service have a default timeout of 4s.
The corresponding tune option are:
The corresponding tune options are:
- tune.lua.session-timeout (fetches, converters and action)
- tune.lua.task-timeout (task)
- tune.lua.service-timeout (services)
The tasks does not have a timeout because it runs in background along the
The task does not have a timeout because it runs in background along the
HAProxy process life.
For example, if an Lua script is executed during 1,1s and the script executes a
sleep of 1 second, the effective measured running time is 0,1s.
For example, if an Lua script is executed during 1.1s and the script executes a
sleep of 1 second, the effective measured running time is 0.1s.
This timeout is useful for preventing infinite loops. During the runtime, it
should never triggered.
should be never triggered.
The stack and the coprocess
---------------------------
@ -284,11 +284,11 @@ Some examples follows. This first one, is a simple addition:
lua_pushnumber(L, 2)
lua_arith(L, LUA_OPADD)
Its easy, we push 1 on the stack, after, we push 2, and finally, we perform an
It's easy, we push 1 on the stack, after, we push 2, and finally, we perform an
addition. The two top entries of the stack are added, poped, and the result is
pushed. It is a classic way with a stack.
Now an example for constructing array and objects. Its little bit more
Now an example for constructing array and objects. It's a little bit more
complicated. The difficult consist to keep in mind the state of the stack while
we write the code. The goal is to create the entity described below. Note that
the notation "*1" is a metatable reference. The metatable will be explained
@ -333,7 +333,7 @@ So, coding for Lua in C, is not complex, but it needs some mental gymnastic.
The object concept and the HAProxy format
-----------------------------------------
The objects seems to not be a native concept. An Lua object is a table. We can
The object seems to be not a native concept. An Lua object is a table. We can
note that the table notation accept three forms:
1. mytable["entry"](mytable, "param")
@ -341,7 +341,7 @@ note that the table notation accept three forms:
3. mytable:entry("param")
These three notation have the same behaviour pattern: a function is executed
with the itself table as first parameter and string "param" as second parameter
with the table itself as first parameter and string "param" as second parameter
The notation with [] is commonly used for storing data in a hash table, and the
dotted notation is used for objects. The notation with ":" indicates that the
first parameter is the element at the left of the symbol ":".
@ -376,8 +376,8 @@ the global part of the main Lua stack, and it is called with the case sensitive
class name. A great part of these class must not be used directly because it
requires an initialisation using the HAProxy internal structs.
The HAProxy objects uses unified conventions. An Lua object is always a table.
In most cases, an HAProxy Lua object need some private data. These are always
The HAProxy objects use unified conventions. An Lua object is always a table.
In most cases, an HAProxy Lua object needs some private data. These are always
set in the index [0] of the array. The metatable entry "__tostring" returns the
object name.
@ -405,7 +405,7 @@ specially useful with embedded environments.
When the memory limit is reached, HAProxy refuses to give more memory to the Lua
scripts. The current Lua execution is terminated with an error and HAProxy
continue its processing.
continues its processing.
The max amount of memory is configured with the option:
@ -419,21 +419,21 @@ from C part or Lua part.
Sometimes, objects using lightuserdata or userdata requires to free some memory
block or close filedescriptor not controlled by the Lua. A dedicated garbage
collection function is providedthrought the metatable. It is referenced with the
collection function is provided through the metatable. It is referenced with the
special entry "__gc".
Generally, in HAProxy, the garbage collector does this job without any
intervention. However some object uses a great amount of memory, and we want to
release as quick as possible. The problem is that only the GC knows if the object
is in use or not. The reason is simple variable containing objects can be shared
between coroutines and the main thread, so an object can used everywhere in
HAProxy.
intervention. However some objects use a great amount of memory, and we want to
release as quickly as possible. The problem is that only the GC knows if the
object is in use or not. The reason is simple variable containing objects can be
shared between coroutines and the main thread, so an object can be used
everywhere in HAProxy.
The only one example is the HAProxy sockets. These are explained later, just for
understanding the GC issues, a quick overview of the socket follows. The HAProxy
socket uses an internal session and stream, these sessions uses resources like
socket uses an internal session and stream, the session uses resources like
memory and file descriptor and in some cases keeps a socket open while it is no
loner used by Lua.
longer used by Lua.
If the HAProxy socket is used, we forcing a garbage collector cycle after the
end of each function using HAProxy socket. The reason is simple: if the socket
@ -449,42 +449,42 @@ The yield concept / longjmp issues
The "yield" is an action which does some Lua processing in pause and give back
the hand to the HAProxy core. This action is do when the Lua needs to wait about
data or other things. The most basically example is the sleep() function. In a
data or other things. The most basically example is the sleep() function. In an
event driven software the code must not process blocking systems call, so the
sleep blocks the software between a lot of time. In HAProxy, an Lua sleep does a
yield, and ask to the scheduler to be waked up in a required sleep time.
Meanwhile, the HAProxy scheduler dos other things, like accepting new connection
or forwarding data.
yield, and ask to the scheduler to be woken up in a required sleep time.
Meanwhile, the HAProxy scheduler does other things, like accepting new
connection or forwarding data.
A yield is also executed regularly, after a lot of Lua instruction processed.
A yield is also executed regularly, after a lot of Lua instructions processed.
This yield permits to control the effective execution time, and also give back
the hand to the haproxy core. When HAProxy finish to process the pending jobs,
the Lua execution continue.
the hand to the HAProxy core. When HAProxy finishes to process the pending jobs,
the Lua execution continues.
This special "yield" uses the Lua "debug" functions. Lua provides a debug method
called "lua_sethook()" which permits to interrupt the execution after some
configured condition and call a function. This condition used in HAProxy is
a number of instruction processed and when a function returns. The function
called controls the effective execution time, and if it is possible send a
a number of instructions processed and when a function returns. The function
called controls the effective execution time, and if it is possible to send a
"yield".
The yield system is based on a couple setjmp/longjmp. In brief, the setjmp()
stores a stack state, and the longjmp restores the stack in its state which had
before the last Lua execution.
Lua can immediately stop is execution if an error occurs. This system uses also
Lua can immediately stop its execution if an error occurs. This system uses also
the longjmp system. In HAProxy, we try to use this sytem only for unrecoverable
errors. Maybe some trivial errors targets an exception, but we try to remove it.
errors. Maybe some trivial errors target an exception, but we try to remove it.
It seems that Lua uses the longjmp system for having a behaviour like the java
try / catch. We can use the function pcall() to executes some code. The function
try / catch. We can use the function pcall() to execute some code. The function
pcall() run a setjmp(). So, if any error occurs while the Lua code execution,
the flow immediately return from the pcall() with an error.
the flow immediately returns from the pcall() with an error.
The big issue of this behaviour is that we cannot do a yield. So if some Lua code
executes a library using pcall for catching errors, HAProxy must be wait for the
end of execution without processing any accept or any stream. The cause is the
yield must be jump to the root of execution. The intermediate setjmp() avoid
yield must be jump to the root of execution. The intermediate setjmp() avoids
this behaviour.
@ -502,7 +502,7 @@ Another issue with the processing of strong errors is the manipulation of the
Lua stack outside of an Lua processing. If one of the functions called occurs a
strong error, the default behaviour is an abort(). It is not acceptable when
HAProxy is in runtime mode. The Lua documentation propose to use another
setjmp/longjmp to avoid the abort(). The goal is to puts a setjmp between
setjmp/longjmp to avoid the abort(). The goal is to put a setjmp between
manipulating the Lua stack and using an alternative "panic" function which jumps
to the setjmp() in error case.
@ -510,10 +510,10 @@ All of these behaviours are very dangerous for the stability, and the internal
HAProxy code must be modified with many precautions.
For preserving a good behaviour of HAProxy, the yield is mandatory.
Unfortunately, some HAProxy part are not adapted for resuming an execution after
a yield. These part are the sample fetches and the sample converters. So, the
Lua code written in these parts of HAProxy must be quickly executed, and can not
do actions which require yield like TCP connection or simple sleep.
Unfortunately, some HAProxy parts are not adapted for resuming an execution
after a yield. These parts are the sample fetches and the sample converters. So,
the Lua code written in these parts of HAProxy must be quickly executed, and can
not do actions which require yield like TCP connection or simple sleep.
HAproxy socket object
---------------------
@ -523,7 +523,7 @@ server, and processing the many errors which can occurs during these exchanges.
HAProxy is not designed for having a third connection established to a third
party server.
The solution consist to puts the main stream in pause waiting for the end of the
The solution consist to put the main stream in pause waiting for the end of the
exchanges with the third connection. This is completed by a signal between
internal tasks. The following graph shows the HAProxy Lua socket:
@ -557,12 +557,12 @@ A more detailed graph is available in the "doc/internals" directory.
The HAProxy Lua socket uses a full HAProxy session / stream for establishing the
connection. This mechanism provides all the facilities and HAProxy features,
like the SSL stack, many socket type, and support for namespaces.
Technically it support the proxy protocol, but there are no way to enable it.
Technically it supports the proxy protocol, but there are no way to enable it.
How compiling HAProxy with Lua
==============================
HAProxy 1.6 requires Lua 5.3. Lua 5.3 offers some features which makes easy the
HAProxy 1.6 requires Lua 5.3. Lua 5.3 offers some features which make easy the
integration. Lua 5.3 is young, and some distros do not distribute it. Luckily,
Lua is a great product because it does not require exotic dependencies, and its
build process is really easy.
@ -582,7 +582,7 @@ The compilation process for linux is easy:
make linux
- install it:
sudo make INSTALL_TOP=/opt/lua-5.3.1
sudo make INSTALL_TOP=/opt/lua-5.3.1 install
HAProxy builds with your favourite options, plus the following options for
embedding the Lua script language:
@ -609,16 +609,16 @@ embedding the Lua script language:
First steps with Lua
====================
Now, its time to using Lua in HAProxy.
Now, it's time to use Lua in HAProxy.
Start point
-----------
The HAProxy global directive "lua-load <file>" allow to load an lua file. This
The HAProxy global directive "lua-load <file>" allows to load an Lua file. This
is the entry point. This load become during the configuration parsing, and the
Lua file is immediately executed.
All the register_*() function must be called at this time because there are used
All the register_*() functions must be called at this time because they are used
just after the processing of the global section, in the frontend/backend/listen
sections.
@ -626,7 +626,7 @@ The most simple "Hello world !" is the following line a loaded Lua file:
core.Alert("Hello World !");
It display a log during the HAProxy startup:
It displays a log during the HAProxy startup:
[alert] 285/083533 (14465) : Hello World !
@ -634,8 +634,8 @@ Default path and libraries
--------------------------
Lua can embed some libraries. These libraries can be included from different
paths. It seems that Lua doesn't like subdirectories. In the following example, I
try to load a compiled library, so the first line is Lua code, the second line
paths. It seems that Lua doesn't like subdirectories. In the following example,
I try to load a compiled library, so the first line is Lua code, the second line
is an 'strace' extract proving that the library was opened. The next lines are
the associated error.
@ -657,7 +657,7 @@ The variable "<libname>" is defined using the content of the variable
/usr/local/lib/lua/5.3/?.so;/usr/local/lib/lua/5.3/loadall.so;./?.so
The "<libname>" is the content which replaces the symbol "<?>". In th previous
The "<libname>" is the content which replaces the symbol "<?>". In the previous
example, its "luac/concat", and obviously the Lua core try to load the function
associated with the symbol "luaopen_luac/concat".
@ -677,7 +677,7 @@ First useful example
return txn.sc:sdbm(salt .. txn.sf:req_fhdr("host") .. txn.sf:path() .. txn.sf:src(), 1)
end)
You will see that these 3 line can generate a lot of explanations :)
You will see that these 3 lines can generate a lot of explanations :)
Core.register_fetches() is executed during the processing of the global section
by the HAProxy configuration parser. A new sample fetch is declared with name
@ -686,16 +686,16 @@ sample fetch will be used calling "lua.my-hash" in the HAProxy configuration
file.
The second parameter is an inline declared anonymous function. Note the closed
parenthesis after the keyword "end" which end the function. The first parameter
of these anonymous function is "txn". It an object of class TXN. It provides
parenthesis after the keyword "end" which ends the function. The first parameter
of this anonymous function is "txn". It is an object of class TXN. It provides
access functions. The second parameter is an arbitrary value provided by the
HAProxy configuration file. This parameter is optional, the developer must
check if its present.
check if it is present.
The anonymous function registration is executed when the HAProxy backend or
frontend configuration references the sample fetch "lua.my-hash".
This example can writed with an other style, like below:
This example can be written with another style, like below:
function my_hash(txn, salt)
return txn.sc:sdbm(salt .. txn.sf:req_fhdr("host") .. txn.sf:path() .. txn.sf:src(), 1)
@ -705,7 +705,7 @@ This example can writed with an other style, like below:
This second form is clearer, but the first one is compact.
The operator ".." is a string concatenation. If one of the two operands are not a
The operator ".." is a string concatenation. If one of the two operands is not a
string, an error occurs and the execution is immediately stopped. This is
important to keep in mind for the following things.
@ -732,22 +732,22 @@ understand that the function "my_hash" will be called for each HAProxy request
using the declared sample fetch. So, this function can be executed many times in
parallel.
By default, Lua uses global variables. so in this example, il the variable "str"
By default, Lua uses global variables. So in this example, if the variable "str"
is declared without the keyword "local", it will be shared by all the parallel
executions of the function and obviously, the content of the requests will be
shared.
This warning is very important. I tried to write useful Lua code like a rewrite
of the statistics page, and its very hard to thing to declare each variable as
of the statistics page, and it is very hard thing to declare each variable as
"local".
I guess than this behaviour will be the cause of many trouble on the mailing
I guess that this behaviour will be the cause of many troubles on the mailing
list.
str = str ..
~~~~~~~~~~~~
Now a parenthesis about the form "str = str ..". This form allow to do string
Now a parenthesis about the form "str = str ..". This form allows to do string
concatenations. Remember that Lua uses a garbage collector, so what happens when
we do "str = str .. 'another string'" ?
@ -755,10 +755,10 @@ we do "str = str .. 'another string'" ?
^ ^ ^ ^
1 2 3 4
Lua execute first the concatenation operator (3), it allocates memory for the
Lua executes first the concatenation operator (3), it allocates memory for the
resulting string and fill this memory with the concatenation of the operands 2
and 4. Next, it free the variable 1, now the old content of 1 can be garbage
collected. and finally, the new content of 1 is the concatenation.
and 4. Next, it frees the variable 1, now the old content of 1 can be garbage
collected. And finally, the new content of 1 is the concatenation.
what the matter ? when we do this operation many times, we consume a lot of
memory, and the string data is duplicated and move many times. So, this practice
@ -779,41 +779,41 @@ sample fetches and converters. The object txn contains 2 members dedicated to
the sample fetches and 2 members dedicated to the converters.
The sample fetches members are "f" (as sample-Fetch) and "sf" (as String
sample-Fetch). These two members contains exactly the same functions. All the
sample-Fetch). These two members contain exactly the same functions. All the
HAProxy native sample fetches are available, obviously, the Lua registered sample
fetches are not available. Unfortunately, HAProxy sample fetches names are not
compatible with the Lua function names, and they are renames. The rename
compatible with the Lua function names, and they are renamed. The rename
convention is simple, we replace all the '.', '+' and '-' by '_'. The '.' is the
object member separator, and the "-" and "+" is math operator.
Now, that I'm writing this article, I known the Lua better than I wrote the
Now, that I'm writing this article, I know the Lua better than I wrote the
sample-fetches wrapper. The original HAProxy sample-fetches name should be used
using alternative manner to call an object member, so the sample-fetch
"req.fhdr" (actually renamed req_fhdr") is should be used like this:
"req.fhdr" (actually renamed req_fhdr") should be used like this:
txn.f["req.fhdr"](txn.f, ...)
However, I think that this form is not elegant.
The "s" collection return a data with a type near to the original returned type.
A string return an Lua string, an integer returns an Lua integer and an IP
A string returns an Lua string, an integer returns an Lua integer and an IP
address returns an Lua string. Sometime the data is not or not yet available, in
this case it returns the Lua nil value.
The "sf" collection guarantee that a string will be always returned. If the data
The "sf" collection guarantees that a string will be always returned. If the data
is not available, an empty string is returned. The main usage of these collection
is to concatenate the returned sample-fetches without testing each function.
The parameters of the sample-fetches are according with the haproxy
The parameters of the sample-fetches are according with the HAProxy
documentation.
The converters runs exactly with the same manner as the sample fetches. The
only one difference is that the fist parameter is the converter entry element.
The converters run exactly with the same manner as the sample fetches. The
only one difference is that the first parameter is the converter entry element.
The "c" collection returns a precise result, and the "sc" collection returns
always a string.
The sample-fetches used in the example function are "txn.sf:req_fhdr()",
"txn.sf:path()" and "txn.sf:src()". The converter are "txn.sc:sdbm()". The same
"txn.sf:path()" and "txn.sf:src()". The converter is "txn.sc:sdbm()". The same
function with the "s" collection of sample-fetches and the "c" collection of
converter should be written like this:
@ -832,7 +832,7 @@ converter should be written like this:
tostring
~~~~~~~~
The function tostring ensure that its parameter is returned as a string. If the
The function tostring ensures that its parameter is returned as a string. If the
parameter is a table or a thread or anything that will not have any sense as a
string, a form like the typename followed by a pointer is returned. For example:
@ -845,7 +845,7 @@ returns:
For objects, if the special function __tostring() is registered in the attached
metatable, it will be called with the table itself as first argument. The
HAProxy objects returns its own type.
HAProxy object returns its own type.
About the converters entry point
--------------------------------
@ -854,12 +854,12 @@ In HAProxy, a converter is a stateless function that takes a data as entry and
returns a transformation of this data as output. In Lua it is exactly the same
behaviour.
So, the registered Lua function doesn't have any special parameters, juste a
So, the registered Lua function doesn't have any special parameters, just a
variable as input which contains the value to convert, and it must return data.
The data required as input by the Lua converter is a string. So HAProxy will
always provide a string as input. If the native sample fetch is not a string it
will ne converted in best effort.
will be converted in best effort.
The returned value will have anything type, it will be converted as sample of
the near HAProxy type. The conversion rules from Lua variables to HAProxy
@ -885,7 +885,7 @@ The task entry point
The function "core.register_task(fcn)" executes once the function "fcn" when the
scheduler starts. This way is used for executing background task. For example,
you can use this functionnality for periodically checking the health of an other
you can use this functionnality for periodically checking the health of another
service, and giving the result to each proxy needing it.
The task is started once, if you want periodic actions, you can use the
@ -895,7 +895,7 @@ Storing Lua variable between function in the same session
---------------------------------------------------------
All the functions registered as action or sample fetch can share an Lua context.
This context is a memory zone in the stack. sample fetch and action uses the
This context is a memory zone in the stack. sample fetch and action use the
same stack, so both can access to the context.
The context is accessible via the function get_priv and set_priv provided by an
@ -922,7 +922,7 @@ HTTP actions
Lua is fast, but my service require more execution speed
========================================================
We can wrote C modules for Lua. These modules must run with HAProxy while they
We can write C modules for Lua. These modules must run with HAProxy while they
are compliant with the HAProxy Lua version. A simple example is the "concat"
module.
@ -949,7 +949,7 @@ The build
---------
The compilation of the source file requires the Lua "include" directory. The
compilation and the link of the object file requires the -fPIC option. Thats
compilation and the link of the object file requires the -fPIC option. That's
all.
cc -I/opt/lua/include -fPIC -shared -o mymod.so mymod.c
@ -961,7 +961,7 @@ You can load this module with the following Lua syntax:
require("mymod")
When you start HAProxy, this module just print "Hello world" when its loaded.
When you start HAProxy, this module just print "Hello world" when it is loaded.
Please, remember that HAProxy doesn't allow blocking method, so if you write a
function doing filesystem access or synchronous network access, all the HAProxy
process will fail.