IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
When a node gets banned, it should go into recovery and freeze all
databases. We rely on the recovery daemon to detect the banned state
and put the node in recovery and freeze all databases.
Recent change in b4357a79d9 took explicit
freezing out of banning code but left the setting of recovery mode
to ACTIVE. Recovery daemon will freeze databases only if the recovery
mode is NORMAL. Recovery mode set to ACTIVE is an indication that the
freeze has started.
Do not set the recovery mode to ACTIVE in banning. Let recovery daemon
take care of it.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 30 10:32:38 CET 2015 on sn-devel-104
This makes sure that all function prototypes in ctdb_client.h are in
the same order of functions defined in ctdb_client.c.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 30 05:04:49 CET 2015 on sn-devel-104
This groups function prototypes for common client/server functions in
common/common.h and removes them from ctdb_private.h.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
When building standalone ctdb from git repo, samba_version_file correctly
includes git sha in VERSION string. When building standalone ctdb from
tarball, samba_version_file puts UNKNOWN in the VERSION string.
Use the packaged include/ctdb_version.h file to set the correct git sha.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Instead of includes.h, include the required header files explicitly.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This groups function prototypes for system specific functions in
common/system.h and removes them from ctdb_private.h.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Wed Oct 28 15:59:02 CET 2015 on sn-devel-104
This was part of libctdb which has been removed.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Since commit 84f5528d9b, CTDB will not
remove an existing socket if it can connect to the existing one.
Instead it will fail to start.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Oct 28 09:44:37 CET 2015 on sn-devel-104
If 2 attempts are made to start CTDB in quick succession then it is
possible for the 2nd attempt to remove a newly created PID file from
the 1st.
If the PID file existed then the PID/SID from ctdbd_is_running() will
be passed to kill_ctdbd(). If the PID file did not exist then there
is no point removing it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
While 0000 is possible from the UNIX/POSIX point of view,
these permissions create problems in an environment with
selinux enabled, which is more strict.
This aligns the perms of the read only tracking db with other
internal dbs.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11577
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Oct 28 06:13:09 CET 2015 on sn-devel-104
Currently only Samba is built with CTDB and since CTDB headers are not
public any more, no other application can build with CTDB. So define
CTDB_SOCKET only during build.
In future CTDB will have public API and proper way of figuring out
CTDB_SOCKET.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The databases are repacked automatically during vacuuming when the
freelist size grows beyond configured threshold (RepackLimit).
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
ctdb_protocol.h was modified to update default capabilities without
corresponding changes in protocol.h.
(Patch dfc84fdd45)
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This is deprecated. For many commands it doesn't make sense. Instead
of "ctdb ip -n all" use "ctdb ip all".
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Add new function delete_ip_from_all_nodes(). Also
ctdb_get_all_pnns().
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
None of these uses of -n are particularly useful. "-n all" will be
deprecated.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Centrally define all the default capabilities to make the defaults
crystal clear. Capability-related command-line options now have a
direct correspondence rather than a reverse correspondence.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
Once the databases are recovered, all the pending calls are resent.
If the vnnmap is not updated, then the nodes can redirect calls to nodes
that are not part of the new vnnmap.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 16 09:31:34 CEST 2015 on sn-devel-104
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Oct 14 05:22:28 CEST 2015 on sn-devel-104
Recovery daemon checks if it is the recovery master before performing
certain checks. During those checks it's possible that re-election can
change the recmaster. In such a case, the recovery daemon should never
do a database recovery.
This is not complete fix since the recovery master can still change
while the recovery is going on. The correct fix is to abort recovery
if the recovery master changes.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Oct 7 17:55:05 CEST 2015 on sn-devel-104
The reason for favouring more connected node is to create a larger
cluster in case of a split brain. In split brain condition, the nodes
are not communicating across partitions and each partition will run its
own election. Among all the partitions, the node which holds the recovery
lock will eventually "win". All the other nodes which won election but
could not grab recovery lock will end up banning themselves.
This also prevents the recovery master role from bouncing between nodes
during startup when the entire cluster is restarted.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
If election occurs during SMB activity, then trying to freeze all the
databases can cause samba/ctdb deadlock which parallel database recovery
is trying to avoid.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Parallel database recovery fixes the samba/ctdb deadlock during recovery.
Many times samba tries to grab multiple record locks in sequence.
Consider a case when samba is already holding a record lock on a database
and tries to get a record lock on second database. If the second record
is not available on the local node, samba asks ctdb to migrate the record.
If recovery occurs at this time (e.g. node becoming inactive), ctdb
cannot freeze all the databases since samba is already holding a lock
and waiting for the second lock. CTDB can process the second record
request only after the recovery is complete, thus causing a deadlock.
In parallel database recovery, each database is frozen and recovered
independent from each other. So as soon as the second database is
recovered, CTDB will resend all the pending migration requests and Samba
can get the second lock. Once samba releases both the locks, ctdb can
freeze the first database and recover it completing recovery process.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Attempt to make common code independent and free from ctdb_context.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The functions defined in common/system_*.c depend on the functions
defined in common/system_common.c. So keep them together.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
These headers are used by the server code and should not be marked public.
Samba builds against the in-tree version of the headers and should not
be built with externally installed CTDB.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
In the parallel database recovery model, all the database will not remain
frozen at the same time. So relax the condition to check if recovery
is active.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Before setting recovery mode to normal, confirm that all the databases are
recovered by matching the database generation with the global generation.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
CTDB verifies the generation in the packet header matches that of the
current generation. However, that check now needs to be done where
database context is available. So add in the check in handlers for
database requests (CTDB_REQ_CALL, CTDB_REQ_DMASTER, CTDB_REPLY_DMASTER
and CTDB_REPLY_CALL).
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The database generation for each database is updated only during recovery.
After recovery is complete the database generation would be the same as
the global generation.
The database generation is required for parallel database recovery.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
It's possible that the databases can get attached after initial freeze.
This typically happens during startup as CTDB will only attach persistent
databases and go in to startup freeze. During recovery, the recovery
master will attach all the missing databases and then send freeze
controls.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Since the banning code does not call this function anymore, it can be
made static.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Once the node is marked as banned, the recovery daemon on that node will
start freezing the databases anyway. This check happens every second
in recovery daemon, so the delay in freezing the databases on ban would
be a second.
The main reason for freezing databases is to prevent any database access
from samba clients. However, banning code also drops the public IPs
and thus disconnecting the clients. In addition, the generation gets
set to INVALID_GENERATION, thus database record migration requests on
that node will not be processed.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This cascades the global transaction controls to individual database
transaction operations. This ensures that the individual database state
is correctly set when processing global transaction controls.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The same structure is required in new controls for database transactions.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Instead of marking all the databases with priority, mark only the database
which is currently being processed.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This changes the locking behaviour when the databases are frozen.
Instead of a single lock helper locking all databases with a priority,
a lock helper is launched for every database.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
These variables are used for state information related to freezing
databases. Instead use the API functions to check if the databases
are frozen.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Use ctdb->freeze_mode only in ctdb_freeze.c and use the functions to
check if databases are frozen everywhere else.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The pending calls are migration requests received from clients (over unix
domain socket) which are under processing. After a recovery is finished,
any requests which are under processing will be dropped since they do
not belong to the current generation. All the pending call requests
are resent with new generation to restart record migrations.
This is in preparation for parallel database recovery.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This iterator only iterates through databases of specified priority.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
These changes enable the default installation prefix settings to
take effect in both Waf 1.5 and 1.8 with no additional code changes.
Signed-off-by: Thomas Nagy <tnagy@waf.io>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Should be an IP address not a filename.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Sep 19 16:39:23 CEST 2015 on sn-devel-104
At the moment they are not installed at all if xsltproc is not
available.
They can still be filtered to have the paths match what was
configured.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Sep 15 11:57:44 CEST 2015 on sn-devel-104
These still say /tmp/ctdb.socket, which is very out-dated. Update
with build-time location.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
In this case: ctdbd_wrapper, onnode, ctdb_diagnostics, ctdb.sudoers.
Set sensible defaults from configure options.
Update documentation to match, trying to fix up anything that has been
missed before.
The onnode unit tests need a symlink to the functions file.
The simple integration tests need to set CTDB_BASE and also
need symlinks to functions/nodes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
fixup
Signed-off-by: Martin Schwenke <martin@meltin.net>
This hasn't existed for a long time.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
This variable points to /etc/ by default.
This distinguishes it from the different variable from wscript, which
points to /etc/ctdb/.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Also fix an unused test to set CTDB_BASE.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Clearly identifies where all state files from scripts should go and
allows that location to be easily changed. This patch should not
change any behaviour (outside of eventscript unit tests, where a
clearer location is now used).
CTDB_VARDIR should no longer be overridden. Continue to set
CTDB_DBDIR and similar to override database location in unit tests.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Have wscript do path substitution.
No need to export this and CTDB_ETCDIR here, but test scripts will
still need to do so.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Format instructions for adding/deleting nodes with appropriate XML and
simplify wording.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Using $CTDB_VARDIR at file scope is dangerous because it doesn't
respect the configuration. Uses of these variables are simple so just
drop the variables and use $CTDB_VARDIR inside functions where it is
safe.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
This directory has been under $CTDB_VARDIR for a long time, so is
already removed. The command is also potentially dangerous if
$ctdb_managed_dir is not set.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
They're not used outside of ctdb_standard_event_helper().
As a consequence, make ctdb_standard_event_helper() do nothing. It is
harder to remove because it is used in many places, perhaps by
external eventscripts where it has been copied from existing scripts.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
To get a similar effect just do something like this:
mmaddcallback ctdb-disable-on-quorumLoss \
--command /usr/bin/ctdb \
--event quorumLoss --parms "disable"
mmaddcallback ctdb-enable-on-quorumReached \
--command /usr/bin/ctdb \
--event quorumReached --parms "enable"
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
60.nfs sets this to a directory where the NFS callout can store state.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Sep 14 08:36:34 CEST 2015 on sn-devel-104
sockets are created in a loop until an unused address is found.
But the unused socket fds were not closed.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Just use ctdb_tcp_connection. It is the same. There are no external
users.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Thu Sep 3 22:12:02 CEST 2015 on sn-devel-104
Always check filesystem usage for the database directories.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Sat Aug 29 20:08:48 CEST 2015 on sn-devel-104
CTDB should warn by default if too much system memory or swap is used.
The tests have also been tweaked. In particular, the filesystem-only
tests need to initialise the memory information to avoid errors where
meminfo isn't set.
Document the defaults, warning against disabling them.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
They are only printed when the percentage usage changes. This should
stop the logs from being filled with warnings.
Add a test for the throttling.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Marking the node unhealthy should cause Samba processes to close,
possible freeing a stack of memory. If not, then it is somebody
else's problem.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
New variables CTDB_MONITOR_MEMORY_USAGE and CTDB_MONITOR_SWAP_USAGE.
Both take a pair of <warn_threshold>:<unhealthy_threshold> where each
theshold is specified as a percentage.
This adds a callout to check_thresholds() that is run when the
unhealthy threshold is reached.
Add some combination tests.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
CTDB_MONITOR_FREE_MEMORY and CTDB_MONITOR_FREE_MEMORY_WARN are now
percentages that specify thresholds of acceptable memory usage.
Memory/swap usage in tests also specified as percentages.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
No need to use 2 different sources of information for similar checks.
Also, output of free has been changed, whereas /proc/meminfo is a
kernel API, which will not change.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This allows both errors (i.e. unhealthy) and warnings for different
thresholds. It replaces CTDB_CHECK_FS_USE.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Drop obvious comments. Use die() for less lines of code. Use a case
statement to avoid forking unnecessary processes for each filesystem
being checked. Drop parentheses around percentages in messages.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Will put all the system monitoring in here, simplifying 00.ctdb.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
To support this, extend the "ip route add" stub to detect duplicate
routes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Aug 7 08:37:38 CEST 2015 on sn-devel-104
When the ctdb statistics is reset, reset per database statistics to keep
it consistent with ctdb statistics.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Some code involved in releasing IPs is not re-entrant. Memory
corruption can occur if, for example, overlapping attempts are made to
ban a node. We haven't been able to recreate the corruption but this
should protect against it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This calls release_all_ips() only once on the first ban. If the node gets
banned again due to event script timeout while running release_all_ips(),
then avoid calling release_all_ips() in re-entrant fashion.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
If a control fails and error message is set, the returned status of the
control is always set to -1 ignoring the status passed by the daemon.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The timed out error is ignored for certain events (start_recovery,
recoverd, takeip, releaseip). If these events time out, then the debug
hung script outputs the following:
3 scripts were executed last releaseip cycle
00.ctdb Status:OK Duration:4.381 Thu Jul 16 23:45:24 2015
01.reclock Status:OK Duration:13.422 Thu Jul 16 23:45:28 2015
10.external Status:DISABLED
10.interface Status:OK Duration:-1437083142.208 Thu Jul 16 23:45:42 2015
The endtime for timed out scripts is not set. Since the status is not
returned as -ETIME for some events, ctdb scriptstatus prints -ve duration.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
No use dying or failing eventscripts if someone sends a random
SIGUSR1.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Jul 21 11:00:17 CEST 2015 on sn-devel-104
We don't expect to maintain an up-to-date copy. NFS Ganesha team
might provide patches.
Also move the Ganesha .check file
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This adds new configuration variable CTDB_RPCINFO_LOCALHOST6.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Due to the missing cast of uint64_t, CONTROL_GET_DB_SEQNUM always returned
seqnum <= 256.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11398
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Jul 14 13:03:25 CEST 2015 on sn-devel-104
This is an optimisation to avoid forking the callout for operations
that are not implemented.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Unhealthy after 1 failed attempt to contact the portmapper.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Now that there is only a single NFS eventscript, other eventscripts no
longer need to load all of this.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is now handled by passing the desired number of threads to the
command specified in the dump_stuck_threads variable in .check files.
Remove unused function nfs_dump_some_threads().
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This isn't a straightforward move of code from 60.ganesha to the
callout. Simplifications have been made to allow better
interoperation with the new NFS checking logic.
The following configuration variables have been removed:
CTDB_GANESHA_REC_SUBDIR
Edit NFS ganesha callout to change this location
CTDB_NFS_SERVER_MODE, NFS_SERVER_MODE
Use CTDB_NFS_CALLOUT instead
CTDB_NFS_SKIP_KNFSD_ALIVE_CHECK, CTDB_SKIP_GANESHA_NFSD_CHECK
Disable the corresponding .check file instead
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
$service_check_cmd specifies a command to run instead of the regular
rpcinfo-based check.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is now always enabled. If nfsd thread monitoring is not required
then make CTDB_NFS_CALLOUT point to a wrapper around
nfs-linux-kernel-callout that does not implement "monitor-post".
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Note that the 60.ganesha RPC checks need to be identical to those in
the nfs-checks.d/ directory. This is because the NFS unit test
infrastructure checks output against what should be produced by the
checks in nfs-checks.d/. This is a minor issue, since one of the aims
of this work is to remove the need for a separate 60.ganesha.
In most cases configuration variable CTDB_NFS_DUMP_STUCK_THREADS is
now ignored. This is now handled by passing the desired number of
threads to the command specified in the service_debug_cmd variable in
a .check file.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Provides a new extensible format for .check files, using simple
variables instead of the unwieldy extended test(1) syntax now used.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Change status, nlockmgr, mountd, rquotad to be unhealthy after 6
rpcinfo check failures and do a verbose restart after every 2
failures. Change 60.ganesha for consistency, since 60.ganesha tests
are broken and depend on the consistency.
Apart from the consistency aspect, the check infrastructure will soon
be simplified so that it only allows the equivalent of "unhealthy" and
"verbose restart:b" actions.
Update tests to have a corresponding numbers of iterations. Run 1
extra iteration in most tests to check there are no unexpected
behaviour changes after the designated number of iterations completes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
They are contrived and hard to read. Better to just enumerate the few
sub-tests in these testcases.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
A useful baseline test to ensure that certain things (e.g. rpcinfo)
aren't consistently broken.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This means that required result will not be calculated on each
iteration. This is useful in baseline tests where, say, all
iterations should succeed and produce no output. This is useful for
confirming that the eventscript and test infrastructure is working
correctly.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Much clearer than using iterate_test() for this purpose. This also
does failover counting by calling rpcinfo in each iteration.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Before commit cbffbb7c2f this was
possible and some users depend on this behaviour.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Warn specifically about cancellation instead of printing a generic
error message. Also pass back an error message for the tool - it
could just rely on the status but it already looks at the error
message.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The eventscript state should never be freed externally, so it should
never be allocated off a temporary context. It will either be freed
by the handler or in the cancellation code.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Pair-programmed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Martin Schwenke <martin@meltin.net>
It needs to have a default for the standalone case, when it is not run
in a loop inside "ip addr show".
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
VLAN interfaces on bonds with a name other than <iface>.<id>@<iface>
are not currently supported. That is, where the VLAN name isn't based
on the underlying bond name. Such VLAN interfaces can be created with
the "ip link" command, as opposed to the "vconfig" command, or by
renaming a VLAN interface.
This is improved by determining the underlying interface name for a
VLAN from the output of "ip link".
No serious attempt is made to support VLANs with '@' in their name,
although this seems to be legal. Why would you do that?
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Commit 6471541d6d broke support for VLAN
interfaces. Releasing a public IP address depends on
ip_maskbits_iface() and for a VLAN interface this will return an
interface of the form <vlan>@<iface>, which can't be fed back into
"ip" commands.
Update ip_maskbits_iface() to drop the '@' and everything after it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Reported-by: Jan Schwaratzki <jschwaratzki@ddn.com>
This has not been used for a long time.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Jul 10 23:41:18 CEST 2015 on sn-devel-104
==20761== Invalid read of size 8
==20761== at 0x11BE30: ctdb_ctrl_dbstatistics (ctdb_client.c:1286)
==20761== by 0x12BA89: control_dbstatistics (ctdb.c:713)
==20761== by 0x1312E0: main (ctdb.c:6543)
==20761== Address 0x713b0d0 is 0 bytes after a block of size 560 alloc'd
==20761== at 0x4C27A2E: malloc (vg_replace_malloc.c:270)
==20761== by 0x5CB0954: _talloc_memdup (talloc.c:615)
==20761== by 0x11395C: ctdb_control_recv (ctdb_client.c:1146)
==20761== by 0x11BDD7: ctdb_ctrl_dbstatistics (ctdb_client.c:1265)
==20761== by 0x12BA89: control_dbstatistics (ctdb.c:713)
==20761== by 0x1312E0: main (ctdb.c:6543)
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
60.nfs backgrounds it so it persists in the background causing
problems. In particular, it causes the "ctdb ip" command stub to be
run in parallel, which produces inconstent results.
Better not to run it at all in the NFS tests.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Jul 9 09:27:02 CEST 2015 on sn-devel-104
If statd-callout is unwanted, so is removed, then this code fails.
Change to an "if" so that it succeeds as intended.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
To do any cleanup before exiting the test, register hooks with
test_cleanup(). Multiple hooks can be registered. All the hooks will
be called before exiting from the test.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This code was copied from onnode unit tests, but not used.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This is now achieved by defining functions extra_header() and
extra_footer().
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This follows the same pattern as the tstore command, and it allows
specifying key strings with a trailing \0 character.
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Jul 6 23:23:22 CEST 2015 on sn-devel-104
Remove the private libraries from builtin_libraries and package them
instead.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Jul 1 07:19:43 CEST 2015 on sn-devel-104
To build CTDB rpms with PCP PMDA, use
make rpm RPM_OPTIONS="--with pmda"
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Pair-Programmed-With: Martin Schwenke <martin@meltin.net>
Reviewed-by: Martin Schwenke <martin@meltin.net>
On IPv4-only or IPv6-only systems one of these files will not exist.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
It is important to know when ctdbd is started with --start-as-stopped
or --start-as-disabled. Given that this only happens once it makes
sense to promote these debug items to NOTICE level.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If a node was previously recovery master (say, 20 years ago) and it
becomes recovery master again then, if IP assignments have changed,
verify_remote_ip_allocation() can produce messages like the following
when called during recovery:
ctdbd: recoverd:Inconsistent IP allocation - node 0 thinks 10.1.1.1 is held by node 0 while it is assigned to node 1
When a node loses an election it should clear all data specific to it
being the recovery master.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This needs to be cleared to avoid stale data when a new recovery
master is elected.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will handle the most obvious cases. It won't handle the case
where the directory is missing and the recovery lock location is
updated at run-time. However, this is a good improvement.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This works around cases where ctdb_transaction gets stuck - this still
needs to be debugged. However, this change will at least cause
individual tests to fail rather than having whole test runs time out.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Michael Adam <obnox at samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Jun 27 13:20:18 CEST 2015 on sn-devel-104
This was used to unlink the socket at daemon exit, which
was removed in ctdb commit b18356764cd49d934eab901e596bb75c6e3ecdf8
(Samba master commit 4259156050).
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Jun 25 18:29:59 CEST 2015 on sn-devel-104
This reverts commit 257311e337.
That commit was due to a misunderstanding, and it
does not fix what it was supposed to fix.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The caller should not dereference lock_ctx after invoking
process_callbacks(), it might be destroyed already.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11293
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Jun 12 15:28:57 CEST 2015 on sn-devel-104
We should not dereference lock_ctx after invoking the callback
in the auto_mark == false case. The callback could have destroyed it.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11293
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Let ctdb_lock_request_destructor() take care of the proper cleanup.
If the request if freed from the callback function, then the lock context
should not be freed. Setting request->lctx to NULL takes care of that
in the destructor.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11293
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
There is already code in the destructor to correctly remove it from the
pending or the active queue. This also ensures that when lock context
is in pending queue and if the lock request gets freed, the lock context
is correctly removed from the pending queue.
Thanks to Stefan Metzmacher for noticing this and suggesting the fix.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11293
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
The code was added to ctdb_lock_context_destructor() to ensure that
the if a lock_ctx gets freed first, the lock_request does not have a
dangling pointer. However, the reverse is also true. When a lock_request
is freed, then lock_ctx should not have a dangling pointer.
In commit 374cbc7b0f, the code for second
condition was dropped causing a regression.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11293
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If the lock request is freed from within the callback, then setting
lock_ctx->request to NULL in ctdb_lock_context_destructor will end up
corrupting memory. In this case, lock_ctx->request could be reallocated
and pointing to something else. This may cause unexpected abort trying
to dereference a NULL pointer.
So, set lock_ctx->request to NULL before processing callbacks.
This avoids the following valgrind problem.
==3636== Invalid write of size 8
==3636== at 0x151F3D: ctdb_lock_context_destructor (ctdb_lock.c:276)
==3636== by 0x58B3618: _talloc_free_internal (talloc.c:993)
==3636== by 0x58AD692: _talloc_free_children_internal (talloc.c:1472)
==3636== by 0x58AD692: _talloc_free_internal (talloc.c:1019)
==3636== by 0x58AD692: _talloc_free (talloc.c:1594)
==3636== by 0x15292E: ctdb_lock_handler (ctdb_lock.c:471)
==3636== by 0x56A535A: epoll_event_loop (tevent_epoll.c:728)
==3636== by 0x56A535A: epoll_event_loop_once (tevent_epoll.c:926)
==3636== by 0x56A3826: std_event_loop_once (tevent_standard.c:114)
==3636== by 0x569FFFC: _tevent_loop_once (tevent.c:533)
==3636== by 0x56A019A: tevent_common_loop_wait (tevent.c:637)
==3636== by 0x56A37C6: std_event_loop_wait (tevent_standard.c:140)
==3636== by 0x11E03A: ctdb_start_daemon (ctdb_daemon.c:1320)
==3636== by 0x118557: main (ctdbd.c:321)
==3636== Address 0x9c5b660 is 96 bytes inside a block of size 120 free'd
==3636== at 0x4C29D17: free (in
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==3636== by 0x58B32D3: _talloc_free_internal (talloc.c:1063)
==3636== by 0x58B3232: _talloc_free_children_internal (talloc.c:1472)
==3636== by 0x58B3232: _talloc_free_internal (talloc.c:1019)
==3636== by 0x58B3232: _talloc_free_children_internal (talloc.c:1472)
==3636== by 0x58B3232: _talloc_free_internal (talloc.c:1019)
==3636== by 0x58AD692: _talloc_free_children_internal (talloc.c:1472)
==3636== by 0x58AD692: _talloc_free_internal (talloc.c:1019)
==3636== by 0x58AD692: _talloc_free (talloc.c:1594)
==3636== by 0x11EC30: daemon_incoming_packet (ctdb_daemon.c:844)
==3636== by 0x136F4A: lock_fetch_callback (ctdb_ltdb_server.c:268)
==3636== by 0x152489: process_callbacks (ctdb_lock.c:353)
==3636== by 0x152489: ctdb_lock_handler (ctdb_lock.c:468)
==3636== by 0x56A535A: epoll_event_loop (tevent_epoll.c:728)
==3636== by 0x56A535A: epoll_event_loop_once (tevent_epoll.c:926)
==3636== by 0x56A3826: std_event_loop_once (tevent_standard.c:114)
==3636== by 0x569FFFC: _tevent_loop_once (tevent.c:533)
==3636== by 0x56A019A: tevent_common_loop_wait (tevent.c:637)
==3636== by 0x56A37C6: std_event_loop_wait (tevent_standard.c:140)
==3636== by 0x11E03A: ctdb_start_daemon (ctdb_daemon.c:1320)
==3636== by 0x118557: main (ctdbd.c:321)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11293
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This avoids rebuilding ctdb after each commit, instead we're using
SAMBA_VERSION_STRING, which may contain "4.3.0pre1-DEVELOPERBUILD",
instead of "4.3.0pre1.GIT.4ccdd39" for the non-install build.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Jun 8 15:51:46 CEST 2015 on sn-devel-104
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
While 'which' is a very common tool, on many distros it is not a requirement
that it be installed. 'type' is a shell built-in specified by the Open Group,
and is found in shells like bash, dash, and ksh across multiple OSes.
Signed-off-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Fri Jun 5 20:39:47 CEST 2015 on sn-devel-104
For all the records listed in VACUUM_FETCH, migration requests are sent
immediately without waiting. This means there can only be a single
VACUUM_FETCH processing active at a time.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
With the processing of one element factored out,
it is more natural to have the actual loop inside the
handler function. This also makes the talloc/free
bracked around the loop more obvious.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
