IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
If the process holding the recovery lock terminates unexpectedly then
the recovery daemon needs to know that the lock is no longer held.
While here, rename hold_reclock_handler() to take_reclock_handler() so
there is a clear difference between the two handler names.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This makes the API more general. If they are needed in a handler then
they can be in the private data.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will allow a simplification of the cluster mutex API, so the
private data can be registered when calling ctdb_cluster_mutex().
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
It won't be called more than once by the cluster mutex code.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
After the first activity on the file descriptor, ignore any subsequent
activity. Single-shot handlers are easier to write.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
It isn't necessarily a file.
Don't bother changing the control, since it doesn't pervade the code.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Setting the recovery lock file at startup can be done more simply.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Support for updating the recovery lock is being removed because it
isn't possible to recover from failure.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If the recovery lock setting is not consistent with that of the
recovery master then abort.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The recovery lock can not be reliably updated at run-time. If it
fails to update on some nodes then split-brain protection is gone and
there is no reasonable way to repair the situation. CTDB will have to
be restarted on all nodes. So, if this feature is being used to avoid
scheduling an outage then an outage will have to be scheduled just in
case!
To update the recovery lock, shut down CTDB on all nodes, reconfigure
the recovery lock and start CTDB again.
Those that *really* want to be able to change the recovery lock at
run-time can still do so. Set CTDB_RECOVERY_LOCK to point to a script
and this script can then be modified at run-time. However, please
don't report bugs if bad things happen...
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11946
Commit 670db6ac1d split tevent-util public
library to create tevent-unix-util public library for standalone ctdb
use. This created a public library dependency between samba and ctdb
for packaging.
Bundle tevent_unix.c in public library tevent-util as before. However,
to avoid the dependencies for packaging, standalone ctdb build will
build tevent-util as a private library with only tevent_unix.c
This simplifies any new subsystems (or libraries) which need tevent-util
and are linked in both samba and ctdb.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
The timeout RecoverTimeout (default 120) is used for control messages
sent during the recovery. If any of the nodes does not respond to any
of the recovery control messages for RecoverTimeout seconds, then it
will cause a failure of recovery of a database. Recovery helper will
retry the recovery for a database 5 times.
In the worst case, if a database could not be recovered within 5 attempts,
a total of 600 seconds would have passed. During this time period other
timeouts will be triggered causing unnecessary failures as follows:
1. During the recovery, even though recoverd is processing events,
it does not send a ping message to ctdb daemon. If a ping message is
not received for RecdPingTimeout (default 60) seconds, then ctdb will
count it as unresponsive recovery daemon. If the recovery daemon
fails for RecdFailCount (default 10) times, then ctdb daemon will
restart recovery daemon. So after 600 seconds, ctdb daemon will
restart recovery daemon.
2. If ctdb daemon stays in recovery for RecoveryDropAllIPs (default 120),
then it will drop all the public addresses. This will cause all
SMB client to be disconnected unnecessarily. The released public
addresses will not be taken over till the recovery is complete.
To avoid dropping of IPs and restarting recovery daemon during a delayed
recovery, adjust RecoverTimeout to 30 seconds and limit number of
retries for recovering a database to 3. If we don't hear from a node
for more than 25 seconds, then the node is considered disconnected.
So 30 seconds is sufficient timeout for controls during recovery.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Jun 6 08:49:15 CEST 2016 on sn-devel-144
If the node becomes stopped or banned after recovery is marked
active, then it will never freeze the databases, and hence the
node will keep banning itself indefinitely, until ctdbd is restarted.
This is a regression from 4.3, introduced with
b4357a79d9
and
d8f3b490bb
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11945
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Wed Jun 1 17:36:12 CEST 2016 on sn-devel-144
This adapts the debug message in local_node_got_banned
to reflect what the function is currently doing.
This message was not adapted when the function was changed.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jun 1 04:30:36 CEST 2016 on sn-devel-144
Both of these expand to 1. However, AF_LOCAL is a Unix domain socket,
which makes no sense when reading the code.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu May 26 11:42:46 CEST 2016 on sn-devel-144
With an empty value the first expression adds a trailing opening
quote, so the second expression doesn't add the closing quote. Handle
this with a special case.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This doesn't need to print the family. Nothing uses it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Stub scripts are more obvious. rc.local should only be used when
strictly necessary.
iptables_wrapper doesn't need to be no-op-ed, provided flock is
installed.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Block and unblock IP addresses using these new functions. This makes
the code more readable.
The case statement in each function is very cheap, so there is no need
to prematurely optimise and pass the family.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
CTDB_INIT_STYLE isn't used in this script.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Kai Blin <kai@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri May 20 21:06:18 CEST 2016 on sn-devel-144
Some Linux distributions don't have a "service" compatibility command.
To avoid breaking working systems, prefer the "service" compatibility
command just in case it does some extra, unexpected magic.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Kai Blin <kai@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Tue May 17 21:21:30 CEST 2016 on sn-devel-144
If this fails, we want to know which states it wanted to move to. Don't do the
return before the debug.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Sat May 14 03:06:05 CEST 2016 on sn-devel-144
Add CTDB_NFS_STATE_FS_TYPE and CTDB_NFS_STATE_MNT config options, show use in
nfs-ganesha-callout. Since the callout script is only an example, we
officially don't have default values for these.
Signed-off-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Comment typos and clarifications, erroneous variable names, corrected
pathnames, reorganizing variables, and squashing a few non-fatal
scripting errors.
Signed-off-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
If a recovery is going to be done then this will be followed by a
takeover run anyway. So, there's no use doing the takeover run
checks, potentially doing a takeover run and then doing a recovery.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The recovery daemon should be less involved in the service monitoring
logic.
The cases handled here are already handled elsewhere:
* When a node becomes unhealthy/healthy the monitoring code will
trigger a takeover run
* When a node is disabled/enabled the ctdb CLI tool will trigger a
takeover run
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
It will just become healthy again in the next monitor cycle.
Instead, let the recovery master ban it if the problem persists.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Banning is now handled by the takeover code sending banning credit
messages.
This commit makes a change in behaviour quite obvious. Takeover runs
were initiated from several locations in the code but banning was only
done from one of these locations. Now banning can be done from any
failed takeover run.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Post-process failues and only send banning credits to the node with
the most failures.
If there is a widespread problem or a problem on the recovery master
node then this should help avoid banning all the nodes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will allow banning credits assignments to be limited according to
some criteria.
Note that this only matters when multiple controls are sent to each
node: RELEASE_IP and TAKEOVER_IP. This doesn't change the behaviour
for IPREALLOCATED.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Banning credits are now assigned by takeover runs called from all
locations in the recovery daemon. Previously this only happened from
one of the callers. When separating out the takeover run code the
behaviour should be consistent.
The callback (and corresponding data) passed to ctdb_takeover_run() is
now ignored. Dropping this will allow the interface between the
recovery daemon and IP takeover to be simplified.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Probably due to oversight, this is currently only used for the
"takeip" step.
This does consistent error handling and provides a layer of
indirection to the passed callback, so use it for "releaseip" and
"ipreallocated" steps too.
The callback data now needs to be initialised before the first
possible jump to "ipreallocated".
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Abstract out the initialisation of the callback data. Later, we'll
need to do it multiple times or move it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The ipreallocated control has been in CTDB for a long time.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
These tests do not use ctdb tool stub anymore.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed May 11 02:19:20 CEST 2016 on sn-devel-144
Since there are no public IPs setup, these tests do not really test the
functionality.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This tested parse_nodestring function from tools/ctdb.c. However,
ctdb.c is soon going to be replaced with the code using new client API.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Now all the IP takeover code for non-master node is in this function.
The function can always be renamed to something more suitable.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri May 6 15:10:59 CEST 2016 on sn-devel-144
Update log levels and messages, comments and wrapping of long lines.
No functional changes.
Note that interfaces_have_changed() already does adequate logging.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
When public IP checking is disabled, verify_local_ip_allocation()
still retrieves known IP addresses and runs through a loop that does
nothing.
Instead, completely skip the retrieval and checking loop.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This makes verify_local_ip_allocation() self-contained and simplifies
main_loop().
Due to indentation changes, this commit is most easily read when
ignoring whitespace.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
There is no need to return one of several states and then trigger an
election for one of those return states. Have the recovery master
validation trigger the election directly and just return whether
monitoring should continue.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Change this to return just 0 or -1. It isn't monitoring anything.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
update_local_flags() never returns MONITOR_ELECTION_NEEDED, so drop
this entire if-statement.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Deferred attach processing is done unconditionally at this point. It
is then done again if recovery lock checking is done and completes
successfuly. If the recovery lock checking fails then it should not
be done at all.
Move this processing so it is done with the early exit when the
recovery lock is not being used.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue May 3 08:08:31 CEST 2016 on sn-devel-144
If the buffer size provided is not sufficient, then return the required
buffer length.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This means that the packet allocation will happen just before push
functions are called.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This follows the convention used for marshalling routines for ctdb data types.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This currently fails exactly when it is needed: when there is a
dangling link pointing into an unmounted filesystem. This happens
because [ -e <dangling-link> ] returns false. The solution is simply
to remove that check.
Now the first condition in the "if" statement catches the backward
compatibility case where $GANRECDIR used to be a directory. It will
also catch other problems, such as if a file has been created in this
location. If nothing exists then it will silently succeed (due to
-f).
The second case catches a symlink pointing to the wrong place (e.g. in
an unmounted filesystem).
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 30 04:28:13 CEST 2016 on sn-devel-144
The cluster mutex code already passes the latency and expects the
handler to update the statistics.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
ctdb_recovery_have_lock(), ctdb_recovery_lock(),
ctdb_recovery_unlock() are only used by recovery daemon, so move them
there.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This means that the cluster mutex handle can now be treated as opaque.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
All of the ctdb_cluster_mutex_* infrastucture can now handle an
arbitrary mutex.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The underlying change is to allow the cluster mutex argstring to
optionally contain a helper command. When the argument string starts
with '!' then the first word is the helper command to run. This is
now the standard way of changing the helper from the default.
CTDB_CLUSTER_MUTEX_HELPER show now only be used to change the location
of the default helper when testing.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is currently just treated as the name of a lock file. However,
it is really some arbitrary arguments to lock helper.
Therefore, it should be parsed and passed as separate arguments to the
lock helper.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Replace the file descriptor for the recovery lock in the CTDB context
with the cluster mutex handle, where non-NULL means locked.
Attempting to take the recovery lock is now asynchronous and no longer
blocks the recovery daemon.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Unlike fcntl(2), some other helper might need to explicitly take
action to release a mutex. This can be done by catching SIGTERM.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This implements the type of fcntl locking that the recovery lock uses.
The intent is to use it for multiple locks and allow the choice of
helper to be configured.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If the reclock is set then print it, otherwise print nothing.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is currently only used to check whether the recovery lock can be
taken. However, name it more generally in anticipation of using it
for general cluster mutex taking and testing.
No functional changes. A couple of debug message simplifications and
code rearrangements.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is used to reply to the recmode control for all the different
cases. The callers can later be generalised to use a pointer, which
can then be used for recovery lock handling in different contexts.
Note that the handle is now freed in set_recmode_handler() rather than
the callbacks.
There is one difference in behaviour. Deferred attach calls are now
processed in the timeout case, where they weren't before. That's a
bug fix!
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
'0' = Child took the mutex
'1' = Unable to take mutex - contention
'2' = Unable to take mutex - timeout
'3' = Unable to take mutex - error
This is a straightforward API. When the child is generalised to an
external helper then this makes it easier for a helper to be, for
example, a simple script.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Use the more general name "cluster mutex", since we are likely to end
up with more than one cluster-wide lock. There will probably be a
dedicated recovery lock, held only during recovery, and also a second
lock that is held by the master node. Currently one lock is used for
both purposes.
At the moment the struct and functions are involved with setting the
recovery mode. However, they'll be abstracted out to more generally
deal with the cluster mutexes, so "recmode" -> "cluster_mutex". Drop
"set" from names, since this is used to test the lock. Also drop
"ctdb" prefix from functions, since they are local to this file. The
struct will eventually be a long-lived handle that will release the
mutex when freed, so name it accordingly.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
LVS and NAT gateway support had bit-rotted. We don't use any of these
in scripts/tests and we very much doubt anyone else uses them.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Apr 25 10:34:47 CEST 2016 on sn-devel-144
This can list the different aspects of status: master, list, status.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Tweak "ctdb natgw natgwlist" to keep output format the same.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
In particular, LVS won't work at all if there are no public IP
addresses.
This is a temporary solution until a generic reconfiguration hook is
implemented.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Why allocate all that memory and transfer all that data across the
socket?
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
When external monitoring is enabled for an NFS service using
service_check_cmd then $ctdb_check_rpc_out is empty because the
internal RPC checking isn't used. This results in empty log messages
like:
60.nfs: ERROR:
or:
60.nfs: WARNING:
Improve this so it at least says:
60.nfs: ERROR: monitoring service "statd" failed
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Some controls are fire-and-forget (CTDB_CTRL_FLAG_NOREPLY). Since there
is no reply received, the opcode in the ctdb_reply_control structure
never gets set.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
These functions were used in the transaction code. These controls did
not use server_id structure defined in samba, so samba would not use them.
Instead check if the process exists for conflicting g_lock entry.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
These controls have never been used and also they do not use the server_id
structure defined in samba. In future, similar controls can be added to
register/unregister using proper server_id structure.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This test sets TakeoverTimeout=90 to avoid banning during takeover.
However, the setting is done on the test node instead of the recovery
master node. During "ctdb reloadips", the recovery master will used
the default value of TakeoverTimeout.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
ctdb->idr and ctdb->srv get initialized as part of ctdb_init() called
from ctdb_cmdline_init().
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Get rid of the range reserved for traversals since it's not used.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Don't do a percentage calculation for either memtotal or swaptotal if they
are zero.
Signed-off-by: Jose A. Rivera <jarrpa@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
ss with a filter is much faster than post-processing output from
netstat. CTDB already has a hard dependency on iproute2 for IP
address handling, so depending on ss is no big deal.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This currently causes monitor failure.
Log a warning instead. If there is a transient issue, such as NFS
being restarted in the background, then the thread count file should
be there the next time around so the count can be adjusted if
necessary.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
For "master", if there is a master then print the PNN, otherwise print
nothing.
For "list", print the PNN and IP addresses without a colon in between.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This simply calls out to the wrapper, so that commands are changed as
follows:
ctdb lvsmaster -> ctdb lvs master
ctdb lvs -> ctdb lvs list
This provides a simple, extensible interface and means that "ctdb lvs
status" is also available.
Unit tests are streamlined so that there is a single test for each
CTDB state. Each test does "master", "list" and "status" sub-tests.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
To keep this commit comprehensible, 91.lvs and the CTDB CLI tool are
temporarily inconsistent. The tool will be made consistent in a
subsequent commit.
LVS now uses a configuration file specified by CTDB_LVS_NODES and
supports the same slave-only syntax as CTDB_NATGW_NODES. LVS also
uses new variable CTDB_LVS_PUBLIC_IFACE instead of
CTDB_PUBLIC_INTERFACE.
Update unit tests and documentation.
Note that the --lvs and --single-public-ip daemon options are no
longer used. These will be removed and relevant documentation
updated in a subsequent commit.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will replace the ctdb CLI tool "lvs" and "lvsmaster" options. It
also makes LVS daemon support unnecessary.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Having both "recovered" and "ipreallocated" means that everything
happens twice when there is a recovery. No need for that.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Basic error redirection improvements before writing tests.
Deleting the service during "startup" will usually fail because the
service has never been setup, so redirect output to avoid logging an
error.
Similarly, deleting the service in "ipreallocated" will always fail
the first time, which would cause an error to be logged. Given the
simplicity of the script, there's no sane way to avoid the error
sometimes and log it if it actually matters. This could potentially
be tidied up in the future by making 91.lvs stateful, in a similar way
to 11.natgw.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Testing indicates that these are good reliable defaults that can kill
many connections in a reasonable amount of time.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Apr 1 08:10:54 CEST 2016 on sn-devel-144
This made sense when connections were individually queued in the
daemon. However, they're now done in batch so just keep an overall
count.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
When previously killing TCP connections via the daemon there was some
latency due to each kill being sent to the daemon via a separate
control. This probably meant that when doing a 2-way kill the tickle
ACKs sent to the client end of a connection would not interfere with
listening for the reply ACK from the server end. Now that there is no
latency, the tickle ACK or RST sent to the client end can be seen as
the reply to the server end tickle ACK, and vice-versa.
To avoid this, throw away packets that look like we sent them.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The end of the connection in parentheses is not the end being killed.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Since they're being done in batch, just schedule an event to traverse
all the connections.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The handler won't be called unless there is something to read.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
ctdb_killtcp will take up to 5 seconds to kill connections, so don't
wait in a loop. Just check if there are remaining connections on
completion and log a message either way.
Also add a test stub.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will be needed for a rewrite of the connection killing code but
it is not used yet.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will allow killing of TCP connections without daemon involvement.
It looks strange that the common code for daemon and helper is in the
server directory. Having it in the server directory means less
temporary changes to the build configuration. This code will move
into the helper itself and will no longer be used by the daemon.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This allows common.h and ctdb_private.h to be dropped.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will be used in a standalone helper.
Don't worry that the API isn't clean and opaque. All of the code will
eventually move into the helper and will no longer be used by the
daemon.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This function knows nothing about CTDB contexts or VNNs, so it can be
used elsewhere.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The destructor used in this instances needs a CTDB context and a VNN.
However, destructors used in other cases may need different data.
For this instance create a local structure to hold the required data.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
We don't want this code to depend on a CTDB context, so don't go
looking there for an event context.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If one or more nodes are misbehaving during recovery, keep track of
failures as ban_credits. If the node with the highest ban_credits exceeds
5 ban credits, then tell recovery daemon to assign banning credits.
This will ban only a single node at a time in case of recovery failure.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Mar 25 06:57:32 CET 2016 on sn-devel-144
This will be called from recovery helper to assign banning credits to
misbehaving node.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This abstraction uses capabilities of the remote nodes to either send
older PUSH_DB controls or newer DB_PUSH_START and DB_PUSH_CONFIRM
controls.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This abstraction depending on the capability of the remote node either
uses older PULL_DB control or newer DB_PULL control.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Also, rename traverse function and traverse state for recdb_records
consistently.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This variable is used to set the dmaster value for each record in
recdb_traverse().
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This will be used to limit the size of record buffer sent in newer
controls for recovery and existing controls for vacuuming.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Interface names that are too long will be truncated by strncpy(3)
later on. It is better to validate the length of each new interface
name to ensure it will be usable.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Mar 17 13:56:41 CET 2016 on sn-devel-144
On a busy system, TRANS3_COMMIT control can take upto or longer than
3 seconds. On timeout, there are few possible outcomes.
1. The transaction has completed on all nodes and TRANS3_COMMIT control
has returned. In such a case, there is no problem.
2. The transaction has completed on the local node, but TRANS3_COMMIT
control is still active. In such a case, ctdb_transaction_commit()
can return successfully. If this is being called from ctdb, then
ctdb will exit. This will cause ctdb daemon to trigger recovery
since the client exited while transaction is active. This will cause
unnecessary recovery.
3. Database recovery was started and ctdb_transaction_commit() will
retry till the recovery completes the transaction.
Increasing the timeout to 30 seconds will avoid the spurious database
recoveries when TRANS3_COMMIT control takes longer to finish.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Mar 11 19:59:53 CET 2016 on sn-devel-144
If set, this was used to setup an IP takeover run on a timer after
certain updates to the public IP address configuration (e.g. "ctdb
addip").
However, "ctdb reloadips" completely manages public IP reconfiguration
and avoids the anomalies that DeferredRebalanceOnNodeAdd was
introduced to work around.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This was a workaround for trying to ensure public IP addresses are
properly rebalanced after running "ctdb addip" on multiple nodes.
"ctdb reloadips" is a better solution.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is undocumented and is not needed. It was a workaround for
trying to ensure public IP addresses are properly rebalanced after
running "ctdb addip" on multiple nodes. "ctdb reloadips" is a better
solution.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
During the recovery process, the timeout value for sending all controls
is decided by RecoverTimeout tunable. So in the recovery process,
first get the tunables, so the control timeout gets set correctly.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Current NFS and CIFS tickle tests do not test the killtcp
functionality on the releasing node. 2-way killing is done for NFS,
so this test explicitly looks for packets from the releasing node.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
tcpdump does not support filtering on MAC address when reading from a
file. Therefore, this is implemented by conditionally using grep to
filter the output of tcpdump.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
There's a tiny chance that the connection information may not be
transferred to other nodes quickly enough, so add an explicit wait.
Also clean up the description and recognise that it is the takeover
node that does the tickling.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If the files section uses %{_libexecdir} then CTDB must also be
configured to build and install with the same path, otherwise rpmbuild
can fail due to a mismatch. "rpmbuild --showrc" indicates that the
default %configure command sets:
--libexecdir=%{_libexecdir} \
A mismatch will occur on SUSE systems, where SLES 12 and OpenSUSE 12
set _libexecdir to %{_exec_prefix}/lib.
The failure was initially seen when testing on Debian where
_libexecdir is set to %{_prefix}/lib/x86_64-linux-gnu, though Debian
is an unlikely platform for building RPMs...
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Mar 7 14:43:15 CET 2016 on sn-devel-144
Use real-time priority only for obtaining record and database locks.
Do not open databases with real-time priority as it can cause thundering
herd on fcntl lock while opening tdb database. Also relinquish real-time
priority after the lock is obtained.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Mar 7 11:29:00 CET 2016 on sn-devel-144
CTDB releases all IPs in following cases: starting up, shutting down,
node gets banned, node does not come out of recovery for a long time.
Always inform samba when CTDB releases IP addresses.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
CTDB already notifies Samba with RELEASE_IP message. Samba can take
appropriate action based on that.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This is unmaintained and misleading.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Mar 7 05:00:15 CET 2016 on sn-devel-144
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11770
From man page of packet(7):
protocol is the IEEE 802.3
protocol number in network byte order. See the <linux/if_ether.h>
include file for a list of allowed protocols. When protocol is set to
htons(ETH_P_ALL), then all protocols are received.
Protocol argument was changed from network order to host order wrongly
in commit 9f8395cb7d.
Specifying "protocol" field to socket(AF_PACKET, ...) call only affects
the packets that are recevied. So use protocol = 0 when sending raw
packets.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Mar 4 12:58:50 CET 2016 on sn-devel-144
Anything starting with CTDB_ is appended to each node's configuration
file.
This is made slightly more difficult because the "env" command doesn't
necessarily quote values for consumption by the shell. The strategy
used here is a bit fragile (assumes double-quotes, no nested quotes)
but this is test code and only developers are expected to use this
override.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Mar 1 08:56:09 CET 2016 on sn-devel-144
This uses a configuration file which can be more easily overridden
that explicit command-line options.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
GNU/kFreeBSD's platform name is 'gnukfreebsd', not just 'kfreebsd'.
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming@samba.org>
Autobuild-Date(master): Tue Mar 1 05:42:23 CET 2016 on sn-devel-144
These have been scattered around the code so that
tevent_loop_allow_nesting() can be called. However, only the main
daemon and some tests currently use nested event loops.
TEVENT_DEPRECATED is already defined in the places where it is needed.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Feb 26 07:11:29 CET 2016 on sn-devel-144
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Feb 24 11:57:23 CET 2016 on sn-devel-144
If a record is not on a local node, then it is migrated from remote node.
However, before the client can get a lock on the record, it's possible
for the record to get migrated away. In that case, repeat migration.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
There is no allocated memory returned from these functions.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
The database context returned is allocated off the client and is not
allocated from user-supplied TALLOC_CTX.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This is similar to ctdb_client_wait() with additional timeout argument.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
It does not make sense to update this statistic for the timeout case,
since this could skew the statistic. To keep it simple, just update
it for the usual case where there is lock contention, since this is
the usual case. So the daemon statistic measures time to test the
lock and the corresponding recovery daemon statistic measures time to
take the lock.
Additionally, the recovery daemon will eventually use this code to
take the lock, and the method of updating the latency statistic will
need to be pushed further out to a configurable handler that depends
on the calling context.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Tue Feb 23 10:32:06 CET 2016 on sn-devel-144
Have 0 indicate that the lock was taken. This allows non-zero values
to be used to indicate why the lock could not be taken. EACCES means
lock contention.
For now use just EACCES to cover all failures, since
ctdb_recovery_lock() returns a bool and details of other errors will
be lost. ctdb_recovery_lock() will undergo some big changes, so don't
try to fix this now.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This currently returns an incorrect error when the expected number of
bytes are not read. Separate out the different cases to clarify the
logic and avoid reporting the wrong error.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is already done before the destructor is assigned.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The callbacks that use this value are only ever called if recovery
mode is being set to NORMAL. So do not check if recmode is NORMAL
either.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This pattern is used quite a few times in the CTDB code. Many
instances use ctdb_kill() but for signal 0 this just calls kill(2)
anyway.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The child process writes the status into the pipe before looping to
wait.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Where possible, this should no longer be used.
struct ctdb_public_ip_list is a fixed size structure and introduces an
extra level of indirection. This means one level of indirection can
be dropped for known_public_ips and available_public_ips.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Feb 12 08:40:21 CET 2016 on sn-devel-144
Confirm that after re-attaching detached database, it is empty.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
_cluster_is_recovered() is used to check if all the nodes in the
cluster have recovery mode set to NORMAL. This is mainly used just
after recovery to wait till the recovery is over. Instead just check
the recovery mode on node 0.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This matches the behaviour during serial database recovery.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Feb 11 08:01:14 CET 2016 on sn-devel-144
This is unnecessary in Samba >= 4.0 because winbindd monitors IP
address itself and no longer needs to be told when they are dropped.
The smbcontrol commands can hang if a node has recovery mode active
because smbcontrol is unable to connect to the registry. Therefore,
the smbcontrol commands should be removed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11719
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Wed Feb 10 14:08:17 CET 2016 on sn-devel-144
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Tue Feb 9 22:28:08 CET 2016 on sn-devel-144
This reverts commit 0ff90f4fac.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11707
The checks against database generation are not required since
the global generation is updated as part of updating vnnmap
before the actual database recovery. This change was done in
5aab31a39a.
Checking only against the database generation is incomplete. It can
cause CTDB to abort if the following sequence of events happen.
- CTDB gets REQ_DMASTER packet (gen1)
This packet processing gets deferred to get a record lock
- CTDB goes into recovery, marks RECOVERY_ACTIVE
CTDB recovery helper updates vnnmap (gen2)
- CTDB processes REQ_DMASTER packet (gen1)
The check against database generation (gen1) succeeds.
The check for lmaster is now invalid because VNNMAP has changed.
This will cause CTDB to abort due to protocol error.
Reverting the patch stops processing packets of older generation before
they get into call processing.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Signed-off-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue Feb 9 12:39:24 CET 2016 on sn-devel-144
ctdb/server/ipalloc_lcp2.c:264:29: warning: 'minimbl' may be used uninitialized
in this function [-Wmaybe-uninitialized]
Guenther
Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Feb 7 00:56:44 CET 2016 on sn-devel-144
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11696
We neither have public headers nor a public library.
Guenther
Signed-off-by: Guenther Deschner <gd@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Jan 29 04:33:36 CET 2016 on sn-devel-144
Nothing checks it anymore.
This means that the NAT gateway capability in the daemon is now
unused.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The "natgwlist" command is no longer marked "auto all" and is also
marked "without daemon". That latter is not strictly true because
ctdb_natgw needs the daemon so a subsequent invocation of "ctdb
nodestatus" will work. However, "without daemon" is used here because
the top-level "ctdb natgwlist" does not need to open a connection to
the daemon. It just needs to invoke ctdb_natgw.
Update tests to suit.
It would make sense to make "ctdb natgw" generally call out to
ctdb_natgw, passing all argument. However, that can be done later.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
These tests deal only with timeouts that can occur retrieving
capabilities. The NAT gateway capability is going away so drop the
tests now to simplify future commits.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
To keep this commit comprehensible, 11.natgw and the CTDB CLI tool are
temporarily inconsistent. The tool will be made consistent in a
subsequent commit.
ctdb_natgw_slave_only() is reimplemented to check for the option in
the appropriate line in $CTDB_NATGW_NODES.
Update unit tests and documentation.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Handle the "slave-only" option in the unit test setup. Reindent
function while touching it.
Also drop a test that no longer makes sense. Specifying both "master"
and "slave-only' is now much more obvious, since they need to be on
the same line, and is now punishable by undefined behaviour.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Some features, such NAT gateway and LVS support, can be implemented
without daemon and (internal) ctdb CLI tool support. These are
non-core features that don't need incredible performance and they
don't need to be in the core code. They can easily be reimplemented
in scripts, along with some configuration changes.
For continuity, the ctdb CLI tool code will call out to helper scripts
so that the current status information can still be provided. Those
helper scripts may then reinvoke the ctdb CLI tool to gather
information.
So, redo the tool testing using a "ctdb" stub command. This will
swallow standard input and feed it to the test program each time the
"ctdb" stub is called.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is intended to replace the use of "ctdb natgwlist" in 11.natgw
and provide different views of the NAT gateway status.
It replaces the use of CTDB_NATGW_SLAVE_ONLY=yes with a "slave-only"
keyword in the NAT gateway nodes file. This means the nodes file must
be consistent on all nodes in a NAT gateway group.
Note that this script is not yet integrated, so there are no behaviour
or documentation changes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This has always been the case. Now it is documented and enforced.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Now its name describes its usage and the code reads better.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Each is now used in only one place and the logic is more obvious
without them.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
... instead of talloc_zero(). This sets the db_id correctly.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Jan 20 07:28:42 CET 2016 on sn-devel-144
The reqid will be deleted when the client control state is freed.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
This allows the client code to optionally clean up and/or re-connect to
CTDB daemon when it the daemon goes away. If no disconnect callback is
registered and CTDB daemon goes away, then the client will terminate.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reduces intentation by using early returns.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu Jan 14 22:41:29 CET 2016 on sn-devel-144
These allocate enough memory but things get confusing if they're used
as a guide when updating the code.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Jan 12 22:19:16 CET 2016 on sn-devel-144
Consider this sequence of events:
1. Instance of script running update_tickles() hangs
2. Script debugging is launched asynchronously
3. New instance of script is launched, creates temporary file(s)
4. Original hung script makes progress before asynchronous script
debugging kills it, so it removes temporary file(s)
5. New instance of script produces error due to missing files(s)
This is obviously rare.
Use more unique filenames to avoid step (4) removing the file(s)
belonging to other instances of the script.
This requires some extra cleanup to avoid too many temporary files
(which is why unique filenames were not originally usd). It is
sufficient to remove files modified at least 10 minutes ago.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
Whitespace and indentation improvements.
Remove comments describing events, since the README covers that much
better.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
The current code uses so many shell idioms that it is difficult to
follow.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
Commit cfa0ffe780 introduced a memory
leak. Never assume...
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
The first element of these structures is a 32-bit PNN. On 64-bit
systems this field can be followed by 32-bits of padding. When the
structures are copied this can cause uninitialised memory to be
copied.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
Thousands of these can be generated each second, rendering INFO level
debugging useless.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
assinged -> assigned
Signed-off-by: Karolin Seeger <kseeger@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Jan 11 21:40:36 CET 2016 on sn-devel-144
This is used by the recovery daemon to validate the current recovery
master. Don't risk being unable to elect a new master if the current
master is inactive but unresponsive.
Note that this client call is currently not used by any other callers.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Sun Dec 13 03:17:10 CET 2015 on sn-devel-104
Add a new option -l to check whether POSIX byte range locks are
working. Usage:
node1$ touch /path/to/cluster-fs/FILE
node1$ ./bin/ping_pong -l /path/to/cluster-fs/FILE
Holding lock, press any key to continue...
You should run the same command on another node now.
node2$ ./bin/ping_pong -l /path/to/cluster-fs/FILE
Output can either be:
Holding lock, press any key to continue...
This means POSIX byte range locks are *not* working.
If you see this instead:
file already locked, calling check_lock to tell us who has it locked...:
check_lock failed: lock held: pid='27375', type='1', start='0', len='0'
Working POSIX byte range locks
Congrats, you have a cluster fs with functional byte range locks!
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec 10 08:48:38 CET 2015 on sn-devel-104
Shorter temporary variables for compactness/readability. "tmp_ip" is
5 characters longer than "t". In each for statement it is used 4
times, so costs 20 characters. Save those extra characters so that
future edits will avoid going over 80 columns.
Tweak whitespace for readability, rewrap some code.
No functional changes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
As per the comment:
If the IP address is hosted on this node then remove the connection.
Otherwise this function has been called because the server IP
address has been released to another node and the client has exited.
This means that we should not delete the connection information.
The takeover node processes connections too.
This doesn't matter at the moment, since the empty connection list for
an IP address that has been released will never be pushed to another
node. However, it matters if the connection information is stored in
a real replicated database.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
In a subsequent commit ctdb_takeover_client_destructor_hook() needs to
know the VNN. So just have both callers of
ctdb_remove_tcp_connection() do the lookup and pass in the VNN.
This should cause no change in behaviour.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Tickle list updates are broadcast to all connected nodes and are
accepted even when received on the same node that sent them. This
could actually lead to lost connection information when information
about new connections is received while an update is in-flight.
Instead, return early when the IP is hosted on the current node, since
it is the only one that could have sent the update.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Move information about TCP connection tracking and resetting into
ctdb.7.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
exportfs can hang when, for example, DNS is flakey. Given that
exports don't change much, it makes sense to cache them.
Don't try to add error handling when exportfs fails but do print a
warning. Proper error handling can be added separately.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
It hasn't worked since commit cda5f02c7c
in 2009, which reworked the banning code. Since then
ctdb_control_modflags() has contained a comment saying:
/* we don't let other nodes modify our BANNED status */
Unbanning all nodes originally occurred here when the recovery master
role moved to a new node. The logic could have been meant for the
case when the old recovery master was malfunctioning, so got banned.
If any other nodes had been banned by this recovery master then they
would be unbanned. However, this would also unban the old recovery
master, which is probably suboptimal. The logic would also trigger if
a node was banned for a good reason and then the recovery master was
stopped. So, apart from doing nothing, the logic is too simplistic so
might as well be removed.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The banning code caters for the case where the node specified in the
bantime data is not the node receiving the control. This never
happens. There are 2 places where ctdb_ctrl_set_ban() is called: the
ctdb CLI tool and the recovery daemon. Both pass the same node in the
bantime data that they are sending the control to. There are no plans
to do anything more elaborate, so just delete the handling of this
special case.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
If the header is specified, then the datalen should be incremented.
There are two ways of marshalling ctdb_rec_data:
1. ctdb_rec_data->header is NULL and ctdb_rec_data->data embeds both
the header and the value. Used in recovery for push/pull of records.
2. ctdb_rec_data->header is not NULL and ctdb_rec_data->data embeds
only the value. Used everywhere else.
In both cases, the wire format includes the header and the value.
There is nothing in the wire format to inform the unmarshalling code
whether to extract the header separately or not.
In the current code the header is extracted depending on whether the
header argument to the extractor routine is NULL or not. This is insane
and highly error-prone.
So in the new API, unmarshalling code never extracts the header. If the
caller requires the header, then it can be extracted separately using
special function ctdb_ltdb_header_extract().
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Nov 25 13:21:48 CET 2015 on sn-devel-104
There can be 0 public addresses.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
ctdb_addr_info->iface can be NULL.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
This is required when ctdb client wants to send arbitrary data as part
of CTDB_REQ_MESSAGE.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
This is a duplicate function to ctdb_addr_to_str, but ctdb-util cannot
be linked with the code using new API mainly because it code in ctdb-util
requires struct ctdb_context.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
Since this code does not need any private structures defined in
ctdb_private.h, protocol/protocol.h should suffice.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
This can be easily decomposed into 2 separate arrays.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Nov 23 05:34:55 CET 2015 on sn-devel-104
This puts all of the memory allocation for ipalloc_state into its init
function. This also simplifies the code because
set_ipflags_internal() can no longer fail because it no longer
allocates memory.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is cleaner than returning ipflags and assigning them into
ipalloc_state afterwards.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Instead of local or passed temporary contexts.
This has the side effect of making ipalloc_state available inside the
modified functions, making future use of ipalloc_state simpler.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The only likely failure is out of memory, so just return boolean
value.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
These should be created elsewhere. If not then something is wrong, so
don't hide the problem.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Nov 20 04:40:26 CET 2015 on sn-devel-104
As do the functions called below it. They no longer need a CTDB
context.
create_merged_ip_list() now takes both a CTDB context and an
ipalloc_state.
Drop ipalloc_state from CTDB context. So the substitution in the
code is:
ctdb->ipalloc_state -> ipalloc_state
Tweak the test code to match.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
For various reasons create_merged_ip_list() needs a CTDB context.
This is difficult to resolve now for a few reasons, including:
* The ip_tree needs somewhere to live.
It isn't very useful in its current form. However, in the future
real remote IP monitoring will probably be added back, so leave it
around.
* It uses node flags from the ctdb_node structure.
This could be changed by putting a node map into ipalloc_state
and referencing that.
For now, it is easier to move it out to where there will be a CTDB
context available for the forseeable future. ctdb_takeover_run() will
need one as long as the current client interface is used.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The IP allocation algorithms need the value of this tunable, so copy
it to avoid needing the CTDB context.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Algorithm-related tunables from the CTDB context no longer need to be
accessed in the allocation logic.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Most of the IP allocation code does not need a CTDB context. However,
temporarily hang this off the CTDB context and make only the changes
relating to known/available IP address. This makes those logic
changes obvious without burying them in function type changes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Various scripts (including debug_locks.sh, 00.ctdb, 05.system) need
CTDB_DBDIR to point to the right place... but it doesn't.
Move the rewriting of CTDB_DBDIR to loadconfig() so that it happens
for all scripts. Have this code set internal variable
CTDB_DBDIR_TMPFS_OPTIONS so that ctdbd_wrapper can do the mount.
This loses the generality that was present in dbdir_tmpfs_start() but
it wasn't being used anyway. If it is needed in the future then it
will be in the git history.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Nov 18 11:51:54 CET 2015 on sn-devel-104
Starting to untangle cluster management, database recovery and public
IP allocation. This is a non-trivial subset of the cluster management
code that runs in the recovery daemon on all nodes.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Mon Nov 16 11:47:45 CET 2015 on sn-devel-104
Capabilities are used when computing an election result so having them
up-to-date seems like a good idea.
Also update several instances of an ambiguous comment.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The VNN map is only needed on the recovery master, so no need for all
recovery daemons to retrieve it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This is already handled in update_recovery_lock(), which is called
immediately before.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The only non-obvious part here is dropping the setting of the nodemap
local variable to NULL. If the following control succeeds then it is
set, otherwise return and it doesn't matter.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
That is, using CTDB_CURRENT_NODE makes this more obvious.
Also fix incorrect error messages.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Each recovery daemon knows who the recmaster is and is in sync with
its local daemon. The recovery master is running this check so do not
bother checking with its local daemon - both agree that it is the
recovery master.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The recovery daemon already knows which node is the master. This
relies on rec->recmaster being correctly initialised and correctly set
during elections.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Recovery should not do cluster management functions. Setting the
recovery master should only be done via an election.
Main loop will determine if recovery master is inconsistent across the
cluster and force an election if necessary.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The recovery daemon pushes knowledge of recovery master election
progress/result to local daemon. It then retrieves that information
again.
Instead, have the recovery daemon reliably track election
progress/result in rec->recmaster so it doesn't need to be retrieved.
Be careful to maintain consistency by only doing this when the local
daemon has been updated.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
There can be no holes in the nodemap. Even if a node has been deleted
it will take a slot in the nodemap. The only exception is that the
nodemap shrinks if nodes are deleted from the end. That should never
include the master because a node should be shutdown before being
deleted, and an election should already have take place.
To avoid walking off the end of the nodemap nodes array just confirm
that the master node's PNN is a valid index into the array. No need
to walk through the nodemap.
After this, in this section of the code j is now invalid. So use the
master's PNN to index into the nodemap. This is safe.
In the process, clean up some log messages to avoid saying "Force
reelection". It's just an "election".
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This only applies to monitor events so renaming clarifies this.
Note that this change is not backward compatible. Users with
CTDB_SET_EventScriptTimeoutCount=<n>
in their configuration will get failures when starting CTDB but the
cause will be clearly logged.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
It is only updated for monitor events, so it is meaningless here.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* The defaults for EventScriptTimeout and EventScriptTimeoutCount are
wrong.
* EventScriptTimeout is the total time for all enabled scripts that
are run for an event, not a single event script.
* EventScriptTimeoutCount only applies to monitor events.
* EventScriptUnhealthyOnTimeout is obsolete, so remove it.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Checking runstates is unnecessary now that nodes that are not RUNNING
will return no available IP addresses. I have no idea why I didn't do
it this way originally.
Tweak the test code to cope with this.
Note that this is a backward-incompatible change. If new and old
versions of CTDB are running together in a cluster and a new node
takes over as recovery master then old nodes will be able to host
public IP addresses before they are in RUNNING runstate. This is
mitigated by the bias towards recovery master stability in elections.
If it is important that nodes do not host IPs until they are RUNNING
then do not restart nodes running the old version.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
The available IPs list is now only non-empty for nodes that are in
RUNNING runstate. So, to avoid running the IP allocation algorithm
when there are no available available IPs, explicitly check for
available IPs rather than checking runstates.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This will allow wonderful simplification (i.e. removal) of some of the
runstate checking in the takeover run code.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Include system_util.c in ctdb-system subsystem.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Nov 16 03:51:36 CET 2015 on sn-devel-104
This makes the function compatible with parse_debug().
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
This allows for backward compatibility with the current code where
DEBUG_ERR corresponds to "ERR".
Fix the test that relies on "ERR" instead of "ERROR".
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
This variable is used for adding a prefix to log entries from various
child processes.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
These functions are only used in the ctdb daemon code.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
This function can only called from ctdb daemon.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Michael Adam <obnox@samba.org>
This is currently done before each IP takeover run, so just factor it
in.
ctdb_reload_remote_public_ips() becomes static.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Nov 12 09:28:45 CET 2015 on sn-devel-104
This will help to untangle known and available public IP lists from
the CTDB context.
verify_remote_ip_allocation() needs a forward declaration.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
