1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

6434 Commits

Author SHA1 Message Date
Amitay Isaacs
c620bf5deb ctdb-daemon: Reset push_started flag once DB_PUSH_CONFIRM is done
Once DB_PUSH_START is processed as part of recovery, push_started
flag tracks if there are multiple attempts to send DB_PUSH_START.
In DB_PUSH_CONFIRM, once the record count is confirmed, all information
related to DB_PUSH should be reset.  However, The push_started flag was
not reset when the push_state was reset.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Jun  8 14:31:52 CEST 2016 on sn-devel-144
2016-06-08 14:31:52 +02:00
Amitay Isaacs
ecb74721e7 ctdb-recoverd: Avoid duplicate recoverd event in parallel recovery
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11956

In do_recovery, after the recovery and takeover is complete, recoverd
event is triggered.  When the parallel database recovery was separated,
ctdb_recovery_helper implemented sending END_RECOVERY control which
causes recoverd event to be triggered.  So when there is parallel database
recovery, recoverd event is triggered twice.

Instead move the call to run_recovered_eventscript() explicitly in
the serial recovery code path.  This avoids the duplication trigger of
recoverd event.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2016-06-08 10:33:19 +02:00
Amitay Isaacs
3d3c15232b ctdb-system: Remove duplicate functions
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2016-06-08 10:33:19 +02:00
Amitay Isaacs
a4ac97d6c0 ctdb-daemon: Use lib/util functions instead of redefinitions
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2016-06-08 10:33:19 +02:00
Amitay Isaacs
e1e14420a6 ctdb-tests: Re-use set_blocking instead of re-definition
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2016-06-08 10:33:19 +02:00
Amitay Isaacs
3031732504 ctdb-tests: Re-use async accept wrapper from async_req
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2016-06-08 10:33:19 +02:00
Martin Schwenke
48d1fd8299 ctdb-scripts: Add eventscript 06.nfs
This generates takeip-pre and releaseip-pre call-out events.

One use is to put NFS into grace before an IP is assigned to an
interface.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 10:33:19 +02:00
Martin Schwenke
6096233919 ctdb-scripts: Move NFS callout-related code to functions file
A second NFS eventscript may be required, so make this code available
to it.

The initialisation code can't be evaluated in the functions file
because service_state_dir isn't yet setup, so put it in a function and
call it with other initialisation code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 10:33:19 +02:00
Amitay Isaacs
b7073d4021 ctdb-cluster-mutex: Fix #endif decoration
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Wed Jun  8 04:52:18 CEST 2016 on sn-devel-144
2016-06-08 04:52:18 +02:00
Martin Schwenke
174449c1e0 ctdb-recoverd: Release recovery lock on exit
The recovery lock helper must exit when it notices its parent is gone.
However, that can take a few seconds.

The usual way of terminating the recovery daemon is for the main ctdbd
to send it a SIGTERM.  Installing a handler is nice and simple.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
75717ac667 ctdb-recoverd: Add handler for lost recovery lock
If the process holding the recovery lock terminates unexpectedly then
the recovery daemon needs to know that the lock is no longer held.

While here, rename hold_reclock_handler() to take_reclock_handler() so
there is a clear difference between the two handler names.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
95a7920d22 ctdb-cluster-mutex: Register an extra handler for when mutex is lost
Pass NULL if not needed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
4f0ca0107c ctdb-cluster-mutex: ctdb_cluster_mutex() registers handler and private data
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
145ddcbe37 ctdb-cluster-mutex: Drop cluster_mutex_handler() ctdb and handle arguments
This makes the API more general.  If they are needed in a handler then
they can be in the private data.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
8cf74f335e ctdb-recovery: Wrap private data for reclock test callback
This will allow a simplification of the cluster mutex API, so the
private data can be registered when calling ctdb_cluster_mutex().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
a192364a12 ctdb-recoverd: Simplify reclock handler
Do the interesting work outside the handler.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
197264dfe7 ctdb-recoverd: Recovery lock handle should be in recovery deamon context
This shouldn't be in the CTDB context.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:29 +02:00
Martin Schwenke
5c4744e69d ctdb-cluster-mutex: Pass a talloc context to allocate the handle off
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
58be187de0 ctdb-recoverd: No need to reset reclock handler
It won't be called more than once by the cluster mutex code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
a47da8246e ctdb-cluster-mutex: Don't call the supplied hander more than once
After the first activity on the file descriptor, ignore any subsequent
activity.  Single-shot handlers are easier to write.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
630f169653 ctdb-recoverd: Fix buggy function return on memory allocation failure
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
dbd4e67aee ctdb-recoverd: Don't expose internal cluster mutex status
Just expose whether the lock was taken.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
fdd214ce6a ctdb-daemon: Rename recovery lock file to just recovery lock
It isn't necessarily a file.

Don't bother changing the control, since it doesn't pervade the code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
f16b26bc97 ctdb-daemon: Drop function ctdb_set_recovery_lock_file()
Setting the recovery lock file at startup can be done more simply.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
5b4dd8c001 ctdb-protocol: CTDB_CONTROL_SET_RECLOCK_FILE is obsolete
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
872eb37eb4 ctdb-protocol: Drop support for SET_RECLOCK
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
312521f264 ctdb-client: Remove support for SET_RECLOCK
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
1127f3ae1e ctdb-recovery: Don't update recovery lock from daemon
It can't change after startup.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
23823f128f ctdb-recovery: Don't sync recovery lock across cluster
Support for updating the recovery lock is being removed because it
isn't possible to recover from failure.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
091d4d2dbb ctdb-recovery: Consistency check reclock in start recovery control
If the recovery lock setting is not consistent with that of the
recovery master then abort.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
d3da6f0eb9 ctdb-tool: Drop support for "ctdb setreclock" command
The recovery lock can not be reliably updated at run-time.  If it
fails to update on some nodes then split-brain protection is gone and
there is no reasonable way to repair the situation.  CTDB will have to
be restarted on all nodes.  So, if this feature is being used to avoid
scheduling an outage then an outage will have to be scheduled just in
case!

To update the recovery lock, shut down CTDB on all nodes, reconfigure
the recovery lock and start CTDB again.

Those that *really* want to be able to change the recovery lock at
run-time can still do so.  Set CTDB_RECOVERY_LOCK to point to a script
and this script can then be modified at run-time.  However, please
don't report bugs if bad things happen...

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Martin Schwenke
a3c2a39ea2 ctdb-tests: Replace "ctdb setrelock" test with "ctdb getreclock" test
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-08 00:51:28 +02:00
Amitay Isaacs
97e1f53dae ctdb-packaging: Remove tevent-unix-util public library
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11946

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2016-06-06 12:26:19 +02:00
Amitay Isaacs
2082c0c3b5 lib/util: Avoid splitting tevent-unix-util as public library
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11946

Commit 670db6ac1d split tevent-util public
library to create tevent-unix-util public library for standalone ctdb
use.  This created a public library dependency between samba and ctdb
for packaging.

Bundle tevent_unix.c in public library tevent-util as before.  However,
to avoid the dependencies for packaging, standalone ctdb build will
build tevent-util as a private library with only tevent_unix.c

This simplifies any new subsystems (or libraries) which need tevent-util
and are linked in both samba and ctdb.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2016-06-06 12:26:19 +02:00
Amitay Isaacs
93dcca2a5f ctdb-recovery: Update timeout and number of retries during recovery
The timeout RecoverTimeout (default 120) is used for control messages
sent during the recovery.  If any of the nodes does not respond to any
of the recovery control messages for RecoverTimeout seconds, then it
will cause a failure of recovery of a database.  Recovery helper will
retry the recovery for a database 5 times.

In the worst case, if a database could not be recovered within 5 attempts,
a total of 600 seconds would have passed.  During this time period other
timeouts will be triggered causing unnecessary failures as follows:

1. During the recovery, even though recoverd is processing events,
   it does not send a ping message to ctdb daemon.  If a ping message is
   not received for RecdPingTimeout (default 60) seconds, then ctdb will
   count it as unresponsive recovery daemon.  If the recovery daemon
   fails for RecdFailCount (default 10) times, then ctdb daemon will
   restart recovery daemon.  So after 600 seconds, ctdb daemon will
   restart recovery daemon.

2. If ctdb daemon stays in recovery for RecoveryDropAllIPs (default 120),
   then it will drop all the public addresses.  This will cause all
   SMB client to be disconnected unnecessarily.  The released public
   addresses will not be taken over till the recovery is complete.

To avoid dropping of IPs and restarting recovery daemon during a delayed
recovery, adjust RecoverTimeout to 30 seconds and limit number of
retries for recovering a database to 3.  If we don't hear from a node
for more than 25 seconds, then the node is considered disconnected.
So 30 seconds is sufficient timeout for controls during recovery.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Mon Jun  6 08:49:15 CEST 2016 on sn-devel-144
2016-06-06 08:49:15 +02:00
Amitay Isaacs
f8141e91a6 ctdb-recoverd: Freeze databases whenever the node is INACTIVE
If the node becomes stopped or banned after recovery is marked
active, then it will never freeze the databases, and hence the
node will keep banning itself indefinitely, until ctdbd is restarted.

This is a regression from 4.3, introduced with

b4357a79d9

and

d8f3b490bb

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11945

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Wed Jun  1 17:36:12 CEST 2016 on sn-devel-144
2016-06-01 17:36:12 +02:00
Michael Adam
4b5eaf9a4e ctdb:banning: Improve debug message in ctdb_ban_node_event()
Make it more clear what happens when reading the logs.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-01 14:02:20 +02:00
Michael Adam
57cb011a0a ctdb:banning: Improve a debug message
This adapts the debug message in local_node_got_banned
to reflect what the function is currently doing.
This message was not adapted when the function was changed.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-06-01 14:02:20 +02:00
Michael Adam
c0b17c3900 ctdb:tests: timedout->timed out in 60.nfs.multi.004 test
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Jun  1 04:30:36 CEST 2016 on sn-devel-144
2016-06-01 04:30:36 +02:00
Michael Adam
2fbf19449e ctdb:eventscript: timedout->timed out in ctdb_event_script_args()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-06-01 00:57:33 +02:00
Michael Adam
b360c72eaf ctdb:banning: timedout->timed out in dbg messages in ctdb_ban_node_event()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-06-01 00:57:32 +02:00
Michael Adam
537d5fc50e ctdb:tcp: add missing spaces in debug message in ctdb_tcp_node_connect()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-06-01 00:57:32 +02:00
Martin Schwenke
4db5711494 ctdb-common: Use correct macro for checking Ethernet hardware family
Both of these expand to 1.  However, AF_LOCAL is a Unix domain socket,
which makes no sense when reading the code.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Thu May 26 11:42:46 CEST 2016 on sn-devel-144
2016-05-26 11:42:46 +02:00
Martin Schwenke
b6e75487ef ctdb-tests: Make sure empty override values are properly quoted
With an empty value the first expression adds a trailing opening
quote, so the second expression doesn't add the closing quote.  Handle
this with a special case.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-05-26 08:04:29 +02:00
Martin Schwenke
cec2c9f36c ctdb-tests: Allow local daemons to be run under valgrind
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-05-26 08:04:29 +02:00
Martin Schwenke
6cc4f78891 ctdb-scripts: Simplify ip_maskbits_iface()
This doesn't need to print the family.  Nothing uses it.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-05-26 08:04:29 +02:00
Martin Schwenke
a412f21d4b ctdb-tests: Drop no-op functions and add an ip6tables stub
Stub scripts are more obvious.  rc.local should only be used when
strictly necessary.

iptables_wrapper doesn't need to be no-op-ed, provided flock is
installed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-05-26 08:04:29 +02:00
Martin Schwenke
d78bf2ed3e ctdb-scripts: Rename get_iface_ip_maskbits_family() to get_iface_ip_maskbits()
The family is not needed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-05-26 08:04:29 +02:00
Martin Schwenke
b4878cfc4d ctdb-scripts: New functions ip_block() and ip_unblock()
Block and unblock IP addresses using these new functions.  This makes
the code more readable.

The case statement in each function is very cheap, so there is no need
to prematurely optimise and pass the family.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2016-05-26 08:04:29 +02:00
Martin Schwenke
02c2958d1d ctdb-scripts: Drop unnecessary detect_init_style() call
CTDB_INIT_STYLE isn't used in this script.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Kai Blin <kai@samba.org>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri May 20 21:06:18 CEST 2016 on sn-devel-144
2016-05-20 21:06:17 +02:00