1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-22 13:34:15 +03:00
Commit Graph

55 Commits

Author SHA1 Message Date
Ronnie Sahlberg
93d902e8f7 test of a change to make ctdbd use "status" event instead of the "monitor" event.
This allows running the actual monitoring asynchronously from ctdbd
and only using "status" to pick up the actual results.

(This used to be ctdb commit 1908bac812650ca25151051f5d86815e0b8ed319)
2009-11-13 12:37:55 +11:00
Volker Lendecke
1fa1830f81 Fix a segfault in the eventscript timeout handler.
The state was freed too early.

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit eda052101728cf922ce892e3c53b4f37e7ceac42)
2009-11-05 11:13:53 +01:00
Ronnie Sahlberg
d379b30182 create a separate context for non-monitor eventscripts so they dont collide
(This used to be ctdb commit 325de818f88f339a16dc4544e899a2d735933c44)
2009-10-28 17:35:15 +11:00
Ronnie Sahlberg
f8a8c0d6e4 return 0 in the event script callback if it was aborted by a different script
(This used to be ctdb commit 8d5cb2586a1d5a0255cc18295430927b914d4527)
2009-10-28 16:40:31 +11:00
Ronnie Sahlberg
e07ca41886 change the eventscript handling to allow EventScriptTimeout for each individual script isntead of for the entire set of scripts
restructure the talloc hierarchy to allow this

(This used to be ctdb commit 64da4402c6ad485f1d0a604878a7b0c01a0ea5f0)
2009-10-28 16:11:54 +11:00
Ronnie Sahlberg
3526bc830d Enhance the logging fromeventscripts.
When a single script is finished, also log the name of the script, the duration it took and the return status.

In the loop where we signal back to the main daemon that the script finished, do this once every 100ms instead of once every 1 second

(This used to be ctdb commit 6a1f7a7b1b3a0b8f89998db8fdad83bbb4e9b5a5)
2009-10-28 09:07:43 +11:00
Ronnie Sahlberg
1d7681709b dont run the monitor event so frequently after a event has failed.
use _exit() instead of exit() when terminating an eventscript.

(This used to be ctdb commit cc30ee2f4f33cb75b2be980c2d4dff6c7c23852f)
2009-10-27 13:51:45 +11:00
Ronnie Sahlberg
c61c655769 when scripts timeout, log pstree to a file in /tmp and just log the filename in the messages file
(This used to be ctdb commit 0785afba8e5cd501b9e0ecb4a6a44edf43b57ab0)
2009-10-23 13:55:21 +11:00
Ronnie Sahlberg
902c476c03 From Volker L
Fix some warnings  and an incorrect check for a talloc failure

(This used to be ctdb commit 27296a47b3d057a6729287acf128b2b67775ecde)
2009-10-22 12:19:40 +11:00
Ronnie Sahlberg
d5fd4fc0ce During tests it is common to add/delete test eventscripts at runtime.
This can race with teh eventascript handling that does a :

list all scripts,   sort them,  then execute them

so trap status code 127 which means the script could not be executed (or /bin/sh does not exist) and treat it as not to cause the node to become unhealthy

(This used to be ctdb commit befabc917edb036ca81f5216f65a6d62b26ee83e)
2009-10-21 16:50:39 +11:00
Ronnie Sahlberg
a92ba7f729 lower the debug levels for the "create FD messages" so we dont fill up the logs.
(This used to be ctdb commit 87146db2769c2ec494813685bf9cec0d2a6336c3)
2009-10-21 15:26:24 +11:00
Ronnie Sahlberg
d788dd3627 From wolfgang Mueller
Add a tuneable so that when scripts starts to hang/timeout, we can make the node unhealthy instead of banned

(This used to be ctdb commit 2e9fc6f0609833c6d8146196011ef780669d615d)
2009-10-20 12:59:48 +11:00
Ronnie Sahlberg
9de3652380 add logging everytime we create a filedescriptor in the main ctdb daemon
so we can spot if there are leaks.

plug two leaks for filedescriptors related to when sending ARP fail
and one leak when we can not parse the local address during tcp connection establish

(This used to be ctdb commit ddd089810a14efe4be6e1ff3eccaa604e4913c9e)
2009-10-15 11:24:54 +11:00
Ronnie Sahlberg
c58a6b39a6 add more debugging output to eventscripts and when a script has timed out,
print a full "pstree -p" to the log.

Example :
        |-ctdbd(29826)-+-ctdbd(29862)
        |              `-ctdbd(31897)-+-00.ctdb(31898)---sleep(31908)

change the default timeout to 60 seconds for eventscripts

(This used to be ctdb commit a3406c10d70f89d332eab25d481083142dff987d)
2009-10-14 14:14:28 +11:00
Ronnie Sahlberg
05137e4718 Fix bug spotted by Metze,
the argument to ctdb_control_event_Script_disabled() is a string not a uint32

(This used to be ctdb commit 687535b51622d1fac7ccb38fa640bf1febd69fd8)
2009-10-09 22:22:11 +11:00
Ronnie Sahlberg
f8334e2f68 we should close this file on exec
(This used to be ctdb commit c1c0ebb8da9a6c29ee83868a311f07f30cb4ed16)
2009-10-02 13:41:54 +10:00
Ronnie Sahlberg
e578bed20d dont force an election just because the ban flag differs across the cluster.
a simple push to resync this flag is sufficient

(This used to be ctdb commit 8903b858ddd3a016d9cf765187839814443a67ca)
2009-09-09 10:57:39 +10:00
Ronnie Sahlberg
cda5f02c7c new prototype banning code
(This used to be ctdb commit 0c4c2240267af183d54ffd4c0aacda208f6eff6a)
2009-09-04 02:20:39 +10:00
Ronnie Sahlberg
1cc79905ad add new controls to make it possible to enable/disable individual eventscripts
update scriptstatus output so it lists disabled scripts

(This used to be ctdb commit 7e799b7523c9699bd65a8a8207f7e03d668b0b81)
2009-08-13 13:04:08 +10:00
Ronnie Sahlberg
e5e9fc48b1 create a new event : stopped.
This event is called when a node is stopped and is used by eventscripts that need to do certain cleanup and removal of configuration or ip addresses or routing ...

Note that a STOPPED node is considered "inactive" and as such will not be running the "recovered" event when the rest of the cluster has recovered.

(This used to be ctdb commit 65e9309564611bf937ded3c74a79abff895d7c59)
2009-07-17 12:26:16 +10:00
Ronnie Sahlberg
5371e3a793 lower the loglevel when we long that we skip an evenscript because it is not executable
(This used to be ctdb commit c265df3c7950aab51b8b6ef17040229b97345c35)
2009-06-01 15:29:36 +10:00
Sumit Bose
2fcedf6dac add missing checks on so far ignored return values
Most of these were found during a review by Jim Meyering <meyering@redhat.com>

(This used to be ctdb commit 3aee5ee1deb4a19be3bd3a4ce3abbe09de763344)
2009-05-21 11:22:21 +10:00
Ronnie Sahlberg
a87e6f56ae we only need to switch into client mode from the eventscript child if we are running the monitor event
(This used to be ctdb commit 13e2c9044950f21918e4610726e73ed3d8f76920)
2009-04-06 14:03:09 +10:00
Ronnie Sahlberg
1f87ee85bc use _exit() and not exit() when we terminate a failed eventscript child process
(This used to be ctdb commit 33b296cee177adc61edc911caec8c24b3efa8441)
2009-04-06 13:16:36 +10:00
root
629d5ee1fa add a new command "ctdb scriptstatus"
this command shows which eventscripts were executed during the last monitoring cycle and the status from each eventscript.

If an eventscript timedout or returned an error we also
show the output from the eventscript.

Example :
[root@rcn1 ctdb-git]# ./bin/ctdb scriptstatus
6 scripts were executed last monitoring cycle
00.ctdb              Status:OK    Duration:0.021 Mon Mar 23 19:04:32 2009
10.interface         Status:OK    Duration:0.048 Mon Mar 23 19:04:32 2009
20.multipathd        Status:OK    Duration:0.011 Mon Mar 23 19:04:33 2009
40.vsftpd            Status:OK    Duration:0.011 Mon Mar 23 19:04:33 2009
41.httpd             Status:OK    Duration:0.011 Mon Mar 23 19:04:33 2009
50.samba             Status:ERROR    Duration:0.057 Mon Mar 23 19:04:33 2009
   OUTPUT:ERROR: Samba tcp port 445 is not responding

Add a new helper function "switch_from_server_to_client()" which both
the recovery daemon can use as well as in the child process we start for running the actual eventscripts.

Create several new controls, both for the eventscript child process to inform the master daemon of the current status of the scripts as well as for the ctdb tool to extract this information from the runninc daemon.

(This used to be ctdb commit c98f90ad61c9b1e679116fbed948ddca4111968d)
2009-03-23 19:07:45 +11:00
Ronnie Sahlberg
b9bd20ce55 add a context and a timed event so that once we have been in recovery
mode for too long we drop all public ip addresses

(This used to be ctdb commit 403c68f96e1380dd07217c688de2730464f77ea0)
2008-10-22 11:04:41 +11:00
Ronnie Sahlberg
5808a7be96 allow multiple eventscripts using the same prefix.
this eases the pain for users that use out of tree eventscripts

(This used to be ctdb commit 8313dfb6fc5404cd2d065af6620412f8664ada11)
2008-10-16 17:57:50 +11:00
Ronnie Sahlberg
0964c59dc6 Do not allow "ctdb eventscript" to start new eventscripts while we are in recovery mode
(This used to be ctdb commit 8140825e1d06053a900fd0adf0a150622c0fc146)
2008-07-17 09:04:15 +10:00
Ronnie Sahlberg
66222af5e4 Fix a very subtle race where we could get a double free of a talloced
memory if ctdb_run_eventscript() would be called
during processing of ctdb_event_script_timeout() for
user unvoked eventscripts. (eventsccripts invoked by "ctdb eventscript ...")

Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>

(This used to be ctdb commit 15bc66ae801b0c69a65a7a2acf5df151e76edc2a)
2008-07-11 10:33:46 +10:00
Ronnie Sahlberg
334db8ccba proper waitpid() fix.
remove all waitpid() calls and use the event system to trap sigchld

(This used to be ctdb commit 77458b2b6b51b2970c12b0e5b097088d3fb9d358)
2008-07-09 14:02:54 +10:00
Ronnie Sahlberg
522830dea8 Revert "waitpid() can block if it takes a long time before the child terminates"
This reverts commit bfba5c7249eff8a10a43b53c1b89dd44b625fd10.

revert the waitpid changes.   we need to waitpid for some childredn so should
refactor the approach completely

(This used to be ctdb commit 702ced6c2fe569c01fe96c60d0f35a7e61506a96)
2008-07-08 17:41:31 +10:00
Ronnie Sahlberg
d67de4a7d2 waitpid() can block if it takes a long time before the child terminates
so we should not call it from the main daemon.

1, set SIGCHLD to SIG_DFL to make sure we ignore this signal

2, get rid of all waitpid() calls

3, change reporting of event script status code from _exit()/waitpid()   to write()/read() one byte across the pipe.

(This used to be ctdb commit bfba5c7249eff8a10a43b53c1b89dd44b625fd10)
2008-07-08 03:48:11 +10:00
Ronnie Sahlberg
6bfbec28a4 use more libral handling of event scripts timing out.
If the event script that timed out was for the "monitor" event, then
even if it timed out we still return SUCCESS back to the guy invoking the eventscript.
Only consider the eventscript for "monitor" to have failed with an error
IFF it actually terminated with an error,   or if it timed out 5 times in a row and hung.

(This used to be ctdb commit 60f3c04bd8b20ecbe937ffed08875cdc6898b422)
2008-07-07 20:38:59 +10:00
Ronnie Sahlberg
779468ab3f if the event scripts hangs EventScriptsBanCount consecutive times in a row
the node will ban itself for the default recovery ban period

(This used to be ctdb commit 7239d7ecd54037b11eddf47328a3129d281e7d4a)
2008-06-13 13:18:06 +10:00
Ronnie Sahlberg
30535c815d when a eventscript has timed out, log the event options (i.e. "monitor" "takeip 1.2..." etc)
to the log

(This used to be ctdb commit dbe31581abf35fc4a32d3cbf487dd34e2b9c937a)
2008-06-13 12:18:00 +10:00
Andrew Tridgell
8ec3665231 put the return in the right place
We were refusing the 'startrecovery' event

(This used to be ctdb commit 788d38812d73729f11d12e9812b16092c0ae4123)
2008-05-14 22:05:09 +10:00
Andrew Tridgell
e465110f95 Fix the chicken and egg problem with ctdb/samba and a registry smb.conf
This attempts to fix the problem of ctdb event scripts blocking due to
attempted access to the ctdb databases during recovery. The changes are:

  - now only the 'shutdown' and 'startrecovery' events can be called
    with the databases locked in recovery. The event scripts must ensure
    that for these two events no database access is attempted

  - the recovered, takeip and releaseip events could previously be called
    inside a recovery. The code now ensures that this doesn't happen, delaying
    the events till after recovery has finished

  - the 50.samba event script now avoids using testparm unless it is really
    needed

This needs extensive testing.

(This used to be ctdb commit e3cdb8f2be6a44ec877efcd75c7297edb008a80b)
2008-05-14 20:57:04 +10:00
Ronnie Sahlberg
e8e67ef576 add a mechanism to force a node to run the eventscripts with arbitrary arguments
ctdb eventscript "command argument argument ..."

(This used to be ctdb commit 118a16e763d8332c6ce4d8b8e194775fb874c8c8)
2008-04-02 11:13:30 +11:00
Andrew Tridgell
f6e53f433b merge from ronnie
(This used to be ctdb commit e7b57d38cf7255be823a223cf15b7526285b4f1c)
2008-02-04 20:07:15 +11:00
Andrew Tridgell
9d6ac0cf55 added debug constants to allow for better mapping to syslog levels
(This used to be ctdb commit 7ba8f1dde318eab03f4257e5a89fd23e7281e502)
2008-02-04 17:44:24 +11:00
Ronnie Sahlberg
12ebb74838 change how we do public addresses and takeover so that we can have
multiple public addresses spread across multiple interfaces on each 
node.

this is a massive patch since we have previously made the assumtion that 
we only have one public address per node.

get rid of the public_interface argument.  the public addresses file 
now explicitely lists which interface the address belongs to

(This used to be ctdb commit 462ebbc791e906a6b874c862defea43235597ca8)
2007-09-04 09:50:07 +10:00
Ronnie Sahlberg
794fb10634 add an extra debug statement when we send a SIGTERM to a process
(This used to be ctdb commit a9c1be9cf9efdc69bfc95657b70e9f8b8230cda8)
2007-08-27 17:33:46 +10:00
Ronnie Sahlberg
b582e13cae make sure that the event script is executable and just ignore it
othervise

(This used to be ctdb commit 65eb7845c70489d654acaaf99cd2c8eac7df11dc)
2007-08-21 09:22:14 +10:00
Andrew Tridgell
405e123ffb removed redundent debug message
(This used to be ctdb commit 9ee742b7cc43be7da6b568308912a3f2cfe4f4d3)
2007-08-20 11:13:38 +10:00
Andrew Tridgell
46639ac19e merged new event script calling code from ronnnie
(This used to be ctdb commit bbacad61b3eee4276ffe44ed2a23949aca8152cf)
2007-08-20 11:10:30 +10:00
Ronnie Sahlberg
7322e82bcb add text to the event script timeout log on how to find out which script
timed out

(This used to be ctdb commit bd6db995fb00ed45c5f0a50bbe6cf5d0fe22a194)
2007-08-15 15:08:42 +10:00
Ronnie Sahlberg
3b9d50f3ee change the now rather small /etc/ctdb/events script into a service
specific script /etc/ctdb/events.d/00.ctdb

get rid of CTDB_EVENTS_SCRIPT and --event-script

(This used to be ctdb commit 81ccfaf838e5772d4a58eb6a70224b7b39aba9f3)
2007-08-15 15:01:31 +10:00
Ronnie Sahlberg
ff58f7c7ea add a comment that the talloc_free also removes the script from the tree
(This used to be ctdb commit ce71f6e9cf983cc4fe66935ad6c18d55dfed03a5)
2007-08-15 14:46:06 +10:00
Ronnie Sahlberg
4023576e50 call the service specific event scripts directly from the forked child
instead for from /etc/ctdb/events so that we can get better debugging 
output in the logs when something fails in the scripts

(This used to be ctdb commit 4ed96b768aea1611e8002f7095d3c4d12ccf77a3)
2007-08-15 14:44:03 +10:00
Ronnie Sahlberg
5a02262a06 comment that ctdb_event_script_v() is called from a forked childs
context and thus can make blocking calls

(This used to be ctdb commit b31d98281f15995ad340d2510e08e04ed46e271a)
2007-08-15 10:48:10 +10:00