1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

454 Commits

Author SHA1 Message Date
Ronnie Sahlberg
e28c652cca Dont store debug level DEBUG_DEBUG in the in-memory ringbuffer.
It is unlikely we will need something this verbose for normal troubleshooting.
This allows us to keep a significantly longer time interval of log messages
in the 500k slots available in the ringbuffer.

(This used to be ctdb commit cc99c05c0c6484ad574039a454e6133852cb41fa)
2009-12-04 11:45:37 +11:00
Ronnie Sahlberg
6bad4a4836 Add a proper function to process a process-exist control in the daemon.
This controls is only used by samba when samba wants to check if a subrecord held by a <node-id>:<smbd-pid> is still valid or if it can be reclaimed.

If the node is banned or stopped, we kill the smbd process and return that the process does not exist to the caller. This allows us to recover subrecords from stopped/banned nodes where smbd is hung waiting for the databases to thaw.

bz58185

(This used to be ctdb commit 157807af72ed4f7314afbc9c19756f9787b92c15)
2009-12-02 13:58:27 +11:00
Ronnie Sahlberg
1c7de7a2ed Add a double linked list to the ctdb_context to store a mapping between client pids and client structures.
Add the mapping to the list everytime we accept() a new client connection
and set it up to remove in the destructor when the client structure is freed.

(This used to be ctdb commit f75d379377f5d4abbff2576ddc5d58d91dc53bf4)
2009-12-02 13:41:04 +11:00
Ronnie Sahlberg
569001afd0 Merge commit 'martins/status-test-2'
Conflicts:

	server/eventscript.c

(This used to be ctdb commit e9b3477a5b9a2eff18f727e7d59338bfb5214793)
2009-12-01 10:53:18 +11:00
Martin Schwenke
a64ccf07c1 Add flag to ctdb_event_script_callback indicating when called by client.
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit a1d654a982ca56fade82552f4e6b5586236d3233)
2009-11-26 15:49:49 +11:00
Rusty Russell
3188df4a88 eventscript: check that ctdb forced script events correct
Now we're doing checking, we might as well make sure the commands from
"ctdb eventscripts" are valid.

This gets rid of the "UNKNOWN" event type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


(This used to be ctdb commit 1d24a3869fe89fc9a109fd9e9b69df5fc665a5f6)
2009-11-25 11:02:29 +10:30
Rusty Russell
2d9254404d eventscript: introduce enum for different event script calls.
Rather than doing strcmp everywhere, pass an explicit enum around.  This
also subtly documents what options are available.  The "options" arg
is now used for extra arguments only.

Unfortunately, gcc complains on empty format strings, so we make
ctdb_event_script() take no varargs, and add ctdb_event_script_args().  We
leave ctdb_event_script_callback() taking varargs, which means callers
have to do "%s", "".

For the moment, we have CTDB_EVENT_UNKNOWN for handling forced scripts
from the ctdb tool.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


(This used to be ctdb commit 8001488be4f2beb25e943fe01b2afc2e8779930d)
2009-11-24 11:16:49 +10:30
Rusty Russell
2763df22de eventscript: put timeout inside ctdb_event_script_callback_v
Everyone uses the same timeout value, so just remove it from the API.
If we ever need variable timeouts, that might as well be central too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


(This used to be ctdb commit 533c3e053293941d2a9484b495e78d45f478bb08)
2009-11-24 11:09:46 +10:30
Ronnie Sahlberg
e6b69fa760 rework and simplify the eventscript handling
This version has no trailing whitespace, and fixed 


(This used to be ctdb commit defbe318152fc479e8076ad70433cdb4971951af)
2009-11-25 11:00:11 +10:30
Ronnie Sahlberg
ae209c74c8 dont reset the event script context everytime we start a new "ctdb eventscript ..."
command.
Use the existing context used for non-monitor events

Multiple concurrent uses of "ctdb eventscript ..." could otherwise lead to a SEGV

(This used to be ctdb commit 80a8d728e9680040e00d24361dfc9367dd372a56)
2009-11-19 11:03:51 +11:00
Ronnie Sahlberg
cc2d81a77c make the ringbuffer logging more efficient and marshall the data by writing to a tmpfile instead of continously talloc resizing a blob
(This used to be ctdb commit 6427f0b68d60b556a023f64e15e156000ba6f943)
2009-11-18 19:10:50 +11:00
Ronnie Sahlberg
bc2675119d add an in memory ringbuffer where we store the last 500000 log entries regardless of log level.
add commandt to extract this in memory buffer and to clear it

(This used to be ctdb commit 29d2ee8d9c6c6f36b2334480f646d6db209f370e)
2009-11-18 12:44:18 +11:00
Ronnie Sahlberg
61de178e0a set up a pipe betweent he main daemon and the child we use for syslogling so that we can clean up the childprocess when we stop ctdbd
(This used to be ctdb commit cb8df973ccd446d87fbdd9a27843e54841ba5d89)
2009-11-16 15:17:32 +11:00
Ronnie Sahlberg
93d902e8f7 test of a change to make ctdbd use "status" event instead of the "monitor" event.
This allows running the actual monitoring asynchronously from ctdbd
and only using "status" to pick up the actual results.

(This used to be ctdb commit 1908bac812650ca25151051f5d86815e0b8ed319)
2009-11-13 12:37:55 +11:00
Ronnie Sahlberg
e33722a569 start the syslog child a little later, after we have forked and detached from the local shell
(This used to be ctdb commit 9ffd54b73c0d64b67e8e736d7cb54490e77ffa78)
2009-10-30 19:39:11 +11:00
Ronnie Sahlberg
5d73f19418 create a child process to write to syslog.
use a udp socket on the ctdbd port to send messages to teh syslog child process for loggign.

we need this when syslog becomes "slow",   like very slow, and on boxes where syslog is limited to 100 lines per second and starts to block after that

(This used to be ctdb commit 1446f4c247310e2ff2d522055bd8927d1a78d017)
2009-10-30 18:53:17 +11:00
Michael Adam
0113744fec server: trans2_active: don't report a transaction active on the node that performs the transaction
Otherwise a node can lock itself out, e.g. when a commit control times out...

Michael

(This used to be ctdb commit cb432e30351d5e5a41e98da3c7b1c2a4d400a3a2)
2009-10-30 09:22:18 +11:00
Ronnie Sahlberg
023d09cd38 Revert "update the "uptime" command to indicate the "time since last" is the time since the last recovery OR failover."
This reverts commit 3b0d44497800a16400d05a30bdaf6e6c285d4b36.

(This used to be ctdb commit cb36bbb5418290e8e5b770d2d836285b15da2a6f)
2009-10-29 10:49:00 +11:00
Ronnie Sahlberg
279b7ca564 update the "uptime" command to indicate the "time since last" is the time since the last recovery OR failover.
(This used to be ctdb commit 3b0d44497800a16400d05a30bdaf6e6c285d4b36)
2009-10-29 10:37:10 +11:00
Michael Adam
abac42ca34 server: add a new ctdb control CTDB_TRANS2_ACTIVE
This aske the daemon wheter a transaction is currently active on a
given DB on that node. More precisely this asks for the transaction_active
flag in the ctdb_db_context that is set in the CTDB_TRANS2_COMMIT
control and cleared in the CTDB_TRANS2_ERROR or CTDB_TRANS2_FINISHED controls.

This will be useful for fixing race conditions in the transaction code.

Michael

(This used to be ctdb commit 8d430ae6968dfe566614379436fc3c56003fcd88)
2009-10-29 10:14:30 +11:00
Ronnie Sahlberg
d379b30182 create a separate context for non-monitor eventscripts so they dont collide
(This used to be ctdb commit 325de818f88f339a16dc4544e899a2d735933c44)
2009-10-28 17:35:15 +11:00
Ronnie Sahlberg
e07ca41886 change the eventscript handling to allow EventScriptTimeout for each individual script isntead of for the entire set of scripts
restructure the talloc hierarchy to allow this

(This used to be ctdb commit 64da4402c6ad485f1d0a604878a7b0c01a0ea5f0)
2009-10-28 16:11:54 +11:00
Ronnie Sahlberg
4d40b86805 for debugging
add a global variable holding the pid of the main daemon.
change the tracking of time() in the event loop to only check/warn when called from the main daemon

(This used to be ctdb commit a10fc51f4c30e85ada6d4b7347b0f9a8ebc76637)
2009-10-27 13:18:52 +11:00
Ronnie Sahlberg
86d1b4c465 Add a mechanism where we can register notifications to be sent out to a SRVID when the client disconnects.
The way to use this is from a client to :
1, first create a message handle and bind it to a SRVID
   A special prefix for the srvid space has been set aside for samba :
   Only samba is allowed to use srvid's with the top 32 bits set like this.
   The lower 32 bits are for samba to use internally.

2, register a "notification" using the new control :
                    CTDB_CONTROL_REGISTER_NOTIFY         = 114,
   This control takes as indata a structure like this :
struct ctdb_client_notify_register {
        uint64_t srvid;
        uint32_t len;
        uint8_t notify_data[1];
};

srvid is the srvid used in the space set aside above.
len and notify_data is an arbitrary blob.
When notifications are later sent out to all clients, this is the payload of that notification message.

If a client has registered with control 114 and then disconnects from ctdbd, ctdbd will broadcast a message to that srvid to all nodes/listeners in the cluster.

A client can resister itself with as many different srvid's it want, but this is handled through a linked list from the client structure so it mainly designed for "few notifications per client".

3, a client that no longer wants to have a notification set up can deregister using control
                    CTDB_CONTROL_DEREGISTER_NOTIFY       = 115,
which takes this as arguments :
struct ctdb_client_notify_deregister {
        uint64_t srvid;
};

When a client deregisters, there will no longer be sent a message to all other clients when this client disconnects from ctdbd.

(This used to be ctdb commit f1b6ee4a55cdca60f93d992f0431d91bf301af2c)
2009-10-23 15:24:51 +11:00
Ronnie Sahlberg
9b8c72c446 When clients have blocked, perhaps because the node is banned or stopped and the client is blocked trying to tdb_fetch() a record, make sure we dont queue up too many REQ_MESSAGES.
Add a new tunable to control the maximum queue size we allow to a blocked client before we start discarding REQ_MESSAGES instead of queueing them for delivery.

    This avoids having queued up very very large number of MESSAGES that samba semds
     between eachother to nodes that are blocked/banned/stopped for extended periods
    .

(This used to be ctdb commit f76d6fed8f9630450263b9fa4b5fdf3493fb1e11)
2009-10-21 15:20:55 +11:00
Ronnie Sahlberg
d788dd3627 From wolfgang Mueller
Add a tuneable so that when scripts starts to hang/timeout, we can make the node unhealthy instead of banned

(This used to be ctdb commit 2e9fc6f0609833c6d8146196011ef780669d615d)
2009-10-20 12:59:48 +11:00
Ronnie Sahlberg
80be59d35e when we change state between healthy/unhealthy, make sure we ask the recovery
master to perform an explicit ip reallocation.

This is more reliable and faster than having the recovery dameon track these
changes, and since we now have an explicit method to ask the recovery daemon
to perform an explicit ip reallocation, we should use this.

(This used to be ctdb commit 3807681e74f4bfe92befdae6ed616ff5f1a99880)
2009-10-14 11:59:16 +11:00
Ronnie Sahlberg
122c423b82 add a new control for explicitely cancelling recovery transactions, i.e. the
transactions we start across all tdb databased during the recovery.

this allows us to properly clean up and delete these tdb transactions on a
recovery failure.

(This used to be ctdb commit b2ce8b900a7d00944c84e0574fea5b371064a06d)
2009-10-12 16:48:05 +11:00
Ronnie Sahlberg
73c0adb029 initial attempt at freezing databases in priority order
(This used to be ctdb commit e8d692590da1070c87a4144031e3306d190ebed2)
2009-10-12 12:08:39 +11:00
Ronnie Sahlberg
d4c98516a2 uptade the freeze/thaw commands to be able to send the requested database priority to freeze/thaw to the daemon.
this is encoded in the srvid field of the request header

(This used to be ctdb commit 0cb3d33caa42ed783e03bc825b181dde4cf63616)
2009-10-12 09:22:17 +11:00
Ronnie Sahlberg
3219f81710 add a control to read the db priority from a database
(This used to be ctdb commit ca6d045e419f308f57e74d4c978907afb05ddb85)
2009-10-10 15:04:18 +11:00
Ronnie Sahlberg
6cf7d8e131 add a control to set a database priority. Let newly created databases default to priority 1.
database priorities will be used to control in which order databases are locked during recovery in.

(This used to be ctdb commit 67741c0ee01916d94cace8e9462ef02507e06078)
2009-10-10 14:26:09 +11:00
Ronnie Sahlberg
166b1c97b4 add a new message to ask the recovery daemon to temporarily disable checking ip address consistency.
This is useful when we are moving addresses using moveip in the cluster since otherwise if we collide with the recovery daemons own check we could cause a recovery

(This used to be ctdb commit 9c63858c0b22c81eaccb9865a414af0bbb2833d4)
2009-10-06 12:11:32 +11:00
Ronnie Sahlberg
71e4259150 add a new function to collect a list of all active nodes EXCEPT a certain node
(This used to be ctdb commit be52954d921e7d443304cf49fbd488c619a9c4ec)
2009-10-06 10:52:31 +11:00
Ronnie Sahlberg
c971d934a9 From Wolfgang Mueller-Friedt
Remove the explicit vacuum/repack commands from the 00.ctdb eventscript
and implement this in the ctdb daemon.

Combine vacuuming and repacking into one
cheap read traverse to enumerate all candidate records
and one write traverse that both repacks the database and also deletes the record locally where we are lmaster and where the records have already been deleted remotely.

this code also adds initial autotuning heuristics for the vacuum intervals and how many records to delete in each iteration.

minor stylish changes made by ronnie s

(This used to be ctdb commit 95a3ee551241aa164967991fe5efe078e1714bde)
2009-09-29 13:27:19 +10:00
Ronnie Sahlberg
cda5f02c7c new prototype banning code
(This used to be ctdb commit 0c4c2240267af183d54ffd4c0aacda208f6eff6a)
2009-09-04 02:20:39 +10:00
Ronnie Sahlberg
1cc79905ad add new controls to make it possible to enable/disable individual eventscripts
update scriptstatus output so it lists disabled scripts

(This used to be ctdb commit 7e799b7523c9699bd65a8a8207f7e03d668b0b81)
2009-08-13 13:04:08 +10:00
Wolfgang Mueller-Friedt
16af87bf25 repack limit tunable
Signed-off-by: Wolfgang Mueller-Friedt <wolfmuel@de.ibm.com>

(This used to be ctdb commit a2768b0732f2ab2e3fafda55587bd2e99eedf0fa)
2009-07-29 13:30:39 +10:00
Ronnie Sahlberg
fb63d27e4e initial part of new vacuuming patch.
create some new fields for ctdb_db and tunables

(This used to be ctdb commit 3a8e7d36cc42aedf4b7665364224140dcbfb3efa)
2009-07-29 13:25:43 +10:00
Michael Adam
4cd06a330e Fix persistent transaction commit race condition.
In ctdb_client.c:ctdb_transaction_commit(), after a failed
TRANS2_COMMIT control call (for instance due to the 1-second
being exceeded waiting for a busy node's reply), there is a
1-second gap between the transaction_cancel() and
replay_transaction() calls in which there is no lock on the
persistent db. And due to the lack of global state
indicating that a transaction is in progress in ctdbd, other nodes
may succeed to start transactions on the db in this gap and
even worse work on top of the possibly already pushed changes.
So the data diverges on the several nodes.

This change fixes this by introducing global state for a transaction
commit being active in the ctdb_db_context struct and in a db_id field
in the client so that a client keeps track of _which_ tdb it as
transaction commit running on. These data are set by ctdb upon
entering the trans2_commit control and they are cleared in the
trans2_error or trans2_finished controls. This makes it impossible
to start a nother transaction or migrate a record to a different
node while a transaction is active on a persistent tdb, including
the retry loop.

This approach is dead lock free and still allows recovery process
to be started in the retry-gap between cancel and replay.
Also note, that this solution does not require any change in the
client side.

This was debugged and developed together with
Stefan Metzmacher <metze@samba.org> - thanks!

Michael

(This used to be ctdb commit f88103516e5ad723062fb95fcb07a128f1069d69)
2009-07-29 11:12:39 +10:00
Ronnie Sahlberg
37d68c58b8 add two commands : setlmasterrole and setrecmasterrole to enable/disable these capabilities at runtime
(This used to be ctdb commit 51aaed0e9e42e901451292e8dd545297ab725a62)
2009-07-28 13:45:13 +10:00
Ronnie Sahlberg
72e2380e92 add a command "setnatgwstate {on|off}" that can be used to indicate if this node is using natgw functionality or not.
(This used to be ctdb commit 89a9bb29a60a6fb1fba55987e6cf0a4baa695e50)
2009-07-28 09:58:11 +10:00
Ronnie Sahlberg
e5e9fc48b1 create a new event : stopped.
This event is called when a node is stopped and is used by eventscripts that need to do certain cleanup and removal of configuration or ip addresses or routing ...

Note that a STOPPED node is considered "inactive" and as such will not be running the "recovered" event when the rest of the cluster has recovered.

(This used to be ctdb commit 65e9309564611bf937ded3c74a79abff895d7c59)
2009-07-17 12:26:16 +10:00
Ronnie Sahlberg
88f3c40d9c add two new controls, CTOP_NODE and CONTINUE_NODE
that are used to stop/continue a node instead of using modflags messages

(This used to be ctdb commit 54b4a02053a0f98f8c424e7f658890254023d39a)
2009-07-09 12:22:46 +10:00
Ronnie Sahlberg
66c8d4fb3d make it possible to start the daemon in STOPPED mode
(This used to be ctdb commit 866aa995dc029db6e510060e9e95a8ca149094ac)
2009-07-09 11:57:20 +10:00
Ronnie Sahlberg
9f0dc4b93b Add a new node flag : STOPPED
This node flag means the node is DISABLED and that all its public ip addresses
are failed over, but also that it has been removed from the VNNmap.

A STOPPED node should be in recovery mode active untill restarted using the continue command.

Adding two new commands "ctdb stop" "ctdb continue"

(This used to be ctdb commit d47dab1026deba0554f21282a59bd172209ea066)
2009-07-09 11:38:18 +10:00
Ronnie Sahlberg
289c58e9b6 add a new command "ctdb ipreallocate", this command will force the recovery master to perform a full ip reallocation process.
the ctdb command will block until the ip reallocation has comleted

(This used to be ctdb commit abad7b97fe0c066b33f6e75d0953bbed892a3216)
2009-07-02 13:00:26 +10:00
Ronnie Sahlberg
93026f4cbf update the handling of debug levels so that we always can use a literal instead of a numeric value.
validate the input values used and refuse setting the debug level to an unknown value

(This used to be ctdb commit daec49cea1790bcc64599959faf2159dec2c5929)
2009-07-01 09:17:13 +10:00
Ronnie Sahlberg
5b235c3999 add a control to set the reclock file
(This used to be ctdb commit 36cc2e586f03fa497ee9b06f3e6afc80219c4aaa)
2009-06-25 14:25:18 +10:00
Ronnie Sahlberg
2b253c094c add a control to read the current reclock file from a node
(This used to be ctdb commit ed6a4cbcdcbb4e0df83bec8be67c30288bf9bd41)
2009-06-25 12:17:19 +10:00