1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-27 14:04:05 +03:00

1220 Commits

Author SHA1 Message Date
Martin Schwenke
1ab2bbb349 recoverd: Backward compatibility for nodes without IPREALLOCATED control
Consider the case of upgrading a cluster node by node, where some
nodes are still running older versions of CTDB without the
IPREALLOCATED control.  If a "new" node takes over as recovery master
and a failover occurs, then it will attempt to send IPREALLOCATED
controls to all nodes.  The "old" nodes will fail in a fairly
nondescript way (result == -1).

To try to handle this situation, fall back to the EVENTSCRIPT control
to handle "ipreallocated".  Only do this on the failed nodes.
However, do not do this on nodes that timed out (they've probably
implemented the control and we should call the regular fail_callback
to get those nodes banned) or for stopped nodes (since they can't
actually run the "ipreallocated" event via the EVENTSCRIPT control).

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit b2654853ce9b7c18c5874b080bc94d3118078a5d)
2013-05-27 15:15:25 +10:00
Amitay Isaacs
a002c6ec12 vacuum: Reduce the priority of non-critical error
Since the complete database is not locked when the receive_records
control is received, it's possible that we may not be able to obtain
lock on a chain.  We will try again to store this record.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 32723c9efdad1c6ca4aa53f308ccd9bef1aadfff)
2013-05-24 14:22:16 +02:00
Michael Adam
d1dd29197e ctdbd: fix comment explaining redirection of CTDB_REQ_CALL redirection.
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit b697625b184227dad1be31a41b7a3fd9bd312e29)
2013-05-24 22:06:24 +10:00
Michael Adam
3f03a3c8a3 ctdbd: remove a nonempty blank line
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit d9e24782a90d9ce29c0e6584b75d2b186142174d)
2013-05-24 22:06:21 +10:00
Michael Adam
a0b20771fe ctdbd: update comment describing ctdb_call_send_redirect()
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 9a21d417c51fb9cad8f2e87e00ca54d379aef860)
2013-05-24 22:06:16 +10:00
Martin Schwenke
f35e9bba9b recoverd: Nodes can only takeover IPs if they are in runstate RUNNING
Currently the order of the first IP allocation, including the first
"ipreallocated" event, and the "startup" event is undefined.  Both of
these events can (re)start services.

This stops IPs being hosted before the "startup" event has completed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f15dd562fd8c08cafd957ce9509102db7eb49668)
2013-05-24 16:27:55 +10:00
Martin Schwenke
7f03618ae4 recoverd: Handle errors carefully when fetching tunables
If a tunable is not implemented on a remote node then this should not
be fatal.  In this case the takeover run can continue using benign
defaults for the tunables.

However, timeouts and any unexpected errors should be fatal.  These
should abort the takeover run because they can lead to unexpected IP
movements.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit c0c27762ea728ed86405b29c642ba9e43200f4ae)
2013-05-24 16:27:55 +10:00
Martin Schwenke
116f62a7b3 recoverd: Set explicit default value when getting tunable from nodes
Both of the current defaults are implicitly 0.  It is better to make
the defaults obvious.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 1190bb0d9c14dc5889c2df56f6c8986db23d81a1)
2013-05-24 16:04:57 +10:00
Martin Schwenke
140f0cfd3b ctdbd: Update the get_tunable code to return -EINVAL for unknown tunable
Otherwise callers can't tell the difference between some other failure
(e.g. memory allocation failure) and an unknown tunable.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 03fd90d41f9cd9b8c42dc6b8b8d46ae19101a544)
2013-05-24 16:04:50 +10:00
Martin Schwenke
e78b064dcc recoverd: Whitespace improvements
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 473cfcb019f0cb4a094bf10397f7414f7923ee57)
2013-05-24 15:55:11 +10:00
Martin Schwenke
1a181a4284 recoverd: Use talloc_array_length() for simpler code
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit f6792f478197774d2f3b2258c969b67c83e017ab)
2013-05-24 15:55:10 +10:00
Martin Schwenke
94b0e8dfeb ctdbd: When the "setup" event fails log an error and exit, don't abort
The "setup" event can fail when one of the eventscripts fails to run
its "setup" event.  If this occurs then the eventscript should log an
error.  The stack trace and core file generated when we abort provides
no useful information.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit c50eca6fbf49a6c7bf50905334704f8d2d3237d7)
2013-05-24 14:08:07 +10:00
Martin Schwenke
6d9667f01c ctdbd: Add new runstate CTDB_RUNSTATE_FIRST_RECOVERY
This adds more serialisation to the startup, ensuring that the
"startup" event runs after everything to do with the first recovery
(including the "recovered" event).

Given that it now takes longer to get to the "startup" state, the
initscript needs to wait until ctdbd gets to "first_recovery".

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit ed6814ff0a59ddbb1c1b3128b505380f60d7aeb7)
2013-05-24 14:08:07 +10:00
Martin Schwenke
77671b9ef5 ctdbd: New control CTDB_CONTROL_GET_RUNSTATE
Also new client function ctdb_ctrl_get_runstate().

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit dc4220e6f618cc688b3ca8e52bcb3eec6cb55bb1)
2013-05-24 14:08:07 +10:00
Martin Schwenke
147f6bb4b8 ctdbd: Start logging process earlier
Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f43fe3a560d5915c1a9893256f4e7bfe3d7e290a)
2013-05-24 14:08:07 +10:00
Martin Schwenke
0e678a73b8 ctdbd: Only start recovery daemon and timed events after setup event
This deconstructs ctdb_start_transport(), which did much more than
starting the transport.

This removes a very unlikely race and adds some clarity.  The setup
event is supposed to set the tunables before the first recovery.
However, there was nothing stopping the first recovery from starting
before the setup event had completed.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit c31feb27dcdb748b5333321c85fe54852dfa1bcf)
2013-05-24 14:08:06 +10:00
Martin Schwenke
63577c96db ctdbd: Replace ctdb->done_startup with ctdb->runstate
This allows states, including startup and shutdown states, to be
clearly tracked.  This doesn't include regular runtime "states", which
are handled by node flags.

Introduce new functions ctdb_set_runstate(), runstate_to_string() and
runstate_from_string().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 8076773a9924dcf8aff16f7d96b2b9ac383ecc28)
2013-05-24 14:08:06 +10:00
Amitay Isaacs
c8d577eb80 locking: Set lock helper path once
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 80fbe9364350d42658f7f8af250ac87eb1afbc21)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
1ddc7b0d10 locking: Remove functions that are not used anymore
These functions were used in locking child process to do the locking.  With
locking helper, these are not required.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit c660f33c3eaa1b4a2c4e951c1982979e57374ed4)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
90c4fa77b9 locking: Remove functions that are not used anymore
These functions were used in locking child process to do the locking.  With
locking helper, these are not required.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 6ea3212a7b177c6c06b1484cf9e8b2f4036653d9)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
ae25420e56 locking: Use separate locking helper binary for locking
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7cde53a6cbe74b1e46f7e1bca298df82c08de866)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
e30978eae1 locking: Create commandline arguments for locking helper
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f665e3d540c90579952e590caa5828acb581ae61)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
30aa825c1e locking: Add a standalone helper to lock record/db
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit a08b6ac19506160f3fb5925ea025027dce07781d)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
c9f4589c13 locking: Use database iterator for unmarking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7630ca4116b476636c27407748088ea335f1a06c)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
65a9195916 locking: Add handler function for unmarking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit adc113055de98fae276f9b501aff5c03cd25ddc8)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
a5133d16e7 locking: Use database iterator for marking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e8ea65b2713417db4a618a9f4633991cfaa93fe6)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
ed359bb1ea locking: Add handler function for marking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f120e40533780e02ff1cdc41cc6d3af1c4c83258)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
c5c79d63f2 locking: Use database iterator for unlocking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 187ed83f9701c7fa8d3cc476d47c5d2a87d5c308)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
b96388f95f locking: Add handler function for unlocking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 725239535f40ca2cca445bb5bf2e181351b330e9)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
403b1eaa6e locking: Use database iterator for locking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d2634d72d9ca0ceeb72cbb1adc95017a234480fd)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
bd6ad3f817 locking: Add handler function for locking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 2a1c933ef7c78ee071e2a640ea10941f1c12e32a)
2013-05-24 09:06:38 +10:00
Amitay Isaacs
4581582a5e locking: Refactor code to iterate over databases based on priority
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit a3275854812aca86032704134fdf6a129069c86a)
2013-05-24 09:06:38 +10:00
Amitay Isaacs
0c9d72eb18 locking: Add newline to debug logs
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d98a861716d5f8c1f4387d21666396d3164551b3)
2013-05-24 09:06:38 +10:00
Amitay Isaacs
7ee9e22a09 ctdbd: Print version string in the daemon startup
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 9d4524d13cbba21bfaf61bd35667984359b379b3)
2013-05-23 16:18:23 +10:00
Martin Schwenke
5fdf71b898 recoverd: takeover_run_core() should not use modified node flags
Modifying the node flags with IP-allocation-only flags is not
necessary.  It causes breakage if the flags are not cleared after use.
ctdb_takeover_run() no longer needs the general node flags - it only
needs the IP flags.

Instead of modifying the node flags in nodemap, construct a custom IP
flags list and have takeover_run_core() use that instead of node
flags.  As well as being safer, this makes the IP allocation code more
self contained and a little bit clearer.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 14bd0b6961ef1294e9cba74ce875386b7dfbf446)
2013-05-23 16:18:23 +10:00
Martin Schwenke
3f37b4418e ctdbd: Update confusing log message
Inactive can also mean stopped.  To add information, just print the
flags instead.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit a8605f7e06076e7edf84e0cc160fd3d9ab5c4b64)
2013-05-23 16:18:23 +10:00
Martin Schwenke
5aeae9744e ctdbd: Log a message when recovery master changes
Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-Programmed-With: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 1f96ea08f9a39dfe537c9b957ac512c84dc76f91)
2013-05-23 16:17:18 +10:00
Martin Schwenke
e769f8575a ctdbd: Log add and delete of IPs
At the moment, when someone deletes all the IPs on a node, all we see
are the release IP messages and we have to guess why.

Some would argue that add/release are more significant than
take/release so they should be logged.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 3c3df1d6afec7e3e721f9bcd4e8b8e008fd6e50b)
2013-05-22 14:24:22 +10:00
Martin Schwenke
0baefba368 ctdbd: Removed bogus comment in ctdb_find_iface()
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 4a8d90d0812a3242f58a2a0e2aa0f528f60f7013)
2013-05-22 14:24:21 +10:00
Martin Schwenke
54e91df60d recoverd: Move IP flags into ctdb_takeover.c
These should never be seen outside the IP allocation code.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit e143abd16ccde2e0edfe103673d31a5fb06b6aef)
2013-05-09 12:55:42 +10:00
Martin Schwenke
50f19b5bd4 recoverd: Clear IP flags after IP allocation algorithm has run
If these flags are left set they will confuse other recovery daemon
code.

Factor the clearing code into new function clear_ipflags().

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 45c776958017ea7001f061842c9e0f60e4a25f23)
2013-05-09 12:55:42 +10:00
Martin Schwenke
530020d83b recoverd: Remove unused mask argument and initial mask calculation
This has been replaced by set_ipflags() and associated functionality.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit d0a3822573db296e73cc897835f783c8abc084b3)
2013-05-07 16:20:47 +10:00
Martin Schwenke
ee7357de51 recoverd: When calculating rebalance candidates don't consider flags
This is really a check to see if a node is already hosting IPs.  If
so, we assume it was previously healthy so it isn't considered as a
rebalance candidate.  There's no need to limit this to healthy node,
since this is checked elsewhere.

Due to this the variable newly_healthy is renamed everywhere to
rebalance_candidates.

The mask argument is now completely unused.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 65e0ea6c2c0629e19349ba4b9affa221fde2b070)
2013-05-07 16:20:47 +10:00
Martin Schwenke
c9056b4f88 recoverd: Remove unused mask argument from IP allocation functions
This is a no-op and is in a separate commit to make the previous
commit less cumbersome.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 107e656bbe24f9d21fbaf886a3e9417da4effe5a)
2013-05-07 16:20:47 +10:00
Martin Schwenke
0445c988e2 recoverd: Fix tunable NoIPTakeoverOnDisabled, rename to NoIPHostOnAllDisabled
This really needs to be per-node.  The rename is because nodes with
this tunable switched on should drop IPs if they become unhealthy (or
disabled in some other way).

* Add new flag NODE_FLAGS_NOIPHOST, only used in recovery daemon.

* Enhance set_ipflags_internal() and set_ipflags() to setup
  NODE_FLAGS_NOIPHOST depending on setting of NoIPHostOnAllDisabled
  and/or whether nodes are disabled/inactive.

* Replace can_node_servce_ip() with functions can_node_host_ip() and
  can_node_takeover_ip().  These functions are the only ones that need
  to look at NODE_FLAGS_NOIPTAKEOVER and NODE_FLAGS_NOIPHOST.  They
  can make the decision without looking at any other flags due to
  previous setup.

* Remove explicit flag checking in IP allocation functions (including
  unassign_unsuitable_ips()) and just call can_node_host_ip() and
  can_node_takeover_ip() as appropriate.

* Update test code to handle CTDB_SET_NoIPHostOnAllDisabled.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 1308a51f73f2e29ba4dbebb6111d9309a89732cc)
2013-05-07 16:20:46 +10:00
Martin Schwenke
ac80824709 recoverd: Factor out new function all_nodes_are_disabled()
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 12aef10e9889760d98f58c8d916f19d069fa381a)
2013-05-07 16:20:46 +10:00
Martin Schwenke
657162fb34 recoverd: Refactor code to get NoIPTakeover tunable from all nodes
Signed-off-by: Martin Schwenke <martin@meltin.net>
Pair-programmed-with: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 1fb5352d2b6918fcc6f630db49275d25a3eebe8d)
2013-05-07 16:20:46 +10:00
Martin Schwenke
17521b31b2 recoverd: Add debug message when dropping IPs in IP allocation
Update tests accordingly.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 91405282ba4abad4ad8e8c5f7ee4c83c75f38280)
2013-05-07 16:20:46 +10:00
Martin Schwenke
3769368a99 ctdbd: Log CTDB startup before creating the PID file
Otherwise the messages are in a stupid order...  :-)

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reported-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit cd87ba85fc6c375758c7d3dfa8dbd4d8a02074b0)
2013-05-06 15:40:30 +10:00
Martin Schwenke
fa16cccf02 ctdbd: Remove the "stopped" event
It isn't used, superceded by "ipreallocated".

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit c2bb8596a8af6406ef50e53953884df9d6246a96)
2013-05-06 13:38:21 +10:00