1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

183 Commits

Author SHA1 Message Date
Martin Schwenke
9aa9abcc19 ctdbd: Avoid unnecessary updateip event
The existing code makes one fatally bad assumption:
vnn->iface->references can never be -1 (or max-unit32_t in this case).
Right now the reference counting is broken so a reference count of -1
is possible and causes a spurious updateip when vnn->iface is the same
as best_face.  This can occur frequently because we get a lot of
redundant takeovers, especially when each IP can only be hosted on one
interface.

This makes the code much more defensive by noting that when best_iface
is the same as vnn->iface there is never a need for an updateip event.
This effectively neuters the updateip code path when IPs can only be
hosted by a single interface.

This should obsolete 6a74515f0a1e24d97cee3ba05d89133aac7ad2b7.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 7054e4ded59c6b8f254dcfefaef64da05f25aecd)
2012-10-10 14:54:53 +11:00
Amitay Isaacs
3c1f656764 Revert "when creating/adding a public ip, set the initial interface to be the first interface specified"
This reverts commit 4308935ba48ac7a29e7523315acf580019715f0f.

This fixes 16_ctdb_config_add_ip.sh test when run against local daemons. When
running against local daemons, if the interface is assigned as soon as an IP is
added, then takeover would never assign this IP address.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 06dfd13604d08910e07cbf927c338d7b9fce9a2f)
2012-10-07 15:25:34 +11:00
Martin Schwenke
7df1da1c91 recoverd: Update a log message that has bit-rotted
This message used to be correct because the ipreallocated event only
handled updating the NAT gateway.  However, that has changed so the
message needs to be updated.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit cc9d96f4248e45ea99c5f00db1526426ac26fbc2)
2012-08-08 16:11:11 +10:00
Martin Schwenke
75a0041567 ctdbd: Fix ctdb_control_release_ip() on local daemons
When running on local daemons no IPs are actually assigned to
interfaces.  Commit 9a806dec8687e2ec08a308853b61af6aed5e5d1e broke
ctdb_control_release_ip() for local daemons because it asks the system
which interface the given IP is on, instead of the old behaviour of
trusting CTDB's internal records.

For local deamons (i.e. !ctdb->do_checkpublicip) revert to the old
behaviour of looking up the interface internally.  This is good
enough, given that the tests don't tend to misconfigure the addresses.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 38e8651b955afdbaf0ae87c24c55c052f8209290)
2012-07-26 22:10:54 +10:00
Amitay Isaacs
e379fc3ea5 Fix compiler warnings.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d29e1880c8ce7219e065d31b47b0e8ad9e83146d)
2012-07-13 14:50:56 +10:00
Ronnie Sahlberg
c7e648c2d1 When we release an ip, get the interface name from the kernel
instead of using the interface where ctdb thinks the ip is hosted at.
The difference is that this now allows us to handle cases where we want to release an ip   but ctdbd does not know which interface the ip is assigned on.
(user has used 'ip addr add...'  and manually assigned an ip to the wrong interface)

(This used to be ctdb commit c6bf22ba5c01001b7febed73dd16a03bd3fd2bed)
2012-06-20 15:11:56 +10:00
Amitay Isaacs
7631830152 server: Replace BOOL datatype with bool, True/False with true/false
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 6e5cbe8fff71985e5a2fc16b7e9f2b868011ff5d)
2012-05-28 11:22:25 +10:00
Ronnie Sahlberg
a57eba2bb4 Track all child process so we never send a signal to an unrelated process (our child died and kernel wrapped the pid-space and reused the pid for a different process
Wrap all creation of child processes inside ctdb_fork() which is used to track all processes we have spawned.
Capture SIGCHLD to track also which child processes have terminated.

Wrap kill() inside ctdb_kill() and make sure that we never send a !0 signal to a child process pid that has already terminated (and might have been replaced with a

(This used to be ctdb commit f73a4b1495830bcdd094a93732a89dd53b3c2f78)
2012-05-03 14:03:26 +10:00
Ronnie Sahlberg
a367fa6138 RELOADIPS: simplify the reloadips code a bit
and also update the "read public address file" to not check if the address exists already locally when we read if from the child process, to stop it
from spamming the logs with "We already host ..."
messages

(This used to be ctdb commit 334ea830f1bf33419f4a1e78f23afd41a852d0f4)
2012-05-01 15:34:26 +10:00
Ronnie Sahlberg
7a1aa560e7 Add new control to reload the public ip address file on a node
Also add a method to use the recovery master/daemon to reload the public ips on all nodes in the cluster.
Reloading the public ips on all node sin the cluster is only suported if all nodes in the cluster are available and healthy.

(This used to be ctdb commit 05603e914f8c12618d7e06943c0f7df207f645b0)
2012-05-01 10:48:08 +10:00
Ronnie Sahlberg
db411aaada Merge remote branch 'amitay/tevent-sync'
(This used to be ctdb commit 17ff3f240b0d72c72ed28d70fb9aeb3b20c80670)
2012-04-26 08:09:23 +10:00
Amitay Isaacs
4392591555 Remove explicit include of lib/tevent/tevent.h.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 0681014ca5ed2a9b56f63fdace7f894beccf8a9a)
2012-04-13 17:28:14 +10:00
Amitay Isaacs
b3d098ced7 ctdbd: Fix spurious warnings when running with --nopublicipcheck
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 67b909a0718d6cfce82ffce0830da3a6ff1f6c4b)
2012-04-13 15:38:11 +10:00
Amitay Isaacs
425b8768ee ctdbd: Fix the error message string
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 15f63ebab9686734f41a6adf38d4a7faa919ac66)
2012-04-13 14:51:13 +10:00
Ronnie Sahlberg
2456f77ca6 NoIPTakeover: change the tunable name for the "dont allow failing addresses over onto the node" to NoIPTakeover
(This used to be ctdb commit 35592e618cfd827b6978af6332f80504f232c46a)
2012-03-22 11:05:15 +11:00
Ronnie Sahlberg
9f31f76805 NoIPFailback: Exclude nodes which have NoIPFailback as failback targets during reallocation
(This used to be ctdb commit c262c29773d1608e7ce04bdfb7f4469df0a9637b)
2012-03-22 09:24:32 +11:00
Ronnie Sahlberg
befa9df152 Make NoIPFailback a node local setting. Nodes that have NoIPFailback set to !0 can not takeover new ip addresses during failover.
Remove the old global setting for this unused tunable and add it as a new node flag. This node flag is only valid/defined within the takeover subsystem in the recovery daemon. Add async functions to collec the NoIPFailback settings for each node.

This will later e used to disqualify certain nodes from being takeover targets when we perform reallocation.

(This used to be ctdb commit 668f3e88a9e5f598706952b7140547640c85a5ed)
2012-03-22 09:09:57 +11:00
Ronnie Sahlberg
ef2bd0b016 When adding ips to nodes, set up a deferred rebalance for the whole node to trigger after 60 seconds in case the normal ipreallocated is not sufficient to trigger rebalance.
(This used to be ctdb commit 4340263b219d75c39f8de22abe3f6f1c1ee63ea2)
2012-02-28 06:56:04 +11:00
Ronnie Sahlberg
91c9371f2d Make KILLTCP structure a child of VNN so that it is freed at the same time
the referenced VNN structure is.

Also, remove the circular reference between the two objects KIPPCTP and VNN

(This used to be ctdb commit 02b62482164a3c69715949074feb7f191a29d534)
2012-02-27 07:21:26 +11:00
Volker Lendecke
5e3b13a32a FreeBSD does not define s6_addr32, only s6_addr
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit d657af4fb68ce3f7c462856f2934f6bf169e120b)
2012-02-13 16:20:12 +01:00
Martin Schwenke
3ae8273d86 Make some ctdb_takeover.c functions static
These were intentionally not static so they could be linked to in unit
test programs.  However, using the CCAN-style unit tests where
relevant code is just included, this is no longer necessary.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit d0e9e8554614bd49ffb9ec3509feaa0e80d0f65d)
2011-11-11 14:41:47 +11:00
Ronnie Sahlberg
8db9b73920 Merge remote branch 'martins/lcp2fix'
(This used to be ctdb commit 7c02d242af552aa732f5c70ea4eeefbc8a8542e2)
2011-11-08 14:06:30 +11:00
Ronnie Sahlberg
0f92fa224c RB_TREE: Add mechanism to abort a traverse
This patch changes the callback signature for traversal
functions to allow a client to abort a traverse before it finishes.
Updates to all callers and examples as well as rb-test tool.

(This used to be ctdb commit 8ab0c63ad36cfbbb1e5fed46a1f4c47b1fdb581f)
2011-11-08 13:40:28 +11:00
Martin Schwenke
c0939af571 LCP IP allocation algorithm - try harder to find a candidate source node
There's a bug in LCP2.  Selecting the node with the highest imbalance
doesn't always work.  Some nodes can have a high imbalance metric
because they have a lot of IPs.  However, these nodes can be part of a
group that is perfectly balanced.  Nodes in another group with less
IPs might actually be imbalanced.

Instead of just trying the source node with the highest imbalance this
tries them in descending order of imbalance until it finds one where
an IP can be moved to another node.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 574091d5aced5e87aefad52f8bc47aa75c25fbf6)
2011-11-02 10:17:00 +11:00
Martin Schwenke
98c27f973d LCP IP allocation algorithm - new function lcp2_failback_candidate()
There's a bug in LCP2.  Selecting the node with the highest imbalance
doesn't always work.  Some nodes can have a high imbalance metric
because they have a lot of IPs.  However, these nodes can be part of a
group that is perfectly balanced.  Nodes in another group with less
IPs might actually be imbalanced.

Factor out the code from lcp2_failback() that actually takes a node
and decides which address should be moved to which node.

This is the first step in fixing the above bug.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 75718c5768b5bb5c0bcd7dd90e0327c6ed22a63d)
2011-11-01 21:01:25 +11:00
Ronnie Sahlberg
d79596ba1a One of the entry points to release an ip reset the pnn field before invoking the eventscript.
this triggered a check for "only run the eventscript if we host the address" to trigger and shortcir=cuit calling the eventscript.

An effect of this would be that 'ctdb delip' would remove the ip from ctdb, but fail to delete it from the interface.

S1028798

(This used to be ctdb commit b82524f240bf21769dd7624ca6026763d38b9396)
2011-09-22 15:17:23 +10:00
Ronnie Sahlberg
4587bdb052 when checking that the interfaces exist in ctdb_add_public_address()
cant talloc off vnn since it is not yet initialized and might not always be NULL

(This used to be ctdb commit 3d37be3e2bfb61ede824028aeebaa18ba304faae)
2011-09-21 11:42:19 +10:00
Ronnie Sahlberg
783ceca07b Interface monitoring: add a event to trigger every 30 seconds to check that all interfaces referenced by the public address list actually exists.
This will make it much easier to root-cause problems such as
S1029023
when an external application deleted the interface while it is still is in use by ctdbd.

(This used to be ctdb commit 9abf9c919a7e6789695490e2c3de56c21b63fa57)
2011-09-06 17:02:19 +10:00
Ronnie Sahlberg
64378fea58 Check interfaces: when reading the public addresses file to create the vnn list
check that the actual interface exist, print error and fail startup if the interface does not exist.

(This used to be ctdb commit cd33bbe6454b7b0316bdfffbd06c67b29779e873)
2011-09-06 16:11:00 +10:00
Volker Lendecke
1cf1670f0a Fix a const warning
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit e25559087c9752502580875f7e33f3c416c05f84)
2011-08-22 17:11:07 +02:00
Ronnie Sahlberg
fea64f65b5 Remove a log message about setting linkstate for an unknown interface.
sometimes we do want to try to set the linkstate for interfaces that are not in use by public addresses right now (but posisbly by other mechanisms) and these messages just spam the logs

S1026357

(This used to be ctdb commit f2fe0a090a9650910ebe49514b3ca01dc593bea3)
2011-08-05 10:05:12 +10:00
Martin Schwenke
5ac67504ca Tests: Initial test code for LCP2 IP allocation algorithm.
Move struct ctdb_public_ip_list to ctdb_private.h and put some
definitions for some functions from ctdb_takeover.c there.  This
allows those functions to be called from unit tests.

Add ctdb_takeover_tests.c and the Makefile support to build it.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 9d34be0233edf3bc022345c0494c4b2a4d7f8480)
2011-07-29 09:01:36 +10:00
Martin Schwenke
ff1a81c872 IP allocation - add LCP2 algorithm.
The current non-deterministic IP allocation algorithm balances IPs
across the whole cluster.  It does not consider different
interfaces/VLANs/subnets, so these different groups of IPs aren't
generally well balanced.

This adds the LCP2 algorithm for IP allocation and allows it to be
enabled by setting the "LCP2PublicIPs" tunable to 1.

The LCP2 algorithm calculates the imbalance of a node by totalling the
squares of the distances between each IP on the node.  The IP distance
is defined as the length longest common prefix (LCP) of bits that is
found when comparing 2 IPs.  The imbalance of a cluster is the maximum
imbalance for any node.  At each step the algorithm selects an
allocation to the IP/node combination that results in the choosing the
allocation that best reduces the imbalance of the cluster.

The implementation splits out the IP allocation part of
ctdb_takeover_run() into new function ctdb_takeover_run_core(), and
then extracts out the basic IP assignment code into new functions
basic_allocate_unassigned() and basic_failback().  3 new functions
lcp2_init(), lcp2_allocate_unassigned() and lcp2_failback() implement
the LCP2 algorithm, and are hooked into ctdb_takeover_run_core().

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 61fc7fbd0235469df22deb6581c6bd47e30bc0be)
2011-07-29 09:01:17 +10:00
Ronnie Sahlberg
e707f23596 Update the delip command
Dont talloc_free(vnn) immediately but postphone it until later when
the eventscript callback has completed.

CQ S1026664

(This used to be ctdb commit 0a99e8742a261b1d3a2c8830f5c19ea6c2c47cad)
2011-07-29 08:50:48 +10:00
Ronnie Sahlberg
c93a968619 When trying to re-balance the ip assignment and shuffle ips from
nodes with many addresses to nodes with few addresses,
loop up to num_ips+5 times instead of only 5 times.

When we have very many public ips per node, we might need to loop more than
5 times or else we will exit without reaching optimal balance.

(This used to be ctdb commit aa8114a625a637277561a66c80bdece3c27e9e20)
2011-07-06 13:14:13 +10:00
Ronnie Sahlberg
f84bd3b5f1 Dont call the UPDATE event if both old and new interface is the same.
CQ S1018175

(This used to be ctdb commit 6a74515f0a1e24d97cee3ba05d89133aac7ad2b7)
2011-05-04 13:29:29 +10:00
Ronnie Sahlberg
c04505724a IFACE handling. Assume links are always good on nstartup (they almost always
Simplify the handling of setting the links in the 10.interface eventscript
and remove the optimization to only call setifacelink on state change
to make the code simpler to read.

If a take ip event fails, flag the node as unhealthy.

Add a check to the interface script to check if the interface exists
or if it has been deleted.
So that we can capture and become UNHELTHY if someone deletes an interface
we are using to host public addresses.

(This used to be ctdb commit 4ab63d2a7262aff30d5eced184c294c9c9dd4974)
2011-04-11 07:40:05 +10:00
Ronnie Sahlberg
f82936402f IP reallocation. If a public address is already hosted on the node when we startup, log a warning message but do not cause the recovery to fail.
CQ S1022356

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 89f8169c24da96c1fdd0ac19b8a1e0e1df01a72a)
2011-03-14 13:35:53 +01:00
Ronnie Sahlberg
93bea39391 IPALLOCATION : If the node is held pinned down in "init" state
by external services failing to start, or blocking CTDBD from finishing the startup phase,
we can encounter a situation where we have not yet fully initialized, but a
remote recovery master tries to release a certain ip clusterwide.

In this situation the node that is pinned down in init/startup phase
would fail to perform the release of the ip address since we are not yet fully operational and not yet host any valid interfaces.

In this situation, we just need to remain unhealthy, there is on need to
also ban the node.

Remove the autobanning for this condition and just let the node remain in
unhealthy mode.
Banning is overkill in this situation when the system is broken and just
draws attention to ctdbd instead of the root cause.

(This used to be ctdb commit d8af74e4c4961deb94c18dde8ba7fc07e944729c)
2011-01-13 09:42:01 +11:00
Ronnie Sahlberg
a9a6ae064d When assigning the single-public-ip during startup,
flag the interface as initially being "link ok"
so that we can add it and startup.

The eventscript can later drop the flag if required

(This used to be ctdb commit 720849b756c825fb8b285f09972a8c39f1888a99)
2010-12-13 14:24:04 +11:00
Ronnie Sahlberg
c2c53db49d during ip allocation, there are failure modes where a node might hold a ip address
but thinks it is still unassigned (-1).

add code to the recovery daemon to detect this case and trigger a reallocation
so that the ip gets covered

and change the takeip code to allow for this condition, taking on an ip address that is
already hosted.

cq s1021073

(This used to be ctdb commit 9020baf27cab7821c9094cda185206fb7af0fee7)
2010-12-03 13:30:39 +11:00
Ronnie Sahlberg
dbcf0de18c Dont exit the update ip function if the old and new interfaces are the same
since if they are the same for whatever reason this triggers the system
to go into an infinite loop and is unrobust

The scriptds have been changed instead to be able to cope with this
situation for enhanced robustness

During takeover_run and when merging all ip allocations across the cluster
try to kepe track of when and which node currently hosts an ip address
so that we avoid extra ip failovers between nodes

(This used to be ctdb commit cf778b5aaf6356401e3985acccc7df9e08ab6930)
2010-11-10 14:55:25 +11:00
Ronnie Sahlberg
6fa8e1fddb when we load the public address file, at the same time check if we are already hosting the public address, if so, set ourselves up as the pnn for that address
(This used to be ctdb commit 0f2a2dac91a61be188c3578c8bb89d47cbf9a0f8)
2010-11-10 14:55:24 +11:00
Ronnie Sahlberg
5f76f3c0e2 Add a new tunable : DisableIPFailover that when set to non 0
will stopp any ip reallocations at all from happening.

(This used to be ctdb commit d8d37493478a26c5f1809a5f3df89ffd6e149281)
2010-11-10 14:55:24 +11:00
Ronnie Sahlberg
87a0ece976 when creating/adding a public ip, set the initial interface to be the first interface specified
(This used to be ctdb commit 4308935ba48ac7a29e7523315acf580019715f0f)
2010-11-10 14:55:23 +11:00
Ronnie Sahlberg
d8d8b9e1d7 add a new serverid to send a message everytime an ip address is taken on the local node
(This used to be ctdb commit 1261f3d9702800a4e59550c881350daf479f00ef)
2010-09-13 15:43:19 +10:00
Ronnie Sahlberg
19211f99c8 remove an unused variable
(This used to be ctdb commit e07fdbaf12bbe84370bc47a1979fe198a06a6cc8)
2010-09-13 13:13:12 +10:00
Ronnie Sahlberg
c95f4258d8 Add a new event "ipreallocated"
This is called everytime a reallocation is performed.

    While STARTRECOVERY/RECOVERED events are only called when
    we do ipreallocation as part of a full database/cluster recovery,
    this new event can be used to trigger on when we just do a light
    failover due to a node becomming unhealthy.

    I.e. situations where we do a failover but we do not perform a full
    cluster recovery.

    Use this to trigger for natgw so we select a new natgw master node
    when failover happens and not just when cluster rebuilds happen.

(This used to be ctdb commit 7f4c591388adae20e98984001385cba26598ec67)
2010-08-30 18:09:30 +10:00
Ronnie Sahlberg
2e8aac6689 Merge commit 'rusty/ports-from-1.0.112' into foo
(This used to be ctdb commit 13e58d92f5f1723e850a82ae030d0ca57e89b1ee)
2010-08-19 13:17:56 +10:00
Ronnie Sahlberg
5aa5f3e7bf Remove the structure ctdb_control_tcp_vnn since this is identical to the structure ctdb_tcp_connection.
Add a new "ctdb deltickle" command to delete tickles from the database.
This can ONLY be used for tickles created by "ctdb addtickle".

Push any "addtickle/deltickle" updates to other nodes every TickleUpdateInterval seconds'

(This used to be ctdb commit acded034e2f0dcae4c2c9e54e16a001caf23caec)
2010-08-18 12:36:03 +10:00