1
0
mirror of https://github.com/samba-team/samba.git synced 2025-03-08 04:58:40 +03:00

937 Commits

Author SHA1 Message Date
Martin Schwenke
c0939af571 LCP IP allocation algorithm - try harder to find a candidate source node
There's a bug in LCP2.  Selecting the node with the highest imbalance
doesn't always work.  Some nodes can have a high imbalance metric
because they have a lot of IPs.  However, these nodes can be part of a
group that is perfectly balanced.  Nodes in another group with less
IPs might actually be imbalanced.

Instead of just trying the source node with the highest imbalance this
tries them in descending order of imbalance until it finds one where
an IP can be moved to another node.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 574091d5aced5e87aefad52f8bc47aa75c25fbf6)
2011-11-02 10:17:00 +11:00
Martin Schwenke
98c27f973d LCP IP allocation algorithm - new function lcp2_failback_candidate()
There's a bug in LCP2.  Selecting the node with the highest imbalance
doesn't always work.  Some nodes can have a high imbalance metric
because they have a lot of IPs.  However, these nodes can be part of a
group that is perfectly balanced.  Nodes in another group with less
IPs might actually be imbalanced.

Factor out the code from lcp2_failback() that actually takes a node
and decides which address should be moved to which node.

This is the first step in fixing the above bug.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 75718c5768b5bb5c0bcd7dd90e0327c6ed22a63d)
2011-11-01 21:01:25 +11:00
Ronnie Sahlberg
19bd9149c3 ReadOnly: fix bug writing incorrect amount of data in delegated record
Fix bug when ctdbd updates the local copy of a delegated record to write the correct
amount of data to the record.

(This used to be ctdb commit 8814d8bc159a5e368afaa236ac7d865165db04b2)
2011-10-28 11:44:19 +11:00
Ronnie Sahlberg
a9e77de9e4 ReadOnly: Dont update the record header from the calling client. While it is convenient since it avoids having to create a child process from the main dameon for writing the updated record it makes the cleitn more complex.
Remove the code in the example client code that writes the record to the local tdb.
Add code to the local ctdbd processing of replies to check if this reply contain a ro delegation and if so, spawn a child process to lock the tdb and then write the data.

(This used to be ctdb commit bf1d429227dc4f5818263cc39401d0a22663cdba)
2011-10-24 13:14:26 +11:00
Ronnie Sahlberg
90c0c235fa Remove debug message
(This used to be ctdb commit db0fdc2281c4742113c92d697371c37815db35a0)
2011-10-24 12:21:55 +11:00
Ronnie Sahlberg
d79596ba1a One of the entry points to release an ip reset the pnn field before invoking the eventscript.
this triggered a check for "only run the eventscript if we host the address" to trigger and shortcir=cuit calling the eventscript.

An effect of this would be that 'ctdb delip' would remove the ip from ctdb, but fail to delete it from the interface.

S1028798

(This used to be ctdb commit b82524f240bf21769dd7624ca6026763d38b9396)
2011-09-22 15:17:23 +10:00
Ronnie Sahlberg
4587bdb052 when checking that the interfaces exist in ctdb_add_public_address()
cant talloc off vnn since it is not yet initialized and might not always be NULL

(This used to be ctdb commit 3d37be3e2bfb61ede824028aeebaa18ba304faae)
2011-09-21 11:42:19 +10:00
Ronnie Sahlberg
e3cdf429b6 ReadOnly: revokechild_active is a list, not a context.
Dont reset the pointer to NULL after deleting the first entry, loop deleting one entry
at a time until they are all gone or we will leak some memory and possibly a process.

(This used to be ctdb commit 8a86ac72088ad9f64ca83218c704f84c9abe00b6)
2011-09-13 18:47:18 +10:00
Ronnie Sahlberg
0dc5584101 Merge branch 'master-readonly-records' into foo
Conflicts:

	Makefile.in
	tools/ctdb.c

(This used to be ctdb commit 0fedef0ffba4178126eee9544c5e2db52f5db893)
2011-09-12 09:34:34 +10:00
Ronnie Sahlberg
1c05db2c9c Merge remote branch 'ddiss/master_pmda_and_client_timeouts'
(This used to be ctdb commit 7bebfc7bad8f36e54003b8e25372fdaf54836e21)
2011-09-08 11:22:53 +10:00
David Disseldorp
5296da5609 client: add timeout argument to ctdb_attach
Rather than using a fixed 2 second CTDB_CONTROL_GETDBPATH timeout.

(This used to be ctdb commit 9e178671560cb95121e11d718a76b05380ecd6c5)
2011-09-06 13:57:04 +02:00
Ronnie Sahlberg
783ceca07b Interface monitoring: add a event to trigger every 30 seconds to check that all interfaces referenced by the public address list actually exists.
This will make it much easier to root-cause problems such as
S1029023
when an external application deleted the interface while it is still is in use by ctdbd.

(This used to be ctdb commit 9abf9c919a7e6789695490e2c3de56c21b63fa57)
2011-09-06 17:02:19 +10:00
Ronnie Sahlberg
64378fea58 Check interfaces: when reading the public addresses file to create the vnn list
check that the actual interface exist, print error and fail startup if the interface does not exist.

(This used to be ctdb commit cd33bbe6454b7b0316bdfffbd06c67b29779e873)
2011-09-06 16:11:00 +10:00
Michael Adam
a3e0079568 Add a tunable "AllowClientDBAttach" with default value 1.
When set to 0, clients will not be able to attach to databases
via the db_attach control. This might can be useful for maintenance
where ctdb should be kept running but clients should not be able
to modify databases.

(This used to be ctdb commit ddfeecda87955b4e46777599f678e6926d37f4c4)
2011-09-05 16:17:39 +10:00
Ronnie Sahlberg
206a3c0c66 ReadOnly: add a new control to activate readonly lock capability for a database.
let all databases default to not support this  until enabled through this control

(This used to be ctdb commit 908a07c42e5135a3ba30a625fc4f4e4916de197a)
2011-09-01 11:08:18 +10:00
Ronnie Sahlberg
a0d4d240c3 ReadOnly: add a readonly flag to the getdbmap control and show the readonly setting in ctdb getdbmap output
(This used to be ctdb commit 4cac9ad7d9c9ca657a247a6c215476399c7d2210)
2011-09-01 10:28:15 +10:00
Ronnie Sahlberg
63dc96cdb2 ReadOnly: Change the ctdb_db structure to keep a uint8_t for flags instead of a boolean for
the persistent flag.
This is the same size as the original boolean but allows ut to add additional flags for the database

(This used to be ctdb commit 7462761638d25880ad46024ad4ef21667eb99a98)
2011-09-01 10:21:55 +10:00
Ronnie Sahlberg
2902203900 Logging: when we log stdout/stderr messages from eventscripts to the system log, prefix every line of output with the name of the eventscript.
CQ S1028412

(This used to be ctdb commit 392363c04185f47a826fc6ed95038342be2150bf)
2011-08-26 09:39:25 +10:00
Ronnie Sahlberg
9729d3e339 ReadOnly: Check the readonly flag instead of whether the tdb pointer is NULL or not
(This used to be ctdb commit 01314c2cb3a480917d6a632b83c39f0a48bba0e7)
2011-08-23 10:41:52 +10:00
Ronnie Sahlberg
de7c3de0a2 ReadOnly: clear out the tracking record once a revoke is completed
(This used to be ctdb commit 7af255551f058d1f6bfdd38ca603e7a19d1bb7ba)
2011-08-23 10:35:56 +10:00
Ronnie Sahlberg
38e8964910 ReadOnly: Add handlign of readonly requests readwrite requests, delegations and revoking of delegation to the processing loop for CALL requests coming in from a local client via domain socket
(This used to be ctdb commit e7cbf5b5d03cc26a73a92066a651f8eab73624b8)
2011-08-23 10:32:42 +10:00
Ronnie Sahlberg
6fd8cc659d ReadOnly: Add processing for ReadOnly delegation requests and revoke requests to the processing loop for CALL packets we receive from different nodes.
This implements the ReadOnly and ReadWrite request processing, delegation and revoking of delegations for all requests coming in across the network from a remote node.

(This used to be ctdb commit 78f2c2ea70e6270cec59db7c3f174a511bf608a9)
2011-08-23 10:32:02 +10:00
Ronnie Sahlberg
59d8d9b695 ReadOnly: Once recovery has finished, make sure to free all revoke child processes and trigger the destructors for all deferred calls to re-queue the original packets to the input packet processing function
(This used to be ctdb commit 530a78aa05910beeca0867c4dbe226d4ce73f946)
2011-08-23 10:30:57 +10:00
Ronnie Sahlberg
07d995ab72 ReadOnly: When releasing all deferred calls that blocked during revoke of all previous delegations, add a 1 second grace/delay for any new readonly delegation requests so that the read-write fetch-lock porcess has a chance to make progress
(This used to be ctdb commit 2a4e9e69850d64dd8aef695f587ebe04393a688f)
2011-08-23 10:30:22 +10:00
Ronnie Sahlberg
dda2616cf5 ReadOnly: Add a function to start a revoke of all delegations for a record.
This triggers a child process to be created to perform the actual potentially blocking calls that are required.

(This used to be ctdb commit 7d575ee92c95bc4aab78a33bc1aac7ff0811ab3a)
2011-08-23 10:27:31 +10:00
Ronnie Sahlberg
1bb855bd52 ReadOnly: Add functions to register CALLs to a context used to handle deferal of processing of CALL commands.
Once the contexts are freed, the deferred calls are re-issued to the input packet processing functions again.
This is needed when/if a CALL can not currently be processed by the main engine due to the record being locked down for revoking of all delegations.

The data is passed through several layers of callbacks, and finally a timed event callback to ensure that the processing of the packet will be restarted again at the topmost eventloop, avoinding event loop nesting.

(This used to be ctdb commit cc6f78efcfa3b8caeffbd68018e6dfbf81488dce)
2011-08-23 10:25:57 +10:00
Ronnie Sahlberg
3d495c48d2 ReadOnly: Add an extra flag to ctdb_call_local to specify whether we want to write the record and header back to the tdb (for example we do when performing dmaster migrations)
(This used to be ctdb commit b935e83255aeb3754b2fd37cf5611e02f7283514)
2011-08-23 10:25:05 +10:00
Ronnie Sahlberg
b01dc029ca ReadOnly: After recovering all databases, make sure to clear out the tracking database used to track delegations and revoke. This is because the recovery will implicitely result in a revoke of all delegations.
(This used to be ctdb commit b5520933b9922d6af6f59f535824e1cdacb9f774)
2011-08-23 10:24:44 +10:00
Ronnie Sahlberg
1441b77cce ReadOnly: Add "readonly" flag to the ctdb_db_context to indicate if this database supports readonly operations or not. Add a private lock-less tdb file to the ctdb_db_context to use for tracking delegarions for records
Assume all databases will support readonly mode for now and se thte flag for all databases. At later stage we will add support to control on a per database level whether delegations will be supported or not.

(This used to be ctdb commit 502f86f79944df4bac9094f716e54110c511dc24)
2011-08-23 10:24:26 +10:00
Ronnie Sahlberg
6ff039d444 ReadOnly: After performing a recovery, clear out all flags related to readonly delegations and revoke
(This used to be ctdb commit 9985a97e11688f3f688bb84e1180fd57c42077f4)
2011-08-23 10:24:18 +10:00
Ronnie Sahlberg
00a870f759 ReadOnly records: Add a new RPC function FETCH_WITH_HEADER.
This function differs from the old FETCH in that this function will also fetch the record header and not just the record data

(This used to be ctdb commit c7196d16e8e03bb2a64be164d15a7502300eae0e)
2011-08-23 10:06:59 +10:00
Volker Lendecke
1cf1670f0a Fix a const warning
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit e25559087c9752502580875f7e33f3c416c05f84)
2011-08-22 17:11:07 +02:00
Ronnie Sahlberg
d1f5177374 Change the default for ip failover to be LCP2 and not DeterministicIPs
(This used to be ctdb commit 038916248a73d6a250108c9235c0c4f76dba8e0c)
2011-08-15 10:43:42 +10:00
Ronnie Sahlberg
fea64f65b5 Remove a log message about setting linkstate for an unknown interface.
sometimes we do want to try to set the linkstate for interfaces that are not in use by public addresses right now (but posisbly by other mechanisms) and these messages just spam the logs

S1026357

(This used to be ctdb commit f2fe0a090a9650910ebe49514b3ca01dc593bea3)
2011-08-05 10:05:12 +10:00
Ronnie Sahlberg
10caf186e1 remove log message we dont need
S1026492

(This used to be ctdb commit c5f6e44b92210519d4bfc24611cae3f9978cc2e5)
2011-08-04 13:49:57 +10:00
Ronnie Sahlberg
a9eba762d7 remove a non-error logmessage about persistent databases being healthy, as expected
S1026492

(This used to be ctdb commit da9e02085523e27fa29e35c60034f6a8aaaa81e8)
2011-08-04 13:49:48 +10:00
Ronnie Sahlberg
629f4da55a remove a log message we dont need about "allow clients to attach to databases"
S1026492

(This used to be ctdb commit 42c3e4c5216000c370814441e38c7a8180047aaf)
2011-08-04 13:49:38 +10:00
Ronnie Sahlberg
44dd9b458d Change the message when we start the daemon to "CTDB starting on node"
S1026492

(This used to be ctdb commit 4dfe85dc19eea7b3382a005f63e1a3ef824aea90)
2011-08-04 13:49:30 +10:00
Martin Schwenke
5ac67504ca Tests: Initial test code for LCP2 IP allocation algorithm.
Move struct ctdb_public_ip_list to ctdb_private.h and put some
definitions for some functions from ctdb_takeover.c there.  This
allows those functions to be called from unit tests.

Add ctdb_takeover_tests.c and the Makefile support to build it.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 9d34be0233edf3bc022345c0494c4b2a4d7f8480)
2011-07-29 09:01:36 +10:00
Martin Schwenke
ff1a81c872 IP allocation - add LCP2 algorithm.
The current non-deterministic IP allocation algorithm balances IPs
across the whole cluster.  It does not consider different
interfaces/VLANs/subnets, so these different groups of IPs aren't
generally well balanced.

This adds the LCP2 algorithm for IP allocation and allows it to be
enabled by setting the "LCP2PublicIPs" tunable to 1.

The LCP2 algorithm calculates the imbalance of a node by totalling the
squares of the distances between each IP on the node.  The IP distance
is defined as the length longest common prefix (LCP) of bits that is
found when comparing 2 IPs.  The imbalance of a cluster is the maximum
imbalance for any node.  At each step the algorithm selects an
allocation to the IP/node combination that results in the choosing the
allocation that best reduces the imbalance of the cluster.

The implementation splits out the IP allocation part of
ctdb_takeover_run() into new function ctdb_takeover_run_core(), and
then extracts out the basic IP assignment code into new functions
basic_allocate_unassigned() and basic_failback().  3 new functions
lcp2_init(), lcp2_allocate_unassigned() and lcp2_failback() implement
the LCP2 algorithm, and are hooked into ctdb_takeover_run_core().

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 61fc7fbd0235469df22deb6581c6bd47e30bc0be)
2011-07-29 09:01:17 +10:00
Ronnie Sahlberg
e707f23596 Update the delip command
Dont talloc_free(vnn) immediately but postphone it until later when
the eventscript callback has completed.

CQ S1026664

(This used to be ctdb commit 0a99e8742a261b1d3a2c8830f5c19ea6c2c47cad)
2011-07-29 08:50:48 +10:00
Rusty Russell
87ea4818bf eventscript: fix callback after free
ctdb_event_script_callback() takes a mem_ctx arg which it doesn't use, but
the implication is pretty clear, that when that mem_ctx is freed, the callback
shouldn't happen.  Indeed, Ronnie reproduced a case where that callback
refers to freed memory, in the ip reallocation code under stress.

So attach the callback to the mem_ctx they give us, and remove it from the
script state structure when that's freed.  It's a bit weird, but it works.

CQ: S1026179
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(This used to be ctdb commit 6fcd867cc835ef1ffc1c50964f135c346503d40c)
2011-07-29 08:50:39 +10:00
Ronnie Sahlberg
c93a968619 When trying to re-balance the ip assignment and shuffle ips from
nodes with many addresses to nodes with few addresses,
loop up to num_ips+5 times instead of only 5 times.

When we have very many public ips per node, we might need to loop more than
5 times or else we will exit without reaching optimal balance.

(This used to be ctdb commit aa8114a625a637277561a66c80bdece3c27e9e20)
2011-07-06 13:14:13 +10:00
Ronnie Sahlberg
f84bd3b5f1 Dont call the UPDATE event if both old and new interface is the same.
CQ S1018175

(This used to be ctdb commit 6a74515f0a1e24d97cee3ba05d89133aac7ad2b7)
2011-05-04 13:29:29 +10:00
Ronnie Sahlberg
ae35e9e5b2 Cleanup of logging messages/spamming
Reduce an infomational message about not performing ip reallocation
from NOTICE(the default) to INFO.
These messages are normal during startup or when stopped/banned when
we will be in recovery mode for a while.

Remove a messager in the loop waiting for initial startup to complete about
the generation being invalid. It is always invalid at this stage before we have
finished initial recovery.

Rate-limit the informational messages for CTDB_WAIT_UNTIL_RECOVERED
so that we only print them once per second for the first 60 seconds and after that only once per 10 minutes.
These messages are normal during startup, but we should not be logging them every second for cases where we will remain in recovery mode during startup for an extended period of time.
Such as if suspended or permabanned.

CQ S1023302

(This used to be ctdb commit 3a0af8780dc595acbed880f288fcbc4f62c862fb)
2011-05-04 10:42:32 +10:00
Ronnie Sahlberg
2f1395ce03 If the eventscript is finished but state->ctdb is NULL,
log an error and return.

(Need to find root cause for this is soo too.)

(This used to be ctdb commit 2e80d53b73fcba58ed5a72bab66c051691ccf719)
2011-04-12 06:36:42 +10:00
Ronnie Sahlberg
c04505724a IFACE handling. Assume links are always good on nstartup (they almost always
Simplify the handling of setting the links in the 10.interface eventscript
and remove the optimization to only call setifacelink on state change
to make the code simpler to read.

If a take ip event fails, flag the node as unhealthy.

Add a check to the interface script to check if the interface exists
or if it has been deleted.
So that we can capture and become UNHELTHY if someone deletes an interface
we are using to host public addresses.

(This used to be ctdb commit 4ab63d2a7262aff30d5eced184c294c9c9dd4974)
2011-04-11 07:40:05 +10:00
Mathieu Parent
a5a6140b7e Correction of spelling errors
* continous -> continuous
* activete  -> activate

(thanks to lintian)

See https://bugzilla.samba.org/show_bug.cgi?id=6935

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit fb6987c2f747d6dbf9bb3899a480124d1c242a90)
2011-03-23 00:35:23 +01:00
Ronnie Sahlberg
653c0901d7 This needs more testing first
Revert "ctdbd: call tdb_reopen_all() in freeze child."

This reverts commit 3d9828861c771a060923f3181fa8224e0122bffc.

(This used to be ctdb commit 55c3446c9ba82d24b1d7db92bc3611fd8027b7fb)
2011-03-21 14:25:53 +11:00
Rusty Russell
88c9dec166 ctdbd: call tdb_reopen_all() in freeze child.
In theory, the ctdbd parent shouldn't be holding any locks, but it's a good
idea to always call tdb_reopen_all() after a fork().

(This used to be ctdb commit 3d9828861c771a060923f3181fa8224e0122bffc)
2011-03-21 13:57:53 +11:00