1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

41 Commits

Author SHA1 Message Date
Ronnie Sahlberg
59565c05cf STATISTICS: Add tracking of the 10 hottest keys per database measured in hopcount
and add mechanisms to dump it using the ctdb dbstatistics command

(This used to be ctdb commit 8307c70ed98996b430c470e9641a09fdeeb81bd8)
2012-06-13 16:19:18 +10:00
Ronnie Sahlberg
7a1aa560e7 Add new control to reload the public ip address file on a node
Also add a method to use the recovery master/daemon to reload the public ips on all nodes in the cluster.
Reloading the public ips on all node sin the cluster is only suported if all nodes in the cluster are available and healthy.

(This used to be ctdb commit 05603e914f8c12618d7e06943c0f7df207f645b0)
2012-05-01 10:48:08 +10:00
Ronnie Sahlberg
2456f77ca6 NoIPTakeover: change the tunable name for the "dont allow failing addresses over onto the node" to NoIPTakeover
(This used to be ctdb commit 35592e618cfd827b6978af6332f80504f232c46a)
2012-03-22 11:05:15 +11:00
Ronnie Sahlberg
befa9df152 Make NoIPFailback a node local setting. Nodes that have NoIPFailback set to !0 can not takeover new ip addresses during failover.
Remove the old global setting for this unused tunable and add it as a new node flag. This node flag is only valid/defined within the takeover subsystem in the recovery daemon. Add async functions to collec the NoIPFailback settings for each node.

This will later e used to disqualify certain nodes from being takeover targets when we perform reallocation.

(This used to be ctdb commit 668f3e88a9e5f598706952b7140547640c85a5ed)
2012-03-22 09:09:57 +11:00
Ronnie Sahlberg
fa3a06246a STICKY: add prototype code to make records stick to a node to "calm" down if they are found to be very hot and accessed by a lot of clients.
This can improve performance and stop clients from having to chase a rapidly migrating/bouncing record

(This used to be ctdb commit d0d98f7e45e5084b81335b004d50bddc80cdc219)
2012-03-20 17:12:19 +11:00
Ronnie Sahlberg
e7e51ddb64 LACOUNT: Add back lacount mechanism to defer migrating a fetched/read copy until after default of 20 consecutive requests from the same node
This can improve performance slightly on certain workloads where smbds frequently read from the same record

(This used to be ctdb commit 035c0d981bde8c0eee8b3f24ba8e2dc817e5b504)
2012-03-20 12:26:22 +11:00
Ronnie Sahlberg
6a493a0b08 STATISTICS: add per-db hop count statistics
(This used to be ctdb commit 1c976d83b1d7dac6f0ef81306774998e4c8b56a1)
2012-03-20 12:11:55 +11:00
Ronnie Sahlberg
038c946e80 add max hop count buckets to see how bad hopcounts are
(This used to be ctdb commit 7d3931298e6477d92f43652c3006b0c426cb1307)
2012-03-20 11:20:53 +11:00
Ronnie Sahlberg
ef2bd0b016 When adding ips to nodes, set up a deferred rebalance for the whole node to trigger after 60 seconds in case the normal ipreallocated is not sufficient to trigger rebalance.
(This used to be ctdb commit 4340263b219d75c39f8de22abe3f6f1c1ee63ea2)
2012-02-28 06:56:04 +11:00
Ronnie Sahlberg
73f8be16c6 ReadOnly: add per-database statistics to view how much delegations/revokes we have
(This used to be ctdb commit 751ed46197661eb841042ab6a02855a51dd0b17c)
2012-02-08 15:29:27 +11:00
Ronnie Sahlberg
1eafa68f0f STATISTICS: add total counts for number of delegations and number of revokes
Everytime we give a delegation to another node we count this as one delegation.
If the same record is delegated to several nodes we count one for each node.

Everytime a record has all its delegations revoked we count this as one revoke.

(This used to be ctdb commit b098bcf8007be63889aaed640a951b0eeaa9d191)
2012-02-08 13:42:30 +11:00
Martin Schwenke
ed8a8ee966 libctdb - add ctdb_getvnnmap()
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit f6039eaece4224b866a98dd49010f278a7b3f015)
2012-02-06 16:00:23 +11:00
Martin Schwenke
8b74037633 ctdb tool - generalise nodestring parsing for -n
Centralise -n nodestring parsing and add the ability to pass a
comma-separated list of node numbers.  Listing a node that is
disconnected or deleted results in failure, similar to the way passing
a single node currently works.  All of the auto_all commands inherit
this functionality.  For now, the non-auto_all commands do not inherit
this - they need to be individually tweaked.  Therefore, we haven't
updated the documentation to advertise this feature.

Implemented via a new function parse_nodestring() that parses an
optional (pass NULL when not available to indicate "current node")
comma-separated list of node numbers or "all".  parse_nodestring() can
be told to be non-fatal for disconnected/deleted nodes so it can also
be used in other contexts (yes, coming soon).  main() is changed to
call this function.

A new magic PNN value CTDB_MULTICAST is added and along with a
corresponding option.nodes structure member (a talloc-ed array of
PNNs).  This is also populated for "all" as well.

control_status() has new function pretty_print_flags() factored out so
pretty-printed flags can be used in error/debug messages.  New
function is_partially_online() is also factored out - this simplifies
some of the logic.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit 920e3a732eb9e09004edde6cfb3c7db8a004016f)
2011-12-08 17:00:17 +11:00
Ronnie Sahlberg
609149bdc8 LibCTDB: Add support for the 'get interfaces' control and update the ctdb tool to use this interface
(This used to be ctdb commit 77dc0c7351071243d9096d3607d7499c82f46ec0)
2011-12-06 13:12:18 +11:00
Michael Adam
ad0de5494e traverse: fix traversing with empty records by adding a new (internal) control CTDB_CONTROL_TRAVERSE_START_EXT
By this, the original CTDB_CONTROL_TRAVERSE_START control that is
used by e.g. samba's smbstatus, is not changed, so that samba
continues working without code change.

The  CTDB_CONTROL_TRAVERSE_START currently just adds the "withemptyrecords"
flag to the state and processon on as CTDB_CONTROL_TRAVERSE_START_EXT.

(This used to be ctdb commit 8281bb210858ed04992eacea7f6d02261e0fc1b1)
2011-12-03 02:15:30 +01:00
Volker Lendecke
5a1da0ac55 Add CTDB_CONTROL_CHECK_SRVID
(This used to be ctdb commit ad64ef2c40a2a12b37dbf39142e95c6781c2fc3b)
2011-11-30 09:02:26 +11:00
Michael Adam
1a31c84348 traverse: add a flag to enable transferring empty records in cluster wide traverse
This will be useful for also printing information about empty/deleted
records in "ctdb catdb", e.g. for debugging vacuuming issues.

(This used to be ctdb commit ddc5da3a0df7701934404192a0a0aa659a806acb)
2011-11-29 10:30:24 +01:00
Ronnie Sahlberg
44de394796 SRVID ranges: Change the ranges for SRVIDs to allow 8 bit prefixes
Update the ranges used for SRVID allocation to allow 8 bit prefixes and thus
56 user-defined bits.
Define the defacto-use of the 0x00 prefix as a SRVID used to register a process id
Upgrade SAMBA/iSCSI/NFS/TEST from a 32 bit prefix each ot a 8 bit prefix each
for private use.

(This used to be ctdb commit 5de9ec2bdf8067406165bc470becdca87f458ae9)
2011-11-09 08:12:44 +11:00
Ronnie Sahlberg
206a3c0c66 ReadOnly: add a new control to activate readonly lock capability for a database.
let all databases default to not support this  until enabled through this control

(This used to be ctdb commit 908a07c42e5135a3ba30a625fc4f4e4916de197a)
2011-09-01 11:08:18 +10:00
Ronnie Sahlberg
17f0e0890c ReadOnly: Add a new flag to call request packet to indicate that the client wants a readonly delegation
(This used to be ctdb commit a3f54a556e97170eedf43708d58dd32446ca5840)
2011-08-23 10:29:40 +10:00
Ronnie Sahlberg
8f63a5dadd ReadOnly: Add 4 new record flags to handle read only delegation and revoking of delegations
(This used to be ctdb commit 875b0bede217547b51f02648b6a28a3c98b6b949)
2011-08-23 10:17:08 +10:00
Ronnie Sahlberg
00a870f759 ReadOnly records: Add a new RPC function FETCH_WITH_HEADER.
This function differs from the old FETCH in that this function will also fetch the record header and not just the record data

(This used to be ctdb commit c7196d16e8e03bb2a64be164d15a7502300eae0e)
2011-08-23 10:06:59 +10:00
Michael Adam
827e871ec4 ctdb_private.h: add record flag CTDB_REC_FLAG_AUTOMATIC
This is a flag that shall signa that a record has been automatically generated by ctdb
and not by an explicit client store operation. This will be used in the ctdb_ltdb_fetch
operation which stores an empty record with default initial header before trying to
migrate the record from the dmaster when the record does not exist in the local tdb.

(This used to be ctdb commit 46381a3cb58ccc11422af8f7798c80ea8d72294f)
2011-03-14 13:35:51 +01:00
Michael Adam
46a05397a4 control: add a new control opcode CTDB_CONTROL_SCHEDULE_FOR_DELETION
(This used to be ctdb commit 4cebfa33db3c7effa087f753530c52b2dd8550e6)
2011-03-14 13:35:49 +01:00
Michael Adam
f7eeb42219 add a new record flag CTDB_REC_FLAG_VACUUM_MIGRATED.
This is to be used internally. The purpose is to flag a record
as been migrated by a VACUUM_MIGRATION, which is triggered by
a VACUUM_FETCH message as part of the vacuuming. The local store
routine will base its decision whether to delete or to store
the record (among other things) upon the value of this flag.

This flag should never be stored in the local database copies.

(This used to be ctdb commit dd2449c422f323f9b5485e45107a9cc5acc09e08)
2011-03-14 13:35:44 +01:00
Michael Adam
f3fbd31d85 call: Move definition of call flags down to the definition of the flags field.
(This used to be ctdb commit 86c844fb08a7fd33e94f56b8d5e43278120e1162)
2011-03-14 13:35:44 +01:00
Michael Adam
a2c11d6edc call: add new call flag CTDB_CALL_FLAG_VACUUM_MIGRATION
This is to be used when the CTDB_SRVID_VACUUM_FETCH message
triggers the migration of deleted records to the lmaster.
The lmaster can then delete records that have not been
migrated with data instead of storing them.

(This used to be ctdb commit 455cc6616e10b7f09589f9b87cb60f591bb502b0)
2011-03-14 13:35:44 +01:00
Ronnie Sahlberg
65f44e159f Add two new flags for the ltdb header.
One of which signals that the record has never been migrated to/from a node
while containing data.
This property "has never been migrated while non-zero" is important later
to provide heuristics on which records we might be able to purge
from the tdb files cheaply, i.e. without having to rely on the full-blown
database vacuum.

These records are belived to be very common and the pattern would look like
this :
1, no record exists at all.
2, client opens a file
3, samba requests the record for this file
4, an empty record is created on the LMASTER
5, the empty record is migrated to the DMASTER
6, samba writes a <sharemode> to the record locally and the record grows
7, client finishes working the file and closes the file
8, samba removes the sharemode and the record becomes empty again.
9, much later : vacuuming will delete the record

At stage 8, since the record has never been migrated onto a node wile being
non-zero it would be safe, and much more efficient to just delete the record
completely from the database and hand it back to the LMASTER.

The flags occupy the same uint32_t as was previously used for laccessor/lacount
in the header. For now, make sure the flags only define/use the top 16 bits
of this field so that we are sure we dont collide with bits set to one
from previous generations of the ctdb cluster database prior to this
change in semantics of this word.

This is a rework of Michaels patch :
commit 2af1a47cbe1a608496c8caf3eb0c990eb7259a0d
Author: Michael Adam <obnox@samba.org>
Date:   Tue Nov 30 17:00:54 2010 +0100

    add a DEFAULT record flag and a MIGRATED_WITH_DATA record flag.

(This used to be ctdb commit e075670dee8e6ecaba54986f87a85be3d0528b6b)
2011-02-18 10:14:56 +11:00
Ronnie Sahlberg
b57bd0f896 Remove LACOUNT and LACCESSOR and migrate the records immediately.
This concept didnt work out and it is really just as expensive as a full migration
anyway, without the benefit of caching the data for subsequence accesses.

Now, migrate the records immediately on first access.
This will be combined with a "cheap vacuum-lite" for special empty records to
prevent growth of databases.

Later extensions to mimic read-only behaviour of records will include proper shared read-only locking of database records, making the laccessor/lacount read-only access to the data obsolete anyway.

By removing this special case and handling of lacount laccessor makes the codapath where shared read-only locking will be be implemented simpler, and frees up space in the ctdb_ltdb header for use by vacuuming flags as well as read-only locking flags.

(This used to be ctdb commit 155dd1f4885fe142c6f8bd09430f65daf8a17e51)
2011-02-18 10:08:32 +11:00
Ronnie Sahlberg
fcd98a7e59 LIBCTDB: add support for traverse
(This used to be ctdb commit 9463e04038ba36792583f83bd95c1af322dc283a)
2011-01-14 17:38:56 +11:00
Ronnie Sahlberg
5ef29f9f25 Update latency countes to show min/max and average
(This used to be ctdb commit 1919e949af4641ffe919123e44b02fb87c13ab9f)
2010-10-11 15:12:24 +11:00
Ronnie Sahlberg
9f66a93f12 Add rolling statistics that are collected across 10 second intervals.
Add a new command "ctdb stats [num]" that prints the [num] most recent statistics intervals collected.

(This used to be ctdb commit e6e16fcd5a45ebd3739a8160c8fb5f44494edb9e)
2010-09-29 12:14:45 +10:00
Ronnie Sahlberg
22ea35f17d adda GETPUBLICIPS control to libctdb and use this in the test example
enhance the test example to show the new releaseip/takeip messages

(This used to be ctdb commit 21cc57883e6c02b0e037211b26d1d866d5d7f03d)
2010-09-15 14:58:11 +10:00
Ronnie Sahlberg
d8d8b9e1d7 add a new serverid to send a message everytime an ip address is taken on the local node
(This used to be ctdb commit 1261f3d9702800a4e59550c881350daf479f00ef)
2010-09-13 15:43:19 +10:00
Ronnie Sahlberg
991a6ae2a0 Update the comment for the range reserved for SAMBA and
define a new symbol to represent this range similarly to NFSD and ISCSID

Keep the old symbol name to be backward compatible with software using
these headers.

(This used to be ctdb commit 2ce34e50d057ba95249117a581658a5ad7e8eb60)
2010-09-13 15:10:36 +10:00
Ronnie Sahlberg
09a08b0da3 define and reserve a range of ctdb message ports for use by nfs and iscsi servers
(This used to be ctdb commit 84a44ac8ee74dd7af15e378c6cafbedb95feec60)
2010-09-13 15:10:24 +10:00
Ronnie Sahlberg
a2c874bd61 Implement a new function GETNODEMAP in libctdb.
This function returns a pointer to a nodemap structure.

The returned structure must later be freed by calling ctdb_free_nodemap().

Move the definition of ctdb_sock_addr from ctdb_client.h to ctdb_protocol.h

Move the definition of the node flags, ctdb_node_and_flags and ctdb_node_map from ctdb_private.h to ctdb_protocol.h

Add both sync and async example for ctdb_getnodemap to the test application libctdb/tst.c

(This used to be ctdb commit 31c10eb2b337fd7d8a97a1f9e69b0e7570fec71d)
2010-09-13 14:32:11 +10:00
Ronnie Sahlberg
c95f4258d8 Add a new event "ipreallocated"
This is called everytime a reallocation is performed.

    While STARTRECOVERY/RECOVERED events are only called when
    we do ipreallocation as part of a full database/cluster recovery,
    this new event can be used to trigger on when we just do a light
    failover due to a node becomming unhealthy.

    I.e. situations where we do a failover but we do not perform a full
    cluster recovery.

    Use this to trigger for natgw so we select a new natgw master node
    when failover happens and not just when cluster rebuilds happen.

(This used to be ctdb commit 7f4c591388adae20e98984001385cba26598ec67)
2010-08-30 18:09:30 +10:00
Ronnie Sahlberg
5aa5f3e7bf Remove the structure ctdb_control_tcp_vnn since this is identical to the structure ctdb_tcp_connection.
Add a new "ctdb deltickle" command to delete tickles from the database.
This can ONLY be used for tickles created by "ctdb addtickle".

Push any "addtickle/deltickle" updates to other nodes every TickleUpdateInterval seconds'

(This used to be ctdb commit acded034e2f0dcae4c2c9e54e16a001caf23caec)
2010-08-18 12:36:03 +10:00
Ronnie Sahlberg
2d4b98381f ctdb_req_control contains 4 padding bytes. Create an explicit pad variable here and set it to 0 when creating a control to keep valgrind happy.
PDUs are padded to 8 byte boundary. If padding is used, memset it to 0
to keep valgrind happy.

(This used to be ctdb commit 8818d5c483558c0faa6a3923ed5e675fdcfc13af)
2010-06-02 16:49:05 +10:00
Rusty Russell
d5f6026a22 libctdb: reorganize headers: remove ctdb.h, add ctdb_client.h and ctdb_protocol.h
ctdb_client.h is the existing internal client interface (which was mainly
in ctdb.h), and ctdb_protocol.h is the information needed for the wire
protocol only.

ctdb.h will be the new, shiny, libctdb API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(This used to be ctdb commit 4bba6b8cd47b352f98d41f9f06258d5ac3c9adef)
2010-05-20 15:18:30 +09:30