1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-25 06:04:04 +03:00

200 Commits

Author SHA1 Message Date
Michael Adam
122682be63 vacuum: add statistics output to the fast and full traverse runs.
(This used to be ctdb commit 3addd28aa73883b3b05888e309d19db0eb67eab9)
2011-03-14 13:35:49 +01:00
Michael Adam
e8b08ce702 vacuum: refactor insert_delete_record_data_into_tree() out of add_record_to_delete_tree()
for reuse in filling the delete_queue.

(This used to be ctdb commit 7bbb12695c24da25671f1c39a411295d35870d2c)
2011-03-14 13:35:48 +01:00
Michael Adam
c95b4f841b vacuum: disable full db-traverse vacuuming runs when VacuumFastPathCount == 0
(This used to be ctdb commit 571683e7c48aeed8ce41c584d016ced7ff0d2e2d)
2011-03-14 13:35:48 +01:00
Michael Adam
28fdb00a22 vacuum: Only run full vacuumig (db traverse) every VacuumFastPathCount times.
(This used to be ctdb commit 23b8c8c5fc8604ee0bd6da1f4b5152277eb5f1c0)
2011-03-14 13:35:48 +01:00
Michael Adam
cab1f75db5 vacuum: reset the fast path count in the event handle if it exceeds the limit.
(This used to be ctdb commit 91e6d36a190b1c9e4c8b18f7833e51c5c9a67574)
2011-03-14 13:35:48 +01:00
Michael Adam
f502e5aa98 vacuum: bump the number of fast-path runs in the vacuum child destructor
(This used to be ctdb commit c0668bfe0bb4e69988ae34d875568d08539e6fb9)
2011-03-14 13:35:47 +01:00
Michael Adam
3c4c560019 vacuum: add a fast_path_count to the vacuum_handle.
(This used to be ctdb commit 53a39d0cc5ea251c2189ec8178ccb769fa046c43)
2011-03-14 13:35:47 +01:00
Michael Adam
8395fcc079 vacuum: traverse the delete_queue befor traversing the database.
(This used to be ctdb commit 04c335f9195a5fd83c91a57d06b1e4eaa511844e)
2011-03-14 13:35:47 +01:00
Michael Adam
bb22135f28 vacuum: add delete_queue_traverse() for traversal of the delete_queue.
(This used to be ctdb commit 5eee05c4d256c08f4ee60a1a69efda6844e39729)
2011-03-14 13:35:47 +01:00
Michael Adam
08ab829f0b vacuum: reduce indentation in add_record_to_delete_tree()
This simplyfies the logical structure a bit by using early return.

(This used to be ctdb commit 4d32908fdcec120426536a761e1d0be60f076198)
2011-03-14 13:35:47 +01:00
Michael Adam
6a1857ee70 vacuum: refactor new add_record_to_delete_tree() out of vacuum_traverse().
This will be reused by the traversal of the delete_queue list.

(This used to be ctdb commit 4407e5a7fb045ce56b6d902f7116de663ea648cb)
2011-03-14 13:35:46 +01:00
Michael Adam
d331d7a7b3 vacuum: skip adding records to list of records to send to lmaster on lmaster
This list is skipped afterwards when the lists are processed.

(This used to be ctdb commit e99834c1a2eea60f7f974c0689ae0a65cfe178ff)
2011-03-14 13:35:46 +01:00
Michael Adam
3d7bfc7a3e vacuum: refactor new add_record_to_vacuum_fetch_list() out of vacuum_traverse().
This is the function that fills the list of records to send to each lmaster
with the VACUUM_FETCH message.

This function will be reused in the traverse function for the delete_queue.

(This used to be ctdb commit d4ab790c1f679e833eb97816762fcfcee15ccb10)
2011-03-14 13:35:46 +01:00
Michael Adam
ee593284d5 server: rename ctdb_repack_db() to ctdb_vacuum_and_repack_db()
(This used to be ctdb commit 6c603f85726d2efac9710af7c4875ded2ca7230e)
2011-03-14 13:35:46 +01:00
Michael Adam
6a1cfe82f1 vaccum: clear the fast-path vacuuming delete_queue after creating the vacuuming child.
Maybe we should keep a copy for the case that the vacuuming fails?

(This used to be ctdb commit f19fe5b45748a6998c6950a5b1db7ec2c4468c1c)
2011-03-14 13:35:45 +01:00
Michael Adam
3cb3620acf vacuum: check lmaster against num_nodes instead of vnn_map->size
When lmaster is bigger than the biggest recorded node number,
then exit the traverse with error.

(This used to be ctdb commit 3930c7796b72bbf275bbca8aaeceec3e705a964b)
2011-03-14 13:35:43 +01:00
Michael Adam
dd75ae8c15 vacuum: reduce indentation of the loop sending VACUUM_FETCH controls
This slightly improves the code structure in that loop.

(This used to be ctdb commit bc4990e600c53433a924a0d70e3488a5a6bdc1ff)
2011-03-14 13:35:43 +01:00
Michael Adam
70fa7394fa vacuum: correctly send TRY_DELETE_RECORDS ctrl to all active nodes
Originally, the control was sent to all records in the vnn_map, but
there was something still missing here:
When a node can not become lmaster (via CTDB_CAPABILITY_LMASTER=no)
then it will not be part of the vnn_map. So such a node would
be active but never receive the TRY_DELETE_RECORDS control from a
vacuuming run.

This is fixed in this change by correctly building the list of
active nodes first in the same way that the recovery process does it.

(This used to be ctdb commit 49247df4a47a8a107fa7dd7b187e69e243e6bdbe)
2011-03-14 13:35:43 +01:00
Michael Adam
33b5d9c1a1 vacuum: in ctdb_vacuum_db, fix the length of the array of vacuum fetch lists
This patch fixes segfaults in the vacuum child when at least one
node has been stopped or removed from the cluster:

The size of the vnn_map is only the number of active nodes
(that can be lmaster). But the node numbers that are referenced
by the vnn_map spread over all configured nodes.

Since the array of vacuum fetch lists is referenced by the
key's lmaster's node number later on, the array needs to
be of size num_nodes instad of vnn_map->size.

(This used to be ctdb commit 136508e3f4dd0acc210dde938ad59ef38b63d3a1)
2011-03-11 23:00:51 +01:00
Michael Adam
8d49f4878c Fix typos in a comment in vacuum_traverse.
(This used to be ctdb commit 6a4df8242ee4d095ff03229a168b83bcd84c8a7a)
2011-03-09 21:14:14 +01:00
Ronnie Sahlberg
c4006ce844 Add ctdb_fork(0 which will fork a child process and drop the real-time
scheduler for the child.

Use ctdb_fork() from callers where we dont want the child to be running
at real-time privilege.

(This used to be ctdb commit 58795a4c9e0624e20fa3e0023b65127053edd103)
2011-01-11 07:40:41 +11:00
Ronnie Sahlberg
ef2e8d3c95 If tdb_open() fails when trying to open the vacuuming database,
print errno so we get some idea of why this failed.

(This used to be ctdb commit 3695462ade497356d1b86c6fd1bc1765f1d11e57)
2010-10-14 14:03:19 +11:00
Ronnie Sahlberg
f98ffde65a Dont log a normal vacuuming message about a missing record and using default vacuuming intervals as an error.
This is normal for a new system until the vacuuming has been initialized.

(This used to be ctdb commit ffd5fdd23b1cb07078759a78cd1d884f92aa4851)
2010-10-07 14:40:24 +11:00
Ronnie Sahlberg
2e8aac6689 Merge commit 'rusty/ports-from-1.0.112' into foo
(This used to be ctdb commit 13e58d92f5f1723e850a82ae030d0ca57e89b1ee)
2010-08-19 13:17:56 +10:00
Ronnie Sahlberg
4c05f1900c Merge commit 'rusty/vacuum-fix-master'
(This used to be ctdb commit dc301b324d2c14a2425a965c076113c4fe97903e)
2010-08-19 13:16:35 +10:00
Rusty Russell
9fbb191b78 logging: give a unique logging name to each forked child.
This means we can distinguish which child is logging, esp. via syslog where we have no pid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(This used to be ctdb commit 68b3761a0874429b90731741f0531f76dcfbb081)
2010-08-18 11:46:32 +09:30
Rusty Russell
5f2d43157d vacuum: disabling vacuuming during a freeze
We shouldn't even think about vacuuming when we've frozen the database
(which is earlier than when we set CTDB_RECOVERY_ACTIVE)

CQ:S1018154 & S1018349
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(This used to be ctdb commit d8df6835a931082af232c4b94f1dede6f16169f9)
2010-08-18 11:01:52 +09:30
Rusty Russell
0b07f91d36 vacuum: fix crash on vacuum abort
Martin Schwenke discovered that 517f05e42f17766b1e8db8f1f4789cbad968e304
("freeze: abort vacuuming when we're going to freeze.") used ctdb_db for
a logging message which is in fact uninitialized, causing a crash (even
if it wasn't actually logged).

Initialize it properly.  Also fix incorrect format in another logging
message introduced in that same change.

CQ:S1019093
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(This used to be ctdb commit 8e518950ba281502318d6300f7a5ec6cdf6b5674)
2010-08-18 11:00:11 +09:30
Rusty Russell
af55c910a4 freeze: abort vacuuming when we're going to freeze.
There are some reports of freeze timeouts, and it looks like vacuuming might
be the culprit.  So we add code to tell them to abort when a freeze is
going on.

(This is based on the 1.0.112 branch version 517f05e42f, but far
 simpler since tdb is now robust against processes being killed during
 transaction commit)

CQ:S1018154 & S1018349
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(This used to be ctdb commit f5d7dc679501e607c2c83a248a89d3cada9df146)
2010-08-18 10:54:28 +09:30
Rusty Russell
f93440c4b7 event: Update events to latest Samba version 0.9.8
In Samba this is now called "tevent", and while we use the backwards
compatibility wrappers they don't offer EVENT_FD_AUTOCLOSE: that is now
a separate tevent_fd_set_auto_close() function.

This is based on Samba version 7f29f817fa939ef1bbb740584f09e76e2ecd5b06.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


(This used to be ctdb commit 85e5e760cc91eb3157d3a88996ce474491646726)
2010-08-18 09:16:31 +09:30
Ronnie Sahlberg
761a075de9 rename ctdb_send_message to ctdb_client_send_message to resolve colission with the function of the same name in libctdb
(This used to be ctdb commit ac3292c12832484a22715f1d46aa23f3b7c8a6f6)
2010-06-02 09:45:21 +10:00
Christian Ambach
3cf183ca40 adjust a vacuum log level
made the severity of the decreasing interval log level the same as for the increasing,
they are both just info logs because they don't report errors

(This used to be ctdb commit fde29921f14a815ea68911d758485c9070f4eb2a)
2010-03-11 18:11:57 +11:00
Ronnie Sahlberg
d7c00d8d7e Drop the debug level for logging fd creation to DEBUG_DEBUG
(This used to be ctdb commit eae1d4f9e52e73b4d8769868fffdafa590d03784)
2010-02-04 06:37:41 +11:00
Ronnie Sahlberg
4c722fe34c fix a conflict in the merge from rusty
Merge commit 'rusty/ctdb-no-setsched'

Conflicts:

	server/ctdb_vacuum.c

(This used to be ctdb commit b4365045797f520a7914afdb69ebd1a8dacfa0d9)
2009-12-17 08:18:04 +11:00
Rusty Russell
f148735928 Add --valgringing flag instead of --nosetsched
The do_setsched was being tested for whether to mmap tdbs: let's make it
explicit.  We can also happily move the kill-child eventscript hack under
this flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> 


(This used to be ctdb commit 2ee86cc1f311d7b7504c7b14d142b9c4f6f4b469)
2009-12-16 20:59:15 +10:30
Stefan Metzmacher
7f05a423e2 server: create vactune.tdb.X with 0600 permissions
metze

(This used to be ctdb commit 21677ed6fb8c589f348321533c608cad58c4ec93)
2009-12-16 08:03:56 +01:00
Stefan Metzmacher
473f02ed48 server: create vactun.tdb.X under /var/ctdb/state
metze

(This used to be ctdb commit 1db17f312558fe59983a3465680e56c9f0c19e36)
2009-12-16 08:03:56 +01:00
Stefan Metzmacher
003985acfd ctdb: pass TDB_DISALLOW_NESTING to all tdb_open/tdb_wrap_open calls
metze

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 1635e931b909c66eb3b1f5357e3a549b1a0da70d)
2009-12-16 08:03:55 +01:00
Christian Ambach
47f8c380d2 reduce vacuuming lognoise
syslog.h says:

LOG_NOTICE      5    normal but significant condition
LOG_INFO        6    informational

several vacuuming related logs logged at NOTICE level although I don't see
any real significance, these are just informational messages for me

Signed-off-by: Christian Ambach <christian.ambach@de.ibm.com>

(This used to be ctdb commit 142111983c103e90ccccbe26fd580c4eb28e949f)
2009-12-10 07:33:59 +11:00
Ronnie Sahlberg
a3d072049e reduce the log level for three vacuuming related log messages
(This used to be ctdb commit fbc453733d53359b9eba34a7ca9123237a7ecca5)
2009-11-24 09:27:22 +11:00
Ronnie Sahlberg
d8f7fd88ac dont use the pointer after it has been talloc_free()d.
(This used to be ctdb commit 1cbf06a126621b3e932925cdad2ef9c009f93d4e)
2009-11-05 16:07:23 +11:00
Ronnie Sahlberg
4bf4e15379 move the check to skip vacuuming on persistent database to the ctdb_vacuuming_init() function
(This used to be ctdb commit fb83dba255fc91413a475b273e374e0c4d538137)
2009-11-03 10:48:27 +11:00
Wolfgang Mueller-Friedt
2c137b7030 vacuuming needed additional check before getting rid of the record; there is a gap between selecting the records and deleting them, therefore we have to check if the records still can be deleted when we actually are about to delete them
(This used to be ctdb commit a6fbc65aca35c41c428a82d7402e43c6eaac1d6e)
2009-10-29 13:45:17 +11:00
Ronnie Sahlberg
5aaa15fdb2 lower the log level of a debug message
(This used to be ctdb commit 496dc2e80b714811c6e69dc928deaad61cf603b1)
2009-10-26 09:35:18 +11:00
Ronnie Sahlberg
28f277acd4 From Wolfgang Mueller
make sure to always create the vactun database and get rid of some annoying log messages

(This used to be ctdb commit 54f9c314a0354f1039208fe6ac7dc159b6db8750)
2009-10-20 13:01:15 +11:00
Ronnie Sahlberg
9de3652380 add logging everytime we create a filedescriptor in the main ctdb daemon
so we can spot if there are leaks.

plug two leaks for filedescriptors related to when sending ARP fail
and one leak when we can not parse the local address during tcp connection establish

(This used to be ctdb commit ddd089810a14efe4be6e1ff3eccaa604e4913c9e)
2009-10-15 11:24:54 +11:00
Ronnie Sahlberg
c971d934a9 From Wolfgang Mueller-Friedt
Remove the explicit vacuum/repack commands from the 00.ctdb eventscript
and implement this in the ctdb daemon.

Combine vacuuming and repacking into one
cheap read traverse to enumerate all candidate records
and one write traverse that both repacks the database and also deletes the record locally where we are lmaster and where the records have already been deleted remotely.

this code also adds initial autotuning heuristics for the vacuum intervals and how many records to delete in each iteration.

minor stylish changes made by ronnie s

(This used to be ctdb commit 95a3ee551241aa164967991fe5efe078e1714bde)
2009-09-29 13:27:19 +10:00
Wolfgang Mueller-Friedt
16af87bf25 repack limit tunable
Signed-off-by: Wolfgang Mueller-Friedt <wolfmuel@de.ibm.com>

(This used to be ctdb commit a2768b0732f2ab2e3fafda55587bd2e99eedf0fa)
2009-07-29 13:30:39 +10:00
Wolfgang Mueller-Friedt
fddfaaaa4b added event repacking
Signed-off-by: Wolfgang Mueller-Friedt <wolfmuel@de.ibm.com>

(This used to be ctdb commit 78466364f22d6a183710338f138b8c808c6b7753)
2009-07-29 13:28:48 +10:00
Ronnie Sahlberg
1653af16a6 vacuum event framework
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Signed-off-by: Wolfgang Mueller-Friedt <wolfmuel@de.ibm.com>

(This used to be ctdb commit 30cdad97706a9e9bb210120699aa939f6b16e8ca)
2009-07-29 13:26:29 +10:00