1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

161 Commits

Author SHA1 Message Date
Michael Adam
e0a11369cf ctdb:vacuum: remove a superfluous and misleading comment
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-06-17 09:33:10 +02:00
Michael Adam
413f99f86e ctdb:vacuum: remove now unused ctdb_repack_tdb().
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-06-17 09:33:10 +02:00
Michael Adam
368683d7af ctdb:vacuum: use plain tdb_repack() instead of ctdb_repack_tdb()
Since we usually have 0 records left for repack-deletion,
repacking is essentially used for the purpose of defragmenting
the freelist, we can use the vanilla tdb_repack function.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-06-17 09:33:10 +02:00
Michael Adam
ec2f1abe48 ctdb:vacuum: remove a comment in ctdb_vacuum_and_repack_db()
The repack operation now mainly defragments the freelist
and does not usually delete any records any more.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-06-17 09:33:10 +02:00
Michael Adam
67e5746b23 ctdb:vacuum: adapt debug message for repacking.
Now we usually have records to delete == 0 after the preceding
vacuum run. Anyways, deletion is not a major aspect any more
of the repack run and will vanish soon.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-06-17 09:33:10 +02:00
Michael Adam
d8e110ed7d ctdb-vacuum: fix delete list counts in delete_marshall_traverse_first
when bumping skipped, decrement left, so the sum is correct

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Mar  6 03:32:33 CET 2014 on sn-devel-104
2014-03-06 03:32:33 +01:00
Michael Adam
5b81848e50 ctdb-vacuum: fix possible cause for delelete_list processing counts left records > 0
We need to have left records == 0 at the end of the delete list processing.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:16 +11:00
Michael Adam
551e9d791c ctdb-vacuum: systematize counters into various structs
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:16 +11:00
Michael Adam
ec3f33c314 ctdb-vacuum: remove unused counter vdata->total
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:15 +11:00
Michael Adam
49eb503c5d ctdb-vacuum: make ctdb_process_delete_list() void
The overall return code was not really used anyways.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:15 +11:00
Michael Adam
776d4e88f2 ctdb-vacuum: make ctdb_process_vacuum_fetch_lists() void.
This constantly returns 0 anyways.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:15 +11:00
Michael Adam
1994870299 ctdb-vacuum: make ctdb_vacuum_traverse_db() void.
Failure in traversal of the DB should not
prevent further processing.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:15 +11:00
Michael Adam
7983946680 ctdb-vacuum: don't stop in process_vacuum_fetch_lists when sending to one node fails.
We should try to continue vacuuming as much as possible.
Failure to send records to one lmaster doesn't mean the
others will fail too.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:15 +11:00
Michael Adam
f3483de240 ctdb-vacuum: catch and log errors to traverse the delete list in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:15 +11:00
Michael Adam
72ea975993 ctdb-vacuum: catch and log error of traverse in ctdb_process_delete_queue()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:14 +11:00
Michael Adam
83fa09e78c ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_marshall_traverse_first()
Spare malloc and free.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:14 +11:00
Michael Adam
974aa73160 ctdb-vacuum: simplify delete_marshall_traverse_first: use tdb_null
we know anyways the record to store is empty at this point.
So skip pointer calculations.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:14 +11:00
Michael Adam
16837bc309 ctdb-vacuum: remove VacuumLimit criterion for triggering a repack
With the new vacuuming, we consider it an error if there are
records left for deletion after processing the various lists.
All records that can be deleted should have been deleted by
tdb_delete calls.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:14 +11:00
Michael Adam
6fdd25008f ctdb-vacuum: treat value 0 of tunable VacuumLimit as turning off repacking
I.e. no number of records found to delete will trigger the
repacking.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:14 +11:00
Michael Adam
1b2fc1f096 ctdb-vacuum: add consistency check for counts at end of process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:13 +11:00
Michael Adam
cf407d208a ctdb-vacuum: log error if records are left for deletion after ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:13 +11:00
Michael Adam
23be632449 ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_record_traverse()
Spare malloc and free.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:13 +11:00
Michael Adam
be2f1a0c79 ctdb-vacuum: update comment for ctdb_process_delete_queue
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:13 +11:00
Michael Adam
cd877b6a21 ctdb-vacuum: rename ctdb_vacuum_db_fast --> ctdb_process_delete_queue
This describes more precisely what this does.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:13 +11:00
Michael Adam
c4478bd40d ctdb-vacuum: update comment for ctdb_vacuum_traverse_db
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:12 +11:00
Michael Adam
0309d5eda4 ctdb-vacuum: rename ctdb_vacuum_db_full --> ctdb_vacuum_traverse_db
This describes more precisely what it actually is, nowadays.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:12 +11:00
Michael Adam
ba49deb234 ctdb-vacuum: change full db traverse vacuuming to fill delete queue
This lets the "fast vacuum" delete queue traverse do the actual work.

On the positive side, we note that this lets the "full vacuuming"
treat the records that have never been migrated with data correctly.
These had previously been added to the delete list for complicated
cross-node deletion instead of directly deleting them.

On the other hand side, there might be a slight overhead
since the records are read again in the delete queu traverse,
but this is OK because this change is in preparation of
untangling the db traverse altogether from the vacuum run,
making it independent.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:12 +11:00
Michael Adam
d0b7b38825 ctdb-vacuum: run the fast vacuum after the db traverse
This in preparation of modifying the db traverse to
fill the delete_queue that is processed by the fast
vacuum run, instead of filling the same lists as the
fast vacuum run for further processing.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:12 +11:00
Michael Adam
04c2115b60 ctdb-vacuum: rename private->private_data in repack_traverse
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:12 +11:00
Michael Adam
951efa1097 ctdb-vacuum: rename private->private_data in vacuum_traverse
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:11 +11:00
Michael Adam
01f359cafc ctdb-vacuum: extract check for full vacuum run out of ctdb_vacuum_db_full()
This is more consistent.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:11 +11:00
Michael Adam
c88fd19714 ctdb-vacuum: add consistency check for counts to ctdb_vacuum_db_fast()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:11 +11:00
Michael Adam
5d5907c7cf ctdb-vacuum: use tdb_parse_record instead of tdb_fetch in delete_queue_traverse()
this spares malloc and free

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:11 +11:00
Michael Adam
fe68b3c494 ctdb-vacuum: simplify delete_record_traverse() - free treats NULL
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:11 +11:00
Michael Adam
593bddf2e8 ctdb-vacuum: simplify delete_queue_traverse() - free treats NULL pointers.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:10 +11:00
Michael Adam
24bec3d31b ctdb-vacuum: reduce indentation in delete_queue_traverse
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:10 +11:00
Michael Adam
48f2d11588 ctdb-vacuum: treat value 0 of tunable RepackLimit as turned off.
I.e. when RepackLimit is set to 0, no size of the freelist
should trigger a repack in vacuuming.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:10 +11:00
Michael Adam
af5568b267 ctdb-vacuum: fix treatment of remaining records and statistics in delete_record_traverse()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:10 +11:00
Michael Adam
b4e0b01a8c ctdb-vacuum: cast freelist_size in comparison.
At this point, it is >= 0 anyways.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:10 +11:00
Michael Adam
6a46a25530 ctdb-vacuum: improve output of delete list statistics
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-03-06 11:31:09 +11:00
Michael Adam
0535f73c3a ctdb:vacuum: move retrieval of freelist to after vacuum run
The fast vacuum run may have increased the freelist size.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Feb 14 03:15:30 CET 2014 on sn-devel-104
2014-02-14 03:15:30 +01:00
Michael Adam
bd474985b1 ctdb:vacuum: fix debug message typo in add_record_to_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-02-14 11:14:31 +11:00
Amitay Isaacs
1c21f37e57 ctdbd: Set process names for child processes
This helps distinguish processes in process list in top, perf, etc.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 2493f57ce268d6fe7e4c40a87852c347fd60d29e)
2013-07-10 14:33:19 +10:00
Mathieu Parent
d82b9ae410 build: Fix tdb.h path to enable building with system TDB library
(This used to be ctdb commit f8bf99de3a5f56be67aaa67ed836458b1cf73e86)
2013-06-14 16:45:27 +10:00
Michael Adam
32b34222b0 vacuum: use CTDB_REC_RO_FLAGS in the vacuuming code
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit a62775334aa20d1d850d2df705eb70303b04ac5c)
2013-04-24 18:48:53 +10:00
Michael Adam
e148458766 vacuum: Update (C)
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 61264debba58355b9716ac1637fdedef5ed249c8)
2013-04-24 18:48:26 +10:00
Michael Adam
6c98664365 vacuum: extend the header comment for ctdb_process_delete_list()
Describe the (new) process more precisely.
And mention that is the last step of the vacuuming process
that is performed on the lmaster.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 06de786c786f1cab4c6721adf47c2cb1e8a72adb)
2013-04-24 18:48:15 +10:00
Michael Adam
b17007ea48 vacuum: turn the vacuuming on lmaster into a three-phase process.
More precisely, before locally deleting an empty record, that has been
migrated with data and that we are dmaster and laster for, we now perform
the deletion on the other nodes in two steps instead of a single step.

- First send out the list of records to be deleted to all
  other nodes with the new RECEIVE_RECORDS control to store
  the lmaster's current empty copy.
- Then send those records that could be deleted on all nodes
  to all nodes again with the TRY_DELETE_RECORDS control
  as before for deletion.
- Finally delete those records locally that were successfully
  deleted remotely in the previous step.

This fixes an old race where a recovery that hits the vacuum process
square between the eyes can create gaps in the record's history and
hence let the records resurrect. In the case of the locking.tdb,
that could mean that a file that was already closed, was recorded as
being open and locked again, so samba clients were locked out of that
file until samba was restarted.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit eee23d44b6427be8ab49bbfcee3abb62f37dfcc7)
2013-04-24 18:47:40 +10:00
Michael Adam
f49d57c21d vacuum: reorder some of ctdb_process_delete_list() more intuitively
Now that the nodemap and its talloc children don't hang off of the
delete_records_list talloc context, we can build the nodemap
and earlier, and move the construction of the delete_records_list
to where it is more obvious what it is used for.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e3740899c1af6962f93c85ad7d1cb71bddce45c6)
2013-04-24 18:47:25 +10:00
Michael Adam
a0e0264986 vacuum: add explicit temporary memory context to ctdb_process_delete_list()
This removes the implicit artificial talloc hierarchy and makes the
code easier to understand.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit b7c3b8cdf92c597e621e3dae28b110d321de5ea8)
2013-04-24 18:47:18 +10:00