1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

118 Commits

Author SHA1 Message Date
Mathieu Parent
d82b9ae410 build: Fix tdb.h path to enable building with system TDB library
(This used to be ctdb commit f8bf99de3a5f56be67aaa67ed836458b1cf73e86)
2013-06-14 16:45:27 +10:00
Michael Adam
32b34222b0 vacuum: use CTDB_REC_RO_FLAGS in the vacuuming code
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit a62775334aa20d1d850d2df705eb70303b04ac5c)
2013-04-24 18:48:53 +10:00
Michael Adam
e148458766 vacuum: Update (C)
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 61264debba58355b9716ac1637fdedef5ed249c8)
2013-04-24 18:48:26 +10:00
Michael Adam
6c98664365 vacuum: extend the header comment for ctdb_process_delete_list()
Describe the (new) process more precisely.
And mention that is the last step of the vacuuming process
that is performed on the lmaster.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 06de786c786f1cab4c6721adf47c2cb1e8a72adb)
2013-04-24 18:48:15 +10:00
Michael Adam
b17007ea48 vacuum: turn the vacuuming on lmaster into a three-phase process.
More precisely, before locally deleting an empty record, that has been
migrated with data and that we are dmaster and laster for, we now perform
the deletion on the other nodes in two steps instead of a single step.

- First send out the list of records to be deleted to all
  other nodes with the new RECEIVE_RECORDS control to store
  the lmaster's current empty copy.
- Then send those records that could be deleted on all nodes
  to all nodes again with the TRY_DELETE_RECORDS control
  as before for deletion.
- Finally delete those records locally that were successfully
  deleted remotely in the previous step.

This fixes an old race where a recovery that hits the vacuum process
square between the eyes can create gaps in the record's history and
hence let the records resurrect. In the case of the locking.tdb,
that could mean that a file that was already closed, was recorded as
being open and locked again, so samba clients were locked out of that
file until samba was restarted.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit eee23d44b6427be8ab49bbfcee3abb62f37dfcc7)
2013-04-24 18:47:40 +10:00
Michael Adam
f49d57c21d vacuum: reorder some of ctdb_process_delete_list() more intuitively
Now that the nodemap and its talloc children don't hang off of the
delete_records_list talloc context, we can build the nodemap
and earlier, and move the construction of the delete_records_list
to where it is more obvious what it is used for.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e3740899c1af6962f93c85ad7d1cb71bddce45c6)
2013-04-24 18:47:25 +10:00
Michael Adam
a0e0264986 vacuum: add explicit temporary memory context to ctdb_process_delete_list()
This removes the implicit artificial talloc hierarchy and makes the
code easier to understand.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit b7c3b8cdf92c597e621e3dae28b110d321de5ea8)
2013-04-24 18:47:18 +10:00
Michael Adam
ebc77602fc vacuum: fix indentation in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 59a887e12469266e514ad7d4e34810e7ea888ba3)
2013-04-24 18:47:14 +10:00
Michael Adam
9778ce4b06 vacuum: free temporary allocated memory correctly in ctdb_process_delete_list().
Add a common exit point for cleanup.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 11d728465a9c635e1829abaae17e2f7720433b69)
2013-04-24 18:47:04 +10:00
Michael Adam
afb22c1e25 vacuum: move variable into scope of use in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 3710dd0f313f551f1b302b4961e0203243e3d661)
2013-04-24 18:46:56 +10:00
Michael Adam
2ead4053da vacuum: move variable into scope of use in ctdb_process_delete_list()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 4640979b526b6dac69a6a0555bfce75fe0206dac)
2013-04-24 18:46:52 +10:00
Michael Adam
79fc6c01d8 vacuum: simplify ctdb_process_delete_list(): reduce indentation
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f3e6e7f8ef22bd70dd2f101d818e2e5ab5ed3cd8)
2013-04-24 18:46:47 +10:00
Michael Adam
0a77ae018c vacuum: add DEBUG to skip conditions in delete_record_traverse()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 817c77a3d0a3546bf46389cec5f6b54778dd1693)
2013-04-24 18:46:42 +10:00
Michael Adam
81de2a13fb vacuum: break line for RO-flags check in delete_record_traverse() for readability
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-By: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 3f7e35ff0db740cdcb6d27c43a59bb6ca6066efb)
2013-04-24 18:46:34 +10:00
Volker Lendecke
83862c5c8d vacuum: Avoid some tallocs in ctdb recovery
In a heavily loaded and volatile database a lot of SCHEDULE_FOR_DELETION
requests can come in between fast vacuuming runs. This can lead to
significant ctdb cpu load due to the cost of doing talloc_free. This
reduces the number of objects a bit by coalescing the two objects
of delete_record_data into one. It will also avoid having to allocate
another talloc header for a SCHEDULE_FOR_DELETION key. Not the full fix
for this problem, but it might contribute a bit.

(This used to be ctdb commit 9a02f61547ddf74629aca21639d8fb61c1df7cbb)
2012-11-26 10:31:53 +11:00
Michael Adam
79468f338a vacuum: fix a comment typo
Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit bab744e3c49efef2e05dc09e8ea9bd3e3fa58716)
2012-11-19 14:53:14 +01:00
Ronnie Sahlberg
a57eba2bb4 Track all child process so we never send a signal to an unrelated process (our child died and kernel wrapped the pid-space and reused the pid for a different process
Wrap all creation of child processes inside ctdb_fork() which is used to track all processes we have spawned.
Capture SIGCHLD to track also which child processes have terminated.

Wrap kill() inside ctdb_kill() and make sure that we never send a !0 signal to a child process pid that has already terminated (and might have been replaced with a

(This used to be ctdb commit f73a4b1495830bcdd094a93732a89dd53b3c2f78)
2012-05-03 14:03:26 +10:00
Amitay Isaacs
4392591555 Remove explicit include of lib/tevent/tevent.h.
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 0681014ca5ed2a9b56f63fdace7f894beccf8a9a)
2012-04-13 17:28:14 +10:00
Ronnie Sahlberg
6f83805183 READONLY: skip vacuuming or deleting records with readonly delegations.
they are hot. wait until they have been revoked before we recall them.

(This used to be ctdb commit 7417d994c2a159f71d27d4bcd2f53684862eece3)
2012-02-29 16:09:24 +11:00
Ronnie Sahlberg
42e477b14e READONLY: only send a control to schedule fast-vacuuming from child context iff we have a connection open to the main daemon
there are some child processes where we do not create a connection to the main daemon (switch_from_server_to_client()) because it is expensive to set up and we normally might not need to talk to the daemon at all via a domainsocket.
but we might want to still call to ctdb_ltdb_store() from such chil processes.

(This used to be ctdb commit 9e372a08c40087e6b5335aa298e94d88273566a5)
2012-02-21 07:03:44 +11:00
Michael Adam
bf99bb65e0 vacuum: improve logging in remove_record_from_delete_queue()
(This used to be ctdb commit 809d1f3b8a9232fce0376cb47e4f633b49237529)
2011-12-23 17:39:13 +01:00
Michael Adam
ca7d2bf63a vacuum: improve logging in delete_record_traverse()
(This used to be ctdb commit 40ca781b3305d9f0bfee5b109d4eefde461a3f13)
2011-12-23 17:39:13 +01:00
Michael Adam
8348c431f2 vacuum: improve logging in insert_record_into_delete_queue()
(This used to be ctdb commit db59ac2ee6a9f5f7f088ebbc5e2800d9e57aae17)
2011-12-23 17:39:13 +01:00
Michael Adam
0a6805eed0 vacuum: improve debugging in delete_queue_traverse()
(This used to be ctdb commit 2c44c6dc583872c603a399afb01936abcbb05158)
2011-12-23 17:39:13 +01:00
Michael Adam
26cac0627e vacuum: add debug message to insert_record_into_delete_queue() for failed insert
(This used to be ctdb commit ae78e8d991636a3a79dd9fbcefc15032afb08f55)
2011-12-23 17:39:12 +01:00
Michael Adam
190b1b240c vacuum: improve debugging in insert_record_into_delete_queue()
(This used to be ctdb commit 6e6aaba28ac560d51d7450ebcf99babf868d0aea)
2011-12-23 17:39:12 +01:00
Michael Adam
cb259e6bd1 vacuum: initialize (to zero) the vacuum fetch list in ctdb_vacuum_init_vacuum_data()
(This used to be ctdb commit 772e08c29ef70a4cc73f7887939e5071a90a98cb)
2011-12-23 17:39:12 +01:00
Michael Adam
d29b2bd8bd vacuum: add statistics output for processing of the delete list
(This used to be ctdb commit bec2ff46402b6d90521c75e8d46b9f6f2f47239a)
2011-12-23 17:39:11 +01:00
Michael Adam
bdf395c2e2 vacuum: add a comment explaining a race in the vacuum code.
(This used to be ctdb commit 07f1ba54b1617fadd6fe83826dd8e76f37cea071)
2011-12-23 17:39:09 +01:00
Michael Adam
c10fc30a38 vacuum: factor out initialization of the vacuum data from ctdb_vacuum_db()
for readability

(This used to be ctdb commit 1963e33859e9dc51e9e88664c6d95e62c7c8b8be)
2011-12-23 17:39:09 +01:00
Michael Adam
4b565b1577 vacuum: streamline ctdb_process_delete_list()
Reducing indentiation and improving use of talloc.

(This used to be ctdb commit 11f52ba18370f1dda983f2263f9af82045b4fdea)
2011-12-23 17:39:08 +01:00
Michael Adam
50a9444626 vacuum: factor processing of the delete list out of ctdb_vacuum_db()
(This used to be ctdb commit cf68b762432a18e4a3cda137591b3f005c196906)
2011-12-23 17:39:08 +01:00
Michael Adam
4e5bdd45a0 vacuum: factor processing of vacuum fetch lists out of ctdb_vacuum_db()
(This used to be ctdb commit 905fc9cd7126ab84c23d4a803c59e32b9c54e6f0)
2011-12-23 17:39:08 +01:00
Michael Adam
1a462df86c vacuum: factor out full vacuum run out of ctdb_vacuum_db() as ctdb_vacuum_db_full()
(This used to be ctdb commit e31594e0c13899e88013ce7eb7e3dc434c2f0723)
2011-12-23 17:39:07 +01:00
Michael Adam
dbede3de7e vacuum: factor out the fast vacuuming run into ctdb_vacuum_db_fast()
for readability

(This used to be ctdb commit 6596f2c4a6b4783b6498502fbbf50f87d4f59085)
2011-12-23 17:39:07 +01:00
Michael Adam
61b336f230 vacuum: rename delete_tree to delete_list
It is not important for the user that this is implemented as a tree.

(This used to be ctdb commit f686bc440560074c1ce866f2037ae3671f182185)
2011-12-23 17:39:07 +01:00
Michael Adam
f96f183ad1 vacuum: appropriately rename vdata.list to vdata.vacuum_fetch_list
(This used to be ctdb commit 09c55b76c96c4ab3fe812763007b81cf0a2ac86c)
2011-12-23 17:39:06 +01:00
Michael Adam
d96df1c374 vacuum: introduce a helper variable for the vacuum fetch list in ctdb_vacuum_db()
(This used to be ctdb commit 019106e035520e6b3e5702b6e76d910626c3df13)
2011-12-23 17:39:06 +01:00
Michael Adam
c9b5396783 vacuum: introduce a helper variable in add_record_to_vacuum_fetch_list() to reduce verbosity
(This used to be ctdb commit 48fe56ea27e3649ae7a67257fdce54f973e7c3c5)
2011-12-23 17:39:06 +01:00
Michael Adam
c7b885c1c9 vacuum: locally delete the records in the delete_tree that have be deleted remotely
This way, especially records added in the fast vacuuming runs
are deleted direclty instead of being handed back to the
repack run which will probably not hit because of the vacuum limit.

(This used to be ctdb commit ea3e640a28178ddcb85285f4efec62ccba2483d9)
2011-12-23 17:39:05 +01:00
Michael Adam
aa04a64679 vacuum: improve vacuuming statistics for handling the delete_tree.
Track the number of records failed to delete remotely with the
TRY_DELETE_RECORDS control.
And add a number of records left to delete locally.

(This used to be ctdb commit 536aad024d52f2c32dd397fc86294c963b91341b)
2011-12-23 17:39:04 +01:00
Michael Adam
1c0bc40961 vacuum: fix and improve a comment
(This used to be ctdb commit 0ed349c24c7399b3b440c1f5d573db77ddb988b3)
2011-12-23 17:39:04 +01:00
Michael Adam
b09bd3e841 vacuum: appropriately rename delete_traverse() to delete_marshall_traverse()
(This used to be ctdb commit 7c5b27bc8dce9e30f2aff05297113da981f7b9a3)
2011-12-23 17:39:04 +01:00
Michael Adam
745eb05e02 vacuum: initialize the delete_count in the vacuum data
(This used to be ctdb commit 970b8fc60857ad2f18f45312d355c571fb2f0246)
2011-12-23 17:39:03 +01:00
Michael Adam
e30d1828f1 vacuum: fix the VacuumInterval tunable's type to uint32_t in get_vacuum_interval()
(This used to be ctdb commit a5a29bf92b23e9b5a1523b06461eced422b22ffe)
2011-12-23 17:39:03 +01:00
Michael Adam
e04fad0ee4 vacuum: add new tunable VacuumInterval and mark Vacuum{Default,Min,Max}Interval obsolete
And use VacuumInterval instead of VacuumDefaultInterval in the code.

(This used to be ctdb commit 78530f40338f511a7cd1d33ada450905742bfa8f)
2011-12-23 17:39:02 +01:00
Michael Adam
1ca3abc5ba vacuum: remove struct vacuum_tuning_data (unused)
(This used to be ctdb commit 1f62afbe1ae0b3e7dbad4081d1b51e3ab6ee5c39)
2011-12-23 17:39:02 +01:00
Michael Adam
9cae401943 vacuum: use get_vacuum_interval() in ctdb_vacuum_event()
(instead of using the tunable directly)

(This used to be ctdb commit e40fa9efd1c77aa35c917aba6e6e4d21d9c5cdea)
2011-12-23 17:39:02 +01:00
Michael Adam
19bbee2acd vacuum: change get_vacuum_interval() to simply return the VacuumDefaultInterval tunable
(This used to be ctdb commit 621c3ce957cb946ca05a2d527c037746ae2f01d3)
2011-12-23 17:39:01 +01:00
Michael Adam
87351d8801 vacuum: remove update_tuning_db()
(This used to be ctdb commit d93750ea3f5cb854e5893e53795fd08656e7c12c)
2011-12-23 17:39:01 +01:00