1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

109 Commits

Author SHA1 Message Date
Amitay Isaacs
d9e4622a44 ctdb-locking: Run debug locks script only if the node is active
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-05 07:05:10 +02:00
Martin Schwenke
c1558adeaa ctdb: Use sys_read() and sys_write() to ensure correct signal interaction
... and avoid compiler warnings in some cases.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-08-21 04:46:13 +02:00
Volker Lendecke
e185ff22ca ctdb-locking: Simplify ctdb_find_lock_context()
I like early returns that avoid else branches :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Aug  6 14:44:31 CEST 2014 on sn-devel-104
2014-08-06 14:44:31 +02:00
Volker Lendecke
9f596c17c7 ctdb-locking: TALLOC_FREE copes with NULL
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-08-06 12:22:13 +02:00
Amitay Isaacs
88f6a6c188 ctdb-locking: Add per database queues for pending and active lock requests
This avoids traversing a single pending queue which is quite expensive
when there are lots of pending lock requests.  This seems to happen
quite a lot on a loaded cluster for notify_index.tdb.

Adding per database queues avoids the need to traverse pending queue
for that database if there are already the maximum number of active
lock requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Aug  4 20:23:45 CEST 2014 on sn-devel-104
2014-08-04 20:23:45 +02:00
Amitay Isaacs
f73adff737 ctdb-locking: Update a comment
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
a890e760bb ctdb-locking: Simplify check for locks on record or database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
aa1ff305f9 ctdb-locking: Decrement pending statistics when lock is scheduled
and not when the lock is obtained.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
dce68a2141 ctdb-locking: Update ctdb statistics for all lock types
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
7189437be4 ctdb-locking: Add DB lock requests to head of the pending queue
This allows to schedule DB locks quickly without having to scan through
the pending lock requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
3aa96c3a3e ctdb-locking: Remove unused variable lock_num_pending
The number of pending locks displayed in ctdb statistics are stored in
ctdb_statistics structure and not ctdb_context.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
59d45ea307 ctdb-locking: Add new tunable LockProcessesPerDB
This allows to change the maximum number of lock processes that can
be active.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
e0d5459451 ctdb-locking: Allocate lock request soon after allocating lock context
This avoids extra work in case lock request allocation fails.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
97a5c57957 ctdb-locking: Remove unused function find_lock_context()
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
c9664b4b17 ctdb-locking: Schedule the next possible lock based on per-db limit
This prevents searching through active lock requests for every pending
lock request to check if the pending lock request can be scheduled or not.
The locks are scheduled in strict first-in-first-out order.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
19b3810b61 ctdb-locking: Remove multiple lock requests per lock context (part 2)
Store only a single request instead of storing a queue in lock context.
Lock request structure does not need to be a linked list any more.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
a89f350879 ctdb-locking: Remove multiple lock requests per lock context (part 1)
This was a bad idea and caused out of order scheduling of lock requests.

The logic to append lock requests to existing lock context is already
commented.  Remove the commented code and there is no need to check if
lock_ctx is NULL, since we are always creating a new one.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
b93d9c0622 ctdb-locking: Remove unused structure members
block_child was used to keep track of a process which was created to debug
why a lock process has blocked.  That logic was replaced to execute an
external debug script.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
8aa6c039ae ctdb-locking: Fix the lock_type_str corresponding to LOCK_ALLDB
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Volker Lendecke
8dede57741 ctdb: Fix a comment typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Apr 30 21:05:09 CEST 2014 on sn-devel-104
2014-04-30 21:05:09 +02:00
Amitay Isaacs
f99a759ce7 ctdb-locking: Instead of comparing key, compare key hash
This replaces memory comparison of the key with integer comparison.
In addition, this also avoids scheduling locks with the same hash.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Mar 28 05:28:58 CET 2014 on sn-devel-104
2014-03-28 05:28:58 +01:00
Amitay Isaacs
d1c8f8d635 ctdb-locking: Check for talloc_memdup error
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-03-28 03:30:09 +01:00
Amitay Isaacs
ff1c85ae27 ctdb-locking: Avoid memory leak
If lock_request could not be allocated, free lock_ctx since there can
only be a single lock_request per lock_ctx.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-03-28 03:30:09 +01:00
Amitay Isaacs
4ea721b2c1 ctdb-locking: Use vfork instead of fork to exec helpers
There is a significant overhead using fork() over vfork(), specially
when the child process execs a helper.  The overhead is in memory space
and time.

    # strace -c ./test_fork 1024 200
    count=1024, size=204800, total=200M
    failed fork=0
    time for fork() = 4879.597000 us
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
    100.00    4.543321        3304      1375       375 clone
      0.00    0.000071           0      1033           mmap
      0.00    0.000000           0         1           read
      0.00    0.000000           0         3           write
      0.00    0.000000           0         2           open
      0.00    0.000000           0         2           close
      0.00    0.000000           0         3           fstat
      0.00    0.000000           0         3           mprotect
      0.00    0.000000           0         1           munmap
      0.00    0.000000           0         3           brk
      0.00    0.000000           0         1         1 access
      0.00    0.000000           0         1           execve
      0.00    0.000000           0         1           arch_prctl
    ------ ----------- ----------- --------- --------- ----------------
    100.00    4.543392                  2429       376 total

    # strace -c ./test_vfork 1024 200
    count=1024, size=204800, total=200M
    failed fork=0
    time for fork() = 82.041000 us
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
     96.47    0.001204           1      1000           vfork
      3.53    0.000044           0      1033           mmap
      0.00    0.000000           0         1           read
      0.00    0.000000           0         3           write
      0.00    0.000000           0         2           open
      0.00    0.000000           0         2           close
      0.00    0.000000           0         3           fstat
      0.00    0.000000           0         3           mprotect
      0.00    0.000000           0         1           munmap
      0.00    0.000000           0         3           brk
      0.00    0.000000           0         1         1 access
      0.00    0.000000           0         1           execve
      0.00    0.000000           0         1           arch_prctl
    ------ ----------- ----------- --------- --------- ----------------
    100.00    0.001248                  2054         1 total

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
0eeb73c187 ctdb-locking: Update current lock statistics when lock is scheduled
When a child process is created for a lock request, the current locks
statistics should be updated immediately.  This will provide accurate
information on number of active lock requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
3879e9991f ctdb-locking: Do not merge multiple lock requests to avoid unfair scheduling
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
094f34e9bf ctdb-locking: Implement active lock requests limit per database
This limit was currently a global limit and not per database.  This
prevents any database freeze lock requests from getting scheduled if
the global limit was reached.

Only individual record requests should be limited and database freeze
requests should always get scheduled.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
6d1b74f052 ctdb-server: Coverity fixes
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-19 17:13:03 +01:00
Martin Schwenke
463a091a77 ctdbd: Debug locks by default with override from enviroment variable
Default is debug_locks.sh, relative to CTDB_BASE.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit c11803e3dcc905a45a08d743595e63f9ca445f0d)
2013-10-22 15:37:54 +11:00
Amitay Isaacs
477a51aba5 locking: Do not create multiple lock processes for the same key
If there are multiple lock helper processes waiting for the same record, then
it will cause a thundering herd when that record has been unlocked.  So avoid
scheduling lock contexts for the same record.  This will also mean that
multiple requests will get queued up behind the same lock context and can be
processed quickly once the lock has been obtained.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit ebecc3a18f1cb397a78b56eaf8f752dd5495bcc9)
2013-08-09 11:04:55 +10:00
Amitay Isaacs
9ba793a80f locking: Move function find_lock_context() before ctdb_lock_schedule()
So that ctdb_lock_schedule() can call this function without requiring extra
prototype declaration.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 68af5405acc123b5a90decd2123e2a02961a8fcf)
2013-08-09 11:04:42 +10:00
Amitay Isaacs
c620457c0b locking: Use external script to debug locking issues
Use an external script to parse /proc/locks and log useful debugging
information about locks rather than doing that in C code.

To use this feature, add configuration variable to /etc/sysconfig/ctdb:

  CTDB_DEBUG_LOCKS=/etc/ctdb/debug_locks.sh

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 2bfb8499366d530f16515b08928056bbda40f781)
2013-07-10 14:33:18 +10:00
Amitay Isaacs
9ae379c91a locking: Update locking bucket intervals
0   < 1 ms
 1   < 10 ms
 2   < 100 ms
 3   < 1 s
 4   < 2 s
 5   < 4 s
 6   < 8 s
 7   < 16 s
 8   < 32 s
 9   < 64 s
10   >= 64 s

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 6fc36a7036933237d09151a0baf4d8ccd2bc2c99)
2013-07-10 14:33:18 +10:00
Amitay Isaacs
1afb7fccb2 locking: Update locks latency in CTDB statistics only for RECORD or DB locks
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit dcc42a75b4638b3aa40c44ed9e0aaae26483e2b0)
2013-07-10 14:33:18 +10:00
Amitay Isaacs
6391f61fbc build: Fix compiler warnings for uninitialized variables
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 5408c5c4050539e5aa06a5e82ceb63a6cb5cef0c)
2013-07-04 20:43:52 +10:00
Amitay Isaacs
c8d577eb80 locking: Set lock helper path once
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 80fbe9364350d42658f7f8af250ac87eb1afbc21)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
1ddc7b0d10 locking: Remove functions that are not used anymore
These functions were used in locking child process to do the locking.  With
locking helper, these are not required.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit c660f33c3eaa1b4a2c4e951c1982979e57374ed4)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
90c4fa77b9 locking: Remove functions that are not used anymore
These functions were used in locking child process to do the locking.  With
locking helper, these are not required.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 6ea3212a7b177c6c06b1484cf9e8b2f4036653d9)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
ae25420e56 locking: Use separate locking helper binary for locking
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7cde53a6cbe74b1e46f7e1bca298df82c08de866)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
e30978eae1 locking: Create commandline arguments for locking helper
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f665e3d540c90579952e590caa5828acb581ae61)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
c9f4589c13 locking: Use database iterator for unmarking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7630ca4116b476636c27407748088ea335f1a06c)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
65a9195916 locking: Add handler function for unmarking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit adc113055de98fae276f9b501aff5c03cd25ddc8)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
a5133d16e7 locking: Use database iterator for marking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e8ea65b2713417db4a618a9f4633991cfaa93fe6)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
ed359bb1ea locking: Add handler function for marking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f120e40533780e02ff1cdc41cc6d3af1c4c83258)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
c5c79d63f2 locking: Use database iterator for unlocking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 187ed83f9701c7fa8d3cc476d47c5d2a87d5c308)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
b96388f95f locking: Add handler function for unlocking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 725239535f40ca2cca445bb5bf2e181351b330e9)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
403b1eaa6e locking: Use database iterator for locking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d2634d72d9ca0ceeb72cbb1adc95017a234480fd)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
bd6ad3f817 locking: Add handler function for locking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 2a1c933ef7c78ee071e2a640ea10941f1c12e32a)
2013-05-24 09:06:38 +10:00
Amitay Isaacs
4581582a5e locking: Refactor code to iterate over databases based on priority
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit a3275854812aca86032704134fdf6a129069c86a)
2013-05-24 09:06:38 +10:00
Amitay Isaacs
0c9d72eb18 locking: Add newline to debug logs
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d98a861716d5f8c1f4387d21666396d3164551b3)
2013-05-24 09:06:38 +10:00
Amitay Isaacs
a73f13ada7 daemon: Add a tunable to enable automatic database priority setting
Samba versions 3.6.x and older do not set the database priority.
This can cause deadlock between Samba and CTDB since the locking order
of database will be different. A hack was added for automatic promotion
of priority for specific databases to avoid deadlock.  This code should
not be invoked with Samba version 4.x which correctly specifies the
priority for each database.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 4a9e96ad3d8fc46da1cd44cd82309c1b54301eb7)
2013-01-05 01:14:57 +01:00
Amitay Isaacs
442d9905fe locking: Do not use RECLOCK for tracking DB locks and latencies
RECLOCK is for recovery lock in CTDB. Do not override the meaning for
tracking locks on databases.  Database lock latency has nothing to do
with recovery lock latency.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 54e24a151d2163954e5a2a1c0f41a2b5c19ae44b)
2012-11-14 15:51:59 +11:00
Martin Schwenke
861d5304ac ctdbd: Fix compilation warning in locking code
Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit cd64035d71ddff6aebe6c15a49e09527283425d2)
2012-10-31 12:33:25 +11:00
Amitay Isaacs
d39fbd60b9 locking: Do not use ctdb_kill() to kill smbd processes
ctdb_kill() is used to terminate processes spawned by CTDB.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7d025281ee70c91ebcd4d9a908de1045a689786b)
2012-10-20 02:48:45 +11:00
Amitay Isaacs
1d83df7516 locking: Add database priority handling for older versions of samba
In samba versions 3.6.x and older, database priorities are not set.
later_db() function implements higher database priority (locking order)
for these databases -
   brlock, g_lock, notify_onelevel, serverid, xattr_tdb

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit edbc8a6669b594d3c413d603e1c9fada9244c2ee)
2012-10-20 02:48:45 +11:00
Amitay Isaacs
3c34207481 locking: Schedule a new lock request everytime a lock is released
Since the number of active lock requests is limited to
MAX_LOCK_PROCESSES_PER_DB (= 100), any new requests won't get scheduled
when they are created. So schedule a pending request once current active
request is done.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit c8eb4a3170ab8524e638047053831ba547e9cce8)
2012-10-20 02:48:44 +11:00
Amitay Isaacs
83306337df ctdbd: locking: Provide non-blocking API for locking of TDB record/db/alldb
This introduces a consistent API for handling locks on single record, complete
db or all dbs. The locks are taken out in a child process. In cases of timeout,
find the processes that currently hold the lock and log.

Callback functions for locking requests take locked boolean to indicate
whether the lock was successfully obtained or not.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 1af99cf0de9919dd89af1feab6d1bd18b95d82ff)
2012-10-20 02:48:44 +11:00
Amitay Isaacs
733bc2c03c Revert "server: locking: Provide a common API for non-blocking locking of TDBs"
This reverts commit 6a92fc2b8da2bba98dca29b781ab459ba4e879a5.

Reverting incomplete changes to ctdb_lock.c

(This used to be ctdb commit 2a3d22431948f849c93762a6ab49344acb687feb)
2012-05-01 12:09:48 +10:00
Amitay Isaacs
c3c5dec52c server: locking: Provide a common API for non-blocking locking of TDBs
(This used to be ctdb commit 6a92fc2b8da2bba98dca29b781ab459ba4e879a5)
2012-05-01 12:09:38 +10:00