1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-21 18:04:06 +03:00

112 Commits

Author SHA1 Message Date
Martin Schwenke
6674949317 ctdb-tests: Mark some functions as static
To avoid warnings when using --enable-developer, which uses
-Wmissing-prototypes.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-09-10 01:36:14 +02:00
Amitay Isaacs
2592ae5a56 ctdb-locking: Remove unused function ctdb_free_lock_request_context
There is no need for a special function to free lock request and
corresponding lock context.  Freeing lock request will free lock
context also.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-05 07:05:10 +02:00
Amitay Isaacs
374cbc7b0f ctdb-locking: Talloc lock request from client specified context
This makes sure that when the client context is destroyed, the lock
request goes away.  If the lock requests is already scheduled, then the
lock child process will be terminated.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-05 07:05:10 +02:00
Amitay Isaacs
d9e4622a44 ctdb-locking: Run debug locks script only if the node is active
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-09-05 07:05:10 +02:00
Martin Schwenke
c1558adeaa ctdb: Use sys_read() and sys_write() to ensure correct signal interaction
... and avoid compiler warnings in some cases.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-08-21 04:46:13 +02:00
Volker Lendecke
e185ff22ca ctdb-locking: Simplify ctdb_find_lock_context()
I like early returns that avoid else branches :-)

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Wed Aug  6 14:44:31 CEST 2014 on sn-devel-104
2014-08-06 14:44:31 +02:00
Volker Lendecke
9f596c17c7 ctdb-locking: TALLOC_FREE copes with NULL
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-08-06 12:22:13 +02:00
Amitay Isaacs
88f6a6c188 ctdb-locking: Add per database queues for pending and active lock requests
This avoids traversing a single pending queue which is quite expensive
when there are lots of pending lock requests.  This seems to happen
quite a lot on a loaded cluster for notify_index.tdb.

Adding per database queues avoids the need to traverse pending queue
for that database if there are already the maximum number of active
lock requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Aug  4 20:23:45 CEST 2014 on sn-devel-104
2014-08-04 20:23:45 +02:00
Amitay Isaacs
f73adff737 ctdb-locking: Update a comment
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
a890e760bb ctdb-locking: Simplify check for locks on record or database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
aa1ff305f9 ctdb-locking: Decrement pending statistics when lock is scheduled
and not when the lock is obtained.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
dce68a2141 ctdb-locking: Update ctdb statistics for all lock types
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
7189437be4 ctdb-locking: Add DB lock requests to head of the pending queue
This allows to schedule DB locks quickly without having to scan through
the pending lock requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
3aa96c3a3e ctdb-locking: Remove unused variable lock_num_pending
The number of pending locks displayed in ctdb statistics are stored in
ctdb_statistics structure and not ctdb_context.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
59d45ea307 ctdb-locking: Add new tunable LockProcessesPerDB
This allows to change the maximum number of lock processes that can
be active.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
e0d5459451 ctdb-locking: Allocate lock request soon after allocating lock context
This avoids extra work in case lock request allocation fails.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
97a5c57957 ctdb-locking: Remove unused function find_lock_context()
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
c9664b4b17 ctdb-locking: Schedule the next possible lock based on per-db limit
This prevents searching through active lock requests for every pending
lock request to check if the pending lock request can be scheduled or not.
The locks are scheduled in strict first-in-first-out order.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
19b3810b61 ctdb-locking: Remove multiple lock requests per lock context (part 2)
Store only a single request instead of storing a queue in lock context.
Lock request structure does not need to be a linked list any more.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
a89f350879 ctdb-locking: Remove multiple lock requests per lock context (part 1)
This was a bad idea and caused out of order scheduling of lock requests.

The logic to append lock requests to existing lock context is already
commented.  Remove the commented code and there is no need to check if
lock_ctx is NULL, since we are always creating a new one.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
b93d9c0622 ctdb-locking: Remove unused structure members
block_child was used to keep track of a process which was created to debug
why a lock process has blocked.  That logic was replaced to execute an
external debug script.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Amitay Isaacs
8aa6c039ae ctdb-locking: Fix the lock_type_str corresponding to LOCK_ALLDB
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-08-04 17:59:52 +02:00
Volker Lendecke
8dede57741 ctdb: Fix a comment typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Apr 30 21:05:09 CEST 2014 on sn-devel-104
2014-04-30 21:05:09 +02:00
Amitay Isaacs
f99a759ce7 ctdb-locking: Instead of comparing key, compare key hash
This replaces memory comparison of the key with integer comparison.
In addition, this also avoids scheduling locks with the same hash.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Mar 28 05:28:58 CET 2014 on sn-devel-104
2014-03-28 05:28:58 +01:00
Amitay Isaacs
d1c8f8d635 ctdb-locking: Check for talloc_memdup error
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-03-28 03:30:09 +01:00
Amitay Isaacs
ff1c85ae27 ctdb-locking: Avoid memory leak
If lock_request could not be allocated, free lock_ctx since there can
only be a single lock_request per lock_ctx.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2014-03-28 03:30:09 +01:00
Amitay Isaacs
4ea721b2c1 ctdb-locking: Use vfork instead of fork to exec helpers
There is a significant overhead using fork() over vfork(), specially
when the child process execs a helper.  The overhead is in memory space
and time.

    # strace -c ./test_fork 1024 200
    count=1024, size=204800, total=200M
    failed fork=0
    time for fork() = 4879.597000 us
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
    100.00    4.543321        3304      1375       375 clone
      0.00    0.000071           0      1033           mmap
      0.00    0.000000           0         1           read
      0.00    0.000000           0         3           write
      0.00    0.000000           0         2           open
      0.00    0.000000           0         2           close
      0.00    0.000000           0         3           fstat
      0.00    0.000000           0         3           mprotect
      0.00    0.000000           0         1           munmap
      0.00    0.000000           0         3           brk
      0.00    0.000000           0         1         1 access
      0.00    0.000000           0         1           execve
      0.00    0.000000           0         1           arch_prctl
    ------ ----------- ----------- --------- --------- ----------------
    100.00    4.543392                  2429       376 total

    # strace -c ./test_vfork 1024 200
    count=1024, size=204800, total=200M
    failed fork=0
    time for fork() = 82.041000 us
    % time     seconds  usecs/call     calls    errors syscall
    ------ ----------- ----------- --------- --------- ----------------
     96.47    0.001204           1      1000           vfork
      3.53    0.000044           0      1033           mmap
      0.00    0.000000           0         1           read
      0.00    0.000000           0         3           write
      0.00    0.000000           0         2           open
      0.00    0.000000           0         2           close
      0.00    0.000000           0         3           fstat
      0.00    0.000000           0         3           mprotect
      0.00    0.000000           0         1           munmap
      0.00    0.000000           0         3           brk
      0.00    0.000000           0         1         1 access
      0.00    0.000000           0         1           execve
      0.00    0.000000           0         1           arch_prctl
    ------ ----------- ----------- --------- --------- ----------------
    100.00    0.001248                  2054         1 total

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
0eeb73c187 ctdb-locking: Update current lock statistics when lock is scheduled
When a child process is created for a lock request, the current locks
statistics should be updated immediately.  This will provide accurate
information on number of active lock requests.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
3879e9991f ctdb-locking: Do not merge multiple lock requests to avoid unfair scheduling
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
094f34e9bf ctdb-locking: Implement active lock requests limit per database
This limit was currently a global limit and not per database.  This
prevents any database freeze lock requests from getting scheduled if
the global limit was reached.

Only individual record requests should be limited and database freeze
requests should always get scheduled.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-27 18:46:16 +01:00
Amitay Isaacs
6d1b74f052 ctdb-server: Coverity fixes
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-11-19 17:13:03 +01:00
Martin Schwenke
463a091a77 ctdbd: Debug locks by default with override from enviroment variable
Default is debug_locks.sh, relative to CTDB_BASE.

Signed-off-by: Martin Schwenke <martin@meltin.net>

(This used to be ctdb commit c11803e3dcc905a45a08d743595e63f9ca445f0d)
2013-10-22 15:37:54 +11:00
Amitay Isaacs
477a51aba5 locking: Do not create multiple lock processes for the same key
If there are multiple lock helper processes waiting for the same record, then
it will cause a thundering herd when that record has been unlocked.  So avoid
scheduling lock contexts for the same record.  This will also mean that
multiple requests will get queued up behind the same lock context and can be
processed quickly once the lock has been obtained.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit ebecc3a18f1cb397a78b56eaf8f752dd5495bcc9)
2013-08-09 11:04:55 +10:00
Amitay Isaacs
9ba793a80f locking: Move function find_lock_context() before ctdb_lock_schedule()
So that ctdb_lock_schedule() can call this function without requiring extra
prototype declaration.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 68af5405acc123b5a90decd2123e2a02961a8fcf)
2013-08-09 11:04:42 +10:00
Amitay Isaacs
c620457c0b locking: Use external script to debug locking issues
Use an external script to parse /proc/locks and log useful debugging
information about locks rather than doing that in C code.

To use this feature, add configuration variable to /etc/sysconfig/ctdb:

  CTDB_DEBUG_LOCKS=/etc/ctdb/debug_locks.sh

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 2bfb8499366d530f16515b08928056bbda40f781)
2013-07-10 14:33:18 +10:00
Amitay Isaacs
9ae379c91a locking: Update locking bucket intervals
0   < 1 ms
 1   < 10 ms
 2   < 100 ms
 3   < 1 s
 4   < 2 s
 5   < 4 s
 6   < 8 s
 7   < 16 s
 8   < 32 s
 9   < 64 s
10   >= 64 s

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 6fc36a7036933237d09151a0baf4d8ccd2bc2c99)
2013-07-10 14:33:18 +10:00
Amitay Isaacs
1afb7fccb2 locking: Update locks latency in CTDB statistics only for RECORD or DB locks
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit dcc42a75b4638b3aa40c44ed9e0aaae26483e2b0)
2013-07-10 14:33:18 +10:00
Amitay Isaacs
6391f61fbc build: Fix compiler warnings for uninitialized variables
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 5408c5c4050539e5aa06a5e82ceb63a6cb5cef0c)
2013-07-04 20:43:52 +10:00
Amitay Isaacs
c8d577eb80 locking: Set lock helper path once
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 80fbe9364350d42658f7f8af250ac87eb1afbc21)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
1ddc7b0d10 locking: Remove functions that are not used anymore
These functions were used in locking child process to do the locking.  With
locking helper, these are not required.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit c660f33c3eaa1b4a2c4e951c1982979e57374ed4)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
90c4fa77b9 locking: Remove functions that are not used anymore
These functions were used in locking child process to do the locking.  With
locking helper, these are not required.

Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 6ea3212a7b177c6c06b1484cf9e8b2f4036653d9)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
ae25420e56 locking: Use separate locking helper binary for locking
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7cde53a6cbe74b1e46f7e1bca298df82c08de866)
2013-05-24 09:06:40 +10:00
Amitay Isaacs
e30978eae1 locking: Create commandline arguments for locking helper
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f665e3d540c90579952e590caa5828acb581ae61)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
c9f4589c13 locking: Use database iterator for unmarking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 7630ca4116b476636c27407748088ea335f1a06c)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
65a9195916 locking: Add handler function for unmarking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit adc113055de98fae276f9b501aff5c03cd25ddc8)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
a5133d16e7 locking: Use database iterator for marking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit e8ea65b2713417db4a618a9f4633991cfaa93fe6)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
ed359bb1ea locking: Add handler function for marking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit f120e40533780e02ff1cdc41cc6d3af1c4c83258)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
c5c79d63f2 locking: Use database iterator for unlocking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 187ed83f9701c7fa8d3cc476d47c5d2a87d5c308)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
b96388f95f locking: Add handler function for unlocking a database
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit 725239535f40ca2cca445bb5bf2e181351b330e9)
2013-05-24 09:06:39 +10:00
Amitay Isaacs
403b1eaa6e locking: Use database iterator for locking databases
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

(This used to be ctdb commit d2634d72d9ca0ceeb72cbb1adc95017a234480fd)
2013-05-24 09:06:39 +10:00