IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This will allow callers to avoid their own reference counting of transactions.
Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This restores the original intent of tdb_traverse_read() in
7dd31288a7
This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process
We add a test to show that a transaction can now start while a read
traverse is in progress
This allows more operations to happen in parallel. The blocking point
is moved to the prepare commit.
This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.
This behaviour was added in 251aaafe3a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This verifies the F_RDLCK => F_WRLCK upgrade logic in the kernel
for conflicting locks.
This is a standalone test to check the traverse_read vs.
allrecord_lock/prepare_commit interaction.
This is based on the example from
https://lists.samba.org/archive/samba-technical/2017-April/119861.html
from Douglas Bagnall <douglas.bagnall@catalyst.net.nz> and Volker Lendecke <vl@samba.org>.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
The current runtime check for robust mutexes in
tdb_runtime_check_for_robust_mutexes() is not thread-safe.
When called in a multi-threaded program where any another thread doesn't
have SIGCHLD blocked, we may end up hung in sigsuspend() waiting for a
SIGCHLD of a child procecss and the signal was delivered to another
thread.
Revert to the previous behaviour of waiting for the child instead of
waiting for the SIGCHLD signal.
Ensure the pid we wait for is not reset to -1 in a toctou race with the
signal handler.
Check whether waitpid() returns ECHILD which can happen if the signal
handler is run by more then one thread in parallel (yes, this can
happen) or if tdb_robust_mutex_wait_for_child() and the signal handler
are racing.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=12593
Pair-programmed-with: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 22 09:16:16 CEST 2017 on sn-devel-144
When a process holds a readlock and wants to upgrade, this needs to be
reflected in the underlying lock. Without this, it is possible to cheat:
One process holds a readlock, and another process wants to write this
record. All the writer has to do is take a readonly lock on the key and
then do the store.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Drop the configure option for --disable-python as it is now
global in wafsamba.
If samba is set to use a system copy of tdb, and tdb wasn't built
with python support, then the system pytevent will not be found. If
samba is being built without python support then pytdb is not needed,
so do not bother to try and find it.
Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* Bug 12455 - tdb mutexes don't work on FreeBSD
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Nov 30 20:02:28 CET 2016 on sn-devel-144
susv4 on mmap has the following snippet:
> The state of synchronization objects such as mutexes, semaphores,
> barriers, and conditional variables placed in shared memory mapped
> with MAP_SHARED becomes undefined when the last region in any process
> containing the synchronization object is unmapped.
This means we can't keep the mutex mmap area unmapped at any point
in time.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12455
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Nov 29 23:59:52 CET 2016 on sn-devel-144
This gains a few percent in tdbbackup
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Jul 15 00:52:00 CEST 2016 on sn-devel-144
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Jul 3 18:11:30 CEST 2016 on sn-devel-144
This comes via a "goto cleanup" before suspend_mask is initialized
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Tue Apr 12 11:39:35 CEST 2016 on sn-devel-144
* avoid a race condition when checking for robust mutexes
(bug #11808)
* Remove use of strcpy in tdb test.
* eliminate deprecation warnings in python tests
* Only set public headers field when installing as a public library.
* Refuse to load a database with hash size 0
* Fix various spelling errors
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Apr 11 18:48:26 CEST 2016 on sn-devel-144
The cleanup logic used six goto lables, at least I'm not able to make
sane modifications to such a beast.
By using state flags that track which objects are initialized and need
cleanup, we get rid of the goto labels. It comes at a cost though: you
have to be careful to correctly set the cleanup flags.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
This fixes a race between calling waitpid() in two places (SIGCHLD the
signal handler and the rendezvous code when waiting for the child to
terminate), by
- blocking SIGCHLD before installing our signal handler
- in the rendezvous code call sigssuspend() which unblocks SIGCHLD and
suspends the thread and waits for signal delivery
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11808
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Mar 29 16:04:19 CEST 2016 on sn-devel-144
assertEquals() is superseded by assertEqual()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Feb 16 19:58:39 CET 2016 on sn-devel-144
This just ensures we reject (rather than div-by-0) a corrupt
DB with a zero hash size.
Found with american fuzzy lop
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Dec 18 08:26:25 CET 2015 on sn-devel-144
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov 6 13:43:45 CET 2015 on sn-devel-104
This reverts commit ea53e86c25.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Sep 18 14:03:12 CEST 2015 on sn-devel-104
The reverse of tdbdump
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 18 03:04:35 CEST 2015 on sn-devel-104
With --disable-python, we should not install any python files.
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Jul 23 18:50:25 CEST 2015 on sn-devel-104
* first fix deadlock in the interaction between fcntl and mutex locking
(bug #11381)
* improved python3 bindings
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This change makes tdb dicts more consistent with Python dicts.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=7416
Signed-off-by: Aurelien Aptel <aaptel@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
- Remove has_key() method, add a fast "in" operator
- Rename iterkeys() to keys()
Signed-off-by: Petr Viktorin <pviktori@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
- Use bytes for all data, text strings for repr()
- Use PyLong instead of PyInt on py3
- Use new module initialization
- Update tests
- Run tests in "make test"
Signed-off-by: Petr Viktorin <pviktori@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
nextkey() was defined to take no arguments but expected one.
Signed-off-by: Petr Viktorin <pviktori@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
config.h may have some flags which affect glibc behavior, e.g.
_FILE_OFFSET_BITS=64. To make sure these flags have the desired
effect, config.h must be included before any glibc header files.
This commit does not fix a specific known bug. It changes the code to
comply with coding conventions.
Signed-off-by: Uri Simchoni <urisimchoni@gmail.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: "Stefan Metzmacher" <metze@samba.org>
This fixes a deadlock in tdb that is a bad interaction between tdb_lockall
and tdb_traverse. This deadlock condition has been around even before
tdb mutexes, it's just that the kernel fcntl EDEADLK detection protected
us from this ABBA lock condition to become a real deadlock stalling
processes. With tdb mutexes, this deadlock protection is gone, so we do
lock dead.
This patch glosses over this particular ABBA condition, making tdb with
mutexes behave the same as tdb without mutexes. Admittedly this is no
real fix, but it works around a real user's problem.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=11381
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
* Fix runtime detection for robust mutexes in the standalone build.
bug #11326
* Possible fix for the build with robust mutexes on solaris 11
bug #11319
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Jun 12 19:55:34 CEST 2015 on sn-devel-104
ABI change: tdb_chainlock_read_nonblock() has been added.
Signed-off-by: Alexander Drozdov <al.drozdov@gmail.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed Apr 29 00:05:29 CEST 2015 on sn-devel-104
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Mar 31 01:56:02 CEST 2015 on sn-devel-104
Working fix that copes with oldact.sa_handler == NULL
if no handler initially set.
Fixes bug #11175 - Lots of winbindd zombie processes on Solaris platform.
https://bugzilla.samba.org/show_bug.cgi?id=11175
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Thu Mar 26 04:29:42 CET 2015 on sn-devel-104
This fails on Linux platforms with robust mutex support with the
following error:
tdb(/home/asn/workspace/projects/samba/git/st/nt4_dc/lockdir/gencache_notrans.tdb):
tdb_mutex_open_ok[/home/asn/workspace/projects/samba/git/st/nt4_dc/lockdir/gencache_notrans.tdb]:
Can use mutexes only with MUTEX_LOCKING or NOLOCK
We also see winbind is not able to start with this error message trying
to open the serverid.tdb.
This reverts commit d191436728.
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Mar 25 14:58:38 CET 2015 on sn-devel-104
Fixes bug #11175 - Lots of winbindd zombie processes on Solaris platform.
https://bugzilla.samba.org/show_bug.cgi?id=11175
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Mar 24 14:43:22 CET 2015 on sn-devel-104
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Mar 11 00:23:20 CET 2015 on sn-devel-104
Transactions are supported with TDB_MUTEX_LOCKING.
This fixes https://bugzilla.samba.org/show_bug.cgi?id=11004
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Fri Dec 19 11:41:26 CET 2014 on sn-devel-104
There's no real reason to disallow transactions as the
allrecord lock is also available with mutexes enabled.
E.g. ctdbd requires transactions also on non-persistent databases
opened with TDB_CLEAR_IF_FIRST and TDB_MUTEX_LOCKING.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=11004
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
This fixes the tdb1-run-mutex-openflags2 test when robust mutexes are available.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=10781
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Dec 2 11:54:28 CET 2014 on sn-devel-104
This used to be invalid in the early developement code, but now we're
able to open a tdb with mutex area and TDB_NOLOCK without problems.
O_RDONLY implies TDB_NOLOCK...
This should have been part of commit c8d05e934e.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=10781
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
When using exit() instead of _exit(), the child will flush buffered stdout
(and other stdio) content that it inherited from the parent process. In
make test, this led to duplicate output from net registry which then
confused the blackbox selftest.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Due to introduction of tdb_runtime_check_for_robust_mutexes().
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
* internal code cleanup
* improved free record detection with a highly contended freelist
on tdb's with dead record support (TDB_VOLATILE).
* implicit defragmentation of the free list.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Sep 16 12:13:31 CEST 2014 on sn-devel-104
../../tdb/tools/tdbtool.c: In function ‘do_command’:
../../tdb/tools/tdbtool.c:717: error: declaration of ‘count’ shadows a global declaration
../../tdb/tools/tdbtool.c:597: error: shadowed declaration is here
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Currently tdbtorture prints the test details (processes, loops, etc.)
from the first forked child process.
When run with -n 1 (one process), the test is run from within the
parent and no details are printed. This change ensures that they are.
Signed-off-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Mon Aug 18 13:26:32 CEST 2014 on sn-devel-104
While we are traversing the freelist anyways, merge a record
with the left if it is also a free list record.
That partially makes up for the fragmentation introduced by
the lack of merging with right records in tdb_free().
Note there is a potential slight downside:
If the left record we merge the current record into was earlier
in the chain and has hence already been met in traverse,
then we can not use the enlarged record even if it might be
a new best fit.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Jun 26 12:16:03 CEST 2014 on sn-devel-104
With the new code, this has the side effect of
merging adjacent records in the freelist if the
database is not read-only.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
So that we automatically defragment the free list when freelist_size is called
(unless the database is read only).
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This is intended to be called to reduce the fragmentation in the
freelist. This is to make up the deficiency of the freelist
to be not doubly linked. If the freelist were doubly linked,
we could easily avoid the creation of adjacent freelist entries.
But with the current singly linked list, it is only possible
to cheaply merge a new free record into a freelist entry on the left,
not on the right...
This can be called periodically, e.g. in the vacuuming process
of a ctdb cluster.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Variant of check_merge_with_left_record() that reads the record
itself if necessary.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Check whether the record left of a given freelist record is
also a freelist record, and if so, merge the two records.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
by using early returns and better variable names,
and reducing indentation.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This allows tdbtorture to run with mutexes.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This adds optional support for locking based on
shared robust mutexes.
The caller can use the TDB_MUTEX_LOCKING flag
together with TDB_CLEAR_IF_FIRST after verifying
with tdb_runtime_check_for_robust_mutexes() that
it's supported by the current system.
The caller should be aware that using TDB_MUTEX_LOCKING
implies some limitations, e.g. it's not possible to
have multiple read chainlocks on a given hash chain
from multiple processes.
Note: that this doesn't make tdb thread safe!
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This makes it possible to have some extra headers before
the real tdb content starts in the file.
This will be used used e.g. to implement locking based on robust mutexes.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This will allow to store a feature mask in the tdb header on disk,
so that openers can check if they can handle the features
other openers are using.
Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This opens the tdb with TDB_NOLOCK.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This opens the tdb with TDB_NOLOCK.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon May 12 21:07:04 CEST 2014 on sn-devel-104
When in tdb_store we re-use a dead record reactivated from the
target hash chain itself, we currently leave it in its place in
the chain. When we re-use a dead record from a different chain or
from the freelist instead, we insert it at the beginning of the
target chain.
This patch changes the behaviour to always newly store a
record at the beginning of the hash chain. This removes
a special case and hence simplifies the allocation code.
On the other hand side, it introduces two additioal tdb_ofs_write
calls for the in-chain-case.
Note the subtelty of the patch that by moving the case of the candidate
record's chain as new case "i=0" into the for loop, we also reverse the
order of the two steps in the for-loop body (non blocking freelist alloc
and searching for dead record in a chain) in order to keep the overall
order of execution identical.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Wed Apr 9 10:37:08 CEST 2014 on sn-devel-104
This makes sure a new test is also added to 'make test'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* internal code cleanups
* always open internal TDBs with incompatible hash
* avoid reallocations in locking code
* systematize output format in tdbtool dump
* reduce freelist contention when allocating new records
- try to find dead records also in other chains
- don't do blocking locks on the freelist
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Mar 18 15:42:48 CET 2014 on sn-devel-104
In a metadata-intensive benchmark we have seen the locking.tdb freelist to be
one of the central contention points. This patch removes most of the contention
on the freelist. Ages ago we already reduced freelist contention by using the
even much older DEAD records: If TDB_VOLATILE is set, don't directly put
deleted records on the freelist, but just mark a few of them just as DEAD. The
next new record can them re-use that space without consulting the freelist.
This patch builds upon the DEAD records: If we need space and the freelist is
busy, instead of doing a blocking wait on the freelist, start looking into
other chains for DEAD records and steal them from there. This way every hash
chain becomes a small freelist. Just wander around the hash chains as long as
the freelist is still busy.
With this patch and the tdb mutex patch (following hopefully some time soon)
you can see a heavily busy clustered smbd run without locking.tdb futex
syscalls.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Will be used soon to unlink a dead record from a chain
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This aligns the tdb_find_dead API with the tdb_allocate API and thus makes it a
bit easier to understand, at least for me.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Hash chains are (or can be made) short enough that a full search for the
best-fitting dead record is feasible. The freelist can become much longer,
there we don't do the full search but accept records which are too large.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
If the freelist is heavily contended, we should avoid accessing it
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb_purge_dead can change the next pointer of "rec" if we purge the record
right behind the current record to be deleted. Just overwrite the magic,
not the whole record with stale data.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This makes them more efficient due to better distribution
of keys across hash chains.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Feb 15 08:26:07 CET 2014 on sn-devel-104
by using the same variable as hash as in the lock.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Feb 15 03:21:07 CET 2014 on sn-devel-104
Make the lock/unlock bracket more obvious by extracting
locking (and finding) from the special cases to the top
of the function. This also lets us take lock and find
the record outside the special case branches (use dead
records or not).
There is a small semantic change implied:
In the dead records case, the record to delete is looked
up before the current dead records are potentially purged.
Hence, if the record to delete is not found, the dead
records are also not purge. This does not make a big
difference though, because purging is only delayed until
directly befor the next record to delete is in fact found.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
In normal operations we have at most 3 entries in this array. Don't
bother with shrinking.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Sat Dec 14 13:19:47 CET 2013 on sn-devel-104
log_ctx.log_private was used uninitialized. Not a real bug here,
as tdb_log does not access it, but tdb_open_ex still moves around
uninitialized data. So this would show up in valgrind as well.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
log_ctx.log_private was used uninitialized. Not a real bug here,
as tdb_log does not access it, but tdb_open_ex still moves around
uninitialized data. So this would show up in valgrind as well.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
In reviewing various files in Samba-4.0.7, I found a number
of instances where malloc()/calloc() were called without the
checking the return value for a value of NULL, which would
indicate failure.
(NB. The changes needed to ccan, iniparser, popt and heimdal
will be reported upstream, not patched inside Samba).
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Simo Source <idra@samba.org>
The original commit 'Add "repack" command to tdbtool.' only added
the documentation to docs-xml/manpages-3/tdbtool.8.xml and forgot
about lib/tdb/manpages/tdbtool.8.xml .
Reviewed-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
* internal code cleanups
* crash fix for pytdb
* fix for 4GB overflow detection
See http://permalink.gmane.org/gmane.network.samba.internals/42906 for
an example of what happens.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Jun 4 16:07:55 CEST 2013 on sn-devel-104
