1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-22 22:04:08 +03:00

618 Commits

Author SHA1 Message Date
Volker Lendecke
5c55c2563d tdb: Consistency check for tdb_storev
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-08-09 22:34:17 +02:00
Stefan Metzmacher
49c45fbb41 tdb: version 1.3.14
* allow tdb_traverse_read before tdb_transaction[_prepare]_commit()
* Improve documentation for tdb_transaction_start()
* Add new function tdb_transaction_active()

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
32702a9745 tdb: Add new function tdb_transaction_active()
This will allow callers to avoid their own reference counting of transactions.

Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
e08d02d027 tdb: Improve documentation for tdb_transaction_start()
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:19 +02:00
Andrew Bartlett
f80076fe43 tdb: Remove locking from tdb_traverse_read()
This restores the original intent of tdb_traverse_read() in
7dd31288a701d772e45b1960ac4ce4cc1be782ed

This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process

We add a test to show that a transaction can now start while a read
traverse is in progress

This allows more operations to happen in parallel.  The blocking point
is moved to the prepare commit.

This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.

This behaviour was added in 251aaafe3a9213118ac3a92def9ab2104c40d12a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2017-07-02 17:35:18 +02:00
Stefan Metzmacher
c581ab3ac5 tdb: add run-fcntl-deadlock test
This verifies the F_RDLCK => F_WRLCK upgrade logic in the kernel
for conflicting locks.

This is a standalone test to check the traverse_read vs.
allrecord_lock/prepare_commit interaction.

This is based on the example from
https://lists.samba.org/archive/samba-technical/2017-April/119861.html
from Douglas Bagnall <douglas.bagnall@catalyst.net.nz> and Volker Lendecke <vl@samba.org>.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-06-15 01:24:25 +02:00
Andrew Bartlett
1fd8ec23c7 tdb: Improve debugging when the allrecord lock fails to upgrade
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-05-23 01:13:24 +02:00
Stefan Metzmacher
77d4e07ef3 tdb: version 1.3.13
* documentation for the tdbbackup -n option
* correctly upgrade F_RDLCK to F_WRLCK locks
* tdbtool: Add "storehex" command
* fix robust mutex detection in threaded applications
  (bug #12593)
* improve debugging of transaction lock failures

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Apr 27 18:50:10 CEST 2017 on sn-devel-144
2017-04-27 18:50:08 +02:00
Andrew Bartlett
3607826334 tdb: Improve debugging in _tdb_transaction_start
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-27 14:52:17 +02:00
Andrew Bartlett
1148e8f040 tdb: Improve debugging when the allrecord lock fails to upgrade
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-27 14:52:17 +02:00
Ralph Boehme
19b193ebc9 tdb: runtime check for robust mutexes may hang in threaded programs
The current runtime check for robust mutexes in
tdb_runtime_check_for_robust_mutexes() is not thread-safe.

When called in a multi-threaded program where any another thread doesn't
have SIGCHLD blocked, we may end up hung in sigsuspend() waiting for a
SIGCHLD of a child procecss and the signal was delivered to another
thread.

Revert to the previous behaviour of waiting for the child instead of
waiting for the SIGCHLD signal.

Ensure the pid we wait for is not reset to -1 in a toctou race with the
signal handler.

Check whether waitpid() returns ECHILD which can happen if the signal
handler is run by more then one thread in parallel (yes, this can
happen) or if tdb_robust_mutex_wait_for_child() and the signal handler
are racing.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=12593

Pair-programmed-with: Stefan Metzmacher <metze@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2017-04-27 14:52:16 +02:00
Volker Lendecke
5d288a9b17 tdbtool: Add "storehex" command
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Böhme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Apr 22 09:16:16 CEST 2017 on sn-devel-144
2017-04-22 09:16:16 +02:00
Volker Lendecke
397d1936ec tdb: Test for readonly lock upgrade bug
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Apr 11 00:33:31 CEST 2017 on sn-devel-144
2017-04-11 00:33:30 +02:00
Volker Lendecke
a6f1532d7f tdb: Do lock upgrades properly
When a process holds a readlock and wants to upgrade, this needs to be
reflected in the underlying lock. Without this, it is possible to cheat:
One process holds a readlock, and another process wants to write this
record. All the writer has to do is take a readonly lock on the key and
then do the store.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-10 20:44:21 +02:00
Volker Lendecke
97cafdcfaa tdb: Fix some signed/unsigned hickups
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-04-10 20:44:21 +02:00
Björn Baumbach
0558715b5b tdb/tools: add documentation for the tdbbackup -n option
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Jeremy Allison <jra@samba.org
2017-03-28 17:45:19 +02:00
Ian Stakenvicius
81ff0f3065 waf: disable-python - align tdb's wscript
Drop the configure option for --disable-python as it is now
global in wafsamba.

If samba is set to use a system copy of tdb, and tdb wasn't built
with python support, then the system pytevent will not be found.  If
samba is being built without python support then pytdb is not needed,
so do not bother to try and find it.

Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-03-10 07:31:12 +01:00
Stefan Metzmacher
78a77d468f tdb: version 1.3.12
* Bug 12455 - tdb mutexes don't work on FreeBSD

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Nov 30 20:02:28 CET 2016 on sn-devel-144
2016-11-30 20:02:28 +01:00
Volker Lendecke
275d9fc7d9 tdb: Fix mutexes on FreeBSD
susv4 on mmap has the following snippet:

> The state of synchronization objects such as mutexes, semaphores,
> barriers, and conditional variables placed in shared memory mapped
> with MAP_SHARED becomes undefined when the last region in any process
> containing the synchronization object is unmapped.

This means we can't keep the mutex mmap area unmapped at any point
in time.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12455

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Nov 29 23:59:52 CET 2016 on sn-devel-144
2016-11-29 23:59:52 +01:00
Volker Lendecke
5ce95abf37 tdb: Only mmap the mutex area if not already mmap'ed
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12455

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-11-29 20:03:25 +01:00
Volker Lendecke
a2843cfd4d tdb: NULL out tdb->mutexes in tdb_mutex_munmap
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12455

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-11-29 20:03:25 +01:00
Volker Lendecke
c27c7d44fb tdb: Use tdb_storev in tdb_append
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:27 +02:00
Volker Lendecke
504b04b817 tdb: Add tdb_storev
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:27 +02:00
Volker Lendecke
647e61b0e8 tdb: Add tdb_trace_1plusn_rec_flag_ret
Needed for tdb_storev

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:27 +02:00
Volker Lendecke
37e644fc20 tdb: Vectorize _tdb_store
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:26 +02:00
Volker Lendecke
f6f4e5e6b5 tdb: Vectorize tdb_update_hash
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:26 +02:00
Volker Lendecke
6295080361 tdb: Allow _v variant in tdb_update_hash_cmp
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:26 +02:00
Volker Lendecke
babd5432ed tdb: Remove unnecessary checks
This has already been done in tdb_find()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:26 +02:00
Volker Lendecke
4913180aad tdb: Do an overflow check
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:26 +02:00
Volker Lendecke
6dc4e294a5 tdb: Fix a signed/unsigned hickup
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2016-08-29 19:03:26 +02:00
Amitay Isaacs
b891feaf2b tdb: Fix format-nonliteral warning
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12168

Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2016-08-24 01:33:50 +02:00
Stefan Metzmacher
2a97fb084f tdb: version 1.3.10
* performance improvements
* make pthread mutex code more portable
* builf fixes

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2016-07-28 05:00:19 +02:00
Volker Lendecke
91a17e5413 tdb: Don't malloc for every record in traverse
This gains a few percent in tdbbackup

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Jul 15 00:52:00 CEST 2016 on sn-devel-144
2016-07-15 00:52:00 +02:00
Bob Campbell
7700ee810b tdb: avoid many fcntl calls when incrementing seqnum
Signed-off-by: Bob Campbell <bobcampbell@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Jul  3 18:11:30 CEST 2016 on sn-devel-144
2016-07-03 18:11:29 +02:00
Jeremy Allison
04967d6e88 s3: tdb: On some platforms pthread_mutex_trylock() returns EBUSY not EDEADLK.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Jun 29 15:14:44 CEST 2016 on sn-devel-144
2016-06-29 15:14:44 +02:00
Michael Adam
0e80775c1b tdb:torture: fix -O3 error unused result of write
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Christian Ambach <ambi@samba.org>
2016-05-13 00:16:15 +02:00
Michael Adam
a860245c4d tdb:torture: fix -O3 error unused result code of read
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Christian Ambach <ambi@samba.org>
2016-05-13 00:16:15 +02:00
Volker Lendecke
e6ed803a3b tdb mutex check: Fix CID 1358473 Uninitialized scalar variable
This comes via a "goto cleanup" before suspend_mask is initialized

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

Autobuild-User(master): Uri Simchoni <uri@samba.org>
Autobuild-Date(master): Tue Apr 12 11:39:35 CEST 2016 on sn-devel-144
2016-04-12 11:39:34 +02:00
Stefan Metzmacher
acf6deb698 tdb: version 1.3.9
* avoid a race condition when checking for robust mutexes
  (bug #11808)
* Remove use of strcpy in tdb test.
* eliminate deprecation warnings in python tests
* Only set public headers field when installing as a public library.
* Refuse to load a database with hash size 0
* Fix various spelling errors

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Apr 11 18:48:26 CEST 2016 on sn-devel-144
2016-04-11 18:48:26 +02:00
Uri Simchoni
ef3d837040 tdb: rework cleanup logic in tdb_runtime_check_for_robust_mutexes()
The cleanup logic used six goto lables, at least I'm not able to make
sane modifications to such a beast.

By using state flags that track which objects are initialized and need
cleanup, we get rid of the goto labels. It comes at a cost though: you
have to be careful to correctly set the cleanup flags.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
2016-04-11 15:22:26 +02:00
Ralph Boehme
a7a77e2f43 tdb: avoid a race condition when checking for robust mutexes
This fixes a race between calling waitpid() in two places (SIGCHLD the
signal handler and the rendezvous code when waiting for the child to
terminate), by

- blocking SIGCHLD before installing our signal handler

- in the rendezvous code call sigssuspend() which unblocks SIGCHLD and
  suspends the thread and waits for signal delivery

BUG: https://bugzilla.samba.org/show_bug.cgi?id=11808

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Tue Mar 29 16:04:19 CEST 2016 on sn-devel-144
2016-03-29 16:04:19 +02:00
Jeremy Allison
a559ac31f7 lib:tdb: Remove use of strcpy in tdb test.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2016-03-22 04:38:24 +01:00
Michael Adam
3330c79324 tdb: eliminate deprecation warnings in python tests
assertEquals() is superseded by assertEqual()

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Feb 16 19:58:39 CET 2016 on sn-devel-144
2016-02-16 19:58:39 +01:00
Jelmer Vernooij
68a3576629 tdb: Only set public headers field when installing as a public library.
Signed-Off-By: Jelmer Vernooij <Jelmer@samba.org>
2016-01-13 04:43:22 +01:00
Andrew Bartlett
dbd87b94aa tdb: Refuse to load a database with hash size 0
This just ensures we reject (rather than div-by-0) a corrupt
DB with a zero hash size.

Found with american fuzzy lop

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Dec 18 08:26:25 CET 2015 on sn-devel-144
2015-12-18 08:26:24 +01:00
Mathieu Parent
c315fce17e Fix various spelling errors
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Nov  6 13:43:45 CET 2015 on sn-devel-104
2015-11-06 13:43:45 +01:00
Stefan Metzmacher
bb0050a332 tdb: version 1.3.8
* Fix broken build with --disable-python
* Minor build fixes

Signed-off-by: Stefan Metzmacher <metze@samba.org>
2015-11-05 18:04:23 +01:00
Ralph Boehme
a36a734717 Revert "tdb: Add tdbdump -u"
This reverts commit ea53e86c256bec08373dbe86e190a7f5a06eff9d.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Sep 18 14:03:12 CEST 2015 on sn-devel-104
2015-09-18 14:03:12 +02:00
Volker Lendecke
ea53e86c25 tdb: Add tdbdump -u
The reverse of tdbdump

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Sep 18 03:04:35 CEST 2015 on sn-devel-104
2015-09-18 03:04:35 +02:00
Martin Schwenke
509c37da13 tdb: Fix broken build with --disable-python
With --disable-python, we should not install any python files.

Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User(master): Amitay Isaacs <amitay@samba.org>
Autobuild-Date(master): Thu Jul 23 18:50:25 CEST 2015 on sn-devel-104
2015-07-23 18:50:25 +02:00