1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-12 09:18:10 +03:00
Commit Graph

405 Commits

Author SHA1 Message Date
Stefan Metzmacher
417489a214 tdb/test: add a "skip()" macro.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:14 +02:00
Stefan Metzmacher
19fb729188 tdb/test: correctly use stderr, not stdout, in fail()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:14 +02:00
Volker Lendecke
9bf4f10b4b tdb/tools: explicitly use TDB_NOLOCK in tdbdump
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:14 +02:00
Volker Lendecke
b4b8e393e5 tdb/tools: add -l option to tdbtool
This opens the tdb with TDB_NOLOCK.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:14 +02:00
Volker Lendecke
781563061d tdb/tools: add -l option to tdbbackup
This opens the tdb with TDB_NOLOCK.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:14 +02:00
Stefan Metzmacher
e77cbe252f tdb: return ENOSYS if the tdb was created with spinlocks.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon May 12 21:07:04 CEST 2014 on sn-devel-104
2014-05-12 21:07:04 +02:00
Stefan Metzmacher
ff2b896edd pytdb: avoid const warnings by using discard_const_p()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-05-12 18:43:07 +02:00
Michael Adam
d9566085c6 tdb: consolidate tdb allocation code - re-use dead records at hash top.
When in tdb_store we re-use a dead record reactivated from the
target hash chain itself, we currently leave it in its place in
the chain. When we re-use a dead record from a different chain or
from the freelist instead, we insert it at the beginning of the
target chain.

This patch changes the behaviour to always newly store a
record at the beginning of the hash chain. This removes
a special case and hence simplifies the allocation code.
On the other hand side, it introduces two additioal tdb_ofs_write
calls for the in-chain-case.

Note the subtelty of the patch that by moving the case of the candidate
record's chain as new case "i=0" into the for loop, we also reverse the
order of the two steps in the for-loop body (non blocking freelist alloc
and searching for dead record in a chain) in order to keep the overall
order of execution identical.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Wed Apr  9 10:37:08 CEST 2014 on sn-devel-104
2014-04-09 10:37:08 +02:00
Stefan Metzmacher
835c95cb77 tdb/wscript: maintain a global list of unit tests
This makes sure a new test is also added to 'make test'

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
2a9d5a3239 tdb/test: avoid const warnings by using 'discard_const_p(uint8_t,' instead of '(void *)'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
6871bf689f tdb/test: remove "\n" in diag() arguments
diag() already adds "\n".

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
71fddde836 tdb/test: always call fflush() after [f]printf() in tap-interface.h
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
e301d1b960 tdb/tools: avoid const warnings in tdbdump.c
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
80dff80ee9 tdb: don't alter errno on success of tdb_open_ex()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Michael Adam
8278d3823a tdb: change version to 1.2.13.
* internal code cleanups
* always open internal TDBs with incompatible hash
* avoid reallocations in locking code
* systematize output format in tdbtool dump
* reduce freelist contention when allocating new records
  - try to find dead records also in other chains
  - don't do blocking locks on the freelist

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Mar 18 15:42:48 CET 2014 on sn-devel-104
2014-03-18 15:42:48 +01:00
Volker Lendecke
3034a5a62b tdb: Reduce freelist contention
In a metadata-intensive benchmark we have seen the locking.tdb freelist to be
one of the central contention points. This patch removes most of the contention
on the freelist. Ages ago we already reduced freelist contention by using the
even much older DEAD records: If TDB_VOLATILE is set, don't directly put
deleted records on the freelist, but just mark a few of them just as DEAD. The
next new record can them re-use that space without consulting the freelist.

This patch builds upon the DEAD records: If we need space and the freelist is
busy, instead of doing a blocking wait on the freelist, start looking into
other chains for DEAD records and steal them from there. This way every hash
chain becomes a small freelist. Just wander around the hash chains as long as
the freelist is still busy.

With this patch and the tdb mutex patch (following hopefully some time soon)
you can see a heavily busy clustered smbd run without locking.tdb futex
syscalls.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
1461362e93 tdb: Make "tdb_purge_dead" internally public
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
92ce9fd9af tdb: Make "tdb_find_dead" internally public
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
4ca018692f tdb: Add "last_ptr" to tdb_find_dead
Will be used soon to unlink a dead record from a chain

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
cb09d7937c tdb: Move adding tailer space to tdb_find_dead
This aligns the tdb_find_dead API with the tdb_allocate API and thus makes it a
bit easier to understand, at least for me.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
255edd1b41 tdb: Do a best fit search for dead records
Hash chains are (or can be made) short enough that a full search for the
best-fitting dead record is feasible. The freelist can become much longer,
there we don't do the full search but accept records which are too large.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
d1ce0110f0 tdb: Don't purge records to a blocked freelist
If the freelist is heavily contended, we should avoid accessing it

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Volker Lendecke
5f7b481349 tdb: Fix a tdb corruption
tdb_purge_dead can change the next pointer of "rec" if we purge the record
right behind the current record to be deleted. Just overwrite the magic,
not the whole record with stale data.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-03-18 13:42:10 +01:00
Michael Adam
001b9582cc tdb: always open internal databases with incompatible hash.
This makes them more efficient due to better distribution
of keys across hash chains.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Feb 15 08:26:07 CET 2014 on sn-devel-104
2014-02-15 08:26:06 +01:00
Michael Adam
41b7acacb3 tdb: in tdb_delete_hash, make lock/unlock bracket more obvious
by using the same variable as hash as in the lock.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Feb 15 03:21:07 CET 2014 on sn-devel-104
2014-02-15 03:21:07 +01:00
Michael Adam
cde8e290c9 tdb: simplify tdb_delete_hash() a bit
Make the lock/unlock bracket more obvious by extracting
locking (and finding) from the special cases to the top
of the function. This also lets us take lock and find
the record outside the special case branches (use dead
records or not).

There is a small semantic change implied:

In the dead records case, the record to delete is looked
up before the current dead records are potentially purged.
Hence, if the record to delete is not found, the dead
records are also not purge. This does not make a big
difference though, because purging is only delayed until
directly befor the next record to delete is in fact found.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-02-14 15:55:46 -08:00
Michael Adam
adb2cd1eee tdb: tdbtool: dump record magic with fixed number of 8 hex digits
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-02-14 15:53:25 -08:00
Michael Adam
057adfae47 tdb: tdbtool: dump record hash with fixed number of 8 hex digits
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-02-14 15:53:25 -08:00
Matthias Dieter Wallnöfer
0c2fbe5a0c samba:python - Py_RETURN_NONE remove compatibility code for releases < 2.4
http://www.python.org/doc//current/c-api/none.html

Reviewed-By: Jelmer Vernooij <jelmer@samba.org>

Autobuild-User(master): Matthias Dieter Wallnöfer <mdw@samba.org>
Autobuild-Date(master): Thu Jan  9 16:27:47 CET 2014 on sn-devel-104
2014-01-09 16:27:47 +01:00
Volker Lendecke
f3556bd03b tdb: Avoid reallocs for lockrecs
In normal operations we have at most 3 entries in this array. Don't
bother with shrinking.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Sat Dec 14 13:19:47 CET 2013 on sn-devel-104
2013-12-14 13:19:47 +01:00
Christian Ambach
6d88bfcab4 lib/tdb: fix compiler warnings
about a variable shadowing a global declaration

Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2013-12-12 14:21:27 -08:00
Jelmer Vernooij
28b5b7e28e tdb tests: Remove custom code for "tdb2", which has been split out into ntdb.
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>

Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2013-12-06 05:21:03 +01:00
Björn Jacke
20999fcaa1 tdb: Fix some typos in comments.
Thanks to Stewart A. Levin for reporting.

fixes bug #10136 (Documentation typos).

Signed-off-by: Bjoern Jacke <bj@sernet.de>
Reviewed-by: Karolin Seeger <kseeger@samba.org>

Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Thu Sep 12 13:54:41 CEST 2013 on sn-devel-104
2013-09-12 13:54:41 +02:00
Volker Lendecke
a6a0d238b3 tdb: Fix CID 1034959 Uninitialized scalar variable
log_ctx.log_private was used uninitialized. Not a real bug here,
as tdb_log does not access it, but tdb_open_ex still moves around
uninitialized data. So this would show up in valgrind as well.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2013-07-22 15:07:21 -07:00
Volker Lendecke
f25449b00a tdb: Fix CID 1034960 Uninitialized scalar variable
log_ctx.log_private was used uninitialized. Not a real bug here,
as tdb_log does not access it, but tdb_open_ex still moves around
uninitialized data. So this would show up in valgrind as well.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2013-07-22 15:06:32 -07:00
Bill Parker
9b58da9866 Fix bug 10025 - Lack of Sanity Checking in calls to malloc()/calloc().
In reviewing various files in Samba-4.0.7, I found a number
of instances where malloc()/calloc() were called without the
checking the return value for a value of NULL, which would
indicate failure.

(NB. The changes needed to ccan, iniparser, popt and heimdal
will be reported upstream, not patched inside Samba).

Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Simo Source <idra@samba.org>
2013-07-17 16:12:19 -07:00
Ralph Wuerthner
3cd640dc5d tdb: Fix typos.
Reviewed-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Jun  7 15:18:47 CEST 2013 on sn-devel-104
2013-06-07 15:18:47 +02:00
Ralph Wuerthner
d273bdf18c Add "repack" command to tdbtool documentation.
The original commit 'Add "repack" command to tdbtool.' only added
the documentation to docs-xml/manpages-3/tdbtool.8.xml and forgot
about lib/tdb/manpages/tdbtool.8.xml .

Reviewed-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2013-06-07 13:25:23 +02:00
Stefan Metzmacher
d1feccb35e tdb: change version to tdb-1.2.12
* internal code cleanups
* crash fix for pytdb
* fix for 4GB overflow detection
  See http://permalink.gmane.org/gmane.network.samba.internals/42906 for
  an example of what happens.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Jun  4 16:07:55 CEST 2013 on sn-devel-104
2013-06-04 16:07:55 +02:00
Volker Lendecke
1f269fcc6e tdb: Add another overflow check to tdb_expand_adjust
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Jun  3 14:08:54 CEST 2013 on sn-devel-104
2013-06-03 14:08:53 +02:00
Volker Lendecke
d9b4f19e73 tdb: Make tdb_recovery_allocate overflow-safe
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:32 +02:00
Volker Lendecke
8b215df445 tdb: Make tdb_recovery_size overflow-safe
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:31 +02:00
Stefan Metzmacher
7ae09a9695 tdb: add proper OOM/ENOSPC handling to tdb_expand()
Failing to do so will result in corrupt tdbs: We will overwrite
the hash chain pointers with 0x42424242.

Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:30 +02:00
Stefan Metzmacher
854c5f0aac tdb: add overflow detection to tdb_expand_adjust()
We round up at maximun to a new size of 4GB,
but still return at least the given size.

The caller has to deal with ENOSPC itself.

Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:28 +02:00
Stefan Metzmacher
e19d46f7e3 tdb: add overflow/ENOSPC handling to tdb_expand_file()
Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:27 +02:00
Stefan Metzmacher
a07ba17e0c tdb: add a 'new_size' helper variable to tdb_expand_file()
Pair-Programmed-With: Volker Lendecke <vl@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:22 +02:00
Volker Lendecke
4483bf143d tdb: Add overflow-checking tdb_add_off_t
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
2013-06-03 10:21:20 +02:00
Rusty Russell
3bd686c5ad tdb: fix logging of offets and lengths.
We can have offsets > 2G, so use unsigned values.  Fixes other prints to be
native types rather than casts, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue May 28 11:22:14 CEST 2013 on sn-devel-104
2013-05-28 11:22:14 +02:00
Andrew Bartlett
cd4b413cb0 build: Remove autoconf build system
We are now confident that that waf build system meets enough of our needs
that we will work to improve it, rather than maintain two build systems.

Andrew Bartlett

Reviewed-by: Jelmer Vernooij <jelmer@samba.org>

Reviewed-by: David Disseldorp <ddiss@samba.org>
2013-05-28 12:17:10 +10:00
Christian Ambach
11f467d0bd tdb: include information about hash function being used in tdbtool info output
makes it possible to easily determine if the tdb under examination
uses jenkins hash or not

Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2013-05-14 14:34:20 +02:00