1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

131 Commits

Author SHA1 Message Date
Michael Adam
1f7da1ca7f tdb: change version to 1.3.1
* internal code cleanup
* improved free record detection with a highly contended freelist
  on tdb's with dead record support (TDB_VOLATILE).
* implicit defragmentation of the free list.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Sep 16 12:13:31 CEST 2014 on sn-devel-104
2014-09-16 12:13:31 +02:00
Michael Adam
47328f535e tdb:build: improve detection of srcdir.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2014-06-20 23:38:10 +02:00
Volker Lendecke
b39e8eac20 tdb/test: add marklock deadlock test
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:15 +02:00
Volker Lendecke
fbaf94bb85 tdb/test: add mutex related tests
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:15 +02:00
Volker Lendecke
db5bda56bf tdb: add TDB_MUTEX_LOCKING support
This adds optional support for locking based on
shared robust mutexes.

The caller can use the TDB_MUTEX_LOCKING flag
together with TDB_CLEAR_IF_FIRST after verifying
with tdb_runtime_check_for_robust_mutexes() that
it's supported by the current system.

The caller should be aware that using TDB_MUTEX_LOCKING
implies some limitations, e.g. it's not possible to
have multiple read chainlocks on a given hash chain
from multiple processes.

Note: that this doesn't make tdb thread safe!

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Pair-Programmed-With: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-22 21:05:15 +02:00
Stefan Metzmacher
835c95cb77 tdb/wscript: maintain a global list of unit tests
This makes sure a new test is also added to 'make test'

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Michael Adam
8278d3823a tdb: change version to 1.2.13.
* internal code cleanups
* always open internal TDBs with incompatible hash
* avoid reallocations in locking code
* systematize output format in tdbtool dump
* reduce freelist contention when allocating new records
  - try to find dead records also in other chains
  - don't do blocking locks on the freelist

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Mar 18 15:42:48 CET 2014 on sn-devel-104
2014-03-18 15:42:48 +01:00
Stefan Metzmacher
d1feccb35e tdb: change version to tdb-1.2.12
* internal code cleanups
* crash fix for pytdb
* fix for 4GB overflow detection
  See http://permalink.gmane.org/gmane.network.samba.internals/42906 for
  an example of what happens.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Jun  4 16:07:55 CEST 2013 on sn-devel-104
2013-06-04 16:07:55 +02:00
Karolin Seeger
ede2aaef28 lib/tdb: Rename manpages/ to man/.
Trying to be more consistent.

Karolin

Reviewed-by: Andreas Schneider <asn@samba.org>
2012-12-03 14:35:11 +01:00
Rusty Russell
90f463b25f tdb: add tdb_rescue()
This allows for an emergency best-effort dump.  It's a little better than
strings(1).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-10-04 09:04:19 +09:30
Rusty Russell
1783fe3443 tdb: make TDB_NOSYNC merely disable sync.
(As suggested by Stefan Metzmacher, based on the change to ntdb.)

Since commit ec96ea690e, we handle the case
where a process dies during a transaction commit.  Unfortunately, TDB_NOSYNC
means this no longer works, as it disables the recovery area as well as the
actual msync/fsync.  We should do everything except the syncs.

This also means we can do a complete test with $TDB_NO_FSYNC set; just
to get more complete coverage, we disable it explicitly for one test
(where we override the actual sync calls anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-06-22 07:35:17 +02:00
Rusty Russell
f7f6992c1e autobuild: always set TDB_NO_FSYNC.
Then we unset it inside the tdb test target itself.  This means that
new code can't accidently forget it, and we can set it in the
'buildnice' script on sn-devel, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-06-22 07:35:16 +02:00
Jelmer Vernooij
bf5934ca1b tdb/wscript: Remove unecessary semicolons. 2012-06-19 02:43:23 +02:00
Amitay Isaacs
593e731097 lib/tdb: Update ABI
Signed-off-by: Amitay Isaacs <amitay@gmail.com>

Autobuild-User: Rusty Russell <rusty@rustcorp.com.au>
Autobuild-Date: Thu Mar 29 13:12:46 CEST 2012 on sn-devel-104
2012-03-29 13:12:46 +02:00
Rusty Russell
583ffeae40 lib/tdb: fix tests for standalone out-of-tree.
Commit 4d58d0fa8f didn't work for lib/tdb
outside the build tree: symlink was pointing to wrong place.

Copy simplification from lib/tdb2, and fix the build farm.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Autobuild-User: Rusty Russell <rusty@rustcorp.com.au>
Autobuild-Date: Sat Mar 10 07:07:45 CET 2012 on sn-devel-104
2012-03-10 07:07:45 +01:00
Rusty Russell
4d58d0fa8f tdb: build and run unit tests in tdb/test/
Now we can build the test binaries: the CCAN style is to compile
everything called "compile_ok*.c", compile and run everything called
"run*.c", compile, link with the module, and run everything called
"api*.c", and link any other C files (presumably test helpers) into
all the tests.

Unfortunately, actually passing that between the various parts of
wscript is painful, so I open-coded the names.

Also, the tests expect to be run in a (temporary) directory they can
pollute, with the test directory found in test/ (to find the canned
TDB files, for example).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Autobuild-User: Rusty Russell <rusty@rustcorp.com.au>
Autobuild-Date: Tue Feb 14 06:53:46 CET 2012 on sn-devel-104
2012-02-14 06:53:46 +01:00
Jelmer Vernooij
86afe83d86 waf: Factor checking for undefined symbol flags out into separate method.
Autobuild-User: Jelmer Vernooij <jelmer@samba.org>
Autobuild-Date: Sun Nov 13 19:38:38 CET 2011 on sn-devel-104
2011-11-13 19:38:38 +01:00
Jelmer Vernooij
774f85649b tdb: Only check for pkg-config file when checking for system tdb. 2011-11-13 18:06:06 +01:00
Jelmer Vernooij
31912781ca wafsamba: Only install .pc files if libraries are public. 2011-08-21 03:22:05 +02:00
Jelmer Vernooij
a5025a3c2f tdb: Install pkg-config file. 2011-08-21 03:22:05 +02:00
Jelmer Vernooij
bf3b2e2aee Support the 'PYTHON' environment variable.
Autobuild-User: Jelmer Vernooij <jelmer@samba.org>
Autobuild-Date: Sat Apr 23 04:19:05 CEST 2011 on sn-devel-104
2011-04-23 04:19:04 +02:00
Andrew Tridgell
c56e3ccfc9 lib: don't install public headers if a private library
for talloc/tevent/tdb, only install headers if we are doing a
standalone build
2011-03-15 12:22:20 +11:00
Andrew Tridgell
7b948a39e1 tdb: use public_headers to install header files 2011-03-15 12:22:19 +11:00
Andrew Tridgell
949427c208 python: use os.environ[] instead of os.putenv()
using os.putenv() causes too much confusion, as it doesn't update
os.getenv()

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2011-02-22 02:51:10 +01:00
Rusty Russell
cac57328a6 tdb: tdb_summary() support.
Autobuild-User: Rusty Russell <rusty@rustcorp.com.au>
Autobuild-Date: Wed Dec 29 10:12:05 CET 2010 on sn-devel-104
2010-12-29 10:12:05 +01:00
Stefan Metzmacher
7ea1b76797 tdb: setup TEST_DATA_PREFIX for make test
metze
2010-12-27 15:37:57 +01:00
Jelmer Vernooij
c754fad571 tdb: Bump version number after symbol versioning changes. 2010-12-21 19:43:04 +01:00
Matthieu Patou
8733738357 build: introduce SAMBA_CHECK_PYTHON_HEADERS
This function is a wrapper around waf's check_python_header.
It avoids searching more than once for the headers bringing a small
speed improvement and a better lisibility of the logs.

But it's mainly to avoid a nasty bug when python libraries are in path
pointed by python_LIBPL (ie. /usr/local/lib/python2.6/config/) instead
of python_LIBDIR (ie. /usr/local/lib).

On the first call waf will correctly find that in order to link with
python libs it needs to add -L$python_LIBPL.

But on the next calls of check_python_headers, waf will use both the
current library path value (ie. -L/usr/local/lib/python2.6/config) and
-L$python_LIBDIR (ie. /usr/local/lib/) which will make him beleive that
python libraries are in $python_LIBDIR which at the end will make the
final link test fails in check_python_headers as it will not use the
good directory.

So by avoiding calling check_python_headers more than once we avoid
making waf fooling itself.
2010-12-10 02:07:18 +03:00
Andrew Tridgell
57f2f1d72a waf: remove the restriction that private libraries must not have a vnum
we need the vnum for ABI checking for public libraries built as
private libraries when bundled

Autobuild-User: Andrew Tridgell <tridge@samba.org>
Autobuild-Date: Thu Dec  9 12:47:41 CET 2010 on sn-devel-104
2010-12-09 12:47:40 +01:00
Andrew Tridgell
ebe2867fc2 waf-abi: auto-generate per-symbol versions from ABI files
This changes our version-script generation to use the ABI files that
are saved in git with each version number change of our public
libraries.

We use these ABI files to generate a linker version script that gives
the exact version number that each symbol was introduced. This
provides us with automatic fine grained symbol versioning.

Pair-Programmed-With: Jelmer Vernooij <jelmer@samba.org>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2010-12-09 13:17:19 +11:00
Andrew Tridgell
f0a472a2d6 waf: added reconfigure targets to our libraries
This allows you to do "make reconfigure" to re-run configure only if
needed
2010-11-03 01:34:04 +00:00
Jelmer Vernooij
50256c01d0 waf: Only specify vnum for non-private libraries. 2010-10-26 10:17:18 -07:00
Jelmer Vernooij
49ef288819 waf: Rename some BUNDLED_ functios to PRIVATE_. 2010-10-23 22:24:06 +00:00
Andrew Tridgell
7cba3cfac8 waf: replace the is_bundled option with private_library
'private_library' better captures what we are trying to get at when we
bundle a library
2010-10-21 19:03:24 +11:00
Andrew Tridgell
713900b812 s4-build: fixed some formatting
indentation was far too deep
2010-10-21 19:03:24 +11:00
Jelmer Vernooij
05c1beb6b4 tdb: Bump version to 1.2.7 after addition of pytdb.__version__.
Autobuild-User: Jelmer Vernooij <jelmer@samba.org>
Autobuild-Date: Sun Oct 17 18:23:16 UTC 2010 on sn-devel-104
2010-10-17 18:23:16 +00:00
Andrew Tridgell
bb0017615d waf: fixed exit status of test suites
use RUN_COMMAND() to handle signal errors and exit status
2010-10-07 14:29:21 +11:00
Jelmer Vernooij
20d39691a8 tdb: Only use system pytdb when using system tdb. 2010-10-04 13:52:09 +02:00
Jelmer Vernooij
e805bf52c9 tdb: Support using system pytdb. 2010-10-04 13:52:09 +02:00
Jelmer Vernooij
6cd722d950 pytdb: Add __version__ attribute. 2010-10-04 13:52:09 +02:00
Rusty Russell
2dcf76c924 tdb: TDB_INCOMPATIBLE_HASH, to allow safe changing of default hash.
This flag to tdb_open/tdb_open_ex effects creation of a new database:
1) Uses the Jenkins lookup3 hash instead of the old gdbm hash if none is
   specified,
2) Places a non-zero field in header->rwlocks, so older versions of TDB will
   refuse to open it.

This means that the caller (ie Samba) can set this flag to safely
change the hash function.  Versions of TDB from this one on will either
use the correct hash or refuse to open (if a different hash is specified).
Older TDB versions will see the nonzero rwlocks field and refuse to open
it under any conditions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-09-27 10:48:28 +09:30
Rusty Russell
3258cf3f11 tdb: add Bob Jenkins lookup3 hash as helper hash.
This is a better hash than the default: shipping it with tdb makes it easy
for callers to use it as the hash by passing it to tdb_open_ex().

This version taken from CCAN and modified, which took it from
http://www.burtleburtle.net/bob/c/lookup3.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-09-27 10:48:28 +09:30
Volker Lendecke
66731461a6 tdb: add restore
Based on an idea by Simon McVittie, largely rewritten
2010-09-23 12:19:09 -07:00
Stefan Metzmacher
d10b2c07be lib/tdb: change version to 1.2.4 after hash checking improvments
lib/tdb: change version to 1.2.4 after hash checking improvments

metze

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-09-13 20:08:58 +09:30
Björn Jacke
20254221d2 tdb/waf: the deps for rt are provided by libreplace 2010-08-31 13:06:39 +02:00
Rusty Russell
11ab43084b tdb: workaround starvation problem in locking entire database.
We saw tdb_lockall() take 71 seconds under heavy load; this is because Linux
(at least) doesn't prevent new small locks being obtained while we're waiting
for a big log.

The workaround is to do divide and conquer using non-blocking chainlocks: if
we get down to a single chain we block.  Using a simple test program where
children did "hold lock for 100ms, sleep for 1 second" the time to do
tdb_lockall() dropped signifiantly.  There are ln(hashsize) locks taken in
the contended case, but that's slow anyway.

More analysis is given in my blog at http://rusty.ozlabs.org/?p=120

This may also help transactions, though in that case it's the initial
read lock which uses this gradual locking routine; the update-to-write-lock
code is separate and still tries to update in one go.

Even though ABI doesn't change, minor version bumped so behavior change
can be easily detected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-08-14 02:31:22 +09:30
Andrew Tridgell
0be1820718 s4-ldb: use CHECK_XSLTPROC_MANPAGES() 2010-06-24 18:46:57 +10:00
Jelmer Vernooij
bd1786f7f1 wafsamba: Fix typo: noextenion -> noextension. 2010-06-19 12:10:29 +02:00
Jelmer Vernooij
04fc459a30 tdb-waf: Don't install binaries when using system tdb. 2010-05-31 04:20:44 +02:00
Jelmer Vernooij
e6129bd596 tdb-waf: Simplify manpage handling. 2010-05-31 04:16:19 +02:00
Jelmer Vernooij
3858b5db8b s4-waf: sanitize library names like the old build system did, only add
-samba4 suffix for libraries that are bundled.
2010-05-31 03:28:42 +02:00
Andrew Tridgell
06fa4c8ebc build: make python development headers not mandatory in standalone libs
This needed an update to the python tool in waf

thanks to Kai for spotting this
2010-04-21 16:17:08 +10:00
Andrew Tridgell
773a8afbba tdb: update tdb ABI to use hide_symbols=True
We now use -fvisibilty=hidden to hide symbols from outside the tdb
shared library.

This also moved tdb_transaction_recover() into the tdb_private.h
header, as it should never have been a public API. For that reason we
are changing the version number. We're only doing a minor version
increment as it is extremely unlikely that anyone was actually using
tdb_transaction_recover() as its locking requirements were rather
unusual.

Pair-Programmed-With: Rusty Russell <rusty@samba.org>
2010-04-20 15:50:27 +10:00
Andrew Tridgell
805f651dbf tdb-waf: added ABI checking for tdb 2010-04-18 15:00:38 +10:00
Andrew Tridgell
c10d267a76 s4-waf: don't try to use the system lib for the library build
when building library FOO, don't try to find the system library FOO
    
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2010-04-13 23:03:27 +10:00
Andrew Tridgell
367ba43b63 tdb-waf: added build of manpages and config options for RPM build
Building a RPM from the fedora spec file now works with minimal changes

Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2010-04-13 20:15:41 +10:00
Andrew Tridgell
7ae4372124 s4-waf: fixed waf distcheck for our standalone libs and s4 2010-04-06 20:27:26 +10:00
Andrew Tridgell
fe060c8f50 s4-waf: added simple 'waf test' support for tdb, talloc and ldb 2010-04-06 20:27:26 +10:00
Andrew Tridgell
c8835f440a s4-waf: use a common pattern for finding buildtools and libs 2010-04-06 20:27:25 +10:00
Andrew Tridgell
553324bc10 s4-waf: move to a universal method of recursing into subdirs
This works with both standalone lib builds and bundled builds
2010-04-06 20:27:25 +10:00
Andrew Tridgell
8dc8d31f4a s4-waf: fixed finding of buildtools and srcdir 2010-04-06 20:27:25 +10:00
Andrew Tridgell
1883ee6dbc s4-waf: avoid having to run waf configure before waf dist
This should be useful for building tarballs from a clean checkout
2010-04-06 20:27:25 +10:00
Andrew Tridgell
9e9e878e8f s4-waf: allow standalone tarball build of libraries 2010-04-06 20:27:25 +10:00
Andrew Tridgell
528d7c7cf3 s4-waf: added 'waf dist' to our standalone libs as well 2010-04-06 20:27:25 +10:00
Andrew Tridgell
874e39cbf0 s4-waf: added implied_deps for system libraries
when we use a system version of a library such as talloc, then we
no longer get the automtica dependency propogation of talloc implying
libreplace. That means we don't get the includes for libreplace, which
means things can fail to build.

To fix this this change adds an implied_deps option to
CHECK_BUNDLED_SYSTEM(), which tells the samba_deps module to add an
implied dependency on the listed targets if the system library is
chosen.
2010-04-06 20:27:17 +10:00
Andrew Tridgell
bfbf43cc36 s4-waf: support the use of system libraries
distros can set --bundled-libraries=NONE to force use of all system
libraries. If the right version isn't found then configure will fail.

Users may choose which libraries to use from the system, and which to
use bundled libs. The default is to try system libs, and use them if
their version matches the one in the source tree.
2010-04-06 20:27:13 +10:00
Andrew Tridgell
c2d17cf978 s4-waf: set default for bundled libraries 2010-04-06 20:27:12 +10:00
Andrew Tridgell
42c9380760 s4-waf: set the bundled library extension for some libs 2010-04-06 20:27:12 +10:00
Andrew Tridgell
f9eae32f4b s4-waf: mark the wscript files as python so vim/emacs knows how to highlight them 2010-04-06 20:27:11 +10:00
Andrew Tridgell
8b14d217c4 s4-waf: fdatasync is in librt on solaris8 2010-04-06 20:27:08 +10:00
Andrew Tridgell
365f29136f build: only enable pytdb on s4 build 2010-04-06 20:26:51 +10:00
Andrew Tridgell
69fb313233 build: enable pytdb 2010-04-06 20:26:51 +10:00
Andrew Tridgell
7ed65d2e12 build: result of hack session with ita 2010-04-06 20:26:40 +10:00
Andrew Tridgell
5ad6369cd0 build: started to split up wafsamba.py into separate modules 2010-04-06 20:26:38 +10:00
Andrew Tridgell
a3421fe6e9 build: added BUILD_SUBDIR() wrapper 2010-04-06 20:26:35 +10:00
Andrew Tridgell
9a1cd09a10 build: added ADD_CFLAGS() and started of Samba4 build 2010-04-06 20:26:34 +10:00
Andrew Tridgell
0316ec1d89 build: tdb does not depend on talloc 2010-04-06 20:26:34 +10:00
Andrew Tridgell
2b5a1cc70c build: added waf build rules for ldb 2010-04-06 20:26:34 +10:00
Andrew Tridgell
8f062f4a0f build: neater way to find libreplace and start on tevent waf build 2010-04-06 20:26:33 +10:00
Andrew Tridgell
2bd0756dc5 build: more binaries for waf tdb build 2010-04-06 20:26:33 +10:00
Andrew Tridgell
8d6eb2a02c build: added waf build for tdb 2010-04-06 20:26:33 +10:00