1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-28 07:21:54 +03:00
Commit Graph

100 Commits

Author SHA1 Message Date
Stefan Metzmacher
31bc9f3ee1 py: Properly increase the reference counter of Py_None. (cherry picked from samba commit d2c70d24e1)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit e7242221c3318a5c312e17ff4074bef80b639ca8)
2009-12-16 08:03:51 +01:00
Jelmer Vernooij
fc9d260310 Make sure to not close tdb database more than once. (cherry picked from samba commit 6fe6983e4c)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 52f78e040749d24058ee1f575d949b57d15f5987)
2009-12-16 08:03:51 +01:00
Jelmer Vernooij
c9907f2634 Implement missing functions in pytdb. (cherry picked from samba commit 2da551bbcc)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 3a671b11770057c91e0ae646499d4714f52bc5c0)
2009-12-16 08:03:51 +01:00
Stefan Metzmacher
50ceb83180 Add simple manually written replacement for the tdb module. (cherry picked from samba commit 2a61fd41e9)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 2231ee0aa163d68383dd9636f25f033fe7c1f3e7)
2009-12-16 08:03:50 +01:00
Jelmer Vernooij
4c69b5e0fd tdb: Add simple reimplementation of tdbdump in Python as an example of the tdb Python bindings. (This used to be commit 47d797f788) (cherry picked from samba commit 6bdd1425b7)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 363c34d4bb488609317794cd3153d85c12643110)
2009-12-16 08:03:50 +01:00
Jeremy Allison
e58068d666 Remove unecessary msync. Jeremy. (cherry picked from samba commit 0bae1ef3de) (This used to be commit db2acaf46f) (cherry picked from samba commit a1cf3ad5d6)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 236dc2fa29b3c0caec51859dbd469f0a13f5917e)
2009-12-16 08:03:50 +01:00
Stefan Metzmacher
61c7444943 The msync manpage reports that msync *must* be called before munmap. Failure to do so may result in lost data. Fix an ifdef check, I really think we meant to check HAVE_MMAP here. (cherry picked from samba commit 74c8575b3f) (This used to be commit 8fd54bb55f) (cherry picked from samba commit b39e332bd7)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 5aa0ab328c36ecd4d7ec03f921e6027340c2ef13)
2009-12-16 08:03:50 +01:00
Volker Lendecke
a3b4f9b59e Attempt to fix bug 5684
With the ctdb checkin dde9f3f006 tdb optimized out write lock checks for
write-enabled transaction. Sadly, this also removed the possibility to ever
remove dead records left over from tdb_delete calls within a transaction.

Tridge, please check this! Did dde9f3f006 have any reason beyond performance
optimizations?

Thanks,

Volker
(cherry picked from samba commit 3f884c4ae3)
(This used to be commit 1d85e0647e)
(cherry picked from samba commit 8c88209c6f)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit b02bf7659f04f1fa203834bd75a2392b48e56c16)
2009-12-16 08:03:50 +01:00
Slava Semushin
06ab3cfe60 lib/tdb/tools/tdbtorture.c: fixed memory leak.
Found by cppcheck:
[lib/tdb/tools/tdbtorture.c:326]: (error) Memory leak: pids
(cherry picked from samba commit 497b9e460b)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 5d4cc4b018a538dc3f1d79fe091f3e6e67003daf)
2009-12-16 08:03:50 +01:00
Andrew Tridgell
3e04b100c1 added basic testing of tdb_transaction_prepare_commit() in tdbtorture (cherry picked from samba commit 84547b8dba)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 923b61fe722c0aec8a5b6ac8dd1df74957dc102b)
2009-12-16 08:03:50 +01:00
Andrew Tridgell
5b6b852691 make tdbbackup use transactions
tdbbackup was originally written before we had transactions, and it
attempted to use its own fsync() calls to make it safe. Now that we
have transactions we can do it in a much safer (and faster!) fashion
(cherry picked from samba commit 2e4247782b)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit cd23d36ada9631095ca68663516de0c8d8c3bbed)
2009-12-16 08:03:49 +01:00
Andrew Tridgell
6eaaa52a1d fixed tdbbackup to give tdb error messages (cherry picked from samba commit 08be1420ba)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 3d44412593b8748a5158e15b83cd9eb548231194)
2009-12-16 08:03:49 +01:00
Rusty Russell
b52a06ffc6 lib/tdb: add -t (always use transactions) option to tdbtorture
This means you can kill it at any time and expect no corruption.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(cherry picked from samba commit 0fc6800005)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit f7278a277ed91587cae5b5e3660dad7124bdb73f)
2009-12-16 08:03:49 +01:00
Rusty Russell
8dcc760f1e lib/tdb: wean off TDB_ERRCODE.
It was a regrettable hack which I used to reduce line count in tdb; in fact it caused confusion as can be seen in this patch.
In particular, ecode now needs to be set before TDB_LOG anyway, and having it exposed in
the header is useless (the struct tdb_context isn't defined, so it's doubly useless).
Also, we should never set errno, as io.c was doing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

(cherry picked from samba commit b77f41d58b)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit a6620f6e74aadc708395b21b42303d1082192fcc)
2009-12-16 08:03:49 +01:00
Rusty Russell
7f857c4d14 lib/tdb: TDB_TRACE support (for developers)
When TDB_TRACE is defined (in tdb_private.h), verbose tracing of tdb operations is enabled.
This can be replayed using "replay_trace" from http://ccan.ozlabs.org/info/tdb.

The majority of this patch comes from moving internal functions to _<funcname> to
avoid double-tracing.  There should be no additional overhead for the normal (!TDB_TRACE)
case.

Note that the verbose traces compress really well with rzip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 703004340c)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit b01b756cb577f32a1ec4597efb00017241e01685)
2009-12-16 08:03:49 +01:00
Andrew Tridgell
805ef91707 tdb: fixed the intermittent failure of tdbtorture in the build farm
There was a race condition that caused the torture.tdb to be left in a
state that needed recovery. The torture code thought that any message
from the tdb code was an error, so the "recovered" message, which is a
TDB_DEBUG_TRACE message, marked the run as being an error when it
isn't.
(cherry picked from samba commit 5dcf0069b6)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 43c97b259b19c42b4edc7f83dbfc5e486568b4e3)
2009-12-16 08:03:49 +01:00
Michael Adam
8fd54bbbe1 tdb:tdbtool: fix indentation.
Michael
(cherry picked from samba commit e440a2e11e)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit c1b8d32b4ef87b9d8f37b451f47fcee2ea753d21)
2009-12-16 08:03:48 +01:00
Stefan Metzmacher
42648556a6 Fix all warnings in source3 with gcc4.3. Jeremy. (cherry picked from samba commit 07e0094365)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit e4d49c182e12c2d429d0414209cc2c8ccc19dc91)
2009-12-16 08:03:48 +01:00
Tim Prouty
a04fecb1c2 s3/s4: Fix "shadows a global declaration" warning (cherry picked from samba commit e48a5cd5d4)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 94f5728a77b8c772fb16c4744c24b45de8016e52)
2009-12-16 08:03:48 +01:00
Tim Prouty
c8366fcfb4 tdb: Fix some recently introduced warnings in tdbtool (cherry picked from samba commit c299833bf8)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 5830a2427b84e1cce74390b58fe12c45b5a056a6)
2009-12-16 08:03:48 +01:00
Andrew Tridgell
e0bed62820 added some more speed tests to tdbtool
This adds 3 simple speed tests to tdbtool, for transaction store,
store and fetch.

On my laptop this shows transactions costing about 10ms
(cherry picked from samba commit e15027155d)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 463279c972fa4538919bdd1dff48ca6b2fb8d49c)
2009-12-16 08:03:48 +01:00
Michael Adam
886cb3e86d tdb:tdbtool: add transaction_start/_commit/_cancel commands.
So one can perform tdbtool operations protected by transactions.

Michael
(cherry picked from samba commit 91e1bab2e9)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 35a5b874b925380f7c227e47aebb590c9db4739e)
2009-12-16 08:03:48 +01:00
Michael Adam
168bb40b4b tdb:tdbtool: add the "speed" command to the help text.
Michael
(cherry picked from samba commit 817383d88d)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit dc287a7d7420cca0b104049e689a73202bc535f8)
2009-12-16 08:03:47 +01:00
Holger Hetterich
0a0281444d Added a simple tdb integrity check to tdbtool. The command "check" runs traverse on the currently open tdb, and returns the number of entries if the integrity check is successful. (cherry picked from samba commit 42366bcbbd)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 02b35ba77672727c96ad004be37c7f6f1d3fe474)
2009-12-16 08:03:47 +01:00
Andrew Tridgell
09f7874151 tdb: allow reads after prepare commit
We previously only allowed a commit to happen after a prepare
commit. It is in fact safe to allow reads between a prepare and a
commit, and the s4 replication code can make use of that, so allow it.
(cherry picked from samba commit 46c99ec2a3)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 5ef5ddb8369e5e76173285fe9a08498dc8dc73ab)
2009-12-16 08:03:46 +01:00
Michael Adam
d05b49aaf2 tdb:mksigs: allow PRINTF_ATTRIBUTE(..) macros function types as funcion args
Michael
(cherry picked from samba commit 55dcf928eb)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit ef1dc585d869a9e48164cd65bafc92c1da245007)
2009-12-16 08:03:46 +01:00
Michael Adam
ab37ff7c04 tdb:mksigs: normalize bool -> _Bool
Michael
(cherry picked from samba commit cfa4e7ec75)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 0ae735b7a2096a40e5e47086ec41d9d45ef6d36b)
2009-12-16 08:03:46 +01:00
Michael Adam
a0dba36390 tdb:mksigs: ignore symbols (like _DEPRECATED_) after closing function parentheses
Michael
(cherry picked from samba commit 25939a627f)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 2e69647404c87c438ae7c180277ac3b532941efd)
2009-12-16 08:03:46 +01:00
Michael Adam
6d9ce0ef50 tdb:mksigs: correctly ignode multiline function typedefs
by first concatenating multilint parentheses and removing typefes afterwards.

Michael
(cherry picked from samba commit 13bfcd5a93)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 37225f1ed3f70d7259c2af2c51c671105c34476a)
2009-12-16 08:03:46 +01:00
Michael Adam
67da31e222 tdb:mksigs: ignore struct forward declarations.
Michael
(cherry picked from samba commit ecd12bfb38)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 66fffa577e051212ac7541be906b6c80f4a7c0c9)
2009-12-16 08:03:45 +01:00
Michael Adam
67d7709140 tdb:mksyms: allow characters after closing functions parenthesis.
Michael
(cherry picked from samba commit 400f08450b)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 310d673b7cb9000d76437d78e43bc2bf133e4e14)
2009-12-16 08:03:44 +01:00
Michael Adam
31b9126d29 tdb:mksyms: allow double pointer return value of functions.
Michael
(cherry picked from samba commit 907e05595f)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit f70e371d70e334a7254649b2bb09aa382e6f09bb)
2009-12-16 08:03:44 +01:00
Günther Deschner
9872a2faab tdb: fix c++ build warning.
Guenther
(cherry picked from samba commit 1c2f4919ab)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 9d5015e6fc68d3eb9e7b7178dbaf8c129dc79471)
2009-12-16 08:03:44 +01:00
Michael Adam
2b29e30df5 One would expect I could spell my name... (cherry picked from samba commit 0d120be36b)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit efa4a795db7fb2bddaab3969850d1554fc5f4da1)
2009-12-16 08:03:43 +01:00
Michael Adam
6e522eb198 tdb: add script/abi_checks.sh. check for abi changes without gcc magic.
USAGE: abi_checks.sh LIBRARY_NAME header1 [header2 ...]

This creates symbol signature lists using the mksyms and mksigs scripts
and compares them with the checked in lists.

Michael
(cherry picked from samba commit 9636e0d373)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 724d71dc838750fff91a45359feeb6e71bf0a4c7)
2009-12-16 08:03:43 +01:00
Michael Adam
52b657756a tdb: add script to extract signatures from header files.
This produces output like the output gcc produces when
invoked with the -aux-info switch.

Run like this: cat include/tdb.h | ./script/mksigs.pl

This simple parser is probably too coarse to handle all
possible header files, but it treats tdb.h correctly...

Michael
(cherry picked from samba commit 0760a04ef9)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 141422d9dc24b15b7b8bc7831adab90367a729f7)
2009-12-16 08:03:43 +01:00
Michael Adam
53336a9cb1 tdb: add scripts to extract library symbols (exports file) from headers
Michael
(cherry picked from samba commit 006fd0c43c)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit aed864dceaf6ec1e6e6066a587c708b485901200)
2009-12-16 08:03:43 +01:00
Rusty Russell
eb9d367843 lib/tdb: don't overwrite TDBs with different version numbers.
In future, this may happen, and we don't want to clobber them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 398d0c2929)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit eebd467961dad6cfb38c2a5d6e4b4dbf86e55e63)
2009-12-16 08:03:43 +01:00
Jeremy Allison
41d4e2dc7c Add define guards around otherwise unused variable. Jeremy. (cherry picked from samba commit 4fc9f9c3f9)
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 6f8614de0f20d4c507aecd744d9c3f6545078127)
2009-12-16 08:03:42 +01:00
Rusty Russell
e1217b7bdb There is one signedness issue in tdb which prevents traverses of TDB records over the 2G offset on systems which support 64 bit file offsets. This fixes that case.
On systems with 32 bit offsets, expansion and fcntl locking on these records
will fail anyway.  SAMBA already does '#define _FILE_OFFSET_BITS 64' in
config.h (on my 32-bit x86 Linux system at least) to get 64 bit file offsets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 252f7da702)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 2d768f664e6db65b3b7e0c732f33ee2b806892f9)
2009-12-16 08:03:42 +01:00
Ronnie Sahlberg
029fd6b00f Revert "try to restart statd everytime it fails, not just the first time"
This reverts commit 4f7b39a4871af28df1c4545ec37db179fa47a7da.

(This used to be ctdb commit db7b96304e4725f29b12398b7582e385daed63ed)
2009-09-15 19:33:35 +10:00
Ronnie Sahlberg
59cacded72 try to restart statd everytime it fails, not just the first time
(This used to be ctdb commit 4f7b39a4871af28df1c4545ec37db179fa47a7da)
2009-09-15 13:35:58 +10:00
Mathieu Parent
f095a64137 Update outdated autotools helper files.
This fixes https://bugzilla.samba.org/show_bug.cgi?id=6370
and http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=536256

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 4899a888d4e2c31b4bd7220f3cd8936168942e61)
2009-09-09 09:52:13 +02:00
Rusty Russell
3cc352f975 tdb: don't alter tdb->flags in tdb_reopen_all()
The flags are user-visible, via tdb_get_flags/add_flags/remove_flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 8f48e37c254e0852d4e2dea54b905ce5ef2b925d)
2009-08-04 16:50:55 +09:30
Rusty Russell
ed2c8e415f tdb: Reimplementation of Metze's "lib/tdb: if we know pwrite and pread are thread/fork safe tdb_reopen_all() should be a noop".
This version just wraps the reopen code, so we still re-grab the lock and do
the normal sanity checks.

The reason we do this at all is to avoid global fd limits, see:
http://forums.fedoraforum.org/showthread.php?t=210393

Note also that this whole reopen concept is fundamentally racy: if the parent
goes away before the child calls tdb_reopen_all, the database can be left
without an active lock and another TDB_CLEAR_IF_FIRST opener will clear it.
A fork_with_tdbs() wrapper could use a pipe to solve this, but it's hardly
elegant (what if there are other independent things which have similar needs?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 8d0d432ab7766d9c0f9868fd77e48b9b5cc5d9f9)
2009-08-04 16:50:52 +09:30
Rusty Russell
6e4577d08f realloc() has that horrible overloaded free semantic when size is 0: current code does a free of the old record in this case, then fail.
(This used to be ctdb commit 8b6a5bba93843cd83b7b386b82949ad88f29884a)
2009-08-04 16:50:48 +09:30
Rusty Russell
1664d75ad3 If the record is at the end of the database, pretending it has length 1 might take us out-of-bounds. Only pretend to be length 1 for the malloc.
(This used to be ctdb commit 6de2823f5f7976d4efa20761e518d6b67753f054)
2009-08-04 16:50:43 +09:30
Rusty Russell
6aa98fb041 Port from SAMBA tdb: commit 54a51839ea Author: Rusty Russell <rusty@rustcorp.com.au> Date: Sat Jul 18 15:28:58 2009 +0930
Make tdb transaction lock recursive (samba version)

    This patch replaces 6ed27edbcd and
    1a416ff13c, which fixed the bug where traversals
    inside transactions would release the transaction lock early.

    This solution is more general, and solves the more minor symptom that nested
    traversals would also release the transaction lock early.  (It was also suggestd in
    Volker's comment in 6ed27ed).

    This patch also applies to ctdb, if the traverse.c part is removed (ctdb's tdb
    code never received the previous two fixes).

    Tested using the testsuite from ccan (adapted to the samba code).  Thanks to
    Michael Adam for feedback.

    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Signed-off-by: Michael Adam <obnox@samba.org>
commit 760104188d
Author: Rusty Russell <rusty@rustcorp.com.au>
Date:   Tue Jul 21 16:23:35 2009 +0930

    tdb: fix locking error

    54a51839ea "Make tdb transaction lock
    recursive (samba version)" was broken: I "cleaned it up" and prevented
    it from ever unlocking.

    To see the problem:
        $ bin/tdbtorture -s 1248142523
        tdb_brlock failed (fd=3) at offset 8 rw_type=1 lck_type=14 len=1
        tdb_transaction_lock: failed to get transaction lock
        tdb_transaction_start failed: Resource deadlock avoided

    My testcase relied on the *count* being correct, which it was.  Fixing that
    now.

    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit ce19658ba13272238058e9b9bc03e62f48b737c0)
2009-08-04 16:43:34 +09:30
Rusty Russell
825391de1d Port from SAMBA tdb: commit a6cc04a200 Author: Andrew Tridgell <tridge@samba.org> Date: Mon Jun 1 13:13:07 2009 +1000
overallocate all records by 25%

    This greatly reduces the fragmentation of databases where records
    tend to grow slowly by a small amount each time. The case where this
    is most seen is the ldb index records. Adding this overallocation
    reduced the size of the resulting database by more than 20x when
    running a test that adds 10k users.

(This used to be ctdb commit e72974e5cefabc7035399d16633f727f868caa61)
2009-08-04 16:43:31 +09:30
Rusty Russell
94ee0cbe3a Port from SAMBA tdb: commit a386173fa1 Author: Andrew Tridgell <tridge@samba.org> Date: Mon Jun 1 13:11:39 2009 +1000
auto-repack in transactions that expand the tdb

    The idea behind this is to recover from badly fragmented free
    lists. Choosing the point where the file expands is fairly arbitrary,
    but seems to work well.

(This used to be ctdb commit 233c52bfb087f636ad61e95c12616c02901f4f83)
2009-08-04 16:43:26 +09:30