IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
To test the case of death of a process during transaction commit, add
a -k (kill random) option to tdbtorture. The easiest way to do this
is to make every worker a child (unless there's only one child), which
is why this patch is bigger than you might expect.
Using -k without -t (always transactions) you expect corruption, though
it doesn't happen every time. With -t, we currently get corruption but
the next patch fixes that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit ececeffd85)
(This used to be ctdb commit e5af0ce79c74f11daae2a8514e398c40535e2e67)
The current recovery code truncates the tdb file on recovery. This is
fine if recovery is only done on first open, but is a really bad idea
as we move to allowing recovery on "live" databases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 8c3fda4318)
(This used to be ctdb commit 65bc926d1a9cb3af18cae6b1462b832a5bcec561)
Now the transaction code uses the standard allrecord lock, that stops
us from trying to grab any per-record locks anyway. We don't need to
have special noop lock ops for transactions.
This is a nice simplification: if you see brlock, you know it's really
going to grab a lock.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 9f295eecff)
(This used to be ctdb commit 6d7093cf51d0256245cc6bab24c9550ed3f1d8a5)
tdb_release_extra_locks() is too general: it carefully skips over the
transaction lock, even though the only caller then drops it. Change
this, and rename it to show it's clearly transaction-specific.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit a84222bbaf)
(This used to be ctdb commit 803035716338170896fee15f15b17c32e7ee777e)
Now the transaction allrecord lock is the standard one, and thus is cleaned
in tdb_release_extra_locks(), _tdb_transaction_cancel() doesn't need to
know what type it is.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit dd1b508c63)
(This used to be ctdb commit 74874ffb2c81e098c1d7935b37557c2151382ca6)
Centralize locking of all chains of the tdb; rename _tdb_lockall to
tdb_allrecord_lock and _tdb_unlockall to tdb_allrecord_unlock, and
tdb_brlock_upgrade to tdb_allrecord_upgrade.
Then we use this in the transaction code. Unfortunately, if the transaction
code records that it has grabbed the allrecord lock read-only, write locks
will fail, so we treat this upgradable lock as a write lock, and mark it
as upgradable using the otherwise-unused offset field.
One subtlety: now the transaction code is using the allrecord_lock, the
tdb_release_extra_locks() function drops it for us, so we no longer need
to do it manually in _tdb_transaction_cancel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit fca1621965)
(This used to be ctdb commit d7fdb38ac05b5f2af9eb485e98673280835273dd)
Records themselves get (read) locked by the traversal code against delete.
Interestingly, this locking isn't done when the allrecord lock has been
taken, though the allrecord lock until recently didn't cover the actual
records (it now goes to end of file).
The write record lock, grabbed by the delete code, is not suppressed
by the allrecord lock. This is now bad: it causes us to punch a hole
in the allrecord lock when we release the write record lock. Make this
consistent: *no* record locks of any kind when the allrecord lock is
taken.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit caaf5c6baa)
(This used to be ctdb commit 7a99cdf5d0a91764a750c1a264e90e5b66f910a1)
We were previously inconsistent with our "global" lock: the
transaction code grabbed it from FREELIST_TOP to end of file, and the
rest of the code grabbed it from FREELIST_TOP to end of the hash
chains. Change it to always grab to end of file for simplicity and
so we can merge the two.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 9341f230f8)
(This used to be ctdb commit 46f2c33357c999c31a8064c159c6162269c28d9d)
This was redundant before this patch series: it mirrored num_lockrecs
exactly. It still does.
Also, skip useless branch when locks == 1: unconditional assignment is
cheaper anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 1ab8776247)
(This used to be ctdb commit 587ac01ce836286aab54bfcb7a693a0170c7ebd3)
This is pure overhead, but it centralizes the locking. Realloc (esp. as
most implementations are lazy) is fast compared to the fnctl anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit d48c3e4982)
(This used to be ctdb commit 2e8512403525c14c9b776ce28891d09c17ada91d)
Use our newly-generic nested lock tracking for the active lock.
Note that the tdb_have_extra_locks() and tdb_release_extra_locks()
functions have to skip over this lock now it is tracked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 4738d474c4)
(This used to be ctdb commit 0a44584963232b0b1c62e30c9bede0439e68ef7d)
This never nests, so it's overkill, but it centralizes the locking into
lock.c and removes the ugly flag in the transaction code to track whether
we have the lock or not.
Note that we have a temporary hack so this places a real lock, despite
the fact that we are in a transaction.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 9136818df3)
(This used to be ctdb commit 6812d81907793299e874f121174d885f6500f374)
Rather than a boutique lock and a separate nest count, use our
newly-generic nested lock tracking for the transaction lock.
Note that the tdb_have_extra_locks() and tdb_release_extra_locks()
functions have to skip over this lock now it is tracked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit e8fa70a321)
(This used to be ctdb commit 4ca1b96a70048b2eaa0d12fb5f0fdb54ec396aa3)
Factor out two loops which find locks; we are going to introduce a couple
more so a helper makes sense.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit ce41411c84)
(This used to be ctdb commit cfb154dd0f189f37b937e90144c2eb9e66a26420)
Move locking intelligence back into lock.c, rather than open-coding the
lock release in transaction.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit db270734d8)
(This used to be ctdb commit d2dd720b51c4032e5d77d30212da8117d3f119df)
In many places we check whether locks are held: add a helper to do this.
The _tdb_lockall() case has already checked for the allrecord lock, so
the extra work done by tdb_have_extra_locks() is merely redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit fba42f1fb4)
(This used to be ctdb commit dda3587dfee598f387c2e696f3645486fac65052)
tdb_transaction_lock() and tdb_transaction_unlock() do nothing if we
hold the allrecord lock. However, the two locks don't overlap, so
this is wrong.
This simplification makes the transaction lock a straight-forward nested
lock.
There are two callers for these functions:
1) The transaction code, which already makes sure the allrecord_lock
isn't held.
2) The traverse code, which wants to stop transactions whether it has the
allrecord lock or not. There have been deadlocks here before, however
this should not bring them back (I hope!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit b754f61d23)
(This used to be ctdb commit 495f3554259b9dbf9ee673c4fe420d98e50e4901)
Because fcntl locks don't nest, we track them in the tdb->lockrecs array
and only place/release them when the count goes to 1/0. We only do this
for record locks, so we simply place the list number (or -1 for the free
list) in the structure.
To generalize this:
1) Put the offset rather than list number in struct tdb_lock_type.
2) Rename _tdb_lock() to tdb_nest_lock, make it non-static and move the
allrecord check out to the callers (except the mark case which doesn't
care).
3) Rename _tdb_unlock() to tdb_nest_unlock(), make it non-static and
move the allrecord out to the callers (except mark again).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 5d9de604d9)
(This used to be ctdb commit 28576ddbd9bf91049db8a4f9e9e7856ac5b8f48a)
The word global is overloaded in tdb. The global_lock inside struct
tdb_context is used to indicate we hold a lock across all the chains.
Rename it to allrecord_lock.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit e9114a7585)
(This used to be ctdb commit a912657fb50a78b9b328c4564281fb9f7f1b3766)
The word global is overloaded in tdb. The GLOBAL_LOCK offset is used at
open time to serialize initialization (and by the transaction code to block
open).
Rename it to OPEN_LOCK.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 7ab422d6fb)
(This used to be ctdb commit a4f83910a485cf56f9b3df1dcf2ad36ebec57473)
Now tdb_open() calls tdb_transaction_cancel() instead of
_tdb_transaction_cancel, we can make it static.
Signed-off-by: Rusty Russell<rusty@rustcorp.com.au>
(Imported from commit a6e0ef87d2)
(This used to be ctdb commit d728a7f65bcd5f1aedbee41d6db5c35f10cb417e)
This is taken from the CCAN code base: rather than using tdb_brlock for
locking and unlocking, we split it into brlock and brunlock functions.
For extra debugging information, brunlock says what kind of lock it is
unlocking (even though fnctl locks don't need this). This requires an
extra argument to tdb_transaction_unlock() so we know whether the
lock was upgraded to a write lock or not.
We also use a "flags" argument tdb_brlock:
1) TDB_LOCK_NOWAIT replaces lck_type = F_SETLK (vs F_SETLKW).
2) TDB_LOCK_MARK_ONLY replaces setting TDB_MARK_LOCK bit in ltype.
3) TDB_LOCK_PROBE replaces the "probe" argument.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 452b4a5a6e)
(This used to be ctdb commit 7b5fdc9c588237c83a1e70e5437e2e5510055b92)
Signed-off-by: Matthias Dieter Wallnöfer <mwallnoefer@yahoo.de>
(Imported from commit 09e756b1d6)
(This used to be ctdb commit b0dff4ed35ab2423b8fcc801cdaaebaa0d7654bb)
If a process (or the machine) dies after just after writing the
recovery head (pointing at the end of file), the recovery record will filled
with 0x42. This will not invoke a recovery on open, since rec.magic
!= TDB_RECOVERY_MAGIC.
Unfortunately, the first transaction commit will happily reuse that
area: tdb_recovery_allocate() doesn't check the magic. The recovery
record has length 0x42424242, and it writes that back into the
now-valid-looking transaction header) for the next comer (which
happens to be tdb_wipe_all in my tests).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit b37b452cb8)
(This used to be ctdb commit 8c8782ecbb347e026f67d82a39555c0e43b1e9f8)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(Imported from commit 6269cdcd15)
(This used to be ctdb commit e0ca2e02120258aabca1e1586a58a8d672484fb5)
This should make it easier to keep all release scripts alined as it will reduce
the difference between them to ideally a few variables
Also moves the tdb script in the scripts directory.
(Imported from commit 6339de7f4f)
(This used to be ctdb commit 8885b2206fba41ec289fda5dfd653ee676aa0dd3)
after recent fixes we need to raise the version to 1.2.1 so that
we can require also the right patched version.
(Imported from commit 70534adee1)
(This used to be ctdb commit 84c971f33c24d32e5599aba7ba83bb474f7ac922)
The TLIST_*() macros are like the DLIST_*() macros, but take both a
head and tail pointer for the list. This means that adding an element
to the end of the list is efficient (it doesn't need to walk the
list).
We should move all uses of the DLIST_*() macros which use
DLIST_ADD_END() to use the TLIST_*() macros instead.
(This used to be ctdb commit 2d05a71349e9ade869b62cf261c2a9a21818a474)
There was a bug in tdb where the
tdb_brlock(tdb, GLOBAL_LOCK, F_UNLCK, F_SETLKW, 0, 1);
(ending the transaction-"mutex") was done before the
/* remove the recovery marker */
This means that when a transaction is committed there is a window where another
opener of the file sees the transaction marker while the transaction committer
is still fully functional and working on it. This led to transaction being
rolled back by that second opener of the file while transaction_commit() gave
no error to the caller.
This patch moves the F_UNLCK to after the recovery marker was removed, closing
this window.
(This used to be ctdb commit 898b5edfe757cb145960b8f3631029bfd5592119)
This finished commit a78b8ea7168e5fdb2d62379ad3112008b2748576.
The logic was missing in events_standard (the one that's used by default).
metze
(This used to be ctdb commit 49f0488a5e60c74b6b8361dffcd09ebb2db740f0)
(cherry picked from commit 4334092cba)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 093f57a2c00f2d629a3b58e58202f1a7e1bbd406)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from samba commit 9776cb0345)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit d1873bd81bfc9f486b88f3a38c65c7de8f5a0909)
metze
(cherry picked from samba commit 5ca0a4bfd6)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 04aeac728f56c65b973762f09977de1b1b99099e)
We need to keep TDB_ALLOW_NESTING as default behavior,
so that existing code continues to work.
However we may change the default together with a major version
number change in future.
metze
(cherry picked from samba commit 3b9f19ed91)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit c1c0ede32dc00ed619d1cf5fda40a9de43995f3a)
metze
(cherry picked from samba commit 85449b7bcc)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 855391c1e37012b0d6c673a304bb8da8a1efcd72)
While studying tdb, I've noticed a couple of mismatches between readme
and actual code:
- tdb_open_ex changed it's log_fn argument to log_ctx
- there is now no tdb_update(), which it seems was transformed into
non-exported tdb_update_hash()
There were other mismatches, but I don't remember them now, sorry.
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 83de5c8263)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 7a88f1df9190674deaf5dcbedad02ae4120a5263)
The reason I do it is that when using older python-tdb as shipped in
Debian Lenny, python interpreter crashes on this test:
(gdb) bt
#0 0xb7f8c424 in __kernel_vsyscall ()
#1 0xb7df5640 in raise () from /lib/i686/cmov/libc.so.6
#2 0xb7df7018 in abort () from /lib/i686/cmov/libc.so.6
#3 0xb7e3234d in __libc_message () from /lib/i686/cmov/libc.so.6
#4 0xb7e38624 in malloc_printerr () from /lib/i686/cmov/libc.so.6
#5 0xb7e3a826 in free () from /lib/i686/cmov/libc.so.6
#6 0xb7b39c84 in tdb_close () from /usr/lib/libtdb.so.1
#7 0xb7b43e14 in ?? () from /var/lib/python-support/python2.5/_tdb.so
#8 0x0a038d08 in ?? ()
#9 0x00000000 in ?? ()
master's pytdb does not (we have a check for self->closed in obj_close()),
but still...
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 71a21393dd)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 03372b4ea8ba2938468a5c0fc234d604966ce070)
So that erroneous double tdb_close() calls do not try to close() same
fd again. This is like SAFE_FREE() but for fd.
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit b4424f8234)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit f5c992bdaeb73ef726ff4728a9922721474cd6f5)
It's Tdb.get(), not Tdb.fetch().
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit cfed5f946d)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 76aacdd8e1106f26565e25903091a757b59cd7e2)
This can help with ldb where we rewrite the index records
(cherry picked from samba commit d4c0e8fdf0)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 470750fa2e3cf987f10de48451b1ee13aab03907)
metze
(cherry picked from samba commit 3b62e250c0)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 03b3682e3fa53c9f5fdf2c4beac8b5d030fd2630)
Also, set logging function so we get more informative messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 0944931159)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 6ac7ef8bf4d384f880c7f483ace70f8e08c15a8b)
ctdb wants a quick way to detect corrupt tdbs; particularly, tdbs with
loops in their hash chains. tdb_check() provides this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 022b4d4aa6)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit df1a3ce0380fa9d8722b2f9b16f65557095e4c83)
We no longer use swig for pytdb, so there is no need for swig make
rules. Also pytdb.c header should be updated.
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit ecbe5ebd8d)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 27611d6a0c313732e438cb24c82b9de126e50156)
This fixes the build on Tru64.
metze
(cherry picked from samba commit 3718cf294a)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 5652e403be099f35cdd29fda8ba4fe2c35de8035)
With the ctdb checkin dde9f3f006 tdb optimized out write lock checks for
write-enabled transaction. Sadly, this also removed the possibility to ever
remove dead records left over from tdb_delete calls within a transaction.
Tridge, please check this! Did dde9f3f006 have any reason beyond performance
optimizations?
Thanks,
Volker
(cherry picked from samba commit 3f884c4ae3)
(This used to be commit 1d85e0647e)
(cherry picked from samba commit 8c88209c6f)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit b02bf7659f04f1fa203834bd75a2392b48e56c16)
Found by cppcheck:
[lib/tdb/tools/tdbtorture.c:326]: (error) Memory leak: pids
(cherry picked from samba commit 497b9e460b)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 5d4cc4b018a538dc3f1d79fe091f3e6e67003daf)
tdbbackup was originally written before we had transactions, and it
attempted to use its own fsync() calls to make it safe. Now that we
have transactions we can do it in a much safer (and faster!) fashion
(cherry picked from samba commit 2e4247782b)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit cd23d36ada9631095ca68663516de0c8d8c3bbed)
This means you can kill it at any time and expect no corruption.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 0fc6800005)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit f7278a277ed91587cae5b5e3660dad7124bdb73f)
It was a regrettable hack which I used to reduce line count in tdb; in fact it caused confusion as can be seen in this patch.
In particular, ecode now needs to be set before TDB_LOG anyway, and having it exposed in
the header is useless (the struct tdb_context isn't defined, so it's doubly useless).
Also, we should never set errno, as io.c was doing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit b77f41d58b)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit a6620f6e74aadc708395b21b42303d1082192fcc)
When TDB_TRACE is defined (in tdb_private.h), verbose tracing of tdb operations is enabled.
This can be replayed using "replay_trace" from http://ccan.ozlabs.org/info/tdb.
The majority of this patch comes from moving internal functions to _<funcname> to
avoid double-tracing. There should be no additional overhead for the normal (!TDB_TRACE)
case.
Note that the verbose traces compress really well with rzip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 703004340c)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit b01b756cb577f32a1ec4597efb00017241e01685)
There was a race condition that caused the torture.tdb to be left in a
state that needed recovery. The torture code thought that any message
from the tdb code was an error, so the "recovered" message, which is a
TDB_DEBUG_TRACE message, marked the run as being an error when it
isn't.
(cherry picked from samba commit 5dcf0069b6)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 43c97b259b19c42b4edc7f83dbfc5e486568b4e3)
Michael
(cherry picked from samba commit e440a2e11e)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit c1b8d32b4ef87b9d8f37b451f47fcee2ea753d21)
This adds 3 simple speed tests to tdbtool, for transaction store,
store and fetch.
On my laptop this shows transactions costing about 10ms
(cherry picked from samba commit e15027155d)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 463279c972fa4538919bdd1dff48ca6b2fb8d49c)
So one can perform tdbtool operations protected by transactions.
Michael
(cherry picked from samba commit 91e1bab2e9)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 35a5b874b925380f7c227e47aebb590c9db4739e)
Michael
(cherry picked from samba commit 817383d88d)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit dc287a7d7420cca0b104049e689a73202bc535f8)
We previously only allowed a commit to happen after a prepare
commit. It is in fact safe to allow reads between a prepare and a
commit, and the s4 replication code can make use of that, so allow it.
(cherry picked from samba commit 46c99ec2a3)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 5ef5ddb8369e5e76173285fe9a08498dc8dc73ab)
Michael
(cherry picked from samba commit 55dcf928eb)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit ef1dc585d869a9e48164cd65bafc92c1da245007)
Michael
(cherry picked from samba commit cfa4e7ec75)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 0ae735b7a2096a40e5e47086ec41d9d45ef6d36b)
Michael
(cherry picked from samba commit 25939a627f)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 2e69647404c87c438ae7c180277ac3b532941efd)
by first concatenating multilint parentheses and removing typefes afterwards.
Michael
(cherry picked from samba commit 13bfcd5a93)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 37225f1ed3f70d7259c2af2c51c671105c34476a)
Michael
(cherry picked from samba commit ecd12bfb38)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 66fffa577e051212ac7541be906b6c80f4a7c0c9)
Michael
(cherry picked from samba commit 400f08450b)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 310d673b7cb9000d76437d78e43bc2bf133e4e14)
Michael
(cherry picked from samba commit 907e05595f)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit f70e371d70e334a7254649b2bb09aa382e6f09bb)
Guenther
(cherry picked from samba commit 1c2f4919ab)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 9d5015e6fc68d3eb9e7b7178dbaf8c129dc79471)
USAGE: abi_checks.sh LIBRARY_NAME header1 [header2 ...]
This creates symbol signature lists using the mksyms and mksigs scripts
and compares them with the checked in lists.
Michael
(cherry picked from samba commit 9636e0d373)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 724d71dc838750fff91a45359feeb6e71bf0a4c7)
This produces output like the output gcc produces when
invoked with the -aux-info switch.
Run like this: cat include/tdb.h | ./script/mksigs.pl
This simple parser is probably too coarse to handle all
possible header files, but it treats tdb.h correctly...
Michael
(cherry picked from samba commit 0760a04ef9)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 141422d9dc24b15b7b8bc7831adab90367a729f7)
Michael
(cherry picked from samba commit 006fd0c43c)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit aed864dceaf6ec1e6e6066a587c708b485901200)
In future, this may happen, and we don't want to clobber them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 398d0c2929)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit eebd467961dad6cfb38c2a5d6e4b4dbf86e55e63)
On systems with 32 bit offsets, expansion and fcntl locking on these records
will fail anyway. SAMBA already does '#define _FILE_OFFSET_BITS 64' in
config.h (on my 32-bit x86 Linux system at least) to get 64 bit file offsets.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 252f7da702)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 2d768f664e6db65b3b7e0c732f33ee2b806892f9)
This might be a bit less efficient, but experience in winbind has shown that
event callbacks can trigger changes in the socket state in very hard to
diagnose ways.
(This used to be ctdb commit a78b8ea7168e5fdb2d62379ad3112008b2748576)
add the __location__ macro to the logs to get a better idea
in which loop the problem occured
Signed-off-by: Christian Ambach <christian.ambach@de.ibm.com>
(This used to be ctdb commit dccb549fd6a6e338063699544e52f2a1a6a966b5)
As we start to use errno more, it's a huge pain if talloc_free() can blatt
it (esp. destructors).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(This used to be ctdb commit 76a0ca77feba14e1e1162c195ffbdf516e62aa4d)
It is unlikely we will need something this verbose for normal troubleshooting.
This allows us to keep a significantly longer time interval of log messages
in the 500k slots available in the ringbuffer.
(This used to be ctdb commit cc99c05c0c6484ad574039a454e6133852cb41fa)
add a global variable holding the pid of the main daemon.
change the tracking of time() in the event loop to only check/warn when called from the main daemon
(This used to be ctdb commit a10fc51f4c30e85ada6d4b7347b0f9a8ebc76637)
log an error if the clock jumps backwards
also log an error if the clock jumps >5 seconds forward (we assume here we will get at least one event every 5 seconds)
(This used to be ctdb commit 11193e1e192bee6f579bdf1303153571a82711d7)
The flags are user-visible, via tdb_get_flags/add_flags/remove_flags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 8f48e37c254e0852d4e2dea54b905ce5ef2b925d)
This version just wraps the reopen code, so we still re-grab the lock and do
the normal sanity checks.
The reason we do this at all is to avoid global fd limits, see:
http://forums.fedoraforum.org/showthread.php?t=210393
Note also that this whole reopen concept is fundamentally racy: if the parent
goes away before the child calls tdb_reopen_all, the database can be left
without an active lock and another TDB_CLEAR_IF_FIRST opener will clear it.
A fork_with_tdbs() wrapper could use a pipe to solve this, but it's hardly
elegant (what if there are other independent things which have similar needs?).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
(This used to be ctdb commit 8d0d432ab7766d9c0f9868fd77e48b9b5cc5d9f9)
Make tdb transaction lock recursive (samba version)
This patch replaces 6ed27edbcd and
1a416ff13c, which fixed the bug where traversals
inside transactions would release the transaction lock early.
This solution is more general, and solves the more minor symptom that nested
traversals would also release the transaction lock early. (It was also suggestd in
Volker's comment in 6ed27ed).
This patch also applies to ctdb, if the traverse.c part is removed (ctdb's tdb
code never received the previous two fixes).
Tested using the testsuite from ccan (adapted to the samba code). Thanks to
Michael Adam for feedback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Michael Adam <obnox@samba.org>
commit 760104188d
Author: Rusty Russell <rusty@rustcorp.com.au>
Date: Tue Jul 21 16:23:35 2009 +0930
tdb: fix locking error
54a51839ea "Make tdb transaction lock
recursive (samba version)" was broken: I "cleaned it up" and prevented
it from ever unlocking.
To see the problem:
$ bin/tdbtorture -s 1248142523
tdb_brlock failed (fd=3) at offset 8 rw_type=1 lck_type=14 len=1
tdb_transaction_lock: failed to get transaction lock
tdb_transaction_start failed: Resource deadlock avoided
My testcase relied on the *count* being correct, which it was. Fixing that
now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Michael Adam <obnox@samba.org>
(This used to be ctdb commit ce19658ba13272238058e9b9bc03e62f48b737c0)
overallocate all records by 25%
This greatly reduces the fragmentation of databases where records
tend to grow slowly by a small amount each time. The case where this
is most seen is the ldb index records. Adding this overallocation
reduced the size of the resulting database by more than 20x when
running a test that adds 10k users.
(This used to be ctdb commit e72974e5cefabc7035399d16633f727f868caa61)
auto-repack in transactions that expand the tdb
The idea behind this is to recover from badly fragmented free
lists. Choosing the point where the file expands is fairly arbitrary,
but seems to work well.
(This used to be ctdb commit 233c52bfb087f636ad61e95c12616c02901f4f83)
imported the tdb_repack() code from CTDB
The tdb_repack() function repacks a TDB so that it has a single
freelist entry. The file doesn't shrink, but it does remove all
freelist fragmentation. This code originated in the CTDB vacuuming
code, but will now be used in ldb to cope with fragmentation from
re-indexing
(This used to be ctdb commit fe3ceb101a5a9c336973c2c6c31406bd8181c2fe)
make TDB_NOSYNC affect all the fsync/msync calls in transactions
During a transaction commit tdb normally uses fsync/msync calls to
make it crash safe. This can be disabled using the TDB_NOSYNC flag,
but it wasn't disabling all the code paths that caused a fsync/msync.
(This used to be ctdb commit e03980add02a28609a7a0a0c87ebc85419b98144)
commit b90863c0b7
Author: Howard Chu <hyc@highlandsun.com>
Date: Tue Mar 31 13:15:54 2009 +1100
Add tdb_transaction_prepare_commit()
Using tdb_transaction_prepare_commit() gives us 2-phase commits. This
allows us to safely commit across multiple tdb databases at once, with
reasonable transaction semantics
Signed-off-by: tridge@samba.org
(This used to be ctdb commit 4c3dac215a088947f645f727343997f5d47e3260)
Make the default be that transaction is not allowed and any attempt to create a nested transaction will fail with TDB_ERR_NESTING.
If an application can cope with transaction nesting and the implicit
semantics of tdb_transaction_commit(), it can enable transaction nesting
by using the TDB_ALLOW_NESTING flag.
(This used to be ctdb commit 3e49e41c21eb8c53084aa8cc7fd3557bdd8eb7b6)
This fixes the autoconf 2.63 warning
"suspicious cache-id, must contain _cv_ to be cached".
Thanks to William Jojo <w.jojo@hvcc.edu> for reporting.
Michael
(This used to be ctdb commit f7be553ed0826d31759812d58db91589ccdf8a8c)
Signed-off-by: Andreas Schneider <anschneider@suse.de>
(cherry picked from commit b39611c36b)
Signed-off-by: Michael Adam <obnox@samba.org>
(This used to be ctdb commit 02893e04154c0590b235082c5a6e5726ceb25dba)
matches commit 7aae6dd80e265aa9402ed507caaff4a5dba55069 in the kernel.
Many thanks to Jim Houston for pointing out this fix to us
(This used to be ctdb commit 85f6032cbdb197f3a003d86f086afa2cee898a4d)
make ctdb uptime print how long the recovery took
in the recovery daemon when we check that the public ip address
allocation on the local node is correct (we have the ips we should have
and we dont have any we shouldnt have) use ctdb uptime and check the
recovery start/stop times and make sure we dont check for ip allocation
inconsistencies during a recovery where the ip address allocation is in flux.
(This used to be ctdb commit f86551580349b7f662f9a07e4eb0c1189e38e429)
for stores into persistent databases, ALWAYS use a lockwait child take out the lock for the record and never the daemon itself.
(This used to be ctdb commit 7fb6cf549de1b5e9ac5a3e4483c7591850ea2464)
in those cases we accept that we couldnt get a transaction and then just writes the data "unsafely" anyway.
(This used to be ctdb commit 56af7df33456bf10d19b97cb005257720ac0b610)
This takes advantage of the fact that we can do left merges but not right merges
By allocating data from the end of the freelist entry rather than the beginning
we can guarantee that if we immediately free the record again it will be merged
with the previous freelist entry, reducing fragmentation
(This used to be ctdb commit 6d159f2c115dd5e9b8bab8798f7ac710b4427593)
ctdb vacuum : vacuums all the databases, deleting any zero length
ctdb records
ctdb repack : repacks all the databases, resulting in a perfectly
packed database with no freelist entries
(This used to be ctdb commit 3532119c84ab3247051ed6ba21ba3243ae2f6bf4)
- make freelist merging more paranoid
- fixed TDB_DATA_STAART() (broken by earlier commit)
(This used to be ctdb commit c3be8eb7d1ca316111ec51f45dbca2fa5609cf93)
