1
0
mirror of https://github.com/samba-team/samba.git synced 2025-02-15 05:57:49 +03:00

265 Commits

Author SHA1 Message Date
Rusty Russell
e1217b7bdb There is one signedness issue in tdb which prevents traverses of TDB records over the 2G offset on systems which support 64 bit file offsets. This fixes that case.
On systems with 32 bit offsets, expansion and fcntl locking on these records
will fail anyway.  SAMBA already does '#define _FILE_OFFSET_BITS 64' in
config.h (on my 32-bit x86 Linux system at least) to get 64 bit file offsets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
(cherry picked from samba commit 252f7da702fd0409f7bfff05ef594911ededa32f)

Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 2d768f664e6db65b3b7e0c732f33ee2b806892f9)
2009-12-16 08:03:42 +01:00
Volker Lendecke
b664a86bc2 Import "talloc_array_length" from upstream talloc
(This used to be ctdb commit 844aa6300ee4d87561e698001ebc15ac1e455528)
2009-12-12 00:45:39 +01:00
Volker Lendecke
a0d9bd3c13 Run only one event for each epoll_wait/select call
This might be a bit less efficient, but experience in winbind has shown that
event callbacks can trigger changes in the socket state in very hard to
diagnose ways.

(This used to be ctdb commit a78b8ea7168e5fdb2d62379ad3112008b2748576)
2009-12-10 07:52:16 +11:00
Christian Ambach
4269d37ce8 improve time jump logging
add the __location__ macro to the logs to get a better idea
in which loop the problem occured

Signed-off-by: Christian Ambach <christian.ambach@de.ibm.com>

(This used to be ctdb commit dccb549fd6a6e338063699544e52f2a1a6a966b5)
2009-12-10 07:31:04 +11:00
Rusty Russell
949803528d talloc: save errno over talloc_free
As we start to use errno more, it's a huge pain if talloc_free() can blatt
it (esp. destructors).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


(This used to be ctdb commit 76a0ca77feba14e1e1162c195ffbdf516e62aa4d)
2009-12-07 23:05:58 +10:30
Ronnie Sahlberg
e28c652cca Dont store debug level DEBUG_DEBUG in the in-memory ringbuffer.
It is unlikely we will need something this verbose for normal troubleshooting.
This allows us to keep a significantly longer time interval of log messages
in the 500k slots available in the ringbuffer.

(This used to be ctdb commit cc99c05c0c6484ad574039a454e6133852cb41fa)
2009-12-04 11:45:37 +11:00
Ronnie Sahlberg
bc2675119d add an in memory ringbuffer where we store the last 500000 log entries regardless of log level.
add commandt to extract this in memory buffer and to clear it

(This used to be ctdb commit 29d2ee8d9c6c6f36b2334480f646d6db209f370e)
2009-11-18 12:44:18 +11:00
Ronnie Sahlberg
4d40b86805 for debugging
add a global variable holding the pid of the main daemon.
change the tracking of time() in the event loop to only check/warn when called from the main daemon

(This used to be ctdb commit a10fc51f4c30e85ada6d4b7347b0f9a8ebc76637)
2009-10-27 13:18:52 +11:00
Ronnie Sahlberg
299b027b8c patch the event loop so we read the current time every iteration.
log an error if the clock jumps backwards
also log an error if the clock jumps >5 seconds forward (we assume here we will get at least one event every 5 seconds)

(This used to be ctdb commit 11193e1e192bee6f579bdf1303153571a82711d7)
2009-10-26 13:20:35 +11:00
Ronnie Sahlberg
029fd6b00f Revert "try to restart statd everytime it fails, not just the first time"
This reverts commit 4f7b39a4871af28df1c4545ec37db179fa47a7da.

(This used to be ctdb commit db7b96304e4725f29b12398b7582e385daed63ed)
2009-09-15 19:33:35 +10:00
Ronnie Sahlberg
59cacded72 try to restart statd everytime it fails, not just the first time
(This used to be ctdb commit 4f7b39a4871af28df1c4545ec37db179fa47a7da)
2009-09-15 13:35:58 +10:00
Mathieu Parent
f095a64137 Update outdated autotools helper files.
This fixes https://bugzilla.samba.org/show_bug.cgi?id=6370
and http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=536256

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 4899a888d4e2c31b4bd7220f3cd8936168942e61)
2009-09-09 09:52:13 +02:00
Michael Adam
ace21b84b7 debug: add debug_add and dump_data functions
Michael

(This used to be ctdb commit 64405bdbebb2ddf0ae980e958ede77df79139000)
2009-08-04 09:46:39 +02:00
Rusty Russell
3cc352f975 tdb: don't alter tdb->flags in tdb_reopen_all()
The flags are user-visible, via tdb_get_flags/add_flags/remove_flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 8f48e37c254e0852d4e2dea54b905ce5ef2b925d)
2009-08-04 16:50:55 +09:30
Rusty Russell
ed2c8e415f tdb: Reimplementation of Metze's "lib/tdb: if we know pwrite and pread are thread/fork safe tdb_reopen_all() should be a noop".
This version just wraps the reopen code, so we still re-grab the lock and do
the normal sanity checks.

The reason we do this at all is to avoid global fd limits, see:
http://forums.fedoraforum.org/showthread.php?t=210393

Note also that this whole reopen concept is fundamentally racy: if the parent
goes away before the child calls tdb_reopen_all, the database can be left
without an active lock and another TDB_CLEAR_IF_FIRST opener will clear it.
A fork_with_tdbs() wrapper could use a pipe to solve this, but it's hardly
elegant (what if there are other independent things which have similar needs?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Stefan Metzmacher <metze@samba.org>

(This used to be ctdb commit 8d0d432ab7766d9c0f9868fd77e48b9b5cc5d9f9)
2009-08-04 16:50:52 +09:30
Rusty Russell
6e4577d08f realloc() has that horrible overloaded free semantic when size is 0: current code does a free of the old record in this case, then fail.
(This used to be ctdb commit 8b6a5bba93843cd83b7b386b82949ad88f29884a)
2009-08-04 16:50:48 +09:30
Rusty Russell
1664d75ad3 If the record is at the end of the database, pretending it has length 1 might take us out-of-bounds. Only pretend to be length 1 for the malloc.
(This used to be ctdb commit 6de2823f5f7976d4efa20761e518d6b67753f054)
2009-08-04 16:50:43 +09:30
Rusty Russell
6aa98fb041 Port from SAMBA tdb: commit 54a51839ea65aa788b18fce8de0ae4f9ba63e4e7 Author: Rusty Russell <rusty@rustcorp.com.au> Date: Sat Jul 18 15:28:58 2009 +0930
Make tdb transaction lock recursive (samba version)

    This patch replaces 6ed27edbcd3ba1893636a8072c8d7a621437daf7 and
    1a416ff13ca7786f2e8d24c66addf00883e9cb12, which fixed the bug where traversals
    inside transactions would release the transaction lock early.

    This solution is more general, and solves the more minor symptom that nested
    traversals would also release the transaction lock early.  (It was also suggestd in
    Volker's comment in 6ed27ed).

    This patch also applies to ctdb, if the traverse.c part is removed (ctdb's tdb
    code never received the previous two fixes).

    Tested using the testsuite from ccan (adapted to the samba code).  Thanks to
    Michael Adam for feedback.

    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Signed-off-by: Michael Adam <obnox@samba.org>
commit 760104188d0d2ed96ec4a70138e6d0bf86d797ed
Author: Rusty Russell <rusty@rustcorp.com.au>
Date:   Tue Jul 21 16:23:35 2009 +0930

    tdb: fix locking error

    54a51839ea65aa788b18fce8de0ae4f9ba63e4e7 "Make tdb transaction lock
    recursive (samba version)" was broken: I "cleaned it up" and prevented
    it from ever unlocking.

    To see the problem:
        $ bin/tdbtorture -s 1248142523
        tdb_brlock failed (fd=3) at offset 8 rw_type=1 lck_type=14 len=1
        tdb_transaction_lock: failed to get transaction lock
        tdb_transaction_start failed: Resource deadlock avoided

    My testcase relied on the *count* being correct, which it was.  Fixing that
    now.

    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit ce19658ba13272238058e9b9bc03e62f48b737c0)
2009-08-04 16:43:34 +09:30
Rusty Russell
825391de1d Port from SAMBA tdb: commit a6cc04a20089e8fbcce138c271961c37ddcd6c34 Author: Andrew Tridgell <tridge@samba.org> Date: Mon Jun 1 13:13:07 2009 +1000
overallocate all records by 25%

    This greatly reduces the fragmentation of databases where records
    tend to grow slowly by a small amount each time. The case where this
    is most seen is the ldb index records. Adding this overallocation
    reduced the size of the resulting database by more than 20x when
    running a test that adds 10k users.

(This used to be ctdb commit e72974e5cefabc7035399d16633f727f868caa61)
2009-08-04 16:43:31 +09:30
Rusty Russell
94ee0cbe3a Port from SAMBA tdb: commit a386173fa1c7c5bcc11ea9260d84b6c52c154b3d Author: Andrew Tridgell <tridge@samba.org> Date: Mon Jun 1 13:11:39 2009 +1000
auto-repack in transactions that expand the tdb

    The idea behind this is to recover from badly fragmented free
    lists. Choosing the point where the file expands is fairly arbitrary,
    but seems to work well.

(This used to be ctdb commit 233c52bfb087f636ad61e95c12616c02901f4f83)
2009-08-04 16:43:26 +09:30
Rusty Russell
3dc37c17c1 Port from SAMBA ctdb: commit 936d76802f98d04d9743b2ca8eeeaadd4362db51 Author: Andrew Tridgell <tridge@samba.org> Date: Tue Dec 16 14:38:17 2008 +1100
imported the tdb_repack() code from CTDB

    The tdb_repack() function repacks a TDB so that it has a single
    freelist entry. The file doesn't shrink, but it does remove all
    freelist fragmentation. This code originated in the CTDB vacuuming
    code, but will now be used in ldb to cope with fragmentation from
    re-indexing

(This used to be ctdb commit fe3ceb101a5a9c336973c2c6c31406bd8181c2fe)
2009-08-04 16:43:21 +09:30
Rusty Russell
d66a7c40da Port from SAMBA tdb: commit 4b4fec65db4e202afa13b2d15867f4d8a54d154e Author: Andrew Tridgell <tridge@samba.org> Date: Thu May 28 16:08:28 2009 +1000
make TDB_NOSYNC affect all the fsync/msync calls in transactions

    During a transaction commit tdb normally uses fsync/msync calls to
    make it crash safe. This can be disabled using the TDB_NOSYNC flag,
    but it wasn't disabling all the code paths that caused a fsync/msync.

(This used to be ctdb commit e03980add02a28609a7a0a0c87ebc85419b98144)
2009-08-04 16:43:17 +09:30
Rusty Russell
f10db37b82 Port from SAMBA tdb: commit a91bcbccf8a2243dac57cacec6fdfc9907580f69 Author: Jim McDonough <jmcd@samba.org> Date: Thu May 21 16:26:26 2009 -0400
Detect tight loop in tdb_find()

(This used to be ctdb commit 5253a0ba3a34fbf5810f363ecc094203d49e835f)
2009-08-04 16:43:14 +09:30
Rusty Russell
c153b85a93 Port from SAMBA tdb: commit 42c0931441ef53a3f977e1334355fa83f05ac184 Author: Tim Prouty <tprouty@samba.org> Date: Tue Mar 31 16:24:07 2009 -0700
tdb: Remove unused variable

(This used to be ctdb commit aa22d1875b1997664af983c0baeabe34e40dd253)
2009-08-04 16:43:10 +09:30
Rusty Russell
166e74a8b1 Port from SAMBA tdb:
commit b90863c0b7b860b006ac49c9396711ff351f777f
Author: Howard Chu <hyc@highlandsun.com>
Date:   Tue Mar 31 13:15:54 2009 +1100

    Add tdb_transaction_prepare_commit()

    Using tdb_transaction_prepare_commit() gives us 2-phase commits. This
    allows us to safely commit across multiple tdb databases at once, with
    reasonable transaction semantics

    Signed-off-by: tridge@samba.org

(This used to be ctdb commit 4c3dac215a088947f645f727343997f5d47e3260)
2009-08-04 16:43:06 +09:30
Ronnie Sahlberg
6feb7a1bf8 New attempt at TDB transaction nesting allow/disallow.
Make the default be that transaction is not allowed and any attempt to create a nested transaction will fail with TDB_ERR_NESTING.

If an application can cope with transaction nesting and the implicit
semantics of tdb_transaction_commit(), it can enable transaction nesting
by using the TDB_ALLOW_NESTING flag.

(This used to be ctdb commit 3e49e41c21eb8c53084aa8cc7fd3557bdd8eb7b6)
2009-05-25 17:04:42 +10:00
Ronnie Sahlberg
c429ca114d Revert "add TDB_NO_NESTING. When this flag is set tdb will not allow any nested transactions and tdb_transaction_start() will implicitely _cancel() any pending transactions before starting any new ones."
This reverts commit 459e4ee135bd1cd24c15e5325906eb4ecfd550ec.

(This used to be ctdb commit f1c6f7dd47bb1081781c0a0d567a92bbbc0aa5d5)
2009-05-25 16:54:25 +10:00
Ronnie Sahlberg
26e1486db7 Whitespace changes and using the CTDB_NO_MEMORY() macro changes to
the previous patch.

(This used to be ctdb commit d623ea7c04daa6349b42d50862843c9f86115488)
2009-05-21 11:49:16 +10:00
Sumit Bose
2fcedf6dac add missing checks on so far ignored return values
Most of these were found during a review by Jim Meyering <meyering@redhat.com>

(This used to be ctdb commit 3aee5ee1deb4a19be3bd3a4ce3abbe09de763344)
2009-05-21 11:22:21 +10:00
Michael Adam
01fb6e326b Remove generated binary files.
Noted by Mathieu Parent <math.parent@gmail.com>

Michael

(This used to be ctdb commit b321dfd1d23492169ac25ed901d49d7c69ad5340)
2009-05-13 16:01:53 +10:00
Ronnie Sahlberg
777c634eae add TDB_NO_NESTING. When this flag is set tdb will not allow any nested transactions and tdb_transaction_start() will implicitely _cancel() any pending transactions before starting any new ones.
(This used to be ctdb commit 459e4ee135bd1cd24c15e5325906eb4ecfd550ec)
2009-04-26 08:38:37 +10:00
Michael Adam
4e12bbbf56 autoconf: Make sure the result of the mkdir_has_mode test gets cached.
This fixes the autoconf 2.63 warning
"suspicious cache-id, must contain _cv_ to be cached".

Thanks to William Jojo <w.jojo@hvcc.edu> for reporting.

Michael

(This used to be ctdb commit f7be553ed0826d31759812d58db91589ccdf8a8c)
2009-01-29 10:24:58 +01:00
Andreas Schneider
4c64dc994e Fix circular dependency error with autoconf 2.6.3.
Signed-off-by: Andreas Schneider <anschneider@suse.de>
(cherry picked from commit b39611c36bb904774fd4032bf2f8003fbdeb5d34)

Signed-off-by: Michael Adam <obnox@samba.org>

(This used to be ctdb commit 02893e04154c0590b235082c5a6e5726ceb25dba)
2009-01-16 20:49:52 +11:00
Andrew Tridgell
9a8d41a992 The author of the upstream code asked for this code to be GPLv2+ not GPLv3
(This used to be ctdb commit 000018f2f4fb9f2452f56731b027dd6a7beda111)
2008-09-30 07:16:17 -07:00
Andrew Tridgell
a8a8c4c1d4 merged a bugfix for the idtree code from the Linux kernel. This
matches commit 7aae6dd80e265aa9402ed507caaff4a5dba55069 in the kernel.

Many thanks to Jim Houston for pointing out this fix to us

(This used to be ctdb commit 85f6032cbdb197f3a003d86f086afa2cee898a4d)
2008-09-30 07:09:06 -07:00
Andrew Tridgell
87e19f1f9d fixed a bug where we would look for a signal past the end of the
signal arrays

This could cause the events code to get into a loop chewing CPU

(This used to be ctdb commit e24152fbd06ba4c2b6cfd473751c7f00a676b9ae)
2008-07-18 13:46:01 +10:00
Ronnie Sahlberg
ef769e7237 track both when we last started and ended a recovery.
make ctdb uptime print how long the recovery took

in the recovery daemon when we check that the public ip address
allocation on the local node is correct (we have the ips we should have
and we dont have any we shouldnt have) use ctdb uptime and check the
recovery start/stop times and make sure we dont check for ip allocation
inconsistencies during a recovery  where the ip address allocation is in flux.

(This used to be ctdb commit f86551580349b7f662f9a07e4eb0c1189e38e429)
2008-07-02 13:55:59 +10:00
Ronnie Sahlberg
ed2cf0291d second try for safe transaction stores into persistend tdb databases
for stores into persistent databases, ALWAYS use a lockwait child take out the lock for the record and never the daemon itself.

(This used to be ctdb commit 7fb6cf549de1b5e9ac5a3e4483c7591850ea2464)
2008-05-22 12:47:33 +10:00
Ronnie Sahlberg
cc5dc9fe31 dont emit the can not start transaction with locks held at all.
it just pollutes the logs

(This used to be ctdb commit c66e44d44f8d1142394ce5c6a68ac09fd67211fe)
2008-05-21 11:23:46 +10:00
Ronnie Sahlberg
26e5f71308 lower the debug level for the "can not start transaction" since we do expect this to happen a few times inside ctdb (since we cant really block and wait for all locks to dissapear before we can write the header, for example when doing a dmaster miration)
in those cases we accept that we couldnt get a transaction and then just writes the data "unsafely" anyway.

(This used to be ctdb commit 56af7df33456bf10d19b97cb005257720ac0b610)
2008-05-21 11:15:14 +10:00
Andrew Tridgell
f6ebcd6a55 carefully step around the recovery area when doing a tdb_wipe_all. This prevents
problems with wipe_all on databases that may need crash recovery

(This used to be ctdb commit e7b1349bf8784c151c2651edd99b3f40ebcece1f)
2008-02-08 14:10:54 +11:00
Andrew Tridgell
fbba202f1a fixed a problem with tdb growing after each recovery
(This used to be ctdb commit d754380961e67271809fed6c44f45356fe7a9c77)
2008-02-07 23:01:06 +11:00
Andrew Tridgell
1c35c8243e fixed a crash bug in the new transaction code
(This used to be ctdb commit 0d298e863c8ab2945fd329661c09ec6889152326)
2008-01-31 09:18:26 +11:00
Andrew Tridgell
ef02267c6e merge from samba4
(This used to be ctdb commit 21245b894b32f232251fd7c0e2fbfd073fb10514)
2008-01-18 15:48:36 +11:00
Andrew Tridgell
ce134656d5 more efficient freelist allocation
This takes advantage of the fact that we can do left merges but not right merges
By allocating data from the end of the freelist entry rather than the beginning
we can guarantee that if we immediately free the record again it will be merged
with the previous freelist entry, reducing fragmentation

(This used to be ctdb commit 6d159f2c115dd5e9b8bab8798f7ac710b4427593)
2008-01-18 13:32:54 +11:00
Andrew Tridgell
b62b7fcde8 added syslog support, and use a pipe to catch logging from child processes to the ctdbd logging functions
(This used to be ctdb commit 1306b04cd01e996fd1aa1159a9521f2ff7b06165)
2008-01-16 22:03:01 +11:00
Andrew Tridgell
840ba4d5f6 minor fix to transaction_write_existing
(This used to be ctdb commit aa4e7578f78449c518730811220666e5e34adc06)
2008-01-15 15:28:07 +11:00
Andrew Tridgell
ddbe31e5aa fixed the bug that caused tdbtorture to fail
It was an error in the new transaction code

(This used to be ctdb commit 27f0dfdfb93d92859de3cbbd3874cfb38c13a169)
2008-01-14 15:11:10 +11:00
Andrew Tridgell
35e8f2ff6c tdb_freelist_size was reporting 1 more than correct size
(This used to be ctdb commit 46214c9234c574b45bce0b88bfb3af437415a69e)
2008-01-10 09:42:44 +11:00
Andrew Tridgell
bc85af6b61 a compromise for freelist scanning - we now will look for other than the first fit, but get exponentially more desperate as we get deeper into the freelist
(This used to be ctdb commit f3319ef84c47dc8bf0bfb4ef1c72cee58ed9d88c)
2008-01-09 15:33:10 +11:00