1
0
mirror of https://github.com/samba-team/samba.git synced 2025-11-22 16:23:49 +03:00
Commit Graph

58 Commits

Author SHA1 Message Date
Andrew Tridgell
a37d9434d1 r13283: added two optimisations to the tdb transactions code. The first is to
more agressively coalesce entries in the linked list of the undo
log. The second is to ensure that writes during a transaction into the
hash table don't cause the size of the undo log linked list to grow.

These optimisations don't affect Samba much, but they make a huge
difference to the use of ldb in kde
2007-10-10 13:51:42 -05:00
Andrew Tridgell
b2934732dd r10891: I noticed that the secrets.db was not being backed up on my system due
to msync/mmap not changing the mtime of the file. This patch ensures
that for successfully completed transactions we update the mtime.

I don't do this on all tdb writes as its too expensive, but doing it
just on transactions is bearable, as those cost quite a lot anyway.
2007-10-10 13:39:40 -05:00
Andrew Tridgell
f02a92787b r10524: SAFE_FREE() in tdb does not need the discard_const_p()
the discard_const_p() was causing problems on openbsd where intptr_t is not defined
2007-10-10 13:39:01 -05:00
Andrew Tridgell
54203aacd1 r10522: finally got the locking working on solaris10. This adds a read lock on
the transaction lock in tdb_traverse_read(). This prevents a pattern
of locks which triggers the deadlock detection code in solaris10. I
suspect solaris10 is trying to prevent lock starvation by granting
locks in the order they were requested, which makes it much easier to
produce deadlocks.
2007-10-10 13:39:00 -05:00
Andrew Tridgell
11739ad31d r10496: - added configure test for sys/time.h
- print length of failed locks
2007-10-10 13:38:56 -05:00
Andrew Tridgell
a11762e7bc r10495: older redhat boxes need sys/time.h for select() 2007-10-10 13:38:56 -05:00
Andrew Tridgell
1b03c4e6c7 r10494: - don't generate a tdb log message for any type of failed lock probe
- if the lock upgrade loop ever fails then log a warning
2007-10-10 13:38:55 -05:00
Andrew Tridgell
144cc3da5e r10493: we need sys/select.h to enable select() in the solaris workaround 2007-10-10 13:38:55 -05:00
Andrew Tridgell
980dd17f7d r10492: work around a bug in solaris which cases lock upgrades to fail with
EDEADLK even when progress can be made. This is not a good solution,
but I can't find anything better.
2007-10-10 13:38:55 -05:00
Andrew Tridgell
196995ea38 r10484: try to fix the pread/pwrite declaration problems 2007-10-10 13:38:53 -05:00
Andrew Tridgell
315653cf1e r10483: fixed some uninitialised variables warnings 2007-10-10 13:38:53 -05:00
Andrew Tridgell
ab953c8c72 r10471: stratos doesn't have getpagesize(), so guess 8k on systems that don't
have it. Overestimating is harmless.
2007-10-10 13:38:51 -05:00
Andrew Tridgell
fc692dc61f r10469: use the older style of structure initialisation for tdb to make it
more portable. tdb is used in more than just Samba4, and I think the
portability/readability balance is a bit different
2007-10-10 13:38:50 -05:00
Andrew Tridgell
c2b1739c63 r10468: - terminate tdbtorture quickly when an error is detected
- more workarounds for aix not handling malloc of size 0
2007-10-10 13:38:50 -05:00
Andrew Tridgell
0177c6ca3e r10467: aix doesn't like zero length malloc :( 2007-10-10 13:38:50 -05:00
Andrew Tridgell
e615f36733 r10466: work around missing pread/pwrite declaration on openbsd 2007-10-10 13:38:49 -05:00
Andrew Tridgell
294ccfd46a r10465: separate out a read_only db from a read-only traversal to ensure we
don't end up doing a mmap read only
2007-10-10 13:38:49 -05:00
Andrew Tridgell
83168c7e76 r10461: fixed tdb build on systems without stdint.h 2007-10-10 13:38:48 -05:00
Andrew Tridgell
bce35ad237 r10460: fixed portability of transaction code to systems with integer
alignment constraints (like sparc)
2007-10-10 13:38:48 -05:00
Andrew Tridgell
03942dd54b r10459: fixed some portability problems 2007-10-10 13:38:48 -05:00
Andrew Tridgell
9b53e04377 r10424: for caller convenience, automatically turn a tdb_traverse() into a
tdb_traverse_read() for read only databases
2007-10-10 13:38:44 -05:00
Andrew Tridgell
7dd31288a7 r10421: following on discussions with simo, I have worked out a way of
allowing searches to proceed while another process is in a
transaction, then only upgrading the transaction lock to a write lock
on commit.

The solution is:

 - split tdb_traverse() into two calls, called tdb_traverse() and
   tdb_traverse_read(). The _read() version only gets read locks, and
   will fail any write operations made in the callback from the
   traverse.

 - the normal tdb_traverse() call allows for read or write operations
   in the callback, but gets the transaction lock, preventing
   transastions from starting inside the traverse

In addition we enforce the following rule that you may not start a
transaction within a traverse callback, although you can start a
traverse within a transaction

With these rules in place I believe all the deadlock possibilities are
removed, and we can now allow for searches to happen in parallel with
transactions
2007-10-10 13:38:44 -05:00
Andrew Tridgell
06bd8abba9 r10405: added transactions into tdb, and hook them into ldb. See my
samba-technical posting for more details on the transactions design.

This also adds a number of command line arguments to tdbtorture,
making it more flexible, and fixes some lock deadlock conditions in
the tdbtorture code.
2007-10-10 13:38:41 -05:00
Tim Potter
360be7028c r10258: Fix an unused/duplicate local variable. 2007-10-10 13:38:12 -05:00
Andrew Tridgell
4673cdd0d2 r10253: a fairly large tdb cleanup and re-organise. Nearly all of this change
just involves splitting up the core tdb.c code into separate files on
logical boundaries, but there are some minor functional changes as well:

 - move the 'struct tdb_context' into tdb_private.h, hiding it from
   users. This was done to allow the structure to change without
   breaking code that uses tdb.

 - added accessor functions tdb_fd(), tdb_name(), and tdb_log_fn() to
   access the elements of struct tdb_context that were used by
   external code but are no longer visible

 - simplied tdb_append() to use tdb_fetch()/tdb_store(), which is just
   as good due to the way tdb locks work

 - changed some of the types (such as tdb_off to tdb_off_t) to make
   syntax highlighting work better

 - removed the old optional spinlock code. It was a bad idea.

 - fixed a bug in tdb_reopen_all() that caused tdbtorture to sometimes
   fail or report nasty looking errors. This is the only real bug
   fixed in this commit. Jeremy/Jerry, you might like to pickup this
   change for Samba3, as that could definately affect smbd in
   Samba3.

The aim of all of these changes is to make the tdb
transactions/journaling code I am working on easier to write. I
started to write it on top of the existing tdb.c code and it got very
messy. Splitting up the code makes it much easier to follow.

There are more cleanups we could do in tdb, such as using uint32_t
instead of u32 (suggested by metze). I'll leave those for another day.
2007-10-10 13:38:12 -05:00
Andrew Tridgell
3c32f24e2c r9774: r11605@blu: tridge | 2005-08-30 12:02:19 +1000
make sure we don't walk off the end of the hash array
2007-10-10 13:34:59 -05:00
Andrew Tridgell
2e14fb893d r9773: r11599@blu: tridge | 2005-08-30 11:55:57 +1000
optimise this case a bit more. The total speedup using non-indexed
 ldbtest is now around a factor of 80x. The code is ugly as hell, but
 I think this speed is worth it.

 Of course, if we only ever do indexed searches in ldb then this
 doesn't help, but it seems all too common that we get unindexable
 searches, so the optimisation is worthwhile
2007-10-10 13:34:59 -05:00
Andrew Tridgell
7f9efaceb6 r9769: r11592@blu: tridge | 2005-08-30 10:40:19 +1000
added a tdb optimisation that speeds up non-indexed ldb by a large
 margin (often 10x or more). I'd be interested in any comments on the
 safety of this optimisation. See the comment in the code for an
 explanation.
2007-10-10 13:34:58 -05:00
Andrew Tridgell
8f222c8c7a r8624: removed valgrind comment on tdb that no longer applies 2007-10-10 13:29:45 -05:00
Jeremy Allison
f9f3037d68 r7641: Fix based on work from "Shlomi Yaakobovich" <Shlomi@exanet.com> to catch
loops in corrupted tdb files.
Jeremy.
2007-10-10 13:18:16 -05:00
Derrell Lipman
a80bced0b9 r7276: - moved static tdb function ltdb_dn_fold() into common/ so that it can be
called from multiple backends.  (ldb_sqlite3 needs it too.)  Added parameter
  for a callback function that determines whether an attribute needs case
  folding.
- begin to prepare for sqlite3 in build process
- work-in-progress updates, on ldb_sqlite3
2007-10-10 13:17:35 -05:00
Andrew Tridgell
162bbe4402 r6528: - in tdb_fetch() we effectively disallowed zero length records by
returning NULL/0, which is the same as we used for a failure. Having
  to look at tdb->ecode (which we never do) is too error prone.

  Instead, tdb_fetch() should behave like malloc() and talloc(), where
  zero length is not special and malloc(0) returns a valid pointer.

- similarly in data_blob(), asking for data_blob(NULL, 0) should
  return a zero blob, but asking for data_blob(ptr, 0) should return a
  zero length blob with a valid pointer, just like talloc() and malloc()

This change fixes the SummaryInformation stream stored in the tdb
backend when manipulated from w2k. The w2k client was using
SET_EOF_INFORMATION to create a zero-length stream, which we return
STATUS_NOT_FOUND on, as the tdb_fetch() gave us back a NULL/0 blob,
which we returned as not-found
2007-10-10 13:16:23 -05:00
Jeremy Allison
d69f7c0546 r6238: Ensure if realloc fails on an internal
tdb we fail gracefully.
Jeremy.
2007-10-10 13:11:28 -05:00
Tim Potter
1ee84e373e r5780: Remove unused arguments. 2007-10-10 13:11:03 -05:00
Tim Potter
dbb56e9a59 r5779: Remove signal and timeout gubbage from tdb. 2007-10-10 13:11:03 -05:00
Jeremy Allison
84f6d2b3dd r5533: Patch to detect infinite loops when traversing a tdb from "Shlomi Yaakobovich" <Shlomi@exanet.com>
Jeremy.
2007-10-10 13:10:53 -05:00
Andrew Tridgell
9db6c79e90 r5298: - got rid of pstring.h from includes.h. This at least makes it a bit
less likely that anyone will use pstring for new code

 - got rid of winbind_client.h from includes.h. This one triggered a
   huge change, as winbind_client.h was including system/filesys.h and
   defining the old uint32 and uint16 types, as well as its own
   pstring and fstring.
2007-10-10 13:09:38 -05:00
Andrew Tridgell
6f79432fe6 r5296: - only include the tdb headers where they are needed
- removed the u32 hack in events.c as I think this was only needed as
  tdb.h defines u32. Metze, can you check that this hack is indeed no
  longer needed on your suse system?
2007-10-10 13:09:38 -05:00
Andrew Tridgell
09e98c8745 r4466: rather than defining "STANDALONE" for building tdb, ldb and talloc
outside the tree, instead defined _SAMBA_BUILD_ inside the Samba
build. This makes it easier to pull code out of Samba for external
use.
2007-10-10 13:07:54 -05:00
Andrew Tridgell
a954e707ef r3480: moved some signal defines into include/system 2007-10-10 13:05:20 -05:00
Andrew Tridgell
b97e395c81 r3463: separated out some more headers (asn_1.h, messages.h, dlinklist.h and ioctl.h) 2007-10-10 13:05:17 -05:00
Andrew Tridgell
10188869ef r3454: moved a few more things out if includes.h into the include/system/ include files.
this brings us down to about 11k lines of headers included with
includes.h, while still retaining the speed of building with pch
2007-10-10 13:05:13 -05:00
Andrew Tridgell
264ce91810 r3447: more include/system/XXX.h include files 2007-10-10 13:05:12 -05:00
Andrew Tridgell
0b53ab85aa r2985: got rid of the unused tdb_lockkeys() and tdb_unlockkeys() functions
they have been broken for 4 years (ever since they were added) and
have been never used, which makes them prime candidates for
destruction.
2007-10-10 12:59:54 -05:00
Jeremy Allison
a7d92301bb r2981: Fix incorrect locks/unlocks in tdb_lockkeys()/tdb_unlockkeys().
Spotted by Taj Khattra <taj.khattra@gmail.com>.
Jeremy.
2007-10-10 12:59:54 -05:00
Andrew Tridgell
56ecda2178 r2634: use discard_const_p() in a few places 2007-10-10 12:59:14 -05:00
Andrew Tridgell
c26c92eb8f r2238: the tdb_debug() function was totally bogus - remove it (you can't
convert a ... varargs function to a va_list by just a cast!!)

also mark the tdb log function with PRINTF_ATTRIBUTE() and fixed some
bad format errors in tdb.c that jim found.
2007-10-10 12:58:34 -05:00
Tim Potter
17a53eaac2 r2154: Merge of Jim's format string fixup in r2130:2131 2007-10-10 12:58:27 -05:00
Jeremy Allison
b43320ee5c r2034: Allow user-selectable hash functions on open only.
Jeremy.
2007-10-10 12:58:18 -05:00
Tim Potter
f690325565 r1053: Make tdb build standalone:
- #include <stdint.h>

  - uint_t isn't a valid type, change back to unsigned int
2007-10-10 12:56:34 -05:00