IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
(ldb releases have been made while this patch set was in train)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We must hold locks not just for the duration of each search, but for the whole search
as our module stack may make multiple search requests to build up the whole result.
This is explains a number of replication and read corruption issues in Samba
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
This allows Samba to provide a consistent view of the DB
despite the use of multiple databases via the partitions module
and over multiple callbacks via a module stack.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This will be used to implement read locking in ldb_tdb
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Currently, a lock is not held against modifications once the final
record is returned via a callback, so modifications can be made
during the DONE callback. This makes it hard to write modules
that interpert an ldb search result and do further processing
so will change in the future to allow the full search to be
atomic.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
A modify or rename during a search must not cause a search to change
output, and attributes having an index should in particular not see
any change in behaviour in this respect
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This is required to pass the test in future, because
otherwise the clean up will fail because we hold locks.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
If we do not do this, then we never take the all record lock, and instead do a lock
for every record as we go, which is very slow during a large search
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
* allow tdb_traverse_read before tdb_transaction[_prepare]_commit()
* Improve documentation for tdb_transaction_start()
* Add new function tdb_transaction_active()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will allow callers to avoid their own reference counting of transactions.
Additionally, this will always line up with the acutal transaction state, even
in the error cases where tdb can cancel the transaction
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
It now references the TDB_ALLOW_NESTING and TDB_DISALLOW_NESTING flags
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This restores the original intent of tdb_traverse_read() in
7dd31288a701d772e45b1960ac4ce4cc1be782ed
This is needed to avoid a deadlock with tdb_lockall() and the
transaction start, as ldb_tdb should take the allrecord lock during a
search (which calls tdb_traverse), and can otherwise deadlock against
a transaction starting in another process
We add a test to show that a transaction can now start while a read
traverse is in progress
This allows more operations to happen in parallel. The blocking point
is moved to the prepare commit.
This in turn permits a roughly doubling of unindexed search
performance, because currently ldb_tdb omits to take the lock due to
an unrelated bug, but taking the allrecord lock triggers the
above-mentioned deadlock.
This behaviour was added in 251aaafe3a9213118ac3a92def9ab2104c40d12a for
Solaris 10 in 2005. But the run-fcntl-deadlock test works also on Solaris 10,
see https://lists.samba.org/archive/samba-technical/2017-April/119876.html.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We need to migrate all pathname based VFS calls to use a struct
to finish modernising the VFS with extra timestamp and flags parameters.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Richard Sharpe <realrichardsharpe@gmail.com>
Using gcc 4.3.2:
cc1: error: unrecognized command line option "-Wno-format-truncation"
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
* Fix mutex locking in tevent_threaded_context_destructor().
* Fix a memleak on FreeBSD.
* Re-init threading in tevent_re_initialise().
* Include the finish location in tevent_req_default_print().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jun 22 17:17:33 CEST 2017 on sn-devel-144
It's verify useful when debugging code without a debugger to
be able to use tevent_req_print() in DEBUG statements.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This has stopped working ages ago. The idea is clear, but if someone
wants to revive it, I think it needs a completely fresh start.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The race is easily reproduced by adding a poll(NULL,0,10) in between the two
pthread_mutex_unlock calls in _tevent_threaded_schedule_immediate.
Before 1828011317b, the main thread was signalled only after the helper
had already unlocked event_ctx_mutex.
Full explaination follows:
-----------------------------------------------------------------
Inside _tevent_threaded_schedule_immediate() we have:
476 ret = pthread_mutex_unlock(&ev->scheduled_mutex);
477 if (ret != 0) {
478 abort();
479 }
HERE!!!!
481 ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
482 if (ret != 0) {
483 abort();
484 }
At the HERE!!! point, what happens is tevent_common_threaded_activate_immediate(),
which is blocked on ev->scheduled_mutex, get released and does:
514 while (ev->scheduled_immediates != NULL) {
515 struct tevent_immediate *im = ev->scheduled_immediates;
516 DLIST_REMOVE(ev->scheduled_immediates, im);
517 DLIST_ADD_END(ev->immediate_events, im);
518 }
- making an immediate event ready to be scheduled.
This then returns into epoll_event_loop_once(), which then calls:
910 if (ev->immediate_events &&
911 tevent_common_loop_immediate(ev)) {
912 return 0;
913 }
which causes the immediate event to fire. This immediate
event is the pthread job terminate event, which was previously
set up in pthreadpool_tevent_job_signal() by:
198 if (state->tctx != NULL) {
199 /* with HAVE_PTHREAD */
200 tevent_threaded_schedule_immediate(state->tctx, state->im,
201 pthreadpool_tevent_job_done,
202 state);
So we now call pthreadpool_tevent_job_done() - which does:
225 TALLOC_FREE(state->tctx);
calling tevent_threaded_context_destructor():
384 ret = pthread_mutex_destroy(&tctx->event_ctx_mutex); <---------------- BOOM returns an error !
385 if (ret != 0) {
386 abort();
387 }
as we haven't gotten to line 481 above (the line after
HERE!!!!) so the tctx->event_ctx_mutex is still
locked when we try to destroy it.
So doing an additional:
ret = pthread_mutex_lock(&tctx->event_ctx_mutex);
ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
(error checking elided) forces tevent_threaded_context_destructor()
to wait until tctx->event_ctx_mutex is unlocked before it locks/unlocks
and then is guaranteed safe to destroy.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This makes the behaviour much more robust, particularly with forest child
domains over one-way forest trusts.
Sadly we don't support this kind of setup with our current ADDC, so
there's no way to have automated tests for this behaviour, but
at least we know it doesn't break any existing tests.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=8630
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
repl_meta_data knows whether linked attributes are appropriately
[un-]duplicated, and this is how it tells ldb_tdb that.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Add efficient function to find duplicate values in ldb messages
(this makes large multi-valued attributes in ldb_tdb more efficient)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
ldb backends need to make sure they are not adding duplicate values to
multi-valued attributes in ADD and MODIFY operations. Until now they
have done this inefficiently using nested loops. Here we add common
functions that deal with large numbers of values in O(n log n) time,
but continue to use the simple methods for small numbers of values.
These functions take a struct ldb_context pointer and an options flag
arguments, although the ldb is not used, and only one bit of the
options has meaning. This is to allow further patches to switch on
schema-aware comparisons.
This entails an ABI jump to add the two new functions.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This helps make some future commits less confusing
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This verifies the F_RDLCK => F_WRLCK upgrade logic in the kernel
for conflicting locks.
This is a standalone test to check the traverse_read vs.
allrecord_lock/prepare_commit interaction.
This is based on the example from
https://lists.samba.org/archive/samba-technical/2017-April/119861.html
from Douglas Bagnall <douglas.bagnall@catalyst.net.nz> and Volker Lendecke <vl@samba.org>.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We do not want the cache or any of the values in it, we want to read the real DB
@INDEX: records.
This matters if a re-index is tiggered in the same transaction
as the modify of the values in the index. Otherwise we won't see
the old index record (it will not show up in the tdb_traverse)
and so fail to remove it.
That in turn can cause a spurious unqiue index violation.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Provide a XSI-compliant strerror_r on GNU based systems.
The default GNU strerror_r is not XSI-compliant, this patch wraps the
GNU-specific call in an XSI-compliant wrapper.
This reverts 18ed32ce0821d11c0c06d82c07ba1c27b0c2b886 which tried to
make Heimdal use roken, rather than libreplace for strerror_r.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Adds macros for preprocessor compares and replaces an incomptatible
compare with one of the new macros.
This fixes a comptability bug on AIX.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11621
Signed-off-by: Guillaume Xavier Taillon <gtaillon@ca.ibm.com>
Reviewed-by: Björn Jacke <bjacke@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Björn Jacke <bj@sernet.de>
Autobuild-Date(master): Tue Jun 13 09:11:56 CEST 2017 on sn-devel-144
We protect setting tctx->event_ctx=NULL with tctx->event_ctx_mutex.
But in _tevent_threaded_schedule_immediate we have the classic
TOCTOU race: After we checked "ev==NULL", looking at
tevent_common_context_destructor the event context can go after
_tevent_threaded_schedule_immediate checked. We need to serialize
things a bit by keeping tctx->event_ctx_mutex locked while we
reference "ev", in particular in the
DLIST_ADD_END(ev->scheduled_immediates,im);
I think the locking hierarchy is still maintained, tevent_atfork_prepare()
first locks all the tctx locks, and then the scheduled_mutex. Also,
I don't think this will impact parallelism too badly: event_ctx_mutex
is only used to protect setting tctx->ev.
Found by staring at code while fixing the FreeBSD memleak due to
not destroying scheduled_mutex.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 9 00:45:26 CEST 2017 on sn-devel-144
FreeBSD has malloc'ed memory attached to mutexes. We need to clean this up.
valgrind really helped here
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
