IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This allows to specify wrapper tevent_contexts, which adds the ability
to run functions before and after the event handler functions.
This can be used to implement impersonation hooks
or advanced debugging/profiling hooks.
We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
We'll undo the 0.9.36 ABI change on the 0.9.37 release
at the end of this patchset.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This will be active while the event is part of the ev->scheduled_immediates
list.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
We don't need to store it. I prefer this as it shows that we must always
get wakeup_fd from the event context at time of use, rather than possibly
storing an out-of-date variable.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Nov 17 12:43:01 CET 2017 on sn-devel-144
We can't rely on tctx to exist after we unlocked the mutex. It took a
while, but this does lead to data corruption. If *tctx is replaced with
something where tctx->wakeup_fd points to a real, existing file
descriptor, we're screwed. And by screwed, this means file corruption
on disk.
Again. I am not tall enough for this business.
http://bholley.net/blog/2015/must-be-this-tall-to-write-multi-threaded-code.html
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13130
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Nov 11 03:20:09 CET 2017 on sn-devel-144
The race is easily reproduced by adding a poll(NULL,0,10) in between the two
pthread_mutex_unlock calls in _tevent_threaded_schedule_immediate.
Before 1828011317, the main thread was signalled only after the helper
had already unlocked event_ctx_mutex.
Full explaination follows:
-----------------------------------------------------------------
Inside _tevent_threaded_schedule_immediate() we have:
476 ret = pthread_mutex_unlock(&ev->scheduled_mutex);
477 if (ret != 0) {
478 abort();
479 }
HERE!!!!
481 ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
482 if (ret != 0) {
483 abort();
484 }
At the HERE!!! point, what happens is tevent_common_threaded_activate_immediate(),
which is blocked on ev->scheduled_mutex, get released and does:
514 while (ev->scheduled_immediates != NULL) {
515 struct tevent_immediate *im = ev->scheduled_immediates;
516 DLIST_REMOVE(ev->scheduled_immediates, im);
517 DLIST_ADD_END(ev->immediate_events, im);
518 }
- making an immediate event ready to be scheduled.
This then returns into epoll_event_loop_once(), which then calls:
910 if (ev->immediate_events &&
911 tevent_common_loop_immediate(ev)) {
912 return 0;
913 }
which causes the immediate event to fire. This immediate
event is the pthread job terminate event, which was previously
set up in pthreadpool_tevent_job_signal() by:
198 if (state->tctx != NULL) {
199 /* with HAVE_PTHREAD */
200 tevent_threaded_schedule_immediate(state->tctx, state->im,
201 pthreadpool_tevent_job_done,
202 state);
So we now call pthreadpool_tevent_job_done() - which does:
225 TALLOC_FREE(state->tctx);
calling tevent_threaded_context_destructor():
384 ret = pthread_mutex_destroy(&tctx->event_ctx_mutex); <---------------- BOOM returns an error !
385 if (ret != 0) {
386 abort();
387 }
as we haven't gotten to line 481 above (the line after
HERE!!!!) so the tctx->event_ctx_mutex is still
locked when we try to destroy it.
So doing an additional:
ret = pthread_mutex_lock(&tctx->event_ctx_mutex);
ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
(error checking elided) forces tevent_threaded_context_destructor()
to wait until tctx->event_ctx_mutex is unlocked before it locks/unlocks
and then is guaranteed safe to destroy.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We protect setting tctx->event_ctx=NULL with tctx->event_ctx_mutex.
But in _tevent_threaded_schedule_immediate we have the classic
TOCTOU race: After we checked "ev==NULL", looking at
tevent_common_context_destructor the event context can go after
_tevent_threaded_schedule_immediate checked. We need to serialize
things a bit by keeping tctx->event_ctx_mutex locked while we
reference "ev", in particular in the
DLIST_ADD_END(ev->scheduled_immediates,im);
I think the locking hierarchy is still maintained, tevent_atfork_prepare()
first locks all the tctx locks, and then the scheduled_mutex. Also,
I don't think this will impact parallelism too badly: event_ctx_mutex
is only used to protect setting tctx->ev.
Found by staring at code while fixing the FreeBSD memleak due to
not destroying scheduled_mutex.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 9 00:45:26 CEST 2017 on sn-devel-144
I did not find a way to do this safely without a mutex per threaded_context.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This is infrastructure to improve our async r/w result handling and latency.
The pthreadpool signalling goes through a pipe. This has downsides: The main
event loop has to go through a read on the pipe before it can ship the result.
Also, it is not guaranteed by poll/epoll that the pthreadpool signal pipe is
handled with top priority. When an async pread/pwrite has finished, we should
immediately ship the result to the client, not waiting for anything else.
This patch enables tevent_immediate structs as job signalling. This means a
busy main tevent loop will handle the threaded job completion before any timed
or file descriptor events. Opposite to Jeremy's tevent_thread_proxy this is
done by a modification of the main event loop by looking at a linked list under
a central mutex.
Regarding performance: In a later commit I've created a test that does nothing
but fire one immediate over and over again. If you add a phread_mutex_lock and
unlock pair in the immediate handler, you lose roughly 25% of rounds per
second, so it is measurable. It is questionable that will be measurable in the
real world, but to counter concerns activation of immediates needs to go
through a new struct tevent_threaded_context. Only if such a
tevent_threaded_context exists for a tevent context, the main loop takes the
hit to look at the mutex'ed list of finished jobs.
This patch by design does not care about talloc hierarchies. The idea is that
the main thread owning the tevent context creates a chunk of memory and
prepares the tevent_immediate indication job completion. The main thread hands
the memory chunk together with the immediate as a job description over to a
helper thread. The helper thread does its job and upon completion calls
tevent_threaded_schedule_immediate with the already-prepared immediate. From
that point on memory ownership is again transferred to the main thread.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
some compilers don't tolerate void-casting for warn_unused_result
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Christian Ambach <ambi@samba.org>
