IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We don't need to store it. I prefer this as it shows that we must always
get wakeup_fd from the event context at time of use, rather than possibly
storing an out-of-date variable.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Nov 17 12:43:01 CET 2017 on sn-devel-144
* Remove unused select backend
* Fix a race condition in tevent_threaded_schedule_immediate()
(bug #13130)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Mon Nov 13 18:02:46 CET 2017 on sn-devel-144
We can't rely on tctx to exist after we unlocked the mutex. It took a
while, but this does lead to data corruption. If *tctx is replaced with
something where tctx->wakeup_fd points to a real, existing file
descriptor, we're screwed. And by screwed, this means file corruption
on disk.
Again. I am not tall enough for this business.
http://bholley.net/blog/2015/must-be-this-tall-to-write-multi-threaded-code.html
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13130
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sat Nov 11 03:20:09 CET 2017 on sn-devel-144
select() is no longer useful on modern systems.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Sep 16 08:35:39 CEST 2017 on sn-devel-144
* make tevent_req_print() more robust against crashes
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
We have the same information available under req->internal.private_type.
This way it's possible to call tevent_req_print() after
tevent_req_received() was called.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
* Fix mutex locking in tevent_threaded_context_destructor().
* Fix a memleak on FreeBSD.
* Re-init threading in tevent_re_initialise().
* Include the finish location in tevent_req_default_print().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jun 22 17:17:33 CEST 2017 on sn-devel-144
It's verify useful when debugging code without a debugger to
be able to use tevent_req_print() in DEBUG statements.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
The race is easily reproduced by adding a poll(NULL,0,10) in between the two
pthread_mutex_unlock calls in _tevent_threaded_schedule_immediate.
Before 1828011317, the main thread was signalled only after the helper
had already unlocked event_ctx_mutex.
Full explaination follows:
-----------------------------------------------------------------
Inside _tevent_threaded_schedule_immediate() we have:
476 ret = pthread_mutex_unlock(&ev->scheduled_mutex);
477 if (ret != 0) {
478 abort();
479 }
HERE!!!!
481 ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
482 if (ret != 0) {
483 abort();
484 }
At the HERE!!! point, what happens is tevent_common_threaded_activate_immediate(),
which is blocked on ev->scheduled_mutex, get released and does:
514 while (ev->scheduled_immediates != NULL) {
515 struct tevent_immediate *im = ev->scheduled_immediates;
516 DLIST_REMOVE(ev->scheduled_immediates, im);
517 DLIST_ADD_END(ev->immediate_events, im);
518 }
- making an immediate event ready to be scheduled.
This then returns into epoll_event_loop_once(), which then calls:
910 if (ev->immediate_events &&
911 tevent_common_loop_immediate(ev)) {
912 return 0;
913 }
which causes the immediate event to fire. This immediate
event is the pthread job terminate event, which was previously
set up in pthreadpool_tevent_job_signal() by:
198 if (state->tctx != NULL) {
199 /* with HAVE_PTHREAD */
200 tevent_threaded_schedule_immediate(state->tctx, state->im,
201 pthreadpool_tevent_job_done,
202 state);
So we now call pthreadpool_tevent_job_done() - which does:
225 TALLOC_FREE(state->tctx);
calling tevent_threaded_context_destructor():
384 ret = pthread_mutex_destroy(&tctx->event_ctx_mutex); <---------------- BOOM returns an error !
385 if (ret != 0) {
386 abort();
387 }
as we haven't gotten to line 481 above (the line after
HERE!!!!) so the tctx->event_ctx_mutex is still
locked when we try to destroy it.
So doing an additional:
ret = pthread_mutex_lock(&tctx->event_ctx_mutex);
ret = pthread_mutex_unlock(&tctx->event_ctx_mutex);
(error checking elided) forces tevent_threaded_context_destructor()
to wait until tctx->event_ctx_mutex is unlocked before it locks/unlocks
and then is guaranteed safe to destroy.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We protect setting tctx->event_ctx=NULL with tctx->event_ctx_mutex.
But in _tevent_threaded_schedule_immediate we have the classic
TOCTOU race: After we checked "ev==NULL", looking at
tevent_common_context_destructor the event context can go after
_tevent_threaded_schedule_immediate checked. We need to serialize
things a bit by keeping tctx->event_ctx_mutex locked while we
reference "ev", in particular in the
DLIST_ADD_END(ev->scheduled_immediates,im);
I think the locking hierarchy is still maintained, tevent_atfork_prepare()
first locks all the tctx locks, and then the scheduled_mutex. Also,
I don't think this will impact parallelism too badly: event_ctx_mutex
is only used to protect setting tctx->ev.
Found by staring at code while fixing the FreeBSD memleak due to
not destroying scheduled_mutex.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 9 00:45:26 CEST 2017 on sn-devel-144
FreeBSD has malloc'ed memory attached to mutexes. We need to clean this up.
valgrind really helped here
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Drop the configure option for --disable-python as it is now
global in wafsamba.
If samba is set to use a system copy of tevent, and tevent wasn't built
with python support, then the system pytevent will not be found. If
samba is being built without python support then pytevent is not needed,
so do not bother to try and find it.
Signed-off-by: Ian Stakenvicius <axs@gentoo.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
These files should not be executable.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Jan 11 20:21:01 CET 2017 on sn-devel-144
The tevent.py is not a executable python script.
And rpmlint consider it as an error if module file
contians shebang
python2-tevent.x86_64: E: non-executable-script
/usr/lib64/python2.7/site-packages/tevent.py 644 /usr/bin/python
python3-tevent.x86_64: E: non-executable-script
/usr/lib64/python3.5/site-packages/tevent.py 644 /usr/bin/python
Signed-off-by: Lukas Slebodnik <lslebodn@redhat.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
* tevent_update_timer() and tevent_req_reset_endtime() have been added
* documentation updates
* it is now safe to talloc_free() a tevent_threaded_context,
all running threads keep running until they're finished,
but we no longer abort().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Oct 5 15:32:35 CEST 2016 on sn-devel-144
We might decide at some point that we don't want a request to
time out
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
I did not find a way to do this safely without a mutex per threaded_context.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This prepares tevent run-down with active threads.
It has the advantage to not depend on talloc'ed structs. It is needed to make
talloc_free(tevent_context) safe when tevent_threaded_contexts are still
around.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This makes the reading end of the signalling pipe special: If we have eventfd,
this is the same as the write fd. Without eventfd, it will have to be a
separate fd. This moves the requirement to #ifdef from the writing end to the
reading end. Why? We'll use the writing end somewhere else too soon, and this
patch avoids an #ifdef in that new place.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This will be a quicker way to time out sending sockets in messaging_dgm. Right
now cleanup of out-sockets is a bit coarse. The ideal would be to kill a socket
after being idle n seconds. This would mean to free and re-install a timer on
every packet. tevent_update_timer will be quite a bit cheaper.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Sep 6 23:16:34 CEST 2016 on sn-devel-144
* add tevent_threaded_context_create() and tevent_threaded_schedule_immediate()
They add a way to pass the thread result from a helper thread into
the main event loop.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
According to the manpage, eventfd is cheaper than a pipe. At least, we can save
a file descriptor and space for it in struct tevent_context :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Purely cosmetic change: This moves closing the signal/thread event pipe
to where it's opened. This prepares the eventfd support, making the
"magic" for eventfd more obvious.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
No functionality change. This just looks better in objdump --disassemble :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This is infrastructure to improve our async r/w result handling and latency.
The pthreadpool signalling goes through a pipe. This has downsides: The main
event loop has to go through a read on the pipe before it can ship the result.
Also, it is not guaranteed by poll/epoll that the pthreadpool signal pipe is
handled with top priority. When an async pread/pwrite has finished, we should
immediately ship the result to the client, not waiting for anything else.
This patch enables tevent_immediate structs as job signalling. This means a
busy main tevent loop will handle the threaded job completion before any timed
or file descriptor events. Opposite to Jeremy's tevent_thread_proxy this is
done by a modification of the main event loop by looking at a linked list under
a central mutex.
Regarding performance: In a later commit I've created a test that does nothing
but fire one immediate over and over again. If you add a phread_mutex_lock and
unlock pair in the immediate handler, you lose roughly 25% of rounds per
second, so it is measurable. It is questionable that will be measurable in the
real world, but to counter concerns activation of immediates needs to go
through a new struct tevent_threaded_context. Only if such a
tevent_threaded_context exists for a tevent context, the main loop takes the
hit to look at the mutex'ed list of finished jobs.
This patch by design does not care about talloc hierarchies. The idea is that
the main thread owning the tevent context creates a chunk of memory and
prepares the tevent_immediate indication job completion. The main thread hands
the memory chunk together with the immediate as a job description over to a
helper thread. The helper thread does its job and upon completion calls
tevent_threaded_schedule_immediate with the already-prepared immediate. From
that point on memory ownership is again transferred to the main thread.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signalling the main event loop will also happen from threads soon, and
that will use the same mechanism. This also keeps the pipe open after the last
signal handler is removed. Threaded jobs will come and go very frequently, and
always setting up and tearing down the pipe for each job will be expensive.
Also, this is "just" two file descriptors, and with eventfd just one.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This adds 40 bytes, but they are needed for correctness :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Jul 22 23:33:57 CEST 2016 on sn-devel-144
This is one of or hottest code paths, I think every bit counts here.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This is one of or hottest code paths, I think every bit counts here.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This reverts commit 2991f77099.
Breaks compile for older (<= 4.4) gccs.
Needs to be done differently.
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
We expect these macros to generate tautological compares
intentionally, so disabling the warning is just fine.
This lets --picky-developer work with gcc6 and newer.
Pair-Programmed-With: Ira Cooper <ira@samba.org>
Signed-off-by: Ira Cooper <ira@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
