IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
glibc's pthread_cond_wait(&c, &m) increments m.__data.__nusers, making
pthread_mutex_destroy return EBUSY. Thus we can't allow any thread waiting for
a job across a fork. Also, the state of the condvar itself is unclear across a
fork. Right now to me it looks like an initialized but unused condvar can be
used in the child. Busy worker threads don't cause any trouble here, they don't
hold mutexes or condvars. Also, they can't reach the condvar because _prepare
holds all mutexes.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13006
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit ff98e3fb66)
When copying large files from the server to the client with aio enabled
we noticed that smbd kept growing RSS and VSZ.
valgrind was reporting:
==2503== 4,093,440 bytes in 6,560 blocks are possibly lost in loss record 460 of 460
==2503== at 0x4C299CE: calloc (vg_replace_malloc.c:711)
==2503== by 0x4011C24: _dl_allocate_tls (in /usr/lib64/ld-2.17.so)
==2503== by 0x4E3C960: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so)
==2503== by 0x9B298AE: pthreadpool_add_job (in /usr/lib64/samba/libmessages-dgm-samba4.so)
==2503== by 0x9B29FDC: pthreadpool_tevent_job_send (in /usr/lib64/samba/libmessages-dgm-samba4.so)
==2503== by 0x56A78EF: ??? (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503== by 0x55D86B7: smb_vfs_call_pread_send (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503== by 0x55F7543: schedule_smb2_aio_read (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503== by 0x5608F57: smbd_smb2_request_process_read (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503== by 0x55FCB6C: smbd_smb2_request_dispatch (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503== by 0x55FD7DC: ??? (in /usr/lib64/samba/libsmbd-base-samba4.so)
==2503== by 0x641B977: ??? (in /usr/lib64/samba/libtevent.so.0.9.31)
The problem seems to be caused by worked threads that are not properly
started in detached state and thus their tls is not reclaimed upon
thread termination.
In pthreadpool.c we prepare a pthread attribute with
PTHREAD_CREATE_DETACHED, but we don't pass it to pthread_create().
Bug: https://bugzilla.samba.org/show_bug.cgi?id=12624
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Mar 10 22:06:02 CET 2017 on sn-devel-144
(cherry picked from commit c9a7a065bb)
Autobuild-User(v4-6-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-6-test): Mon Mar 13 13:30:49 CET 2017 on sn-devel-144
Yes, there is one.... I've seen two flaky builds on sn-devel with
pthreadpool after the coverity checks went in. They were in the
ret = pthread_mutex_unlock(&pool->mutex);
assert(ret == 0);
in pthreadpool_parent() and pthreadpool_child(). No idea what that was,
I could not really reproduce that. A build attempt on FreeBSD also gave
an erratic error, this time it was an EINVAL in
ret = pthread_mutex_lock(&pool->mutex);
assert(ret == 0);
pthreadpool_parent(). EINVAL means that the mutex is not a proper
mutex. What happened: Someone (a detached thread) does the
pthreadpool_free behind our back, while we are in pthreadpool_parent,
preparing the fork. Unfortunately the mutex was already destroyed before
we came to lock it.
The fix is simple: Remove the obsolete struct pthreadpool from the
linked list before the mutex is destroyed.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Coverity ID 1373624 says we have a deadlock between pthreadpool_prepare and
pthreadpool_destroy. Coverity somehow misses that pthreadpool_free unlocks
pool->mutex, so I think this is a false positive. Nevertheless this re-arranges
the code a bit for more clarity, hoping that Coverity now can better track the
locks and unlocks. Also, the human reader might have to jump between routines a
bit less.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
So far we used joinable threads. This prevented pthreadpool_destroy with
blocked threads. This patch converts pthreadpool to detached threads. Now
pthreadpool_destroy does not have to wait for the idle threads to finish, it
can immediately return. pthreadpool_destroy will tell all threads to exit, and
the last active thread will actually free(pthreadpool).
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This essentially reverts 1c4284c739. We now call an alien function from
within pthreadpool, and we should not hold a mutex during that call. The alien
function could (and pthreadpool_tevent_job_signal actually does) lock a mutex.
We can't guarantee proper lock ordering here, so in theory we could deadlock. I
haven't seen it in the wild yet, but I could imagine that both _parent pieces
in pthreadpool and tevent could trigger such a deadlock.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Aug 30 04:06:20 CEST 2016 on sn-devel-144
The pthreadpool_tevent wrapper will need this
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Some systems required an explicit <signal.h>, which comes
via "system/wait.h"
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Aug 16 19:09:55 CEST 2016 on sn-devel-144
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 12 04:26:09 CEST 2016 on sn-devel-144
Using the new msg_source program with 1.500 instances against a single
msg_sink I found the msg_source process to spawn two worker threads for
synchronously sending the data towards the receiving socket. This should
not happen: Per destination node we only create one queue. We strictly
only add pthreadpool jobs one after the other, so a single helper thread
should be perfectly sufficient.
It turned out that under heavy overload the main sending thread was
scheduled before the thread that just had finished its send() job. So
the helper thread was not able to increment the pool->num_idle variable
indicating that we don't have to create a new thread when the new job
is added.
This patch moves the signalling write under the mutex. This means that
indicating readiness via the pipe and the pool->num_idle variable happen both
under the same mutex lock and thus are atomic. No superfluous threads anymore.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This can avoid syscalls when multiple jobs are finished simultaneously
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
pthreadpool_add_job is in our hottest code path for r/w intensive workloads, so
we should avoid anything CPU-intensive. pthreadpool used to malloc each job and
free it in the worker thread. This patch adds a FIFO queue for jobs that helper
threads copy from, avoiding constant malloc/free. This cuts user space
CPU in the local-bench-pthreadpool benchmark by roughly 10% on my system.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The current could would crash if a pthreadpool was created, deleted and the
process then fork()s. "pthreadpools" is NULL in this case, but the
pthread_atfork handlers are in place. This fixes walking the pthreadpools list
in reverse.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kamen Mazdrashki <kamenim@samba.org>
Reviewed-by: Simo Sorce <simo@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Tue Jan 28 19:06:40 CET 2014 on sn-devel-104
on success, errno on fail and return the jobid in a separate variable.
I need this fix for my vfs_aio_pthread.c module.
Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Thu Dec 22 12:12:33 CET 2011 on sn-devel-104
The -framework CoreFoundation is required by the charset_macosxfs module
The system/time.h header is required to access the replacement clock_gettime()
Andrew Bartlett
Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Fri Sep 23 10:58:02 CEST 2011 on sn-devel-104
