1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-24 21:34:56 +03:00
Commit Graph

24 Commits

Author SHA1 Message Date
Volker Lendecke
1c4284c739 pthreadpool: Slightly serialize jobs
Using the new msg_source program with 1.500 instances against a single
msg_sink I found the msg_source process to spawn two worker threads for
synchronously sending the data towards the receiving socket. This should
not happen: Per destination node we only create one queue. We strictly
only add pthreadpool jobs one after the other, so a single helper thread
should be perfectly sufficient.

It turned out that under heavy overload the main sending thread was
scheduled before the thread that just had finished its send() job. So
the helper thread was not able to increment the pool->num_idle variable
indicating that we don't have to create a new thread when the new job
is added.

This patch moves the signalling write under the mutex. This means that
indicating readiness via the pipe and the pool->num_idle variable happen both
under the same mutex lock and thus are atomic. No superfluous threads anymore.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-08-23 00:24:18 +02:00
Volker Lendecke
c5d07df6ab pthreadpool: Allow multiple jobs to be received
This can avoid syscalls when multiple jobs are finished simultaneously

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-27 06:06:11 +01:00
Volker Lendecke
84aa2ddd86 pthreadpool: Avoid a malloc/free per job
pthreadpool_add_job is in our hottest code path for r/w intensive workloads, so
we should avoid anything CPU-intensive. pthreadpool used to malloc each job and
free it in the worker thread. This patch adds a FIFO queue for jobs that helper
threads copy from, avoiding constant malloc/free. This cuts user space
CPU in the local-bench-pthreadpool benchmark by roughly 10% on my system.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-27 06:06:11 +01:00
Volker Lendecke
7e12bfc6a8 pthreadpool: Add test for fork crash
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kamen Mazdrashki <kamenim@samba.org>
Reviewed-by: Simo Sorce <simo@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-03 14:31:33 +01:00
Volker Lendecke
ccc187ff5e pthreadpool: Fix pthreadpools with fork
The current could would crash if a pthreadpool was created, deleted and the
process then fork()s. "pthreadpools" is NULL in this case, but the
pthread_atfork handlers are in place. This fixes walking the pthreadpools list
in reverse.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Kamen Mazdrashki <kamenim@samba.org>
Reviewed-by: Simo Sorce <simo@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-03 14:29:24 +01:00
Volker Lendecke
58865d96f1 pthreadpool: Fix a comment, "quit"->"shutdown"
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Tue Jan 28 19:06:40 CET 2014 on sn-devel-104
2014-01-28 19:06:40 +01:00
Volker Lendecke
7027c6aca9 pthreadpool: Fix CID 710828 Sizeof not portable
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ira Cooper <ira@samba.org>
2013-05-12 15:56:32 +02:00
Andrew Bartlett
f22e15d9d5 build: Do not install testing binaries
These binaries are for developer or selftest use, and are not
supported for installation onto the system.  The autoconf build does
not install these binaries, and so neither should the waf build.

Andrew Bartlett

Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Nov 22 12:00:36 CET 2012 on sn-devel-104
2012-11-22 12:00:36 +01:00
Volker Lendecke
572701f9c8 s3: Fix compilation of pthreadpool_sync.c on platforms without pthread.h
Signed-off-by: Bjoern Jacke <bj@sernet.de>

Autobuild-User(master): Björn Jacke <bj@sernet.de>
Autobuild-Date(master): Thu Jun 28 16:43:13 CEST 2012 on sn-devel-104
2012-06-28 16:43:13 +02:00
Volker Lendecke
8b35a4de38 s3: Add a dummy implementation for the pthreadpool API
Signed-off-by: Jeremy Allison <jra@samba.org>
2012-06-19 10:27:24 -07:00
Volker Lendecke
8a907c9c65 s3: Fix the pthreadpool build on OS/X
OS/X does not have clock_gettime, and without replace.h we do not
get the replacement macro
2012-04-09 18:05:02 +02:00
Ira Cooper
e64b1188bc s3: Update waf build to include missed dependancy on Lion.
Autobuild-User: Ira Cooper <ira@samba.org>
Autobuild-Date: Thu Mar 15 07:34:43 CET 2012 on sn-devel-104
2012-03-15 07:34:43 +01:00
Volker Lendecke
957ec28139 s3: Fix a typo
Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Thu Jan 19 13:43:07 CET 2012 on sn-devel-104
2012-01-19 13:43:07 +01:00
Jeremy Allison
711c18c230 Change the signature of pthreadpool_finished_job() to return 0
on success, errno on fail and return the jobid in a separate variable.

I need this fix for my vfs_aio_pthread.c module.

Autobuild-User: Jeremy Allison <jra@samba.org>
Autobuild-Date: Thu Dec 22 12:12:33 CET 2011 on sn-devel-104
2011-12-22 12:12:33 +01:00
Andreas Schneider
7b5fb7d9e8 replace: Add don't include unistd.h directly and add uid_wrapper. 2011-10-27 13:32:02 +02:00
Andrew Bartlett
2a789c8442 build: Fix waf build on MacOS X
The -framework CoreFoundation is required by the charset_macosxfs module

The system/time.h header is required to access the replacement clock_gettime()

Andrew Bartlett

Autobuild-User: Andrew Bartlett <abartlet@samba.org>
Autobuild-Date: Fri Sep 23 10:58:02 CEST 2011 on sn-devel-104
2011-09-23 10:58:02 +02:00
Andreas Schneider
8b02b78044 s3-waf: Fix pthreadpool build which needs librt now.
Autobuild-User: Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date: Mon Jun  6 23:31:09 CEST 2011 on sn-devel-104
2011-06-06 23:31:09 +02:00
Björn Jacke
4778d35c34 s3/pthreadpool: replace bad portable gettimeofday by clock_gettime
Signed-off-by: Stefan Metzmacher <metze@samba.org>
2011-06-06 11:48:10 +02:00
Volker Lendecke
a8a6433fec s3: Properly clean up in pthreadpool_init in case of failure
Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Wed Apr 27 23:57:19 CEST 2011 on sn-devel-104
2011-04-27 23:57:19 +02:00
Günther Deschner
66b26195d2 s3-waf: add pthreadpool support.
Volker, please check.

Guenther

Autobuild-User: Günther Deschner <gd@samba.org>
Autobuild-Date: Wed Apr 27 17:57:32 CEST 2011 on sn-devel-104
2011-04-27 17:57:32 +02:00
Volker Lendecke
dbc36befb5 s3: Allow unlimited parallelism in pthreadpool 2011-04-26 12:41:56 +02:00
Volker Lendecke
f4a0f856f3 s3: pthreadpool_sig_fd->pthreadpool_signal_fd 2011-04-25 09:50:32 +02:00
Volker Lendecke
3c405f5e1d s3: Tiny doc for pthreadpool 2011-04-25 09:50:32 +02:00
Volker Lendecke
62689d8166 s3: Many pthreadpool fixes
In particular, this makes it fork-safe
2011-04-25 09:50:32 +02:00