1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

33 Commits

Author SHA1 Message Date
Andreas Schneider
8c32a475e4 lib:pthreadpool: Fix code spelling
Best reviewed with: `git show --word-diff`.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2023-04-11 09:06:36 +00:00
Ralph Boehme
8e03cbe868 Revert "pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state"
This reverts commit 245d684d28.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
44900b0433 Revert "pthreadpool: add pthreadpool_tevent_job_cancel()"
This reverts commit 791c05144e.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
9e4bc1f4a9 Revert "pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs"
This reverts commit 25756425aa.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
991ca9b564 Revert "pthreadpool: add some lockless coordination between the main and job threads"
This reverts commit 9656b8d8ee.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
d818e51174 Revert "pthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()"
This reverts commit 9b73fda926.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
7882941b73 Revert "pthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue"
This reverts commit aa9b64eccf.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
42e2ab7e99 Revert "pthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_destructor()"
This reverts commit f23cac39b3.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:14 +01:00
Ralph Boehme
622ba5191d Revert "pthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()"
This reverts commit 12a45ee1a6.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:13 +01:00
Ralph Boehme
9b7d225799 Revert "pthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure"
This reverts commit f9745d8b52.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:13 +01:00
Ralph Boehme
96332ed100 Revert "pthreadpool: we need to use pthreadpool_tevent_per_thread_cwd() on the callers pool"
This reverts commit ff863f2d98.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:12 +01:00
Ralph Boehme
a75f5de15d Revert "pthreadpool: ignore the return value of poll(NULL, 0UL, 1)"
This reverts commit 6da0d68f49.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:12 +01:00
Ralph Boehme
57c7aaa36d Revert "pthreadpool: reset monitor_fd after calling tevent_fd_set_auto_close()"
This reverts commit f68b5ee75f.

See the discussion in

https://lists.samba.org/archive/samba-technical/2018-December/131731.html

for the reasoning behind this revert.

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2019-01-11 23:11:12 +01:00
Stefan Metzmacher
f68b5ee75f pthreadpool: reset monitor_fd after calling tevent_fd_set_auto_close()
This tries to convince Coverity that we don't have a resource leak:

CID 1438157:    (RESOURCE_LEAK)
Handle variable "monitor_fd" going out of scope leaks the handle.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-07-31 11:26:16 +02:00
Stefan Metzmacher
6da0d68f49 pthreadpool: ignore the return value of poll(NULL, 0UL, 1)
Otherwise Coverity reports this:

CID 1438160:    (CHECKED_RETURN)
Calling "poll(NULL, 0UL, 1)" without checking return value. This
library function may fail and return an error code.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-07-31 11:26:16 +02:00
Stefan Metzmacher
ff863f2d98 pthreadpool: we need to use pthreadpool_tevent_per_thread_cwd() on the callers pool
In pthreadpool_tevent_job_send() we remember if the job will be chdir
safe. It means we means we need to ask the callers pool when calling
pthreadpool_tevent_per_thread_cwd(), as the callers pool might
be a wrapper using pthreadpool_tevent_force_per_thread_cwd().

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-27 13:07:14 +02:00
Ralph Boehme
3fd1a41f68 pthreadpool: add a missing include
Reported-by: David Disseldorp <ddiss@samba.org>
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2018-07-25 17:49:05 +02:00
Stefan Metzmacher
f9745d8b52 pthreadpool: implement pthreadpool_tevent_wrapper_create() infrastructure
This can be used implement a generic per thread impersonation
for thread pools.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:28 +02:00
Stefan Metzmacher
12a45ee1a6 pthreadpool: add pthreadpool_tevent_[current_job_]per_thread_cwd()
This can be used to check if worker threads run with
unshare(CLONE_FS).

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:27 +02:00
Stefan Metzmacher
f23cac39b3 pthreadpool: add a comment about a further optimization in pthreadpool_tevent_job_destructor()
This seems to be a really rare race, it's likely that the immediate
event will still trigger and cleanup.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:27 +02:00
Stefan Metzmacher
aa9b64eccf pthreadpool: maintain a list of job_states on each pthreadpool_tevent_glue
We should avoid traversing a linked list within a thread without holding
a mutex!

Using a mutex would be very tricky as we'll likely deadlock with
the mutexes at the raw pthreadpool layer.

So we use somekind of spinlock using atomic_thread_fence in order to
protect the access to job->state->glue->{tctx,ev} in
pthreadpool_tevent_job_signal().

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:27 +02:00
Stefan Metzmacher
9b73fda926 pthreadpool: add helgrind magic to PTHREAD_TEVENT_JOB_THREAD_FENCE_*()
This avoids the expected helgrind/drd warnings on the job states which
are protected by the thread fence.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:27 +02:00
Stefan Metzmacher
9656b8d8ee pthreadpool: add some lockless coordination between the main and job threads
In the direction from the main process to the job thread, we have:

- 'maycancel', which is set when tevent_req_cancel() is called,
- 'orphaned' is the job request, tevent_context or pthreadpool_tevent
  was talloc_free'ed.

The job function can consume these by using:

   /*
    * return true - if tevent_req_cancel() was called.
    */
   bool pthreadpool_tevent_current_job_canceled(void);

   /*
    * return true - if talloc_free() was called on the job request,
    * tevent_context or pthreadpool_tevent.
    */
   bool pthreadpool_tevent_current_job_orphaned(void);

   /*
    * return true if canceled and orphaned are both false.
    */
   bool pthreadpool_tevent_current_job_continue(void);

In the other direction we remember the following points
in the job execution:

- 'started'  - set when the job is picked up by a worker thread
- 'executed' - set once the job function returned.
- 'finished' - set when pthreadpool_tevent_job_signal() is entered
- 'dropped'  - set when pthreadpool_tevent_job_signal() leaves with orphaned
- 'signaled' - set when pthreadpool_tevent_job_signal() leaves normal

There're only one side writing each element,
either the main process or the job thread.

This means we can do the coordination with a full memory
barrier using atomic_thread_fence(memory_order_seq_cst).
lib/replace provides fallbacks if C11 stdatomic.h is not available.

A real pthreadpool requires pthread and atomic_thread_fence() (or an
replacement) to be available, otherwise we only have pthreadpool_sync.c.
But this should not make a real difference, as at least
__sync_synchronize() is availabe since 2005 in gcc.
We also require __thread which is available since 2002.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-24 17:38:26 +02:00
Stefan Metzmacher
3eee52b44d pthreadpool: allocate glue->tctx on glue as memory context.
This means it will go aways together with glue and thte event context.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Jul 12 17:18:01 CEST 2018 on sn-devel-144
2018-07-12 17:18:00 +02:00
Stefan Metzmacher
25756425aa pthreadpool: maintain a global list of orphaned pthreadpool_tevent_jobs
Instead of leaking the memory forever, we retry the cleanup,
if other pthreadpool_tevent_*() functions are used.

pthreadpool_tevent_cleanup_orphaned_jobs() could also be called
by external callers.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
fa070d9007 pthreadpool: make use of pthreadpool_stop() in pthreadpool_tevent_destructor()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
791c05144e pthreadpool: add pthreadpool_tevent_job_cancel()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
245d684d28 pthreadpool: split out pthreadpool_tevent_job from pthreadpool_tevent_job_state
This makes it much easier to handle orphaned jobs,
we either wait for the immediate tevent to trigger
or we just keep leaking the memory.

The next commits will improve this further.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
cdbad9041b pthreadpool: let pthreadpool_tevent_job_send() fail with an invalid pool
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:19 +02:00
Stefan Metzmacher
e4dfd3da94 pthreadpool: add pthreadpool_tevent_max_threads() and pthreadpool_tevent_queued_jobs()
These can be used to implement some kind of flow control in the caller.
E.g. unless pthreadpool_tevent_queued_jobs() is lower than
pthreadpool_tevent_max_threads() is good to prepare new jobs.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Stefan Metzmacher
19e4a08577 pthreadpool: correctly handle pthreadpool_tevent_register_ev() failures
It returns errno values instead of setting 'errno'.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-07-12 14:25:18 +02:00
Ralph Boehme
3b16bfe483 pthreadpool: create a tevent_threaded_context per registered event context
We just need one tevent_threaded_context per unique combintation of
tevent event contexts and pthreadpool_tevent pools, not multiple copies
for identical combinations of a tevent contexts and a pthreadpool_tevent
pools.

With this commit we register tevent contexts in a list in the
pthreadpool_tevent structure and will only have one
tevent_threaded_context object per tevent context per pool.

With many pthreadpool_tevent_job_send reqs this pays off, I've seen a
small decrease in cpu-ticks with valgrind callgrind and a modified
local.messaging.ping-speed torture test. The test modification ensured
messages we never directly send, but always submitted via
pthreadpool_tevent_job_send.

Pair-Programmed-With: Jeremy Allison <jra@samba.org>

Signed-off-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Nov 17 02:35:52 CET 2017 on sn-devel-144
2017-11-17 02:35:52 +01:00
Matthieu Patou
dd25d75b96 Move pthreadpool to top of the tree.
Signed-off-by: Matthieu Patou <mat@matws.net>
Reviewed-by: Jeremy Allison <jra@samba.org>
2017-02-09 20:04:12 +01:00