1
0
mirror of https://github.com/samba-team/samba.git synced 2025-03-01 04:58:35 +03:00

6511 Commits

Author SHA1 Message Date
Volker Lendecke
8e71945691 messaging3: Fix 80-char line limit
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-08 09:10:12 +02:00
Volker Lendecke
8d65512e69 dbwrap: Use messaging_filtered_read
This does not really save any code lines, but IMHO the code is simpler
this way. Also, in case we have lots of watchers this will be slightly
cheaper, because we don't have to re-establish a tevent_req.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-08 09:10:12 +02:00
Volker Lendecke
7a266c575a messaging3: Add messaging_filtered_read
This delegates the decision whether to read a message to a callback

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-08 09:10:12 +02:00
Garming Sam
98dbc38efa move str_list_make_v3 out of s3 code
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-07 19:49:16 +02:00
Garming Sam
8d4821168f param: use a single command line set_option in the code
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-05-07 19:49:16 +02:00
Volker Lendecke
dcb3f2150e messaging: Nobody uses G_LOCK_RETRY anymore
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue May  6 23:01:22 CEST 2014 on sn-devel-104
2014-05-06 23:01:22 +02:00
Volker Lendecke
2995b93440 poll_funcs: Fix a comment typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): David Disseldorp <ddiss@samba.org>
Autobuild-Date(master): Sun May  4 22:19:53 CEST 2014 on sn-devel-104
2014-05-04 22:19:53 +02:00
Volker Lendecke
d0590eec7e messaging3: Fix messaging_read_send/recv
messaging_read_send/recv was okay for just one handler in the queue. For
multiple handlers it was pretty broken.

A handler that deletes itself as part of the callback (pretty typical use
case...) drops the message for a subsequent handler that responds to the same
message type. In messaging_dispatch_rec we walk the array, however
messaging_read_cleanup has already changed the array. tevent_req_defer_callback
does not help here: It only defers the callback, it does not defer the cleanup
function.

This also happens when a callback deletes a different handler

A handler that re-installs itself in the callback might get a message twice.

This patch changes the code such that only messaging_dispatch_rec adds records
to msg_ctx->waiters, new waiters are put into a staging area first
(msg_ctx->new_waiters). Also messaging_read_cleanup does not move anything
around in msg_ctx->waiters, it only nulls out itself. messaging_dispatch_rec is
changed to cope with this.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-30 14:52:08 +02:00
Volker Lendecke
8117dcd5fb messaging3: Make "presult" optional in messaging_read_recv
Callers might not be interested in the rec, just the fact that something
arrived

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-30 14:52:08 +02:00
Volker Lendecke
93796a38af messaging3: Fix formatting
This went over the 80-char limit

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-30 14:52:08 +02:00
Volker Lendecke
843f094628 s3: messaging: Add infrastructure to clean up orphaned sockets every 15 minutes as a background task.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:09 +02:00
Volker Lendecke
895d0ac56d smbd: Call the msg_ctx destructor for background jobs
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:09 +02:00
Volker Lendecke
6874e4a0f2 messaging_dgm: Add messaging_dgm_wipe
This walks all sockets and wipes the left-overs

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:09 +02:00
Volker Lendecke
cdc99c145d messaging3: Add messaging_send_iov
This uses a copy, will be replaced by a direct iovec call through to
sendmsg on the unix domain socket

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
e2683605b0 lib: Add iov_buf
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
7009b4921c lib: Introduce iov_buflen
.. with overflow protection

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
b022038cdc lib: Remove messages_local
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
29603d1cd9 lib: Add messaging_dgm
Messaging based on unix domain datagram sockets

This makes every process participating in messaging bind on a unix domain
datagram socket, similar to the source4 based messaging. The details are a bit
different though:

Retry after EWOULDBLOCK is done with a blocking thread, not by polling. This
was the only way I could in experiments avoid a thundering herd or high load
under Linux in extreme overload situations like many thousands of processes
sending to one blocked process. If there are better ideas to do this in a
simple way, I'm more than happy to remove the pthreadpool dependency again.

There is only one socket per process, not per task. I don't think that per-task
sockets are really necessary, we can do filtering in user space. The message
contains the destination server_id, which contains the destination task_id. I
think we can rebase the source4 based imessaging on top of this, allowing
multiple imessaging contexts on top of one messaging_context. I had planned to
do this conversion before this goes in, but Jeremy convinced me that this has
value in itself :-)

Per socket we also create a fcntl-based lockfile to allow race-free cleanup of
orphaned sockets. This lockfile contains the unique_id, which in the future
will make the server_id.tdb obsolete.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
3e24e07467 lib: Move full_path_tos to util_str.c
This can be useful elsewhere

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
6dcf2c7eab lib: Add unix_msg
This is a messaging layer based on unix domain datagram sockets.

Sending to an idle socket is just one single nonblocking sendmsg call. If the
recv queue is full, we start a background thread to do a blocking call. The
source4 based imessaging uses a polling fallback. In a situation where
thousands of senders beat one single blocked socket, this will generate load on
the system due to the constant polling. This does not happen with a threaded
blocking send call.

The threaded approach has another advantage: We save become_root() calls on the
retries. The access checks are done when the blocking socket is connected, the
threaded blocking send call does not check permissions anymore.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Volker Lendecke
bafdecdf1f lib: Add poll_funcs
This is an abstraction for a tevent loop. It will be used in low-level
messaging with the goal to make low-leve our low-level messaging routines
usable also for other projects which are not based on tevent.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-23 22:33:08 +02:00
Andreas Schneider
15feb84273 s3-lib: Add root_mode() which can deal with uid_wrapper.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-04-17 14:56:06 +02:00
Björn Baumbach
fae7e5d771 lib-util: rename memdup to smb_memdup and fix all callers
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-04-16 20:39:08 +02:00
Jelmer Vernooij
9de0dc792d Typo: Specifing -> specifying.
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Change-Id: I80286441b32cd0b8384610b5b6db7790b8a22d48
Reviewed-on: https://gerrit.samba.org/221
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-13 16:55:05 +02:00
Stefan Metzmacher
f71780202a s3:lib: use stack buffers in drain_socket() and default_sys_recvfile()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-04-11 21:41:07 +02:00
Jeremy Allison
4f59580331 s3-lib/util: fix logic inside set_namearray loops.
Additional fix for bug #10544 - s3-lib/util: set_namearray reads across end of namelist string.

Not strictly needed as the initial fix addresses
the problem, but corrects the internal logic
inside the loops.

https://bugzilla.samba.org/show_bug.cgi?id=10544

Signed-off-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
2014-04-09 01:29:07 +02:00
Björn Baumbach
8f46b130c5 s3-lib/util: fix read across end of namelist string
If the namelist is not terminated with a '/', we try to read
the next character after the string termination '\0'.

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Apr  8 21:44:16 CEST 2014 on sn-devel-104
2014-04-08 21:44:15 +02:00
Jeremy Allison
5963519ec3 s3: smbd/nmbd/winbindd - fix append on trailing slash on system paths.
The xx_path() function incorrectly uses talloc_asprintf_append()
instead of talloc_asprintf() on a path that may have been modified
by the trim_string() call previously. talloc_asprintf_append()
always sticks the new text at the *end* of the allocated buffer,
not at the end of the string.

Fix bug #10538 - Daemons crashing when lock/state/cache directory parameter has a trailing slash

https://bugzilla.samba.org/show_bug.cgi?id=10538

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sat Apr  5 13:40:30 CEST 2014 on sn-devel-104
2014-04-05 13:40:29 +02:00
Jeremy Allison
837671f476 s3: messages: Implement cleanup of dead records.
When a smbd process dies, pending messages.tdb records for this process
might not get cleaned up. Implement a cleanup for dead records that is
triggered after a smbd dies uncleanly; the records for that PID are
deleted.

Based on a patchset from Christof Schmitt <cs@samba.org>.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
2014-04-03 06:22:13 +02:00
Stefan Metzmacher
80c57ab797 s3:lib/netapi/examples: remove unused variables
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:43 +02:00
Stefan Metzmacher
7d3b35c155 s3:lib/netapi/examples: fix const warnings in netlogon_control2.c
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:43 +02:00
Stefan Metzmacher
59394b3682 s3:lib/netapi/examples: fix invalid pointer value warnings in getjoininformation.c
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
18c19d51d1 s3:lib/netapi: fix some incompatible pointer type warnings in NetGetJoinableOUs_l()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
defe9aa675 s3:lib/smbconf: fix const warnings in testsuite.c
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Stefan Metzmacher
fc95672bcb s3:lib: fix a const warning in client_match()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-04-02 09:03:42 +02:00
Volker Lendecke
d6db35d7a5 tdb_wrap: Remove tdb_wrap_open_ again
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-31 22:52:14 +02:00
Volker Lendecke
328afb8d3e mutex: Avoid passing lp_ctx to tdb_wrap_open in grab_named_mutex
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-31 22:52:13 +02:00
Volker Lendecke
994d75574b dbwrap: Avoid passing lp_ctx to tdb_wrap_open in db_open_ctdb
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-31 22:52:13 +02:00
Volker Lendecke
604b970d67 serverid: Avoid passing lp_ctx to tdb_wrap_open in serverid_parent_init
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-31 22:52:13 +02:00
Volker Lendecke
12b8df1ba2 messaging: Avoid passing lp_ctx to tdb_wrap_open in messaging_tdb_parent_init
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-31 22:52:13 +02:00
Volker Lendecke
d169d0d9e2 messaging: Avoid passing lp_ctx to tdb_wrap_open in messaging_tdb_init
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-31 22:52:13 +02:00
Volker Lendecke
c35fec883c asys: Allow multiple results to be received
This makes use of C99 dynamic arrays. In this performance-sensitive code, I
would like to avoid malloc/free, and I think 15 years after the standard we
might be able to use this feature. Alternatively, we could use the "results"
memory area and store the jobids in the upper range, playing some cast-tricks.
Should work as well.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-27 06:06:12 +01:00
Volker Lendecke
c5d07df6ab pthreadpool: Allow multiple jobs to be received
This can avoid syscalls when multiple jobs are finished simultaneously

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-27 06:06:11 +01:00
Volker Lendecke
84aa2ddd86 pthreadpool: Avoid a malloc/free per job
pthreadpool_add_job is in our hottest code path for r/w intensive workloads, so
we should avoid anything CPU-intensive. pthreadpool used to malloc each job and
free it in the worker thread. This patch adds a FIFO queue for jobs that helper
threads copy from, avoiding constant malloc/free. This cuts user space
CPU in the local-bench-pthreadpool benchmark by roughly 10% on my system.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-27 06:06:11 +01:00
Stefan Metzmacher
74b359c616 s3:build: remove global CLUSTER_SUPPORT define
All ctdb specific code is isolated in samba-cluster-support.so now.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Mar 24 19:08:44 CET 2014 on sn-devel-104
2014-03-24 19:08:44 +01:00
Stefan Metzmacher
e93b85dbd8 s3:wscript: only build ctdb_dummy.c if we have no cluster support
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-24 17:09:11 +01:00
Stefan Metzmacher
1c96c068ad s3:serverid: use ctdb_serverids_exist_supported() runtime check
This is better than a compile time check for
HAVE_CTDB_CONTROL_CHECK_SRVIDS_DECL.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-24 17:09:10 +01:00
Stefan Metzmacher
4574602057 s3:ctdb_conn: only use ctdb_protocol.h or ctdb_private.h in ctdb_conn.c
We don't need them in ctdb_conn.h.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-24 17:09:10 +01:00
Stefan Metzmacher
fc04cad01a s3:lib: add cluster_support_[available|features]()
This allows runtime selection of libsamba-cluster-support.so

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-24 17:09:10 +01:00
Stefan Metzmacher
24054b5f8b s3:param: move lp_ctdbd_socket() to ctdbd_conn.c
This allows us to use CTDB_PATH only in ctdbd_conn.c

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-03-24 17:09:10 +01:00