IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The current messaging implementation is based on a tdb indexed by server_id. If
we have more than one messaging context in a process, messages might not arrive
at the right context and be dropped, depending on which signal handler is
triggered first.
This is the same patch as bd55fdb lifted to messaging.c
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This also changes the layering
messaging_send_iov -> messaging_send_buf -> messaging_send
to
messaging_send_buf -> messaging_send -> messaging_send_iov
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Tue May 27 04:41:43 CEST 2014 on sn-devel-104
In the future we will have multiple task id's per process. They should all be
able to benefit from the self-send local optimization.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
In the messaging_read receivers we already defer the callback: We need to
reply on potentially different tevent contexts, thus the defer_callback.
The callback case in messaging_dispatch_rec was direct before this
patch. This changes messaging_dispatch_rec to also defer the callback
in the self-send case.
Now we need only two roundtrips in local-messaging-read1 :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
If we register an additional tevent context, we can now properly do
nested event contexts, listening for just one message type inside a
tevent_req_poll.
At this point this only enhances things without ctdb, but I'm working fixing
that soon.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This delegates the decision whether to read a message to a callback
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
messaging_read_send/recv was okay for just one handler in the queue. For
multiple handlers it was pretty broken.
A handler that deletes itself as part of the callback (pretty typical use
case...) drops the message for a subsequent handler that responds to the same
message type. In messaging_dispatch_rec we walk the array, however
messaging_read_cleanup has already changed the array. tevent_req_defer_callback
does not help here: It only defers the callback, it does not defer the cleanup
function.
This also happens when a callback deletes a different handler
A handler that re-installs itself in the callback might get a message twice.
This patch changes the code such that only messaging_dispatch_rec adds records
to msg_ctx->waiters, new waiters are put into a staging area first
(msg_ctx->new_waiters). Also messaging_read_cleanup does not move anything
around in msg_ctx->waiters, it only nulls out itself. messaging_dispatch_rec is
changed to cope with this.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Callers might not be interested in the rec, just the fact that something
arrived
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This uses a copy, will be replaced by a direct iovec call through to
sendmsg on the unix domain socket
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Messaging based on unix domain datagram sockets
This makes every process participating in messaging bind on a unix domain
datagram socket, similar to the source4 based messaging. The details are a bit
different though:
Retry after EWOULDBLOCK is done with a blocking thread, not by polling. This
was the only way I could in experiments avoid a thundering herd or high load
under Linux in extreme overload situations like many thousands of processes
sending to one blocked process. If there are better ideas to do this in a
simple way, I'm more than happy to remove the pthreadpool dependency again.
There is only one socket per process, not per task. I don't think that per-task
sockets are really necessary, we can do filtering in user space. The message
contains the destination server_id, which contains the destination task_id. I
think we can rebase the source4 based imessaging on top of this, allowing
multiple imessaging contexts on top of one messaging_context. I had planned to
do this conversion before this goes in, but Jeremy convinced me that this has
value in itself :-)
Per socket we also create a fcntl-based lockfile to allow race-free cleanup of
orphaned sockets. This lockfile contains the unique_id, which in the future
will make the server_id.tdb obsolete.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
When a smbd process dies, pending messages.tdb records for this process
might not get cleaned up. Implement a cleanup for dead records that is
triggered after a smbd dies uncleanly; the records for that PID are
deleted.
Based on a patchset from Christof Schmitt <cs@samba.org>.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
All ctdb specific code is isolated in samba-cluster-support.so now.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Mon Mar 24 19:08:44 CET 2014 on sn-devel-104
This is made to replace the msg_channel abstraction.
msg_channel was created to not miss any messages. For this, some
complex queueing was installed. This complexity has caused quite a
few problems in the past (see bug 10284 for example).
messaging_read_send/recv is able to achieve the same goal with a
lot less complexity. The messaging_read_send atomically installs
the reader into the messaging_context, we will not miss any messages
while this installed. messaging_send_recv will deinstall that
listener, but in the callback function you can directly call
messaging_read_send again without going through the tevent_loop_once.
As long as this is always made sure, no messages will be lost.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Immediate tevents don't free themselves as timed events do :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jan 10 01:20:04 CET 2014 on sn-devel-104
... not as a speed improvement, it saves the second NULL check
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
We've always had the corresponding deregister message
Autobuild-User: Volker Lendecke <vlendec@samba.org>
Autobuild-Date: Tue Jan 24 15:27:51 CET 2012 on sn-devel-104
This has been broken since ff0ac5b0 (May 2007).
Basically all messages were belonging to the General class except for CTDB
messages.
This fixed the message_send_all() function to correctly compute the class, and
fixes registrations to include all they need to cope with the fact not all
messages are of calss general (registrations rotted a bit because as long as
FLAG_MSG_GENERAL was defined the process woould receive all messages).
Signed-off-by: Andreas Schneider <asn@samba.org>
The FLAG_MSG_PRINT_NOTIFY class is actually obsolete and never used, as the
only message belonging to it is not used either.
Signed-off-by: Andreas Schneider <asn@samba.org>
the daemons themselves. Allows client utilities to silently
fail to create a messaging context due to access denied on the
messaging tdb (which I need for the following patch).
Jeremy.
If we call messaging_ctdbd_connection() we end up with the wrong vnn in our
messaging context.
This is a bit of a hack, get_my_vnn() needs to go eventually along with
procid_self()
When a samba server process dies hard, it has no chance to clean up its entries
in locking.tdb, brlock.tdb, connections.tdb and sessionid.tdb.
For locking.tdb and brlock.tdb Samba is robust by checking every time we read
an entry from the database if the corresponding process still exists. If it
does not exist anymore, the entry is deleted. This is not 100% failsafe though:
On systems with a limited PID space there is a non-zero chance that between the
smbd's death and the fresh access, the PID is recycled by another long-running
process. This renders all files that had been locked by the killed smbd
potentially unusable until the new process also dies.
This patch is supposed to fix the problem the following way: Every process ID
in every database is augmented by a random 64-bit number that is stored in a
serverid.tdb. Whenever we need to check if a process still exists we know its
PID and the 64-bit number. We look up the PID in serverid.tdb and compare the
64-bit number. If it's the same, the process still is a valid smbd holding the
lock. If it is different, a new smbd has taken over.
I believe this is safe against an smbd that has died hard and the PID has been
taken over by a non-samba process. This process would not have registered
itself with a fresh 64-bit number in serverid.tdb, so the old one still exists
in serverid.tdb. We protect against this case by the parent smbd taking care of
deregistering PIDs from serverid.tdb and the fact that serverid.tdb is
CLEAR_IF_FIRST.
CLEAR_IF_FIRST does not work in a cluster, so the automatic cleanup does not
work when all smbds are restarted. For this, "net serverid wipe" has to be run
before smbd starts up. As a convenience, "net serverid wipedbs" also cleans up
sessionid.tdb and connections.tdb.
While there, this also cleans up overloading connections.tdb with all the
process entries just for messaging_send_all().
Volker
The change notify code registered a separate message handler for each
tree connect. This registration uses the global messaging context.
The messaging code would consider a 2nd registration for the same
messaging type as being an 'update' of the handler, rather than a new
handler. It also would only call the first handler in the linked list
for a given message type when dispatching messages.
This patch changes the messaging code to allow for multiple
registrations of the same message type, and allow for multiple calls
to different messaging handler for one incoming message.
This fixes the problem with the test_notify_tcon() test that I
recently committed to the S4 smbtorture
bugs in various places whilst doing this (places that assumed
BOOL == int). I also need to fix the Samba4 pidl generation
(next checkin).
Jeremy.
(This used to be commit f35a266b3c)
I'm 100% certain I've forgotten to merge something, but the main code
should be in. It's mainly in dbwrap_ctdb.c, ctdbd_conn.c and
messages_ctdbd.c.
There should be no changes to the non-cluster case, it does survive make
test on my laptop.
It survives some very basic tests with ctdbd enables, I did not do the
full test suite for clusters yet.
Phew...
Volker
(This used to be commit 15553d6327)
branch, please check if it fulfils your needs.
Two changes: The validation is not done inside the brlock.c traverse_fn,
it's done as a separate routine.
Secondly, this patch does not call the checker routines in smbcontrol
directly but depends on a running smbd.
(This used to be commit 7e39d77c1f)
This removes message_block / message_unblock. I've talked to Jeremy and
Günther, giving them my reasons why I believe they have no effect.
Neither could come up with a counter-argument, so they go :-)
(This used to be commit a925e0991f)
replaced by MSG_FLAG_LOWPRIORITY or'ed into the msg_type. To enable this,
changed the msg_type definitions to hexadecimal.
This way we could theoretically add the MSG_FLAG_NODUPLICATES again, but I
would rather not do this, because that one is racy and can't be guaranteed at
all.
(This used to be commit 3f5eb8a960)
doing this because for the clustering the marshalling is needed in more
than one place, so I wanted a decent routine to marshall a message_rec
struct which was not there before.
Tridge, this seems about the same speed as it used to be before, the
librpc/ndr overhead in my tests was under the noise.
Volker
(This used to be commit eaefd00563)
message_send_pid is used anymore. Two users of duplicates_allowed: winbind and
the printer notify system.
I don't thing this really changes semantics: duplicates_allowed is hell racy
anyway, we can't guarantee that we don't send the same message in sequence
twice, and I think the only thing we can harm with the print notify is
performance.
For winbind I talked to Günther, and he did not seem too worried.
Volker
(This used to be commit 75b3ae6a76)
the timeouts on the individual message send calls with an overall timeout on
all the calls.
The timeout in message_send_pid_with_timeout() did not make much sense IMO
anyway, because the tdb_fetch() for the messages_pending_for_pid was blocking
in a readlock anyway, we "just" did the timeout for the write lock.
This new code goes through the full wait for the write lock once and then
breaks out of sending the notifies instead of running into the timeout per
target.
Jerry, please check this!
Thanks,
Volker
(This used to be commit 697099f06e)
locking/locking.c we have to send retry messages to timed lock holders.
The majority of this patch passes a "struct messaging_context" down
there. No functional change, survives make test.
(This used to be commit bbb5084146)
and connections_forall. This centralizes all the routines that did individual
tdb_open("connections.tdb") and direct tdb_traverse.
Volker
(This used to be commit e43e94cda1)
This changes "struct process_id" to "struct server_id", keeping both is
just too much hassle. No functional change (I hope ;-))
Volker
(This used to be commit 0ad4b1226c)
it needs the specific error message.
Make messages.c return NTSTATUS and specificially NT_STATUS_INVALID_HANDLE if
sending to a non-existent process.
Volker
(This used to be commit 3f620d181d)
void message_register(int msg_type,
void (*fn)(int msg_type, struct process_id pid,
- void *buf, size_t len))
+ void *buf, size_t len,
+ void *private_data),
+ void *private_data)
{
struct dispatch_fns *dfn;
So this adds a (so far unused) private pointer that is passed from
message_register to the message handler. A prerequisite to implement a tiny
samba4-API compatible wrapper around our messaging system. That itself is
necessary for the Samba4 notify system.
Yes, I know, I could import the whole Samba4 messaging system, but I want to
do it step by step and I think getting notify in is more important in this
step.
Volker
(This used to be commit c8ae60ed65)
per type - this is all we use right now and makes
re-entrancy problems with deleting handlers with
a message dispatch loop go away.
Jeremy.
(This used to be commit 2e9b6faeae)
in tdb message processing. If we're inside a dispatch
function and we delete our own handler we'd walk onto
the next pointer from a deleted memory block. Fixes
crash bug in winbindd (and goodness knows where else).
Jeremy.
(This used to be commit 27a4c11214)
calls make it :
become_root_uid_only()
operation
unbecome_root_uid_only()
saving errno across the second call. Most of our internal
change calls can be replaced with these simple calls.
Jeremy
(This used to be commit 4143aa83c0)
fix the messaging code to call the efficient calls :
save_re_uid()
set_effective_uid(0);
messaging_op
restore_re_uid();
instead of using heavyweight become_root()/unbecome_root()
pairs around all messaging code. Fixup the messaging
code to ensure sec_init() is called (only once) so that non-root
processes still work when sending messages.
This is a lighter weight solution to become_root()/unbecome_root()
(which swaps all the supplemental groups) and should be more
efficient. I will migrate all server code over to using this
(a similar technique should be used in the passdb backend
where needed).
Jeremy.
(This used to be commit 4ace291278)
* \PIPE\unixinfo
* winbindd's {group,alias}membership new functions
* winbindd's lookupsids() functionality
* swat (trunk changes to be reverted as per discussion with Deryck)
(This used to be commit 939c3cb5d7)
a big relief on messages.tdb contention as ignoring processes with >1000
messages in printing/notify.c should work correctly now.
Jeremy, Jerry told me to ask you about printer scalability torture tests, this
might be a reason why you implemented the message_send_pid_with_timeout
using the signal (shudder) in the first place. :-)
While looking at that... Wouldn't it be better to not use the signal but have
an overall timeout for print_notify_send_messages using GetTimeOfDay & friends
and not use the alarm signal deep inside tdb.c?
Volker
(This used to be commit b5e82bb512)
printmig.exe work
* merge the sys_select_signal(char c) change from trunk
in order to keeo the winbind code in sync
(This used to be commit a112c5570a)
functions so we can funnel through some well known functions. Should help greatly with
malloc checking.
HEAD patch to follow.
Jeremy.
(This used to be commit 620f2e608f)
the amount of time we hold tdb locks). Gulp down all messages at once rather
than reading/re-writing one at a time. NOTE: All dispatch routines *must*
be able to cope with incoming message on *odd* byte boundaries (all current
handlers do).
Jeremy.
(This used to be commit b752c0340f)
this is needed because W2K will send a TCP reset to any open
connections that have not done a negprot when a second connection is
made. This meant that under heavy netlogon load a Samba domain member
would fail authentications.
Jeremy, you may wish to port this to 2.2.x
(This used to be commit eb196070e6)
source/rpc_server/srv_spoolss_nt.c
- add an access check to _spoolss_deleteprinter() to stop random
users and passers by from deleting printers.
source/lib/messages.c
- converted global msg_all struct to a local in message_send_all()
function.
source/include/smb.h
- added a success error code to the spoolss return codes.
source/include/proto.h
source/param/loadparm.c
source/printing/printing.c
- Added new parameter "total print jobs" to limit the total number
of print jobs across all queues. Currently individual queues are
limited by "max print jobs".
(This used to be commit 02f154e729)
testsuite/printing/psec.c
- Use lock directory from smb.conf parameter when peeking at the
ntdrivers.tdb file.
source/rpc_parse/parse_sec.c
- fix typo in debug message
source/script/installbin.sh
- create private directory as part of 'make install'.
source/nsswitch/winbindd_cache.c
source/nsswitch/winbindd_idmap.c
source/passdb/secrets.c
source/smbd/connection.c
- always convert tdb key to unix code-page when generating.
source/printing/nt_printing.c
- always convert tdb key to unix code-page when generating.
- don't prepend path to a filename that is NULL in
add_a_printer_driver_3().
source/rpc_server/srv_spoolss_nt.c
- always convert tdb key to unix code-page when generating.
- don't prepend server name to a path/filename that is NULL in the
fill_printer_driver_info functions.
source/printing/printing.c
- always convert tdb key to unix code-page when generating.
- move access check for print_queue_purge() outside of job delete
loop.
source/smbd/unix_acls.c
- fix for setting ACLs (this got missed earlier)
source/lib/messages.c
- trivial sync with appliance_head
(This used to be commit 376601d17d)
a --with-spinlocks option to configure, this does mean the on-disk tdb
format has changed, so 2.2alphaX sites will need to re-create their
tdb's. The upside is no more tdb fragmentation and a +5% on netbench.
Swings and roundabouts....
Jeremy.
(This used to be commit 9dea7b7c25)
