1
0
mirror of https://github.com/samba-team/samba.git synced 2025-03-12 20:58:37 +03:00

333 Commits

Author SHA1 Message Date
Michael Adam
925625b528 dbwrap_ctdb: avoid smbd/ctdb deadlocks: check whether we can work locally in db_ctdb_parse_record()
If the same process tries to re-lock the same record
it has already locked, don't go to the ctdbd again.

There are situations where we already have a lock on a record
and then do a dbwrap_parse_record() on that record, for instance
in locking code:

do_lock()
  -> grabs lock on brl record with brl_get_locks()
    -> calls brl_lock()
      -> calls brl_lock_posix or _windows_default()
        -> calls contend_level2_oplocks_begin()
          -> calls brl_locks_get_read_only()
            -> calls dbwrap_parse_record on the same brl record as above

In the local (tdb) case, this is not a problem, because
identical fcntl locks in the same process don't contend each other,
but calling out to ctdb for this lets smbd and ctdb deadlock.

db_ctdb_fetch_lock() already correclty checks first
whether we can simply try to lock locally. But db_ctdb_parse_record()
failed to do so for empty records, i.e. records that only
consist of the ctdb record header. (These can be deleted records
but can also be freshly created and still empty records.)

This patch lets db_ctdb_parse_record() not skip local access
for empty records, hence fixing the deadlock.

PLAN: In the long run, it would be better to solve this
generically on the dbwrap_layer, i.e. root the notion of
an already locked record there, and skip any call to the
db (tdb or ctdb backend) if we have it. This would also
solve the problem for all calls like fetch_locked, parse_record
and possibly others.  But this is the urgent fix for now.

Pair-Programmed-With: Volker Lendecke <vl@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Tested-by: Björn Baumbach <bb@sernet.de>
2014-03-03 12:56:38 +01:00
Volker Lendecke
5e07731cf6 dbwrap: Slightly simplify dbwrap_watch_record_stored
We have messaging_send_buf, so there's no point to use an explicit
DATA_BLOB here.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Feb 25 23:27:21 CET 2014 on sn-devel-104
2014-02-25 23:27:21 +01:00
Michael Adam
b210e7aade dbwrap: use DBWRAP_LOCK_ORDER_VALID() in db_open()
instead of the hand written test.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Feb  7 18:05:51 CET 2014 on sn-devel-104
2014-02-07 18:05:51 +01:00
Michael Adam
7e766a0a8a dbwrap: add dbwrap_flags argument to dbwrap_local_open()
To be consistent with db_open() and prepare for future
possible extensions.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-02-07 16:06:07 +01:00
Stefan Metzmacher
a20c977c7a dbwrap_open: add 'dbwrap_optimize_readonly:* = yes' option
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2014-02-07 16:06:06 +01:00
Stefan Metzmacher
a97b588b63 dbwrap_ctdb: implement DBWRAP_FLAG_OPTIMIZE_READONLY_ACCESS
For non-persistent databases we try to use CTDB_CONTROL_SET_DB_READONLY
in order to make use of readonly records.

Pair-Programmed-With: Michael Adam <obnox@samba.org>

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Michael Adam <obnox@samba.org>
2014-02-07 16:06:06 +01:00
Michael Adam
6def1c3f6e dbwrap: add a dbwrap_flags argument to db_open_ctdb()
This is in preparation of directly supporting ctdb read only
record copies when opening a ctdb database from samba.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-02-07 16:06:06 +01:00
Michael Adam
cf0cb0add9 dbwrap: add a dbwrap_flags argument to db_open()
This is in preparation to support handing flags to backends,
in particular activating read only record support for ctdb
databases. For a start, this does nothing but adding the
parameter, and all databases use DBWRAP_FLAG_NONE.

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2014-02-07 16:06:06 +01:00
Volker Lendecke
017609767f dbwrap_watch: Use messaging_read_send/recv
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-01-21 08:11:01 +01:00
Volker Lendecke
5f3ccfc16b messaging3: Add messaging_read_send/recv
This is made to replace the msg_channel abstraction.

msg_channel was created to not miss any messages. For this, some
complex queueing was installed. This complexity has caused quite a
few problems in the past (see bug 10284 for example).

messaging_read_send/recv is able to achieve the same goal with a
lot less complexity. The messaging_read_send atomically installs
the reader into the messaging_context, we will not miss any messages
while this installed. messaging_send_recv will deinstall that
listener, but in the callback function you can directly call
messaging_read_send again without going through the tevent_loop_once.
As long as this is always made sure, no messages will be lost.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-01-21 08:10:41 +01:00
Christof Schmitt
55b431fad5 s3:dbwrap: Use milliseconds for "Held tdb lock" message
This is consistent with the parameter using milliseconds and the other
warnings in the same file also using milliseconds.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Jan 16 11:19:39 CET 2014 on sn-devel-104
2014-01-16 11:19:39 +01:00
Christof Schmitt
32f232d9ad s3:dbwrap: Store warning thresholds in db_ctdb_ctx
Avoid the parameter lookup for the warning thresholds in the hot code
path by reading them in db_open_ctdb and storing them in the
db_ctdb_ctx.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-01-16 09:18:45 +01:00
Christian Ambach
ef239302ed s3:dbwrap include the hashchain in the logs
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-01-16 09:18:45 +01:00
Christian Ambach
63a8f7e441 s3:dbwrap report time for chainlock and CTDB migrate
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2014-01-16 09:18:45 +01:00
Volker Lendecke
b64abab807 dbwrap_ctdb: Instrument chainunlock timing
We need an indication whether we run into the fcntl thundering
herd. fcntl unlock should be blindingly fast in the normal case. If it
takes longer than 5 milliseconds, warn. The timeout can be adapted by
setting

ctdb:unlock_warn_threshold = <number-of-milliseconds>

Reviewed-by: Christof Schmitt <cs@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
2014-01-16 09:18:45 +01:00
Volker Lendecke
34bfd0b6d2 ctdb_conn: Log long fetch_lock calls
With this patch, the number of fetch_lock attempts before dbwrap_ctdb
logs that it took x attempts to get a record is configurable with

net conf setparm global ctdb:migrate_attempts 10

This patch also adds

net conf setparm global ctdb:migrate_duration 5000

to trigger the same log message if it took longer than x milliseconds
to retrieve a record.

Reviewed-by: Christof Schmitt <cs@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
2014-01-16 09:18:45 +01:00
Volker Lendecke
1cae59ce11 dbwrap_ctdb: Treat empty records as non-existing
This is a patch implementing the workaround Christian mentioned in
https://bugzilla.samba.org/show_bug.cgi?id=10008#c5

Bug: https://bugzilla.samba.org/show_bug.cgi?id=10008
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Christian Ambach <ambi@samba.org>
2013-08-28 23:37:08 +02:00
Christian Ambach
9910b8050c s3:lib/dbwrap add missing curly braces
violation of README.Coding

Signed-off-by: Christian Ambach <ambi@samba.org>

Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2013-05-16 23:22:19 +02:00
Volker Lendecke
e13021e3af dbwrap: Allow dbwrap_record_watch_recv to not lock the record
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2013-04-26 15:17:23 -07:00
Volker Lendecke
c2b26a43ff dbwrap-ctdb: Avoid a talloc_stackframe()
We have only a single allocation in this routine, so I think we can live
without a stackframe.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-03-26 11:19:59 +01:00
Volker Lendecke
1409895d9e dbwrap-ctdb: Use ctdbd_parse in db_ctdb_parse_record
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-03-26 11:19:19 +01:00
Volker Lendecke
bbb4068c93 dbwrap: Use tdb_null in db_ctdb_delete
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Christian Ambach <ambi@samba.org>

Autobuild-User(master): Christian Ambach <ambi@samba.org>
Autobuild-Date(master): Mon Mar 25 19:42:30 CET 2013 on sn-devel-104
2013-03-25 19:42:30 +01:00
Michael Adam
ff65500f2b s3:dbrwap_ctdb: ZERO_STRUCT(rec) just to be sure in traverse_persistent_callback_read()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-02-04 17:14:22 +01:00
Michael Adam
3949854549 s3:dbwrap_ctdb: ZERO_STRUCT(rec) just to be sure in traverse_read_callback()
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-02-04 17:14:22 +01:00
Stefan Metzmacher
a09f3a3551 s3:dbwrap_ctdb: add "db_context" to "db_record"
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-02-04 17:14:22 +01:00
Stefan Metzmacher
25bdab9c61 s3:dbwrap_ctdb: setup result->name in db_open_ctdb()
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
2013-02-04 17:14:22 +01:00
Volker Lendecke
aea49ed37a dbwrap: Use INCOMPATIBLE_HASH for dbwrap_watchers.tdb
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2013-01-11 17:10:48 +01:00
Volker Lendecke
2f38a77a2d dbwrap: Do not rely on dbwrap_record_get_value to return a talloc object
db_tdb_fetch_locked returns the value as part of a larger talloc object
that also contains the key.  This means we can not realloc, but have to
freshly alloc.

Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Thu Nov 29 20:21:51 CET 2012 on sn-devel-104
2012-11-29 20:21:50 +01:00
Volker Lendecke
343ce16e0b s3: Remove db_ctdb_fetch
Note that this also makes the request for read only copies
much more explicity visible in the code.

Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:18:49 +01:00
Volker Lendecke
6b8a37f5ca s3: Directly parse local existing records in db_ctdb_parse_record
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:10:13 +01:00
Volker Lendecke
9dd047d67e s3: Factor out db_ctdb_can_use_local_hdr from db_ctdb_can_use_local_copy
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:08:06 +01:00
Volker Lendecke
55d75a3fbe s3: Remove unused code for fetching persistent ctdb records
The only entry point here is parse_record, and this catches the persistent
case with a direct parse now

Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:07:26 +01:00
Volker Lendecke
7b43362a9e s3: Avoid db_ctdb_fetch for persistent databases
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:05:25 +01:00
Volker Lendecke
93219f92bb s3: Factor out parse_newest_in_marshall_buffer from pull_newest_from_marshall_buffer
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:03:59 +01:00
Volker Lendecke
850d5de967 s3: reduce db_ctdb_marshall_loop_next to specialized db_ctdb_marshall_buf_parse
now that the db_ctdb_marshall_loop_next_key has been factored out.

Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 18:00:02 +01:00
Volker Lendecke
a742b6edd5 s3: Factor out db_ctdb_marshall_loop_next_key from db_ctdb_marshall_loop_next
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:55:19 +01:00
Volker Lendecke
664b170b13 s3: Slightly simplify db_ctdb_marshall_loop_next
Both callers give a key argument

Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:52:34 +01:00
Volker Lendecke
af608cdd8c s3: Use db_ctdb_ltdb_parse in db_ctdb_fetch_db_seqnum_from_db
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:52:02 +01:00
Volker Lendecke
7127344863 s3: Use db_ctdb_ltdb_parse in db_ctdb_ltdb_fetch
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:50:36 +01:00
Volker Lendecke
560ed93147 s3: Add db_ctdb_ltdb_parse
Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:43:02 +01:00
Volker Lendecke
6e0259d8ec s3: Slightly simplify db_ctdb_transaction_commit
Avoid an unnecessary "else".

Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:42:20 +01:00
Volker Lendecke
b206b52748 s3: Remove header==NULL code from db_ctdb_marshall_record
The only call chain (via db_ctdb_marshall_add) has header != NULL

Reviewed-by: Michael Adam <obnox@samba.org>
2012-11-29 17:41:46 +01:00
Volker Lendecke
611675fb37 s3: Fix some blank line endings
Reviewed-by: Michael Adam <obnox@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Nov 20 19:18:33 CET 2012 on sn-devel-104
2012-11-20 19:18:33 +01:00
Gregor Beck
bfafb0ce83 s3:dbwrap_ctdb: set errno = ENOSYS if clustering is not supported
Signed-off-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Sep 21 18:05:56 CEST 2012 on sn-devel-104
2012-09-21 18:05:56 +02:00
Volker Lendecke
25bdc36418 s3: Fix a comment
Signed-off-by: Jeremy Allison <jra@samba.org>
2012-09-04 15:15:00 -07:00
Christof Schmitt
424492a963 s3:dbwrap_ctdb: Add DB name and key to warning message
When a operation takes too long, it is useful for debugging to know the
DB and the key.

Signed-off-by: Christian Ambach <ambi@samba.org>
2012-08-31 11:10:14 +02:00
Stefan Metzmacher
ec69eb41bf s3:dbwrap_ctdb: initialize db_record->db
metze
2012-07-31 09:34:20 +02:00
Michael Adam
01863df25d s3_dbrwap_ctdb: improve a comment in db_ctdb_can_use_local_record()
Pair-Programmed-With: Gregor Beck <gbeck@sernet.de>
2012-06-29 15:21:21 +02:00
Michael Adam
b46977e4f6 s3:dbwrap_ctdb: fix a comment in db_ctdb_fetch()
Pair-Programmed-With: Gregor Beck <gbeck@sernet.de>
2012-06-29 15:21:20 +02:00
Michael Adam
92808a6005 s3:dbwrap_ctdb: expand the comment for the db_ctdb_can_use_local_copy() function
Pair-Programmed-With: Gregor Beck <gbeck@sernet.de>
2012-06-29 15:21:19 +02:00