1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-11 05:18:09 +03:00
Commit Graph

123 Commits

Author SHA1 Message Date
Volker Lendecke
72e9b8ceed lib: Fix a debug typo in g_lock.c
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-11-11 19:08:37 +00:00
Volker Lendecke
023f68cb76 g_lock: Add extensive debug information
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2021-03-09 22:36:28 +00:00
Volker Lendecke
654c18a244 g_lock: Fix uninitalized variable reads
If dbwrap_watched_watch_recv() returns IO_TIMEOUT, "blockerdead" might
be an uninitialized non-false, and further down we'll remove the wrong
exclusive locker.

Bug: https://bugzilla.samba.org/show_bug.cgi?id=14636
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Mar  5 11:22:07 UTC 2021 on sn-devel-184
2021-03-05 11:22:07 +00:00
Stefan Metzmacher
6c68c75b21 s3:g_lock: avoid very expensive generate_random_buffer() in g_lock_parse()
We don't require a sequence number that is incremented,
we just need a value that's not reused.
We use the new generate_unique_u64(), which is much cheaper!

Using smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 500000
under valgrind --tool=callgrind...

This change replaces this:

 13,129,925,659  PROGRAM TOTALS

  4,125,752,958  ???:_nettle_sha256_compress [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
  1,257,005,866  ???:_nettle_aes_encrypt [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
    590,000,773  bin/default/../../lib/tdb/common/lock.c:tdb_lock_list
    571,503,429  ???:_nettle_aes_set_key [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
    479,000,608  bin/default/../../lib/tdb/common/lock.c:tdb_unlock
    ...

by this:

  6,877,826,377  PROGRAM TOTALS

    590,000,773  bin/default/../../lib/tdb/common/lock.c:tdb_lock_list
    479,000,608  bin/default/../../lib/tdb/common/lock.c:tdb_unlock
    ...
     12,500,033  bin/default/../../lib/util/genrand_util.c:generate_unique_u64
    ...
     8,996,970  ???:_nettle_sha256_compress [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]

time smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 5000000

gives:

   537426 locks/sec

  real    0m19,071s
  user    0m15,061s
  sys     0m3,999s

vs.

   900956 locks/sec

  real    0m11,155s
  user    0m8,293s
  sys     0m2,860s

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2020-07-08 09:42:39 +00:00
Volker Lendecke
3bf576bb2a lib: Add g_lock_dump_send/recv
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
0a043e64b6 lib: Add g_lock_set_lock_order()
Optionally allow a database with g_lock format to participate in the dbwarp
lock order check. Will be used once locking.tdb is based upon g_lock.c

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
6fb6a6992d lib: Add g_lock_writev_data
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
4f2117d687 lib: Allow additional dbufs in g_lock_store()
Preparation for g_lock_writev_data()

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
3d0cfb82ec lib: Add g_lock_seqnum()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
c38f37147d lib: Add g_lock_wake_watchers()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
46e8e0cd7e lib: Add g_lock_watch_data_send/recv()
Same concept as dbwrap_watched_watch_send/recv: Get informed if the
underlying data of a record changes. This utilizes the watched
database that g_lock is based upon anyway. To avoid spurious wakeups
by pure g_lock operations this patch adds a sequence number for the
data that is stored in the g_lock data field.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2020-05-15 00:48:32 +00:00
Volker Lendecke
40ecc2f6f0 lib: Fix contending with a READ lock
When contending a WRITE with an existing READ, the contender puts
himself into the exclusive slot, waiting for the READers to go
away. If the async lock request is canceled before we got the lock, we
need to remove ourselves again. This is done in the destructor of the
g_lock_lock_state. In the successful case, the destructor needs to go
away.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Dec 22 18:57:17 UTC 2019 on sn-devel-184
2019-12-22 18:57:17 +00:00
Volker Lendecke
39f21a58b0 g_lock: Speed up sync g_lock_lock()
The comment "this is used in very hot code paths" is not true right now, but
will get true soon....

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-26 21:28:30 +00:00
Volker Lendecke
ba852a3d92 g_lock: Move a variable inside the block were it's used
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-26 21:28:30 +00:00
Volker Lendecke
f849d8c0cf g_lock: Factor out g_lock_cleanup_shared()
This function will find a second user soon

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-26 21:28:30 +00:00
Volker Lendecke
d734547488 lib: Change the g_lock data model
Now we have one fixed field for the exclusive lock holder and an array
of shared locks. This way we now prioritize writers over readers: If a
pending write comes in while readers are active, it will put itself
into the exclusive slot. Then it waits for the readers to vanish. Only
when all readers are gone the exclusive lock request is granted. New
readers will just look at the exclusive slot and see it's taken. They
will then line up as watchers, retrying whenever things change.

This also means that it will be cheaper to support many shared locks:
Granting a shared lock just means to extend the array. We don't have
to walk the array for possible conflicts.

This also adds explicit UPGRADE and DOWNGRADE operations for better
error checking.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:48 +00:00
Volker Lendecke
effad89f62 lib: Add g_lock_ctx_init_backend()
This will allow using the g_lock.c logic on other databases as well

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:48 +00:00
Volker Lendecke
97ba1ee2f9 lib: Make struct g_lock_rec private to g_lock.c
This is no longer exposed in the API

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:48 +00:00
Volker Lendecke
c9b34cd205 g_lock: Change prototype of g_lock_dump
Soon the g_lock database format will change. There will be one
exclusive entry and an array of shared entries. In that format,
there's no need to attach a lock_type to each entry in the g_lock
database. Reflect this change in the g_lock_dump API

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:47 +00:00
Volker Lendecke
78c96f6995 g_lock: Pass "data" down to g_lock_trylock()
Avoid a call to dbwrap_record_get_value(), dbwrap_do_locked() already
gave us the value.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
caef82b1fe dbwrap: Pass "value" to dbwrap_do_locked() callback
I want to reduce dbwrap_record_get_value(). It makes the caller believe it can
make a copy of the TDB_DATA returned and that the value remains constant. It's
not, as you can always do a dbwrap_record_store().

This patch removes one requirement for getting the value out of a
db_record via dbwrap_record_get_value(). You can still make a copy, but from an
API perspective to me it's more obvious that "value" as a parameter to the
callback has a limited lifetime.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-22 23:57:46 +00:00
Volker Lendecke
dd8aba3048 lib: Avoid includes.h in g_lock.c
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-06 20:36:34 +00:00
Volker Lendecke
8febbc2bcd lib: Remove g_lock_do()
This puts too much logic into this lowlevel infrastructure module,
given the two minor external users.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-06 20:36:34 +00:00
Volker Lendecke
aaad4d012d g_lock: Avoid sys_random(), use generate_random()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-06 20:36:34 +00:00
Volker Lendecke
54d673742f lib: Remove #define serverid_equal server_id_equal
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-11-06 20:36:34 +00:00
Noel Power
e3e9330609 s3/lib: clang: Fix 'passed-by-value struct argument contains uninit-ed data'
Fixed:

g_lock.c:358:21: warning: Passed-by-value struct argument contains uninitialized data (e.g., field: 'pid') <--[clang]
        state->watch_req = dbwrap_watched_watch_send(
                           ^
1 warning generated.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-07-16 22:52:25 +00:00
Volker Lendecke
b93ae77a6f g_lock: Fix a typo
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed May  8 17:47:39 UTC 2019 on sn-devel-184
2019-05-08 17:47:39 +00:00
Volker Lendecke
8992cf4aec g_lock: Apply some const
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-05-08 16:33:25 +00:00
Volker Lendecke
14f9e6f833 dbwrap: Clarify db_open_watched API
Point out in the API that "backend" talloc_moves into the watched
database.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Aug 17 21:29:15 CEST 2018 on sn-devel-144
2018-08-17 21:29:15 +02:00
Volker Lendecke
fea8ebce48 g_lock: Fix DEBUG messages
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-08-17 18:25:08 +02:00
Volker Lendecke
f986a73b24 lib: Pass mem_ctx to lock_path()
Fix a confusing API: Many places TALLOC_FREE the path where it's not
clear you have to do it.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2018-08-17 11:30:10 +02:00
Volker Lendecke
00513daf78 g_lock: Simplify g_lock_trylock
While chasing a bug in g_lock (not in master) I saw some opportunity to
simplify g_lock_trylock a bit. This is array handling, and array
handling is just extremely error-prone. This *might* be a little less
efficient or large numbers of READ locks, but this remains to be
seen. For now, simplify the code.

First, we make two passes now: One to remove ourselves, and the other
one to search for conflicts. Mixing up both made it pretty hard for me
to follow the code.

Second, I've removed the _mylock and mylock pointer/struct logic and
replaced it with the "mylock.pid.pid != 0 ? &mylock : NULL" when calling
g_lock_store. To me, this focuses the logic whether to add ourselves in
one place instead of spreading it around in the whole routine.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Aug 14 11:42:10 CEST 2018 on sn-devel-144
2018-08-14 11:42:10 +02:00
Volker Lendecke
6b3cc7916b g_lock: Avoid a double call to serverid_exist
If we try to G_LOCK_READ while a G_LOCK_WRITE is active, we do the
serverid_exists call twice. Avoid that.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-08-14 08:54:17 +02:00
Volker Lendecke
a37d9a45d1 tevent: Fix callers of tevent_req_set_endtime
tevent_req_set_endtime internally already calls tevent_req_nomem and thus sets
the error status correctly.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-04-24 19:41:15 +02:00
Volker Lendecke
be3c8d08ec lib: Make g_lock_locks use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Feb  8 14:50:49 CET 2018 on sn-devel-144
2018-02-08 14:50:49 +01:00
Volker Lendecke
67fcc7dbb9 lib: Make g_lock_dump use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-02-08 10:01:50 +01:00
Volker Lendecke
a6c749e76c lib: Make g_lock_do use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-02-08 10:01:50 +01:00
Volker Lendecke
ed3521d172 lib: Make g_lock_write_data use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-02-08 10:01:50 +01:00
Volker Lendecke
a104e08171 lib: Make g_lock_unlock use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-02-08 10:01:50 +01:00
Volker Lendecke
bdeb7e7d81 lib: Make g_lock_lock use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-02-08 10:01:50 +01:00
Volker Lendecke
3bc87a20e9 lib: Make g_lock_lock_send use TDB_DATA
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-02-08 10:01:50 +01:00
Stefan Metzmacher
22e309e541 s3:g_lock: keep old mylock on error and don't store new mylock on error
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2018-01-10 01:01:23 +01:00
Stefan Metzmacher
576fb4fb5d g_lock: fix cleanup of stale entries in g_lock_trylock()
g_lock_trylock() always incremented the counter 'i', even after cleaning a stale
entry at position 'i', which means it skipped checking for a conflict against
the new entry at position 'i'.

As result a process could get a write lock, while there're still
some read lock holders. Once we get into that problem, also more than
one write lock are possible.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13195

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Wed Dec 20 20:31:48 CET 2017 on sn-devel-144
2017-12-20 20:31:48 +01:00
Andreas Schneider
b3d50723b0 s3:glock: Move sanity check in g_lock_parse()
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Sun Dec 10 00:46:26 CET 2017 on sn-devel-144
2017-12-10 00:46:26 +01:00
Volker Lendecke
2f8055f676 dbwrap_watch: Remove the "prec" parameter from watch_recv
The initial idea was to have some "atomicity" in this API. Every
caller interested in a record would have to do something with
it once it changes. However, only one caller really used this
feature, and that is easily changed to not use it. So
remove the complexity.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-11-29 16:59:15 +01:00
Volker Lendecke
667307cd78 g_lock: remove unused functions g_lock_get() and g_lock_get_talloc()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:17 +02:00
Volker Lendecke
21df1401ab g_lock: Use parse_record in g_lock_dump
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:17 +02:00
Volker Lendecke
145b9cbbf2 g_lock: remove unused functions g_lock_put() and g_lock_record_store()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:17 +02:00
Volker Lendecke
241f31dbd4 g_lock: Use dbwrap_do_locked in g_lock_write_data
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:17 +02:00
Volker Lendecke
fc891872c2 g_lock: Use dbwrap_do_locked in g_lock_lock_retry
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2017-07-25 17:43:17 +02:00