IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
This changes the talloc hierarchy for a few callers, but as
talloc_tos() was initially designed exactly for this purpose (printing
SIDs in DEBUG), it should be okay.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
If the record is already locked check if the requested timeout is zero
and fail directly with NT_STATUS_LOCK_NOT_GRANTED.
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
We now have only one code path that stores the fully
granted lock.
This is not change in behavior, but it will simplify further
changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This prepares some helper functions in order to
allow callers of g_lock_lock() to pass in a callback function
that will run under the tdb chainlock when G_LOCK_WRITE was granted.
The idea is that the callers callback function would run with
g_lock_ctx->busy == true and all key based function are not be allowed
during the execution of the callback function. Only the
g_lock_lock_cb_state based helper function are allowed to be used.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This prepares some helper functions in order to
allow callers of g_lock_lock() to pass in a callback function
that will run under the tdb chainlock when G_LOCK_WRITE was granted.
The idea is that the callers callback function would only be allowed
to run these new helper functions, while all key based function are
not to be allowed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
First we fully check if we'll get the lock
and then store the lock.
This is not change in behavior, but it will simplify further changes.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
In the common case we don't have any shared lock holders,
so there's no need to allocate memory for the empty array.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This improves 'time smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 50000000'
from ~1.400.000 to ~3.400.000 operations per second any a testsystem.
As we also use TDB_VOLATILE for locking.tdb, this is a much more
realistic test now.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Unless the unique_lock_epoch changes via g_lock_lock()/g_lock_unlock()
we try to keep our existing watch instance alive while waiting
for unique_data_epoch to change.
This will become important in the following commits when the
dbwrap_watch layer will only wake up one watcher at a time
and each woken watcher will wakeup the next one. Without this
commit we would trigger an endless loop as none of the watchers
will ever change unique_data_epoch.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
It changes with every lock and unlock.
This will be needed in future in order to differentiate between
lock and data changed.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
The key points are:
1. We keep our position in the watcher queue until we got what
we were waiting for. It means the order is now fair and stable.
2. We only wake up other during g_lock_unlock() and only if
we detect that an pending exclusive lock is able to make progress.
(Note: read lock holders are never waiters on their own)
This reduced the contention on locking.tdb records drastically,
as waiters are no longer woken 3 times (where the first 2 times were completely useless).
The following test with 256 commections all looping with open/close
on the same inode (share root) is improved drastically:
smbtorture //127.0.0.1/m -Uroot%test smb2.create.bench-path-contention-shared \
--option='torture:bench_path=' \
--option="torture:timelimit=60" \
--option="torture:nprocs=256"
From some like this:
open[num/s=50,avslat=6.455775,minlat=0.000157,maxlat=55.683846]
close[num/s=50,avslat=4.563605,minlat=0.000128,maxlat=53.585839]
to:
open[num/s=80,avslat=2.793862,minlat=0.004097,maxlat=46.597053]
close[num/s=80,avslat=2.387326,minlat=0.023875,maxlat=50.878165]
Note the real effect of this commit will releaved together
with a following commit that only wakes one waiter at a time.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This matches the behavior of g_lock_cleanup_shared(), which also
only operates on the in memory struct g_lock.
We do a g_lock_store() later during g_lock_trylock() anyway
when we make any progress.
In the case we where a pending exclusive lock holder
we now force a g_lock_store() if g_lock_cleanup_dead()
removed the dead blocker.
This will be useful for the following changes...
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
The destructor triggered by dbwrap_watched_watch_recv() will
remove the watcher instance via a dedicated dbwrap_do_locked(),
just calling dbwrap_watched_watch_remove_instance() inside.
But the typical caller triggers a dbwrap_do_locked() again after
dbwrap_watched_watch_recv() returned. Which means we call
dbwrap_do_locked() twice.
We now allow dbwrap_watched_watch_recv() to return the existing
instance id (if it still exists) and removes the destructor.
That way the caller can pass the given instance id to
dbwrap_watched_watch_remove_instance() from within its own dbwrap_do_locked(),
when it decides to leave the queue, because it's happy with the new
state of the record. In order to get the best performance
dbwrap_watched_watch_remove_instance() should be called before any
dbwrap_record_storev() or dbwrap_record_delete(),
because that will only trigger a single low level storev/delete.
If the caller found out that the state of the record doesn't meet the
expectations and the callers wants to continue watching the
record (from its current position, most likely the first one),
dbwrap_watched_watch_remove_instance() can be skipped and the
instance id can be passed to dbwrap_watched_watch_send() again,
in order to resume waiting on the existing instance.
Currently the watcher instance were always removed (most likely from
the first position) and re-added (to the last position), which may
cause unfair latencies.
In order to improve the overhead of adding a new watcher instance
the caller can call dbwrap_watched_watch_add_instance() before
any dbwrap_record_storev() or dbwrap_record_delete(), which
will only result in a single low level storev/delete.
The returned instance id is then passed to dbwrap_watched_watch_send(),
within the same dbwrap_do_locked() run.
It also adds a way to avoid alerting any callers during
the current dbwrap_do_locked() run.
Layers above may only want to wake up watchers
during specific situations and while it's useless to wake
others in other situations.
This will soon be used to add more fairness to the g_lock code.
Note that this commit only prepares the api for the above to be useful,
the instance returned by dbwrap_watched_watch_recv() is most likely 0,
which means the watcher entry was already removed, but that will change
in the following commits.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15125
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
If dbwrap_watched_watch_recv() returns IO_TIMEOUT, "blockerdead" might
be an uninitialized non-false, and further down we'll remove the wrong
exclusive locker.
Bug: https://bugzilla.samba.org/show_bug.cgi?id=14636
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Mar 5 11:22:07 UTC 2021 on sn-devel-184
We don't require a sequence number that is incremented,
we just need a value that's not reused.
We use the new generate_unique_u64(), which is much cheaper!
Using smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 500000
under valgrind --tool=callgrind...
This change replaces this:
13,129,925,659 PROGRAM TOTALS
4,125,752,958 ???:_nettle_sha256_compress [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
1,257,005,866 ???:_nettle_aes_encrypt [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
590,000,773 bin/default/../../lib/tdb/common/lock.c:tdb_lock_list
571,503,429 ???:_nettle_aes_set_key [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
479,000,608 bin/default/../../lib/tdb/common/lock.c:tdb_unlock
...
by this:
6,877,826,377 PROGRAM TOTALS
590,000,773 bin/default/../../lib/tdb/common/lock.c:tdb_lock_list
479,000,608 bin/default/../../lib/tdb/common/lock.c:tdb_unlock
...
12,500,033 bin/default/../../lib/util/genrand_util.c:generate_unique_u64
...
8,996,970 ???:_nettle_sha256_compress [/usr/lib/x86_64-linux-gnu/libnettle.so.6.4]
time smbtorture3 //foo/bar -U% local-g-lock-ping-pong -o 5000000
gives:
537426 locks/sec
real 0m19,071s
user 0m15,061s
sys 0m3,999s
vs.
900956 locks/sec
real 0m11,155s
user 0m8,293s
sys 0m2,860s
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Optionally allow a database with g_lock format to participate in the dbwarp
lock order check. Will be used once locking.tdb is based upon g_lock.c
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Same concept as dbwrap_watched_watch_send/recv: Get informed if the
underlying data of a record changes. This utilizes the watched
database that g_lock is based upon anyway. To avoid spurious wakeups
by pure g_lock operations this patch adds a sequence number for the
data that is stored in the g_lock data field.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
When contending a WRITE with an existing READ, the contender puts
himself into the exclusive slot, waiting for the READers to go
away. If the async lock request is canceled before we got the lock, we
need to remove ourselves again. This is done in the destructor of the
g_lock_lock_state. In the successful case, the destructor needs to go
away.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Dec 22 18:57:17 UTC 2019 on sn-devel-184
The comment "this is used in very hot code paths" is not true right now, but
will get true soon....
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Now we have one fixed field for the exclusive lock holder and an array
of shared locks. This way we now prioritize writers over readers: If a
pending write comes in while readers are active, it will put itself
into the exclusive slot. Then it waits for the readers to vanish. Only
when all readers are gone the exclusive lock request is granted. New
readers will just look at the exclusive slot and see it's taken. They
will then line up as watchers, retrying whenever things change.
This also means that it will be cheaper to support many shared locks:
Granting a shared lock just means to extend the array. We don't have
to walk the array for possible conflicts.
This also adds explicit UPGRADE and DOWNGRADE operations for better
error checking.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This will allow using the g_lock.c logic on other databases as well
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Soon the g_lock database format will change. There will be one
exclusive entry and an array of shared entries. In that format,
there's no need to attach a lock_type to each entry in the g_lock
database. Reflect this change in the g_lock_dump API
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Avoid a call to dbwrap_record_get_value(), dbwrap_do_locked() already
gave us the value.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
I want to reduce dbwrap_record_get_value(). It makes the caller believe it can
make a copy of the TDB_DATA returned and that the value remains constant. It's
not, as you can always do a dbwrap_record_store().
This patch removes one requirement for getting the value out of a
db_record via dbwrap_record_get_value(). You can still make a copy, but from an
API perspective to me it's more obvious that "value" as a parameter to the
callback has a limited lifetime.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This puts too much logic into this lowlevel infrastructure module,
given the two minor external users.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Wed May 8 17:47:39 UTC 2019 on sn-devel-184
