IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Add limited nested transaction support to the back ends to make the key value
operations atomic (for those back ends that support nested transactions).
Note: that only the lmdb backend currently supports nested transactions.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We need to check for the errors given by ldb_unpack() et al by preserving
the error code from kv_ctx->parser() called by tdb_parse_record() in
ltdb_parse_record().
Otherwise we will silently accept corrupt records and segfault later.
Likewise new pack formats will confuse the parser but not be
detected except by the incomplete struct ldb_message.
With this patch, the user will see a message like:
Invalid data for index DN=@BASEINFO
Failed to connect to 'st/ad_dc/private/sam.ldb' with backend 'tdb': Unable to load ltdb cache records for backend 'ldb_tdb backend'
Failed to connect to st/ad_dc/private/sam.ldb - Unable to load ltdb cache records for backend 'ldb_tdb backend'
This can be refined in the future by a specific check for
pack format versions in a higher caller, but this much is
needed regardless to detect corrupt records.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We want to reuse the reindex context struct for repacking, but it has an
unnecessary module pointer on it. Turns out the existing code doesn't
need it either, so this patch deletes the pointer.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This makes LDAP bind significantly faster in the case of having many
members, due to large size of these records (with tens of thousands of
member links). During the nested group calculation, you are only
interested in memberOf not the member links.
(We add a bit-field to determine whether or not the backend actually
supports pointing into database memory. For some reason TDB pointers
aren't stable, so for now we set this option just on LMDB backends.)
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Adding iterate_range to LDB API and implementing in LMDB. This
operation takes a start_key and end_key and returns all records between
the two, inclusive of both. This will be used to implementing indexing
for <= and >= expressions.
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add the get_size method to the ldb_key_value layer, this will allow the
reindexing code to get an estimate of the number of records in the
database.
The lmdb backend returns an accurate count of the number of records in
the database withe the mdb_env_stat call.
The tdb backend does not provide a low cost method to determine the
number of records on the database. It does provide a tdb_summary call
however this this walks the entire database.
So for tdb we use the map size divided by 500, this over estimates the counts
for small domains, but the extra memory allocated for the cache should
not be significant.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Move the key value code to a separate subdirectory.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename ltdb_parse_data_unpack_ctx to ldb_kv_parse_data_unpack_ctx, as
it's a key value level structure and not ltdb specific.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename ltdb_idxptr to ldb_kv_idxptr as it's key value level and not tdb
specific.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename tdb_key_ctx to key_ctx, as it's key value level and not tdb
specific.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename ltdb_cache to ldb_kv_cache as it's key value level and not tdb
specific
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Tidy up the code format after the rename of ltdb_private to
ldb_kv_private
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename ltdb_private to ldb_kv_private as it contains key value operation
context.
Note there is still some tdb specific context that can be refactored into a
separate structure along the lines of the lmdb context.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fix up the code formatting after the rename of ltdb_req_spy to
ldb_kv_req_spy
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename ltdb_req_spy to ldb_kv_req_spy, as it is key value level and not
tdb specific.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename ltdb_context to ldb_kv_context as it is a key value level
structure and not tdb specific.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fix up the formatting after the rename of ltdb_reindex_context to
ldb_kv_reindex_context.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename struct ltdb_reindex_context to ldb_kv_reindex_context, as this is
a key value level structure and not a tdb specific structure.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Clean up the code format after the rename in the previous commit.
Hopefully doing a rename commit followed by a reformat commit makes the
code easier to review.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Rename the ldb key value functions from ltdb_* to ldb_kv_*. The renaming
is preparation for the separation of the tdb specific code from the key
value code. This work is a follow on from the addition of the lmdb
backend.
Note that the next commit tidies up the code formatting.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
there are also mem leaks in this function
Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13475
Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 15 23:07:25 CEST 2018 on sn-devel-144
After a duplicated add a small amount of memory can be leaked onto a
long-term context.
Found by Andrej Gessel https://github.com/andigese8fb45125e (commitcomment-29334102)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13471
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 15 04:39:41 CEST 2018 on sn-devel-144
Introduced in e8cdacc509
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13459
Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 1 11:10:24 CEST 2018 on sn-devel-144
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
When no search filter is specified, the code falls back to using
'(|(objectClass=*)(distinguishedName=*)'. ltdb_index_dn() then failed
because matching against '*' is not indexed. The error return then
caused the code to fallback to a full-scan of the DB, which could have a
considerable performance hit.
Instead, we want to continue on and do the ltdb_index_filter() over the
indexed results that were returned.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448
Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Otherwise Samba modules like subtree_rename can fail as they modify the
index during the callback.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13452
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This ensures we do not corrupt such an index by making changes to the
main database without knowing that the index values are now in a
sub-database.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Otherwise we rely on the caller doing tdb_reopen_all() which should
not be their job.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We may relax this restriction in the future, but for now do not assume
that the caller has done a tdb_reopen_all() at the right time.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
This will ensure we match LMDB behaviour and avoid a repeat of the per-record locking
issues (compared with full DB locking) we had before Samba 4.7.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This avoids keeping a counter, which can be error-prone.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
The ldb_cache startup code would previously not take a read lock
nor a sufficiently wide write transaction.
The new code takes a read lock, and if it needs to write takes a
write lock (transaction) and re-reads before continuing.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
We need to enforce the GUID index mode so end-users do not get a supprise
in mid-operation and we enforce a max key length of 511 so that the
index key trunctation is done correctly.
Otherwise the DB will appear to work until a very long key (DN or index)
is used, after which it will be sad.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This avoids magic numbers and also is careful against overflow
from a long attr_for_dn.
This is done as a distinct commit to make the previous behaviour
change more clear, and to show that this does not change the
calculations, only improves the overflow check.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
The re-index traverse can abort part-way though and we need to ensure
that the transaction is never committed as that will leave an un-useable db.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13335
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
