IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
* copy the ldb_options passed to ldb_connect onto the ldb_context,
making them more generally available.
* fix index buffering.
As a performance enhancement the indexes are cached in memory during a
transaction, and written to disk as part of the prepare commit. The
indexes could become corrupt in the event of a failed operation.
* fix read beyond buffer
Calling the "ldb_parse_tree" function with a filter consisting of
exactly a single space (" ") would trigger a read beyond the input
buffer.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13900
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Copy the options supplied to to ldb_connect, and place them on the
ldb_context. This allows backend options i.e. lmbd map size to be passed
cleanly from the callers.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fixes:
lib/ldb/common/ldb.c:1091:3: warning: Value stored to 'ret' is never read <--[clang]
ret = 0;
^ ~
1 warning generated.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
This came about because in py2 we had to check for strings and unicode.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Mon Jun 24 18:48:53 UTC 2019 on sn-devel-184
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
We no longer need Samba to be py2/py3 compatible so we choose to return to the standard
function names.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <noel.power@suse.com>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jun 21 05:45:02 UTC 2019 on sn-devel-184
As a performance enhancement the key value layer maintains a cache of
the index records, which is written to disk as part of a prepare commit.
This patch adds an extra cache at the operation layer to ensure that the
cached indexes remain consistent in the event of an operation failing.
Add test to test for index corruption in a failed modify.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
The callers will soon have two possible parents for this pointer, so
we need to remove this check, which was added out of caution given
the rather strange pattern of putting an active memory pointer into a
TDB (as a hash map).
That is, the only callers that did call this with "true" would
have to call this with "false", so just remove the complexity.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add test exercising the sub/nested transactions wrapping the key value
operations.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Use the nested transaction support added to the key value back ends to
make key value operations atomic. This will ensure that rename
operation failures, which delete the original record and add a new
record, leave the database in a consistent state.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add limited nested transaction support to the back ends to make the key value
operations atomic (for those back ends that support nested transactions).
Note: that only the lmdb backend currently supports nested transactions.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Remove the "DeprecationWarning: Please use assertEqual instead."
warnings from api.py
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add a test to document that ldb does not currently support nested
transactions.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_search.c: scope_hint: In
function ‘search_func’ lib/ldb/ldb_key_value/ldb_kv_search.c:308:47:
warning: unused parameter ‘ldb_kv’ [-Wunused-parameter]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Jun 13 08:27:12 UTC 2019 on sn-devel-184
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_index_add1’
lib/ldb/ldb_key_value/ldb_kv_index.c:2601:15: warning: comparison of
integer expressions of different signedness: ‘int’ and ‘unsigned int’
[-Wsign-compare]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_index_traverse_store’
lib/ldb/ldb_key_value/ldb_kv_index.c:779:60: warning: unused parameter
‘tdb’ [-Wunused-parameter]
By tagging the tdb_context parameter in ldb_kv_index_traverse_store as
_UNUSED_
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes csbuild error.
Error: COMPILER_WARNING:
lib/ldb/ldb_key_value/ldb_kv_index.c: scope_hint: In function
‘ldb_kv_key_dn_from_idx’
lib/ldb/ldb_key_value/ldb_kv_index.c:545:15: warning: comparison of
integer expressions of different signedness: ‘int’ and ‘unsigned int’
[-Wsign-compare]
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Add lib/util/attr.h to the ldb distribution, to make the _UNUSED_ macro
available.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Fixes:
lib/tevent/tevent_wrapper.c:295:3: warning: Access to field 'next' results in a dereference of a null pointer (loaded from field 'prev') <--[clang]
Additionally fix similar instance of the same macro
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Gary Lockyer gary@catalyst.net.nz
Calling the "ldb_parse_tree" function with a filter consisting of
exactly a single space (" ") would trigger a read beyond the input
buffer. A unittest is included.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13900
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
* Default pack format changed to version 2 for GUID Indexed databases
(essentially Samba's sam.ldb). This means faster unpacking for records
with lots of attributes.
* Enforce link between all new database features, following toggling of
GUID indexing. If user toggles GUID indexing on/off, target pack format
is changed. Likewise the special ORDERED_INTEGER index format is used
(when GUID indexing on) or behaves like INTEGER (when GUID indexing off).
* Database repacked if packing format not as expected.
* In the case of MDB, since GUID indexing is mandatory, feature toggling
provided by pack_format_override LDB option.
* Check for errors from ldb_unpack_data() in ldb_tdb
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 29 05:56:59 UTC 2019 on sn-devel-184
For TDB databases, toggling GUID indexing mode will also toggle
pack format version 2. This provides a convenient downgrade path for
Samba databases, but the process doesn't work for MDB databases because
GUID indexing cannot be disabled when the MDB backend is used. This patch
addresses that corner case by providing support for a pack_format_override
option which will force the database to use pack format version 2.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Python test to confirm that after enabling or disabling GUID indexing,
the database is repacked on the next transaction with V1 if GUID
indexing was disabled, or V2 if it was enabled.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
VERY IMPORTANT PATCH
Now that we have a new packing format, we need to enable it by repacking
the database. We've decided to link all new database features together,
so once GUID indexing is enabled, the database will be repacked with
version 2 format. Repacking is done following the same iterate pattern as
reindexing.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Only enable ORDERED_INTEGER and index_format_fn functionality if GUID
indexing is enabled. This is in line with the rest of ldb_kv which binds
the new ORDERED_INTEGER to GUID indexed databases, and allows a practical
way to create the old index format (by disabling the GUID index).
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
We need to check for the errors given by ldb_unpack() et al by preserving
the error code from kv_ctx->parser() called by tdb_parse_record() in
ltdb_parse_record().
Otherwise we will silently accept corrupt records and segfault later.
Likewise new pack formats will confuse the parser but not be
detected except by the incomplete struct ldb_message.
With this patch, the user will see a message like:
Invalid data for index DN=@BASEINFO
Failed to connect to 'st/ad_dc/private/sam.ldb' with backend 'tdb': Unable to load ltdb cache records for backend 'ldb_tdb backend'
Failed to connect to st/ad_dc/private/sam.ldb - Unable to load ltdb cache records for backend 'ldb_tdb backend'
This can be refined in the future by a specific check for
pack format versions in a higher caller, but this much is
needed regardless to detect corrupt records.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Parse errors aren't passed up correctly by the tdb backend. This
patch modifies a test to expose the issue, next patch will fix it.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Ensure that the referrals returned in a search request use the same
scheme as the request, i.e. referrals recieved via ldap are prefixed
with "ldap://" and those over ldaps are prefixed with "ldaps://"
BUG: https://bugzilla.samba.org/show_bug.cgi?id=12478
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 24 05:12:14 UTC 2019 on sn-devel-184
For testing we need to know the actual KV level key of records and each
record's pack format version. This patch makes ldbdump add comments with
that info. We will parse it out in python tests.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 22 05:58:17 UTC 2019 on sn-devel-184
In future commits we'll be adding more logging to LDB, which breaks the
ldb_key_value_test suite. By removing the debug handler, a bug
involving an expired debug_string variable being written to is avoided.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
We want to reuse the reindex context struct for repacking, but it has an
unnecessary module pointer on it. Turns out the existing code doesn't
need it either, so this patch deletes the pointer.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* Checking pack format is version 1 and erroring if not (will change soon)
* Pack format routines for unpack and pack version 2 (but not used)
* Test fixes for issues caused by upcoming repack functionality for upgrade
* Making ldbdump print out pack format info and keys so we have low level visibility for testing in python
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Pack function for new pack format with values separated from other data
so that while unpacking, the value section (which is probably large)
doesn't have to be loaded into cache/memory.
The new format is disabled for now.
Two tests are added that operate on a detailed binary breakdown of the
new format.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Unpack function for new pack format with values separated from other
data so that while unpacking, the value section (which is probably large)
doesn't have to be loaded into cache/memory.
Additionally, width of length field can now vary per-element to save space.
The old unpack routine is still present and is called if the old pack
format version number is found.
LDB torture suite is modified to run relevant tests on both old and new
pack format.
NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Since we're about to introduce a new packing format, it's a good time to
improve our code style and change some magic numbers into explicit
constants.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
