IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Fix indentation of list members and fix lines > 79 characters to remove
PEP8 warnings.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add cmocka tests to confirm lmdb's handling of the free list.
As a result of lmdb's MVCC (Multiversion Concurrency Control) long
running read transactions or stale readers (read transactions where the
process exited without ending the transaction) can cause the database to
run out of space.
Items in the free list are only reused when they would not be visible in
a read transaction. So long running read transactions prevent entries
in the free list being reused, and the database can run out of space.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Feb 26 23:11:39 UTC 2020 on sn-devel-184
* Bug 14270:
Samba 4.11 and later give incorrect results for SCOPE_ONE searches
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(master): Tue Feb 25 12:59:02 UTC 2020 on sn-devel-184
Thankfully this only fails if the DB is corrupt and has a duplicate record.
The test was at the wrong end of the loop, and was for the
wrong boundary condition. A write after the end of the array would
occour before the condition was hit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13695
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Wed Jul 10 05:48:52 UTC 2019 on sn-devel-184
* add ldb_options_get
add a function to get the options passed in ldb connect.
* add "batch_mode" option.
This options stops sub transactions being started for key value
operations. It is intended to improve the performance in batch
operations. As it bypasses the protections on operations if an
operation fails, the entire transaction will be aborted by a commit.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_tdb can be a module, but the test is actually looking for ltdb_err_map() in
ldb_tdb_err_map.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* copy the ldb_options passed to ldb_connect onto the ldb_context,
making them more generally available.
* fix index buffering.
As a performance enhancement the indexes are cached in memory during a
transaction, and written to disk as part of the prepare commit. The
indexes could become corrupt in the event of a failed operation.
* fix read beyond buffer
Calling the "ldb_parse_tree" function with a filter consisting of
exactly a single space (" ") would trigger a read beyond the input
buffer.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13900
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add test exercising the sub/nested transactions wrapping the key value
operations.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add lib/util/attr.h to the ldb distribution, to make the _UNUSED_ macro
available.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Calling the "ldb_parse_tree" function with a filter consisting of
exactly a single space (" ") would trigger a read beyond the input
buffer. A unittest is included.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13900
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Tim Beale <timbeale@catalyst.net.nz>
* Default pack format changed to version 2 for GUID Indexed databases
(essentially Samba's sam.ldb). This means faster unpacking for records
with lots of attributes.
* Enforce link between all new database features, following toggling of
GUID indexing. If user toggles GUID indexing on/off, target pack format
is changed. Likewise the special ORDERED_INTEGER index format is used
(when GUID indexing on) or behaves like INTEGER (when GUID indexing off).
* Database repacked if packing format not as expected.
* In the case of MDB, since GUID indexing is mandatory, feature toggling
provided by pack_format_override LDB option.
* Check for errors from ldb_unpack_data() in ldb_tdb
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 29 05:56:59 UTC 2019 on sn-devel-184
Python test to confirm that after enabling or disabling GUID indexing,
the database is repacked on the next transaction with V1 if GUID
indexing was disabled, or V2 if it was enabled.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* Checking pack format is version 1 and erroring if not (will change soon)
* Pack format routines for unpack and pack version 2 (but not used)
* Test fixes for issues caused by upcoming repack functionality for upgrade
* Making ldbdump print out pack format info and keys so we have low level visibility for testing in python
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* Rework pack routines to remove unused features:
- ldb_unpack_data_only_attr_list_flags() is removed
- LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC is now always implied
This improves the unpack performace significantly.
* Improve search performance via new internal flag
LDB_UNPACK_DATA_FLAG_READ_LOCKED which removes a memdup
* Improve search performance during full scan by ignoring
index records early.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* Version bump for adding index_format_fn to the schema syntax structure.
* Range index support added, allowing <= and >= operations to be indexed
* Improved reindex performance by setting the in-memory TDB hash size correctly
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Add the get_size method to the ldb_key_value layer, this will allow the
reindexing code to get an estimate of the number of records in the
database.
The lmdb backend returns an accurate count of the number of records in
the database withe the mdb_env_stat call.
The tdb backend does not provide a low cost method to determine the
number of records on the database. It does provide a tdb_summary call
however this this walks the entire database.
So for tdb we use the map size divided by 500, this over estimates the counts
for small domains, but the extra memory allocated for the cache should
not be significant.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pass the index cache size to ldb_kv_index_transaction_start. This will
allow it to be set for reindex and join operations, where the current
defaults result in a significant performance penalty on large databases.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Remove Python 2.x support except to build just the bare C library
* Remove --extra-python (a build time mode to produce Python2 and Python3
bindings at the same time)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 21 05:08:49 UTC 2019 on sn-devel-144
We do this by removing the confusing mandatory option to
conf.SAMBA_CHECK_PYTHON{,_HEADERS}(), instead just use the value of
--disable-python internally
This follows the default minimum of Python 3.4 and keeps things consistent
with the main Samba build where --disable-python is required to skip building
python bindings.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This option is quite invasive in waf and was mainly for the python3 transition.
Testing with multiple python versions can be done by testing a full compile against
multiple versions, likewise multiple different binding versions can be created
the same way.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
* Fix standalone build of ldb.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Feb 26 12:10:40 CET 2019 on sn-devel-144
* CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Mon Feb 25 22:54:13 CET 2019 on sn-devel-144
Add cmocka tests for ldb_wildcard_match.
Running test_wildcard_match under valgrind reproduces
CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)
valgrind --suppressions=lib/ldb/tests/ldb_match_test.valgrind\
bin/ldb_match_test
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* pyldb: make ldb.connect() url mandatory
* New version number for master (Samba 4.11 eventually)
The 1.5.x series will be maintained in the v4-10-test branch
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Feb 1 07:02:56 CET 2019 on sn-devel-144
* Build fixes
* dirsync: Allow arbitrary length cookies
(bug #13686)
* The build uses python3 by default:
* --extra-python would take python2 now
* To build with python2 only use:
PYTHON=python2 ./configure
PYTHON=python2 make
PYTHON=python2 make install
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Karolin Seeger <kseeger@samba.org>
Replacing paged results module to use GUID list instead of storing
result list in memory, in order to improve memory performance.
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This will require memset_s() because of a later commit moving
ZERO_STRUCT to use memset_s().
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
waf code is making assumptions that main build is py2 and extra build
is py3, this results in wrong library names being used.
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* New API ldb_dn_add_child_val() avoids passing untrusted input to
ldb_dn_add_child_fmt() (bug 13466)
* Free memory nearer to the allocation in calls made by ldbsearch
* Do not overwrite ldb_transaction_commit failure error messages
with a pointless del_transaction()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This is safer for untrusted input than ldb_dn_add_child_fmt()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Move the key value code to a separate subdirectory.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* add some missing return value checks
* Fix several mem leaks in ldb_index ldb_search ldb_tdb (bug#13475)
* ldb_tdb: Use mem_ctx and so avoid leak onto long-term memory
on duplicated add. (bug#13471)
* ldb: Fix memory leak on module context (bug#13459)
* Refused build of Samba 4.8 with ldb 1.4 (bug #13519)
* Prevent similar issues in the future at configure time (bug #13519)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Jul 12 07:43:22 CEST 2018 on sn-devel-144
Samba is not compatible with new versions of ldb (except release versions)
Other users would not notice the breakages, but Samba makes many
more assuptions about the LDB internals than any other package.
(Specifically, LDB 1.2 and 1.4 broke builds against released
Samba versions)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13519
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
* New LMDB backend (experimental)
* Comprehensive tests for index behaviour
* Enforce transactions for writes
* Enforce read lock use for all reads
* Fix memory leak in paged_results module.
We hold at most 10 outstanding paged result cookies
(bug #13362)
* Fix compiler warnings
* Python3 improvements
* Restore --disable-python build
* Fix for performance regression on one-level searches
(bug #13448)
* Samba's subtree_rename could fail to rename some entries
(bug #13452)
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
The way to know if we are in a transaction is if there is a non-NULL
transaction handle.
This allows the ldb_mdb_kv_ops_test test to be run.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
ldb_mdb is now able to pass the full ldb_mod_op_test when compiled against lmdb.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Tests to ensure that the mdb_env wrapping code correctly handles
multiple ldb's point to the same physical database file.
The test_ldb_close_with_multiple_connections tests are in
ldb_mod_op_test due to the utility code it uses from
elsewhere in that test.
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
We need to show that despite the internal cache of TDB pointers that it
is safe to open a ldb_tdb after a fork()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
