1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-26 10:04:02 +03:00

563 Commits

Author SHA1 Message Date
Garming Sam
c9f2ff275b ldb_tdb: Use key value ops for fetch command
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
68423e9f4a ldb_tdb: factor out the (to be) common init code
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Gary Lockyer
141148edd3 ldb_tdb: Add errorstr to the key value ops
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
c66a0054fb ldb_tdb: Remove tdb_get_seqnum and use a generic 'has_changed'
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
e21a476c27 ldb_tdb: Add lock_read and unlock_read to key value ops
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
448101a3fd ldb_tdb: Replace tdb transaction code with generic key value ones
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
e33fb2c633 ldb_tdb: Replace exists, name and error_map with key value ops
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
9c8f00aaba ldb_tdb: Begin abstracting out the base key value operations
This will allow us to change the backend from tdb to lmdb.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Gary Lockyer
c4c64ff51f ldb_mod_op_test: Fix core dump on ldb_case_attrs_index_test_teardown
With no schema syntax, this would occasionally crash as it dereferenced
some possibly NULL sequence of memory.

Note: Removing all tests except this one, made it crash reliably.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:14 +01:00
Andreas Schneider
db4d72d21c ldb: Directly return an error and do not fall through
Detected by -Wimplicit-fallthrough.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlet <abartlet@samba.org>
2018-03-05 16:01:17 +01:00
Gary Lockyer
4c0c888b57 ldb_tdb: Add tests for truncated index keys
Tests for the index truncation code as well as the GUID index
format in general.

Covers truncation of both the DN and equality search keys.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Mar  3 09:58:40 CET 2018 on sn-devel-144
2018-03-03 09:58:40 +01:00
Gary Lockyer
9a4fd23407 ldb_tdb: Combine identical not GUID index and special DN cases
Fold together two identical cases to simplify the code.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Gary Lockyer
ee74d4bb80 ldb_tdb: Refuse to store a value in a unique index that is too long
Rather than add many special cases, over-long unique values are simply banned.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Gary Lockyer
1d86a0895a ldb_tdb: Do not give the warning of duplicate attributes in truncation
In the truncation case a duplicate is perfectly expected.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Gary Lockyer
0918068997 ldb_tdb: Cope with key truncation
Modify the indexing code to handle a maximum key length, index keys
greater than the maximum length will be truncated to the maximum length.
And the unuque index code has been altered to handle multiple records
for the same index key.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Gary Lockyer
5c1504b94d ldb_tdb: Do not fail in GUID index mode if there is a duplicate attribute
It is not the job of the index code to enforce this, but do give a
a warning given it has been detected.

However, now that we do allow it, we must never return the same
object twice to the caller, so filter for it in ltdb_index_filter().

The GUID list is sorted, which makes this cheap to handle, thankfully.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Gary Lockyer
0bfbcdb691 ldb_tdb: Add support for an option to restrict the key length
Allow the setting of the maximum key length, this allows the testing of
index key truncation code.  Index key truncation is required to allow
the samba indexing scheme to be used with backends that enforce a
maximum key length.

This will allow emulation of a length-limited key DB for testing.

This is a testing-only feature, as the index format changes
based on this value.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Andreas Schneider
7adf497443 lib:ldb: Add FALL_THROUGH statements in ldb_map/ldb_map_outbound.c
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-01 04:37:41 +01:00
Andreas Schneider
9d2296e070 lib:ldb: Add FALL_THROUGH statements in ldb_map/ldb_map.c 2018-03-01 04:37:41 +01:00
Andreas Schneider
3429e9d9e5 lib:ldb: Add FALL_THROUGH statements in ldb_map/ldb_map_inbound.c
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-01 04:37:41 +01:00
Andreas Schneider
2b01bd0475 lib:ldb: Add FALL_THROUGH statements in common/ldb_dn.c
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-01 04:37:41 +01:00
Stefan Metzmacher
cb58e18800 ldb: version 1.3.2
* Expose the SHOW_BINARY, ENABLE_TRACING and DONT_CREATE_DB flag constants
  in the python api.
* Extend dn.is_child_of() test.
* Don't load LDB_MODULESDIR as a module file.
* Fix binary data in debug log (bug #13185).

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Feb 28 04:54:21 CET 2018 on sn-devel-144
2018-02-28 04:54:21 +01:00
Gary Lockyer
2a85bcb3f4 ldb_debug tests: Fix binary data in debug log
Tests to ensure:
    When duplicate objects are added, the GUID was printed in the debug log
    are passed through the escape function.
    And that duplicate DN's do not generate debug log entries.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13185

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Feb 26 07:29:49 CET 2018 on sn-devel-144
2018-02-26 07:29:49 +01:00
Andrew Bartlett
c5a14306c8 ldb_debug: Fix binary data in debug log
When duplicate objects were added, the GUID was printed in the debug log
The GUID was not escaped and therefore displayed as binary content.

This patch splits out the duplicate DN creation error and the duplicate
GIUD error.  Duplicate DN's are a normal event and don't require debug
logging.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13185

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-02-26 02:32:09 +01:00
Gary Lockyer
1ed693423d ldb tests: fix null test on incorrect variable
Fix up tests that were  performing a null check on the wrong variable
after a call to ldb_msg_new

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Feb 24 15:50:35 CET 2018 on sn-devel-144
2018-02-24 15:50:35 +01:00
Timur I. Bakeyev
3450dd6a50 Don't load LDB_MODULESDIR as a module file
We are setting modules directory here(LDB_MODULESDIR) so treat it this
way, no need to attempt to load it as a module file.

Signed-off-by: Timur I. Bakeyev <timur@iXsystems.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
2018-02-19 07:36:23 +01:00
Björn Baumbach
cf338b8260 pyldb: extend dn.is_child_of() test: dn is child of itself
Add this test so ensure that this (unclear) behaviour does
not change silently.

Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-02-08 04:58:09 +01:00
Gary Lockyer
736e438bf4 pyldb: Expose extra flags
Expose the SHOW_BINARY, ENABLE_TRACING and DONT_CREATE_DB flag constants
in the python api.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-02-07 23:45:23 +01:00
Stefan Metzmacher
bf19b6ccdc ldb: version 1.3.1
* Intersect the index from SCOPE_ONELEVEL with the index for the search expression
  (bug #13191)
* smaller/greater comparison tests
* Show the last successful DN when failing to parse LDIF
* ldb_index: Add an attriubute flag to require a unique value.
* silence some clang warnings in picky developer mode

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2018-01-12 22:42:22 +01:00
Andrew Bartlett
ef240aaca0 ldb: Intersect the index from SCOPE_ONELEVEL with the index for the search expression
This helps ensure we do not have to scan all objects at this level
which could be very many (one per DNS zone entry).

However, due to the O(n*m) behaviour in list_intersect() for older
databases, we only do this in the GUID index mode, leaving the behaviour
unchanged for existing callers that do not specify the GUID index mode.

NOTE WELL: the behaviour of disallowDNFilter is enforced
in the index code, so this fixes SCOPE_ONELEVEL to also
honour disallowDNFilter, hence the additional tests.

The change to select the SUBTREE index in the absense of
the ONELEVEL index enforces this.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-12-20 04:22:09 +01:00
Matthias Dieter Wallnöfer
e72a8793ae LDB:test-generic.sh - fix smaller/greater comparison tests
The comparison result has been ignored, which is not good. Also remove
the "ldbsearch" command in the error branch which has not much sense.

The scripts needs to be run through test-tdb.sh, test-ldap.sh or
test-sqlite3.sh which I didn't realise before. Hence less changes are needed
and this is a reduced version of the patch published on the mailing list.

Signed-off-by: Matthias Dieter Wallnöfer <mdw@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Matthias Dieter Wallnöfer <mdw@samba.org>
Autobuild-Date(master): Tue Dec 19 03:09:12 CET 2017 on sn-devel-144
2017-12-19 03:09:12 +01:00
Andrew Bartlett
4ea7aa9265 ldb: Show the last successful DN when failing to parse LDIF
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-12-14 08:20:14 +01:00
Gary Lockyer
2599f29144 ldb ldb_index: Add an attriubute flag to require a unique value.
Add attribute flag LDB_FLAG_INTERNAL_UNIQUE_VALUE, to request that the
added attribute is unique on the index.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13004

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2017-12-10 00:47:29 +01:00
Uri Simchoni
3cca62a2ac ldb: silence some clang warnings in picky developer mode
Avoid const in casting since it doesn't increase code
safety in this case and causes clang to generate const-qual
warning. Also initialize a pointer to NULL to silence clang
uninitialized variable warning.

Signed-off-by: Uri Simchoni <uri@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2017-11-22 10:20:20 +01:00
Andrew Bartlett
c40531d9b9 ldb: Release ldb 1.3.0
* GUID Index support.

  NOTE: When activated by setting @IDXGUID in the @INDEXLIST dn, all
  entries in the DB are re-keyed in a way that is NOT visible to
  ldb 1.2.2 and earlier.  To re-key back to the previous format, remove
  the @IDXGUID attribute from @INDEXLIST using ldb 1.2.2 or later.

  (ldb 1.2.2 can re-key, but not otherwise read, the new DB format).

* Give LDB_ERR_CONSTRAINT_VIOLATION, not LDB_ERR_ENTRY_ALREADY_EXISTS
  when a duplicate value is detected in a unique index

* Print status information during a > 10,000 entry re-index
  (as this can be slow)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Sat Sep 23 01:24:19 CEST 2017 on sn-devel-144
2017-09-23 01:24:19 +02:00
Andrew Bartlett
303739e801 ldb_tdb: Treat distinguishedName and objectGUID (in Samba) as unique
This avoids loading any second index for these cases.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
644b3ffb14 ldb: Ensure we do not run out of File descriptors in autobuild
The python TestCase API will keep a reference to the test object until the end
of the tests, long after we need the actual LDB or the fd.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
a0b46ecdd2 ldb: Do not make an ldb file for API tests
An ldb context is valid without a backing file for tests of ldb.Message and ldb.MessageElement

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
8565ca1c6d ldb: Add test showing a search against the index is not possible
This is not actually a great test, as the filter would
fail to match these anyway, but it at least checks the
codepath is safe.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
f0624d3890 ldb_tdb: Also ban a (indexed) search against like @IDXDN=foo
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
f4f9acf781 ldb_tdb: Update comments for the delete_index() pass of ltdb_reindex()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:25 +02:00
Andrew Bartlett
61b66b8d0a ldb_tdb: Print progress messages on re-index
A re-index of 10,000 entries is slow enough and rare enought that we can
justify the message being at LDB_DEBUG_WARNING as otherwise the administrator
will be sure the "lockup" was one.

The default for ldb is to print LDB_DEBUG_WARNING in comand-line tools
and the default for Samba is to log it at level 2.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
c71ddab974 ldb_tdb: Remove incorrect early return from re-index
The ltdb->cache->attribute_indexes test is not correct with the GUID index mode
so for consistency remove it.  This will make re-index on a large un-indexed
database slower, but that is better than making the wrong choice on a large
GUID-indexed database.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
0503f0984a ldb: Add more tests covering in-transaction GUID index and unique index behaviour
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
00410ba99f ldb_tdb: Add UNIQUE_INDEX as a possible flag
This allows easy testing of our unique index code and behaivour from python

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
561ef20634 ldb_tdb: Remove LTDB_FLAG_HIDDEN and ignore "HIDDEN" in @ATTRIBUTES
This was (unintentionally) disabled by
6ef61825541131e16a03975cdbd344e2bbebf810 in 2006.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
29c303f21b ldb_tdb: Clean up index records on ltdb_index_add_new() failure.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
83bc607463 ldb_tdb: Describe index format and control points
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
3113d871bd ldb_tdb: Give a good error message on add without an objectGUID
(or whatever the @IDX_GUID value is)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
b9d561b023 ldb_tdb: Avoid canonicalise and base64 work for DN values, these are already OK
This is important with the GUID index, as a DN lookup is much more common now.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00