1
0
mirror of https://github.com/samba-team/samba.git synced 2025-02-25 17:57:42 +03:00

63 Commits

Author SHA1 Message Date
Gary Lockyer
19be0be232 lib ldb: move key value code to lib/ldb/ldb_key_value
Move the key value code to a separate subdirectory.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:54 +02:00
Gary Lockyer
9191d3baf5 lib ldb: remove unused function prototypes
Remove unused function prototypes

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:54 +02:00
Gary Lockyer
d3bfd374a6 lib ldb: rename ltdb_idxptr to ldb_kv_idxptr
Rename ltdb_idxptr to ldb_kv_idxptr as it's key value level and not tdb
specific.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:54 +02:00
Gary Lockyer
72724f75ce lib ldb: rename ltdb_cache to ldb_kv_cache
Rename ltdb_cache to ldb_kv_cache as it's key value level and not tdb
specific

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:53 +02:00
Gary Lockyer
b8c9c305b3 lib ldb: format rename ldb_kv_private
Tidy up the code format after the rename of ltdb_private to
ldb_kv_private

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:53 +02:00
Gary Lockyer
e0186d1f2d lib ldb: rename ltdb_private to ldb_kv_private
Rename ltdb_private to ldb_kv_private as it contains key value operation
context.

Note there is still some tdb specific context that can be refactored into a
separate structure along the lines of the lmdb context.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:53 +02:00
Gary Lockyer
17a84d12e2 lib ldb: rename ltdb_req_spy to ldb_kv_req_spy
Rename ltdb_req_spy to ldb_kv_req_spy, as it is key value level and not
tdb specific.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:53 +02:00
Gary Lockyer
c782d710e9 lib ldb: rename ltdb_context to ldb_kv_context
Rename ltdb_context to ldb_kv_context as it is a key value level
structure and not tdb specific.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:53 +02:00
Gary Lockyer
e8aa764ea7 lib ldb: rename struct ltdb_reindex_context
Rename struct ltdb_reindex_context to ldb_kv_reindex_context, as this is
a key value level structure and not a tdb specific structure.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:52 +02:00
Gary Lockyer
9e6294668a lib ldb: fix formatting of ldb_kv rename.
Clean up the code format after the rename in the previous commit.
Hopefully doing a rename commit followed by a reformat commit makes the
code easier to review.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:52 +02:00
Gary Lockyer
1c8ea099ce lib ldb: Rename functions to ldb_kv
Rename the ldb key value functions from ltdb_* to ldb_kv_*. The renaming
is preparation for the separation of the tdb specific code from the key
value code.  This work is a follow on from the addition of the lmdb
backend.

Note that the next commit tidies up the code formatting.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-07-30 14:31:52 +02:00
Andrew Bartlett
e99c199d81 ldb: Add tests for when we should expect a full scan
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13448

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2018-05-30 04:23:28 +02:00
Andrew Bartlett
cb5da7a5c6 ldb: Reject a possible future ldb_mdb with the index in a sub-database
This ensures we do not corrupt such an index by making changes to the
main database without knowing that the index values are now in a
sub-database.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2018-05-23 02:27:12 +02:00
Garming Sam
5ec491040c ldb_mdb: Implement the lmdb backend for ldb
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-05-23 02:27:10 +02:00
Andrew Bartlett
1174b52b91 ldb_tdb: Prevent ldb_tdb reuse after a fork()
We may relax this restriction in the future, but for now do not assume
that the caller has done a tdb_reopen_all() at the right time.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2018-05-09 04:29:48 +02:00
Gary Lockyer
f04bbd3ec3 ldb_tdb: Disallow TDB nested transactions and use tdb_transaction_active()
This avoids keeping a counter, which can be error-prone.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
2018-05-03 08:17:44 +02:00
Gary Lockyer
39e5faa77c ldb: make key/value backends expose if there is an active transaction
Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
2018-05-03 08:17:44 +02:00
Garming Sam
7bf853950d ldb: Change remaining fetch prototypes to remove TDB_DATA
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-04-06 02:08:44 +02:00
Garming Sam
7885181650 ldb: Change some prototypes to using ldb_val instead of TDB_DATA
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-04-06 02:08:44 +02:00
Andrew Bartlett
e481e4f30f ldb_tdb: Ensure we can not commit an index that is corrupt due to partial re-index
The re-index traverse can abort part-way though and we need to ensure
that the transaction is never committed as that will leave an un-useable db.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13335

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2018-04-05 04:50:16 +02:00
Garming Sam
4014499b12 ldb_tdb: Build a key value operation library
This allows sharing of the originally ldb_tdb operations to the new
ldb_mdb backend.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Mar  6 01:39:16 CET 2018 on sn-devel-144
2018-03-06 01:39:16 +01:00
Garming Sam
4d5180e45c ldb_tdb: Implement a traversal function in key value ops
This can handle both read-only and writable traverses.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
c9f2ff275b ldb_tdb: Use key value ops for fetch command
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
68423e9f4a ldb_tdb: factor out the (to be) common init code
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Gary Lockyer
141148edd3 ldb_tdb: Add errorstr to the key value ops
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
c66a0054fb ldb_tdb: Remove tdb_get_seqnum and use a generic 'has_changed'
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
e21a476c27 ldb_tdb: Add lock_read and unlock_read to key value ops
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
448101a3fd ldb_tdb: Replace tdb transaction code with generic key value ones
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Garming Sam
9c8f00aaba ldb_tdb: Begin abstracting out the base key value operations
This will allow us to change the backend from tdb to lmdb.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-05 20:50:15 +01:00
Gary Lockyer
0bfbcdb691 ldb_tdb: Add support for an option to restrict the key length
Allow the setting of the maximum key length, this allows the testing of
index key truncation code.  Index key truncation is required to allow
the samba indexing scheme to be used with backends that enforce a
maximum key length.

This will allow emulation of a length-limited key DB for testing.

This is a testing-only feature, as the index format changes
based on this value.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-03-03 05:20:35 +01:00
Andrew Bartlett
d198a43b55 ldb_tdb: Avoid allocation of a DN between the GUID index and the DB lookup
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
dfe85ecaca ldb_tdb: Use the DN extracted from the DB to filter the message later
This should ensure that the upper or lower case the user chooses does not impact
on the filtering, at least for database that have checkBaseOnSearch set.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:24 +02:00
Andrew Bartlett
e4562e58f9 ldb_tdb: Add ltdb_idx_to_key() and use it in ltdb_index_filter()
This will allow a common point to parse index records into a TDB key,
allowing them to be a GUID or DN in the future

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
6ad4bdc9de ldb_tdb: Add mem_ctx to ltdb_key_dn() and ltdb_key_msg()
This follows modern Samba coding style where memory
returned is allocated on a supplied memory context.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
e394b9a856 ldb_tdb: Add an index shortcut for a <GUID= DN
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
ccb9443664 ldb_tdb: Add a function to get the GUID key for a DN
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
f26d1a8c5d ldb_tdb: Add a function to take a GUID and make the TDB_DATA key
This allows us to format the TDB key as DN=GUID=f7c953ee-cf9c-433f-b423-21ce04d09591
and so be compatible with an un-indexed search and a re-index with an old ldb.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
aa68957f7f ldb_tdb: Load the syntax of the GUID index attr during ltdb_cache_load()
This allows us to use the ldif_write function later to create a string GUID for the TDB key.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
8555f6ccc2 ldb_tdb: add control points for the new GUID index mode
The @IDXGUID attribute in the @INDEXLIST will be objectGUID
in Samba.

The @IDX_DN_GUID attribute in the @INDEXLIST will be GUID
in Samba.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:22 +02:00
Andrew Bartlett
ec30439c43 ldb_tdb: Add/remove a GUID index of the DN during ltdb_index_add_all()/ltdb_index_delete()
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
b2aff12366 ldb_tdb: Prepare to handle rename with GUID index by using ltdb_search_base()
This will allow use of a GUID TDB key in the future.  While ltdb_search_base()
might be marginally slower than tdb_exists(), no allocation is done for the
attributes or DN, and renmaes are not a very common operation.

This allows a check if the target DN exists even when the direct DN -> key
link is broken.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
9d9ae1c005 ldb_tdb: Pass the full ldb_message to ldb index funtions
This allows the objectGUID, rather than the DN, to be the index key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
e8cdacc509 ldb_tdb: modify ltdb_delete_noindex() to take a struct ldb_message
This will make it easier to delete records with the GUID TDB key

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:21 +02:00
Andrew Bartlett
367b7ab748 ldb_tdb: Move constants into ldb_tdb.h
This helps ensure we keep these all in sync.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
26dd6f6365 ldb_tdb: Add GUID_index_attribute to ltdb_private
This will be used to determine if we are in GUID index mode

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
3d952157d7 ldb_tdb: Provide struct ltdb_private to index routines
This will make it easier to switch the GUID index mode on and off

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
6db8095835 ldb_tdb: Add ltdb_search_key()
This allows us to slowly split out the tdb key in the DB from being the DN

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
1b310ad99c ldb_tdb: provide ldb_key_dn() and ldb_key_msg()
This will in time allow us to generate a TDB key from
the msg, eg from an objectGUID.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2017-09-22 21:20:20 +02:00
Andrew Bartlett
22854f9b8d ldb_tdb: Change ltdb_connect() NOT to request a kernel-level read only TDB
We support opening and LDB multiple times in a process, but do not support this in tdb.

As we can open the ldb with different flags, we must ensure a later read-write
open is possible.

Additionally, a read-only TDB will refuse the all-record lock, preventing
the ldb from even loading.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13033

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-20 02:25:30 +02:00
Andrew Bartlett
fec666b334 ldb_tdb: Create a common ltdb_key_is_record() allowing multiple key forms
If backported, this allows old ldb versions to full-search and re-index newer databases
and in current code allows the transition to and from a GUID or incrementing ID based index

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13016

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2017-09-07 06:56:26 +02:00