1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-07 17:18:11 +03:00
Commit Graph

921 Commits

Author SHA1 Message Date
Aaron Haslett
73763acf49 ldb: repack old format database if GUID indexing enabled
VERY IMPORTANT PATCH
Now that we have a new packing format, we need to enable it by repacking
the database. We've decided to link all new database features together,
so once GUID indexing is enabled, the database will be repacked with
version 2 format. Repacking is done following the same iterate pattern as
reindexing.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-05-29 04:41:24 +00:00
Aaron Haslett
6005c8cbad ldb: only used a->syntax->index_format_fn if GUID indexing is enabled
Only enable ORDERED_INTEGER and index_format_fn functionality if GUID
indexing is enabled.  This is in line with the rest of ldb_kv which binds
the new ORDERED_INTEGER to GUID indexed databases, and allows a practical
way to create the old index format (by disabling the GUID index).

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
2019-05-29 04:41:24 +00:00
Andrew Bartlett
a3101b9704 ldb: Fix segfault parsing new pack formats
We need to check for the errors given by ldb_unpack() et al by preserving
the error code from kv_ctx->parser() called by tdb_parse_record() in
ltdb_parse_record().

Otherwise we will silently accept corrupt records and segfault later.

Likewise new pack formats will confuse the parser but not be
detected except by the incomplete struct ldb_message.

With this patch, the user will see a message like:

 Invalid data for index  DN=@BASEINFO

 Failed to connect to 'st/ad_dc/private/sam.ldb' with backend 'tdb': Unable to load ltdb cache records for backend 'ldb_tdb backend'
 Failed to connect to st/ad_dc/private/sam.ldb - Unable to load ltdb cache records for backend 'ldb_tdb backend'

This can be refined in the future by a specific check for
pack format versions in a higher caller, but this much is
needed regardless to detect corrupt records.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-05-29 04:41:24 +00:00
Aaron Haslett
2de0aebed6 ldb: test for parse errors
Parse errors aren't passed up correctly by the tdb backend. This
patch modifies a test to expose the issue, next patch will fix it.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13959

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-05-29 04:41:24 +00:00
Gary Lockyer
1958cd8a7f ldap server: generate correct referral schemes
Ensure that the referrals returned in a search request use the same
scheme as the request, i.e. referrals recieved via ldap are prefixed
with "ldap://" and those over ldaps are prefixed with "ldaps://"

BUG: https://bugzilla.samba.org/show_bug.cgi?id=12478

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri May 24 05:12:14 UTC 2019 on sn-devel-184
2019-05-24 05:12:14 +00:00
Aaron Haslett
a666a99e4d ldb: ldbdump key and pack format version comments
For testing we need to know the actual KV level key of records and each
record's pack format version. This patch makes ldbdump add comments with
that info. We will parse it out in python tests.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed May 22 05:58:17 UTC 2019 on sn-devel-184
2019-05-22 05:58:17 +00:00
Aaron Haslett
365838311b ldb: ldb_key_value_test fix
In future commits we'll be adding more logging to LDB, which breaks the
ldb_key_value_test suite. By removing the debug handler, a bug
involving an expired debug_string variable being written to is avoided.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:29 +00:00
Aaron Haslett
64cdd0383f ldb: removing unnecessary module pointer
We want to reuse the reindex context struct for repacking, but it has an
unnecessary module pointer on it. Turns out the existing code doesn't
need it either, so this patch deletes the pointer.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:29 +00:00
Aaron Haslett
bea24253f0 ldb: Release ldb 2.0.2
* Checking pack format is version 1 and erroring if not (will change soon)
 * Pack format routines for unpack and pack version 2 (but not used)
 * Test fixes for issues caused by upcoming repack functionality for upgrade
 * Making ldbdump print out pack format info and keys so we have low level visibility for testing in python

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:29 +00:00
Aaron Haslett
8c0724fa10 ldb: pack function for new pack format
Pack function for new pack format with values separated from other data
so that while unpacking, the value section (which is probably large)
doesn't have to be loaded into cache/memory.
The new format is disabled for now.
Two tests are added that operate on a detailed binary breakdown of the
new format.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:28 +00:00
Aaron Haslett
38feff0731 ldb: unpack function for new pack format
Unpack function for new pack format with values separated from other
data so that while unpacking, the value section (which is probably large)
doesn't have to be loaded into cache/memory.
Additionally, width of length field can now vary per-element to save space.
The old unpack routine is still present and is called if the old pack
format version number is found.
LDB torture suite is modified to run relevant tests on both old and new
pack format.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:28 +00:00
Aaron Haslett
5bf6f0ae32 ldb: replacing length increments with constants in pack
Since we're about to introduce a new packing format, it's a good time to
improve our code style and change some magic numbers into explicit
constants.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:28 +00:00
Aaron Haslett
df1f883204 ldb: push and pull macros for pack format
Replacing push and pull functions (which may cause issues with Undefined
Sanitizer) with Andreas Schneider's excellent macros which are a work in
progress and not yet merged into master. Once his work is upstream, I'll
rebase and change this code to import his headers.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:28 +00:00
Aaron Haslett
474e555232 ldb: baseinfo pack format check on init
We will be adding a new packing format in forthcoming commits and there
may be more versions in the future. We need to make sure the database
contains records in a format we know how to read and write.
Done by fetching the @BASEINFO record and reading the first 4
bytes which contain the packing format version.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-22 04:42:28 +00:00
Fabrice Fontaine
519d72c578 Fix uClibc build on 64bit platforms by including stdint.h
Fixes an error detected by buildroot autobuilders:
http://autobuild.buildroot.net/results/573/573e2268e205e10d1352fa81122d8f225fdb4575/build-end.log

/home/rclinux/rc-buildroot-test/scripts/instance-1/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27:
error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../lib/ldb/tests/ldb_msg.c:17:0:
../third_party/cmocka/cmocka.h:126:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;

The define __WORDSIZE is missing when cmocka.h decides how to
define uintptr_t, this patch includes stdint.h when needed.

Patch sent upstream:
https://lists.samba.org/archive/samba-technical/2018-January/125306.html

[updated for samba-4.8.4, v2 sent upstream
 https://lists.samba.org/archive/samba-technical/2018-August/129732.html,
 updated for samba-4.9.1]

Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
[Retrieved from:
https://git.buildroot.net/buildroot/tree/package/samba4/0003-Fix-uClibc-build-on-64bit-platforms-by-including-std.patch]
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Ralph Böhme <slow@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2019-05-19 17:34:20 +00:00
Noel Power
27d99eefe7 lib/ldb: Fix incorrect return type for (setter) func type
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13948

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-05-16 17:55:17 +00:00
Noel Power
51f146de5a lib/ldb: squash 'cast between incompatible function types' warning
To avoid warning above produced by using
-Wcast-function-type we;

  + ensure PyCFunctions of type METH_NOARGS defined dummy arg
  + ensure PyCFunctions of type METH_KEYWORDS use PY_DISCARD_FUNC_SIG
    macro

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-05-16 17:55:17 +00:00
Andrew Bartlett
85b6f71c65 ldb: Release ldb 2.0.1
* Rework pack routines to remove unused features:
  - ldb_unpack_data_only_attr_list_flags() is removed
  - LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC is now always implied

  This improves the unpack performace significantly.

* Improve search performance via new internal flag
  LDB_UNPACK_DATA_FLAG_READ_LOCKED which removes a memdup

* Improve search performance during full scan by ignoring
  index records early.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:37 +00:00
Aaron Haslett
15f0c4969a ldb: remove unpack only attr list functionality
Unpack functions currently take an attribute list to restrict the set of
attributes to be returned in the constructed message. This
functionality is never used and complicates implementation of
forthcoming new pack format. This patch removes that functionality.
Using the unpack function then filtering the result turns
out not to be any slower.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:37 +00:00
Andrew Bartlett
9310d9088d ldb: move ldb_kv's filter into pack code
This patch moves ldb_kv's filter code into the pack code to replace
'only attr list' functionality which will be removed in forthcoming
commit. Unpacking data then filtering the result is not any slower
than the removed 'only attr list' approach.
'only attr list' test repurposed to test unpack -> filter flow.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:37 +00:00
Aaron Haslett
9f9bbb5aac ldb: removing msg and dn copying from filter attrs
Optimising filter_attrs by removing msg and dn allocation/copying. The
caller can construct the msg and possibly steal the dn.
Also giving the function an ldb for future use.

NOTE: Configure with --abi-check-disable to build this commit. This
patch is part of a set of LDB ABI changes, and the version update is
done on the last commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:36 +00:00
Aaron Haslett
ea7fd52a78 ldb: removing alloc from unpack_data
Making unpack flag LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC required
behaviour, since allocating data during unpack is slow and unnecessary
in all current usages. In any future unpack usage, if editing of
returned memory is required, some function that duplicates the message
should be used, such as one of the filter_attrs functions, or msg_copy.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:36 +00:00
Andrew Bartlett
73bf2949e8 ldb_kv: Use ldb_msg_add_steal_value() in msg_add_distinguished_name()
This more modern routine allocates a nice talloc tree.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:36 +00:00
Andrew Bartlett
94989e4e99 ldb_kv: Remove incorrect reference to LDB_UNPACK_DATA_FLAG_NO_DN
The DN is now exploded so as to improve other aspects of the search handling.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-15 04:03:36 +00:00
Douglas Bagnall
06a02cb88c ldb_msg: remove_element() checks element array bounds
Previously we half-heartedly checked one end.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:18 +00:00
Douglas Bagnall
652a4015e6 ldb_kv: use ldb_msg_remove_element()
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:18 +00:00
Douglas Bagnall
85b6674aeb ldb/tools/ldbtest: initialise msg object
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:18 +00:00
Douglas Bagnall
ce2bf5c72b ldb_ldap: use ldb_msg API to add elements
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:18 +00:00
Douglas Bagnall
2a144ea70e ldb_ldap: trust db_msg_new() to return empty message
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:18 +00:00
Douglas Bagnall
8f081b7f78 ldb: use ldb_msg_new(), not talloc/talloc_zero
ldb_msg_new() is currently the same as talloc_zero(), but it might
not always be.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:17 +00:00
Douglas Bagnall
18966b6eee kdb_kv_search: spell 'linearized'
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:17 +00:00
Douglas Bagnall
985a4b7871 ldb.h: spelling of 'means'
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:17 +00:00
Douglas Bagnall
c2e882b25f ldb.h: improve comment for LDB_ATTR_FLAG_INDEXED
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-05-10 01:15:17 +00:00
Douglas Bagnall
47f2264c5e ldb modules: paged_search checks control is not NULL (CID 241355)
It is unlikely to be NULL, since we're in the callback.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-09 22:39:27 +00:00
Douglas Bagnall
bfcfae72a5 ldb_mdb: check fcntl return values (CID 1435851)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-09 22:39:27 +00:00
Douglas Bagnall
3de7f430db ldb_map: check a return value (CID 241354)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-09 22:39:27 +00:00
Douglas Bagnall
5dc8603883 ldb_ldif: avoid NULL dereference with unexpected arguments (CID 1107195)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-09 22:39:27 +00:00
Douglas Bagnall
aa18f62a8a ldb: avoid NULL deref in ldb_dn_from_ldb_val (CID 1034730)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-05-09 22:39:27 +00:00
Douglas Bagnall
76967b33eb pyldb: avoid segfault when adding an element with no name
We don't want to see this:

python3 -c "import sys
sys.path.insert(0, 'bin/python')
import ldb
m = ldb.Message()
e = ldb.MessageElement('q')
try:
    m.add(e)
except ldb.LdbError:
    pass
print(m)
"
Segmentation fault (core dumped)

instead we want this:

Traceback (most recent call last):
File "<string>", line 7, in <module>
ValueError: The element has no name

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>

Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Tue Apr 23 19:03:35 UTC 2019 on sn-devel-144
2019-04-23 19:03:35 +00:00
Andrew Bartlett
075a078d36 ldb: Avoid calling talloc_get_type() in ldb_kv_parse_data_unpack()
We have the ldb_kv in the caller, just fill it into the context and
so avoid the cost of the talloc_get_type().

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-04-12 04:38:14 +00:00
Garming Sam
74091115f7 tests/ldb_kv: Add another case for completeness
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Apr 11 05:25:02 UTC 2019 on sn-devel-144
2019-04-11 05:25:02 +00:00
Garming Sam
a76d286537 ldb_kv: Avoid memdup of database records in the case of base searches
This makes LDAP bind significantly faster in the case of having many
members, due to large size of these records (with tens of thousands of
member links). During the nested group calculation, you are only
interested in memberOf not the member links.

(We add a bit-field to determine whether or not the backend actually
supports pointing into database memory. For some reason TDB pointers
aren't stable, so for now we set this option just on LMDB backends.)

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-11 04:17:11 +00:00
Garming Sam
d6b1d55659 ldb_mdb: Change function declaration as per README.coding
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-11 04:17:11 +00:00
Garming Sam
65303bf55d ldb_kv: Remove unnecessary space
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-11 04:17:11 +00:00
Andrew Bartlett
49b77d8df2 ldb_kv: Skip @ records early in a search full scan
@ records like @IDXLIST are only available via a base search on the specific name
but the method by which they were excluded was expensive, after the unpack the
DN is exploded and ldb_match_msg_error() would reject it for failing to match the
scope.

This uses the fact that @ records have the DN=@ prefix on their TDB/LMDB key
to quickly exclude them from consideration.

Based on analysis by Garming Sam.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13893

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 10 06:23:39 UTC 2019 on sn-devel-144
2019-04-10 06:23:39 +00:00
Aaron Haslett
15d1ecdca6 ldb: version 2.0.0
* Version bump for adding index_format_fn to the schema syntax structure.
* Range index support added, allowing <= and >= operations to be indexed
* Improved reindex performance by setting the in-memory TDB hash size correctly

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-04-08 02:07:23 +00:00
Garming Sam
9d0b003623 ldb_mdb: Add some warnings about poorly constructed callbacks
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:23 +00:00
Garming Sam
2e05fd785a ldb: tests for <= and >= integer indexing with duplicates
We need to make sure that duplicates are correctly returned (uSNChanged
for instance is UNIQUE but, we should be able to index on attributes
which are not unique).

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:23 +00:00
Aaron Haslett
18438c8af2 ldb: tests for <= and >= integer indexing
Testing max, min and negative values for indexed 32 and 64 bit types.
This has to be done in two different files because the 64 bit type is
LDB_SYNTAX_INTEGER which is implemented at the ldb level, while the 32
bit is added in the ldb-samba module.  Schema syntax binding added for
ldb-samba.

We also need to make sure that full scans are not invoked for LMDB.

Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:23 +00:00
Garming Sam
6062d7241d ldb: Add ORDERED_INTEGER to the proto-schema handling
Adding ordered integer proto schema handling in kv index cache.  This
allows ordered 64 bit integers to be used in cached fields like
@ATTRIBUTES

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:23 +00:00
Garming Sam
db584d5081 schema_syntax: Add comments for our index format functions
We had to devise our own scheme for writing integers in a human readable
format which also sorted correctly numerically. This might look a bit
confusing to outsiders, so here's a large comment as a peace offering.

Pair-programmed-with: Tim Beale <timbeale@catalyst.net.nz>

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:23 +00:00
Aaron Haslett
c9b2a37268 ldb: activating <= and >= indexing for integers
Activating <= and >= mdb indexing in samba for int32 and int64 attributes by:
1. Adding index_format_fn to LDB_SYNTAX_SAMBA_INT32 in ldb_samba
2. Cloning the 64bit LDB_SYNTAX_INTEGER type as LDB_SYNTAX_ORDERED_INTEGER
3. Adding index_format_fn to the new type
4. Modifying LargeInteger use the new type in samba schema
5. Bumping the index version to trigger reindexing

Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:23 +00:00
Garming Sam
f775606516 ldb_kv_index: Add a giant comment in regards to index_format_fn
The reason we needed it in the first place was that the original
canonicalize is being used for non-index functions and it never produced
the right order originally (at least for integers).

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:22 +00:00
Garming Sam
a894515229 ldb_kv_index: Make the edge keys slightly cleaner and generic
It makes no difference in our standard case because \0 will always go
before any value for our index_format_fn, but this is better for
correctness (in case we do mess up our NUL terminations elsewhere).

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:22 +00:00
Aaron Haslett
9b3021b8d6 ldb: <= and >= indexed searching
Full implementation of <= and >= indexed searching using iterate_range
backend operation.  Adds index_format_fn to ldb_schema_syntax so
requires an ABI version bump.  The function must be provided for any
type for which <= and >= indexing is required, and must return a
lexicographically ordered canonicalization of a value.  This causes
index entries to be written in correct order to the database, so
iterate_range on the index DNs can be used.

ldb_kv_index_key is modified to return an index DN with attribute name
but without value if an empty value is provided.  This is needed for
constructing keys that match the beginning or end of an index DN range.

Pair-programmed-with: Garming Sam <garming@catalyst.net.nz>

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:22 +00:00
Aaron Haslett
1b5df44331 lmdb: iterate_range cmocka testing
Cmocka testing for LMDB iterate_range operation added in previous commit.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-04-08 02:07:22 +00:00
Garming Sam
e4ea408f52 lmdb: iterate_range implementation
Adding iterate_range to LDB API and implementing in LMDB.  This
operation takes a start_key and end_key and returns all records between
the two, inclusive of both.  This will be used to implementing indexing
for <= and >= expressions.

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-08 02:07:22 +00:00
Douglas Bagnall
2da9d7d130 ldb_kv_search: avoid handling uninitialised dn
If ldb_kv_filter_attrs() fails, we don't know that the dn of filtered_msg
is OK.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Apr  5 05:46:55 UTC 2019 on sn-devel-144
2019-04-05 05:46:55 +00:00
Gary Lockyer
69408654b7 lib ldb key_value: Set index cache size on open
Set the default index cache from the passed option
"transaction_index_cache_size" on open.  This allows the default cache
size to be overridden when processing large transactions i.e. joining a
large domain.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-04 06:40:17 +00:00
Gary Lockyer
0952f98e41 lib ldb key_value: set the cache size for re-indexing
Set the index cache size to the number of records in the databse when
reindexing.

This significantly improves reindex performance.  For a domain with
100,000 users the reindex times are reduced from 17 minutes to 45
seconds.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-04 06:40:17 +00:00
Gary Lockyer
6129a05ca0 lib ldb key_value: Add get_size method
Add the get_size method to the ldb_key_value layer, this will allow the
reindexing code to get an estimate of the number of records in the
database.

The lmdb backend returns an accurate count of the number of records in
the database withe the mdb_env_stat call.

The tdb backend does not provide a low cost method to determine the
number of records on the database.  It does provide a tdb_summary call
however this this walks the entire database.

So for tdb we use the map size divided by 500, this over estimates the counts
for small domains, but the extra memory allocated for the cache should
not be significant.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-04 06:40:17 +00:00
Gary Lockyer
8f7bf13b96 lib ldb key_value: Pass index cache size
Pass the index cache size to ldb_kv_index_transaction_start.  This will
allow it to be set for reindex and join operations, where the current
defaults result in a significant performance penalty on large databases.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-04 06:40:16 +00:00
Gary Lockyer
267cc71f5d lib ldb key_value: Remove index cache lazy initialisation
Remove the lazy initialisation of the index cache. This make setting
the size of the cache for re-indexing easier, which will be done in
later commits.

Performance testing shows that the removal of lazy initialisation makes
no appreciable difference to performance.

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-04-04 06:40:16 +00:00
Mathieu Parent
959a4837b5 Spelling fixes s/verson/version/
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-04-02 01:12:11 +00:00
Mathieu Parent
882d8a5bf4 Spelling fixes s/conficts/conflicts/
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-04-02 01:12:11 +00:00
Mathieu Parent
6064543d64 Spelling fixes s/overrided/overridden/
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Martin Schwenke <martin@meltin.net>
2019-04-02 01:12:11 +00:00
Mathieu Parent
af3253013a Fix tests whithout lmdb
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13630

Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-03-26 03:03:23 +00:00
Andrew Bartlett
81648d576d ldb: Release ldb 1.6.3
* Remove Python 2.x support except to build just the bare C library
* Remove --extra-python (a build time mode to produce Python2 and Python3
  bindings at the same time)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 21 05:08:49 UTC 2019 on sn-devel-144
2019-03-21 05:08:49 +00:00
Andrew Bartlett
a574e8f517 build: Standardise on calling conf.SAMBA_CHECK_PYTHON() in libraries
We do this by removing the confusing mandatory option to
conf.SAMBA_CHECK_PYTHON{,_HEADERS}(), instead just use the value of
--disable-python internally

This follows the default minimum of Python 3.4 and keeps things consistent
with the main Samba build where --disable-python is required to skip building
python bindings.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-03-21 04:06:16 +00:00
Andrew Bartlett
24127acae0 ABI: Remove unused .py3*.sigs files
These are no longer used by the build system so avoid
confusion by removing them from the tree.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-03-21 04:06:15 +00:00
Andrew Bartlett
fdb6305c86 build: Remove bld.gen_python_environments()
This was part of --extra-python support.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-03-21 04:06:14 +00:00
Andrew Bartlett
270d04e4ea build: Remove --extra-python
This option is quite invasive in waf and was mainly for the python3 transition.

Testing with multiple python versions can be done by testing a full compile against
multiple versions, likewise multiple different binding versions can be created
the same way.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-03-21 04:06:14 +00:00
Mathieu Parent
c059b8cb82 Enable make test even without lmdb
Bug: https://bugzilla.samba.org/show_bug.cgi?id=13630

Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-03-06 03:16:14 +00:00
Stefan Metzmacher
09d281d69b ldb: version 1.6.2
* Fix standalone build of ldb.

Signed-off-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Tue Feb 26 12:10:40 CET 2019 on sn-devel-144
2019-02-26 12:10:40 +01:00
David Mulder
5ed5c33764 Search for location of waf script
When calling make from the ldb, talloc, tdb, and
tevent bundles, we need to first find the
location of the waf script. Currently the build
fails since it can't find waf.

Fixes regression caused by a660b7f.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-26 10:39:24 +01:00
Gary Lockyer
de3bb5cd52 CVE-2019-3824 ldb: Release ldb 1.6.1
* CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>

Autobuild-User(master): Gary Lockyer <gary@samba.org>
Autobuild-Date(master): Mon Feb 25 22:54:13 CET 2019 on sn-devel-144
2019-02-25 22:54:13 +01:00
Gary Lockyer
45b75db50f CVE-2019-3824 ldb: Add tests for ldb_wildcard_match
Add cmocka tests for ldb_wildcard_match.

Running test_wildcard_match under valgrind reproduces
 CVE-2019-3824 out of bounds read in wildcard compare (bug 13773)

 valgrind --suppressions=lib/ldb/tests/ldb_match_test.valgrind\
          bin/ldb_match_test

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-25 21:26:12 +01:00
Gary Lockyer
42f0f57eb8 CVE-2019-3824 ldb: wildcard_match end of data check
ldb_handler_copy and ldb_val_dup over allocate by one and add a trailing '\0'
to the data, to make them safe to use the C string functions on.

However testing for the trailing '\0' is not the correct way to test for
the end of a value, the length should be checked instead.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-25 21:26:12 +01:00
Gary Lockyer
34383981a0 CVE-2019-3824 ldb: wildcard_match check tree operation
Check the operation type of the passed parse tree, and return
LDB_INAPPROPRIATE_MATCH if the operation is not LDB_OP_SUBSTRING.

A query of "attribute=*" gets parsed as LDB_OP_PRESENT, checking the
operation and failing ldb_wildcard_match should help prevent confusion
writing tests.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-25 21:26:11 +01:00
Gary Lockyer
8d34d17209 CVE-2019-3824 ldb: ldb_parse_tree use talloc_zero
Initialise the created ldb_parse_tree with talloc_zero, this ensures
that it is correctly initialised if inadvertently passed to a function
expecting a different operation type.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-25 21:26:11 +01:00
Andrew Bartlett
9427806f72 CVE-2019-3824 ldb: Improve code style and layout in wildcard processing
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-02-25 21:26:11 +01:00
Andrew Bartlett
745b99fc6b CVE-2019-3824 ldb: Extra comments to clarify no pointer wrap in wildcard processing
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-02-25 21:26:11 +01:00
Lukas Slebodnik
3674b0891a CVE-2019-3824 ldb: Out of bound read in ldb_wildcard_compare
There is valgrind error in few tests tests/test-generic.sh
 91 echo "Test wildcard match"
 92 $VALGRIND ldbadd $LDBDIR/tests/test-wildcard.ldif  || exit 1
 93 $VALGRIND ldbsearch '(cn=test*multi)'  || exit 1
 95 $VALGRIND ldbsearch '(cn=*test_multi)'  || exit 1
 97 $VALGRIND ldbsearch '(cn=test*multi*test*multi)'  || exit 1

e.g.
  ==3098== Memcheck, a memory error detector
  ==3098== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
  ==3098== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
  ==3098== Command: ./bin/ldbsearch (cn=test*multi)
  ==3098==
  ==3098== Invalid read of size 1
  ==3098==    at 0x483CEE7: memchr (vg_replace_strmem.c:890)
  ==3098==    by 0x49A9073: memmem (in /usr/lib64/libc-2.28.9000.so)
  ==3098==    by 0x485DFE9: ldb_wildcard_compare (ldb_match.c:313)
  ==3098==    by 0x485DFE9: ldb_match_substring (ldb_match.c:360)
  ==3098==    by 0x485DFE9: ldb_match_message (ldb_match.c:572)
  ==3098==    by 0x558F8FA: search_func (ldb_kv_search.c:549)
  ==3098==    by 0x48C78CA: ??? (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x48C7A60: tdb_traverse_read (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x557B7C4: ltdb_traverse_fn (ldb_tdb.c:274)
  ==3098==    by 0x558FBFA: ldb_kv_search_full (ldb_kv_search.c:594)
  ==3098==    by 0x558FBFA: ldb_kv_search (ldb_kv_search.c:854)
  ==3098==    by 0x558E497: ldb_kv_callback (ldb_kv.c:1713)
  ==3098==    by 0x48FCD58: tevent_common_invoke_timer_handler (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==    by 0x48FCEFD: tevent_common_loop_timer_delay (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==    by 0x48FE14A: ??? (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==  Address 0x4b4ab81 is 0 bytes after a block of size 129 alloc'd
  ==3098==    at 0x483880B: malloc (vg_replace_malloc.c:309)
  ==3098==    by 0x491048B: talloc_strndup (in /usr/lib64/libtalloc.so.2.1.15)
  ==3098==    by 0x48593CA: ldb_casefold_default (ldb_utf8.c:59)
  ==3098==    by 0x485F68D: ldb_handler_fold (attrib_handlers.c:64)
  ==3098==    by 0x485DB88: ldb_wildcard_compare (ldb_match.c:257)
  ==3098==    by 0x485DB88: ldb_match_substring (ldb_match.c:360)
  ==3098==    by 0x485DB88: ldb_match_message (ldb_match.c:572)
  ==3098==    by 0x558F8FA: search_func (ldb_kv_search.c:549)
  ==3098==    by 0x48C78CA: ??? (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x48C7A60: tdb_traverse_read (in /usr/lib64/libtdb.so.1.3.17)
  ==3098==    by 0x557B7C4: ltdb_traverse_fn (ldb_tdb.c:274)
  ==3098==    by 0x558FBFA: ldb_kv_search_full (ldb_kv_search.c:594)
  ==3098==    by 0x558FBFA: ldb_kv_search (ldb_kv_search.c:854)
  ==3098==    by 0x558E497: ldb_kv_callback (ldb_kv.c:1713)
  ==3098==    by 0x48FCD58: tevent_common_invoke_timer_handler (in /usr/lib64/libtevent.so.0.9.38)
  ==3098==
  # record 1
  dn: cn=test_multi_test_multi_test_multi,o=University of Michigan,c=TEST
  cn: test_multi_test_multi_test_multi
  description: test multi wildcards matching
  objectclass: person
  sn: multi_test
  name: test_multi_test_multi_test_multi
  distinguishedName: cn=test_multi_test_multi_test_multi,o=University of Michiga
   n,c=TEST

  # returned 1 records
  # 1 entries
  # 0 referrals

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13773

Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
2019-02-25 21:26:11 +01:00
Noel Power
0c15c4b1db Make sure results from GetAttrString are decref'ed where needed
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett abartlet@samba.org

Autobuild-User(master): Noel Power <npower@samba.org>
Autobuild-Date(master): Wed Feb 13 14:51:12 CET 2019 on sn-devel-144
2019-02-13 14:51:12 +01:00
Douglas Bagnall
d21801b888 ldb_dn: don't free a known NULL pointer
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-13 04:15:15 +01:00
Douglas Bagnall
d4ebe00688 ldb_dn: remove unreachable code in dn_explode
Every time I look at this file, I spend a few minutes wondering how
these bits of code are ever run. Never again.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-13 04:15:14 +01:00
Noel Power
2814690d8f Cleanup (decref) some objects added to list.
PyList_Append doesn't steal references, so if the item created is
a temp object, created just to be added to the list we need to
 decref the item appended in order for it to be released.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-02-07 13:44:30 +01:00
Noel Power
1be9b0cf1b Examine result of SetList (and prevent sending NULL to PyList_SetItem)
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-02-07 13:44:30 +01:00
Noel Power
a8e10a1249 Decrement references to python objects passed to Py_BuildValue
Py_BuildValue when processing format 'O' will
  'Pass a Python object untouched (except for its reference count,
   which is incremented by one'

Basically this means if you are using a new reference to a PyObject
to pass to BuildValue (to be used with the 'O' format) the reference
*isn't* stolen so you really do need to DECREF it in order to ensure
it gets cleaned up.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-02-07 13:44:30 +01:00
Andrew Bartlett
545c3e6b12 ldb: Release ldb 1.6.0
* pyldb: make ldb.connect() url mandatory
* New version number for master (Samba 4.11 eventually)

The 1.5.x series will be maintained in the v4-10-test branch

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Feb  1 07:02:56 CET 2019 on sn-devel-144
2019-02-01 07:02:56 +01:00
Douglas Bagnall
0e419a2163 pyldb: make ldb.connect() url mandatory
The call fails without it, so we might as well fail sooner

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-01 03:36:18 +01:00
Andrew Bartlett
5e716c0256 ldb: Release ldb 1.5.3
* Avoid inefficient one-level searches (bug 13762)
* The test api.py should not rely on order of entries in dict (bug 13772)

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13772

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-02-01 03:36:15 +01:00
Andrew Bartlett
e7f524fd21 ldb: Add even more comments on what strict does to the list intersections
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2019-02-01 03:36:15 +01:00
Tim Beale
57a565b2fd ldb: Rename variable
The old name confused me because it's not really related to the
one-level index at all. It's the result from evaluating the indexed
search specified in the ac->tree.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-01 03:36:15 +01:00
Tim Beale
132600685b ldb: Elaborate on ldb_kv_search_indexed() comments
Disclaimer: this is based on my limited understanding of what the code
is doing.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-01 03:36:15 +01:00
Tim Beale
7292844482 ldb: Remove comment that no longer makes sense
This comment was written before the GUID_index_attribute block of code
existed. So we now *do* load the index values and *do* check for a
strict intersect, so the comment is redundant.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-01 03:36:15 +01:00
Tim Beale
9a893f9613 ldb: Avoid inefficient one-level searches
Commit 88ae60ed18 introduced a problem that made one-level
searches inefficient if there were a lot of child objects in the same
level, and the requested object didn't exist. Basically, it ignored the
case where ldb_kv_index_dn() returned LDB_ERR_NO_SUCH_OBJECT, i.e. the
indexed lookup was successful, but didn't find a match. At which point,
there was no more processing we needed to do.

The behaviour after 88ae60ed18 was to fall-through and run the
ldb_kv_index_filter() function over *all* the children. This still
returned the correct result, but could be costly if there were a lot of
children.

The case 88ae60ed18 was trying to fix was where we could not do
an indexed search (e.g. trying to match on a 'attribute=*' filter). In
which case we want to ignore the LDB_ERR_OPERATIONS_ERROR and just run
ldb_kv_index_filter() over all the children. This is still more
efficient than the fallback of doing a full database scan.

This patch adds in a short-circuit for the NO_SUCH_OBJECT case, so we
can skip the unnecessary ldb_kv_index_filter() work.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13762

Signed-off-by: Tim Beale <timbeale@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2019-02-01 03:36:15 +01:00
Lukas Slebodnik
7a7a5ccf50 ldb: The test api.py should not rely on order of entries in dict
Test failed on s390x but there is a simple reproducer for any
architecture.

The built-in function repr returns the canonical string representation
of the object. We needn't care about order attributes in string
representation. Therefore test should pass for any order.

    for i in {1..30}; do
        PYTHONHASHSEED=random \
        python2 -c 'import ldb; msg = ldb.Message(); msg.dn = ldb.Dn(ldb.Ldb(), "dc=foo29"); msg["dc"] = b"foo"; print(repr(msg)) '
    done

  ======================================================================
  FAIL: test_repr (__main__.LdbMsgTests)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "tests/python/api.py", line 2322, in test_repr
      "Message({'dn': Dn('dc=foo29'), 'dc': MessageElement(['foo'])})")
  AssertionError: "Message({'dc': MessageElement(['foo']), 'dn': Dn('dc=foo29')})" != "Message({'dn': Dn('dc=foo29'), 'dc': MessageElement(['foo'])})"
  ----------------------------------------------------------------------
  Ran 1025 tests in 29.146s
  FAILED (failures=1)

Signed-off-by: Lukas Slebodnik <lslebodn@fedoraproject.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
2019-01-29 22:09:07 +01:00
Andreas Schneider
146ed0894e lib:ldb: Use correct C99 initializer for 'struct tm'
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-01-28 10:29:21 +01:00
Andreas Schneider
3cd5d4b397 lib:ldb: Use C99 initializer for tdb_logging_context
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-01-28 10:29:21 +01:00
Andreas Schneider
85a5dc56e3 lib:ldb: Use C99 initializer for PyGetSetDef in pyldb
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-01-28 10:29:20 +01:00
Stefan Metzmacher
340cb9ca97 ldb: version 1.5.2
* Build fixes
* dirsync: Allow arbitrary length cookies
  (bug #13686)
* The build uses python3 by default:
* --extra-python would take python2 now
* To build with python2 only use:
  PYTHON=python2 ./configure
  PYTHON=python2 make
  PYTHON=python2 make install

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Karolin Seeger <kseeger@samba.org>
2019-01-15 11:29:56 +01:00
Noel Power
8900e0b4cb lib/ldb: Use new PYARG_ES format for parseTuple
While 'es' format works great for unicode (in python2) and
str (in python3) The behaviour with str (in python2) is unexpected.
In python2 the str type is (re-encoded) with the specified encoding.
In python2 the 'et' type would be a better match, that ensures 'str'
type is treated like it was with 's' (no reencoding) and unicode is
encoded with the specified encoding. However in python3 'et' allows
byte (or bytearray) params to be accepted (with no reencoding), we
don't want this. This patch adds a new PYARG_STR_UNI format code which
is a hybrid, in python2 it evaluates to 'et' and in python3 'es' and
so gives the desired behaviour for each python version.

Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sun Jan 13 03:53:00 CET 2019 on sn-devel-144
2019-01-13 03:53:00 +01:00
Noel Power
f8758b3b1f lib/ldb/tests/python: Add test to pass utf8 encoded bytes to ldb.Dn
This test should demonstrate an error with the 'es' format in python
where a 'str' byte-string is passed (containing utf8 encoded bytes)
with some characters that cannot be decoded as ascii. The same
code if run in python3 should generate an error (needs string not
bytes)

Also Add knownfail for ldb.Dn passed utf8 encoded byte string
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2019-01-13 00:40:26 +01:00
Aaron Haslett
975807001e paged results: new paged results module using GUID list
Replacing paged results module to use GUID list instead of storing
result list in memory, in order to improve memory performance.

Signed-off-by: Aaron Haslett <aaronhaslett@catalyst.net.nz>
Reviewed-by: Gary Lockyer <gary@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-21 07:21:19 +01:00
Andreas Schneider
7a35a6874a waf: Add missing libreplace deps to pyldb*
This will require memset_s() because of a later commit moving
ZERO_STRUCT to use memset_s().

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-20 12:16:39 +01:00
Joe Guo
a07279b78a PY3: change shebang to python3 in lib dir
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-12-14 14:40:20 +01:00
Andrew Bartlett
5ddff307b4 build: Move python detection back into waf (instead of in configure and Makefile)
This avoids creating a mini-configure in the configure script.

Users wishing to use python2 to build need to specify PYTHON=
to both ./configure and make

After we merged the python3 change, it became clear that relying on systems prefixing
the correct python just causes trouble and make debugging harder, so only use $PYTHON
for the override, not the default case

This essentially reverts a660b7fb8e but
leaves the files more consistent.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Noel Power <npower@samba.org>
2018-12-14 14:40:19 +01:00
Andrew Bartlett
19a4d3ca69 build: Workaround python3 hash order issues (for now)
This works around python3 having a new hash seed each time it starts to allow
a second "make" not to rebuild the world.

This should probably be reverted once we find the hash that is causing
the issue, but should reduce frustration for now.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-12-13 01:49:30 +01:00
Noel Power
a660b7fb8e PY3: switch current build to use python3
Make sure default make and configure for all now defaults
to building with python3.

To build a samba (or sub component e.g. talloc etc.) with python3
  ./configure && make

To build a samba (or sub component e.g. talloc etc.) with python2
  PYTHON=python ./configure && PYTHON=python make

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:25 +01:00
Noel Power
447276e297 adjust ldb config for py3 build
waf code is making assumptions that main build is py2 and extra build
is py3, this results in wrong library names being used.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-12-10 10:38:24 +01:00
Garming Sam
ad8bb6fcd0 ldb_controls: Add some talloc error checking for controls
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13686

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:16 +01:00
Garming Sam
b7a0d3b110 dirsync: Allow arbitrary length cookies
The length of the cookie is proportional to the number of DCs ever in
the domain (as it stores the uptodateness vector which has stale
invocationID).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13686

Signed-off-by: Garming Sam <garming@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-11-20 22:14:16 +01:00
Andreas Schneider
0595129b8e libldb: Fix const char in pyldb
../lib/ldb/pyldb.c: In function ‘PyDict_AsMessage’:
../lib/ldb/pyldb.c:90:22: error: initialization discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
 #define PyStr_AsUTF8 PyUnicode_AsUTF8
                      ^~~~~~~~~~~~~~~~
../lib/ldb/pyldb.c:1359:19: note: in expansion of macro ‘PyStr_AsUTF8’
   char *key_str = PyStr_AsUTF8(key);
                   ^~~~~~~~~~~~
../lib/ldb/pyldb.c: In function ‘py_ldb_msg_getitem_helper’:
../lib/ldb/pyldb.c:3336:7: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
  name = PyStr_AsUTF8(py_name);
       ^
../lib/ldb/pyldb.c: In function ‘py_ldb_msg_setitem’:
../lib/ldb/pyldb.c:3502:12: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
  attr_name = PyStr_AsUTF8(name);
            ^

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Uri Simchoni <uri@samba.org>
2018-11-14 08:45:19 +01:00
Jeremy Allison
698a9ef36e lib: ldb: Remove use of talloc_autofree_context().
Just use NULL in test case. talloc_autofree_context() is deprecated.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Amitay Isaacs <amitay@gmail.com>
2018-11-06 07:16:15 +01:00
Douglas Bagnall
78ead841bf ldb/tests/py/index: remove dup value in dictionary
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:11 +01:00
Douglas Bagnall
fc64f2586d ldb/tests/py/api: use proper name for ldb.LdbError
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <noel.power@suse.com>
2018-11-01 05:08:11 +01:00
Douglas Bagnall
d2fadfd6e4 ldb/tests/py/api: reveal shadowed casefold test
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:59 +02:00
Douglas Bagnall
eef11c4479 ldb/tests/_ldb_text: remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Noel Power <npower@samba.org>
2018-10-25 21:45:55 +02:00
Douglas Bagnall
a0bad1364a ldb_ldif: avoid strlen(NULL)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Martin Schwenke <martin@meltin.net>

Autobuild-User(master): Martin Schwenke <martins@samba.org>
Autobuild-Date(master): Fri Oct 19 03:43:58 CEST 2018 on sn-devel-144
2018-10-19 03:43:58 +02:00
Douglas Bagnall
ce518f387f ldb_ldif: be less horribly efficient in debugging
perf said all the time was in strlen.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Thu Oct 18 13:17:30 CEST 2018 on sn-devel-144
2018-10-18 13:17:30 +02:00
Noel Power
cddd54e865 lib/ldb: Ensure ldb.Dn can accept utf8 encoded unicode
Additionally remove the associated known fail.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13616
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-27 01:54:26 +02:00
Noel Power
d1492ab919 lib/ldb/tests: add test for ldb.Dn passed utf8 unicode
object dn format should be a utf8 encoded string
Note: Currently this fails in python2 as the c python binding for
      the dn string param uses PyArg_ParseTupleAndKeywords() with 's'
      format, this will accept str *or* unicode in the default encoding.
      The default encoding in python2 is... ascii.

Also adding here a knownfail to squash the error produced by the test.

Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-27 01:54:26 +02:00
Noel Power
d786e1fca9 lib/ldb: Test correct variable for no mem condition
Signed-off-by: Noel Power <noel.power@suse.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-27 01:54:26 +02:00
Amitay Isaacs
3b94d31d24 ldb: Fix dependency of ldb module
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13626
Signed-off-by: Amitay Isaacs <amitay@gmail.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-21 20:04:22 +02:00
Joe Guo
8c2c9794ec PEP8: fix E128: continuation line under-indented for visual indent
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Sep  6 15:50:17 CEST 2018 on sn-devel-144
2018-09-06 15:50:17 +02:00
Alexander Bokovoy
0c703f1d3a lib/ldb/wscript: update to handle waf 2.0.4
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2018-09-05 06:37:22 +02:00
Bernd Kuhls
7c89edfe54 Fix uClibc build on 64bit platforms by including stdint.h
Fixes an error detected by buildroot autobuilders:
http://autobuild.buildroot.net/results/573/573e2268e205e10d1352fa81122d8f225fdb4575/build-end.log

/home/rclinux/rc-buildroot-test/scripts/instance-1/output/host/mips64el-buildroot-linux-uclibc/sysroot/usr/include/stdint.h:122:27:
error: conflicting types for 'uintptr_t'
 typedef unsigned long int uintptr_t;
                           ^
In file included from ../lib/ldb/tests/ldb_msg.c:17:0:
../third_party/cmocka/cmocka.h:126:28: note: previous declaration of 'uintptr_t' was here
       typedef unsigned int uintptr_t;

The define __WORDSIZE is missing when cmocka.h decides how to
define uintptr_t, this patch includes stdint.h when needed.

Patch sent upstream:
https://lists.samba.org/archive/samba-technical/2018-January/125306.html

Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Aug 24 17:22:10 CEST 2018 on sn-devel-144
2018-08-24 17:22:09 +02:00
Joe Guo
7b031b01a1 PEP8: fix E502: the backslash is redundant between brackets
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
4d529c400b PEP8: fix E306: expected 1 blank line before a nested definition, found 0
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
115f2a71b8 PEP8: fix E305: expected 2 blank lines after class or function definition, found 1
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
fd6b2086cb PEP8: fix E303: too many blank lines (2)
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:30 +02:00
Joe Guo
211c9a5f85 PEP8: fix E302: expected 2 blank lines, found 1
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:29 +02:00
Joe Guo
542e91ef92 PEP8: fix E301: expected 1 blank line, found 0
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:29 +02:00
Joe Guo
12d3fbe15c PEP8: fix E231: missing whitespace after ','
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:28 +02:00
Joe Guo
87bbc2df97 PEP8: fix E226: missing whitespace around arithmetic operator
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:28 +02:00
Joe Guo
ba0827b560 PEP8: fix E203: whitespace before ':'
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
5d532543ab PEP8: fix E128: continuation line under-indented for visual indent
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:27 +02:00
Joe Guo
bbb9f57603 PEP8: fix E127: continuation line over-indented for visual indent
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:26 +02:00
Joe Guo
dba0c7ebe5 PEP8: fix E124: closing bracket does not match visual indentation
Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-24 07:49:26 +02:00
Andrew Bartlett
bdbb9422c0 ldb: Release LDB 1.5.1
* New API ldb_dn_add_child_val() avoids passing untrusted input to
  ldb_dn_add_child_fmt() (bug 13466)
* Free memory nearer to the allocation in calls made by ldbsearch
* Do not overwrite ldb_transaction_commit failure error messages
  with a pointless del_transaction()

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:24 +02:00
Andrew Bartlett
9d46795b20 ldb: extend API tests
These additional API tests just check that an invalid base DN
is never accepted.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:24 +02:00
Andrew Bartlett
2dafbd3213 ldb: Add new function ldb_dn_add_child_val()
This is safer for untrusted input than ldb_dn_add_child_fmt()

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13466

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:24 +02:00
Andrew Bartlett
542e7c1791 ldb_tdb: Remove pointless check of ldb_dn_is_valid()
If the DN is not valid the ltdb_search_dn1() will catch it with ldb_dn_validate() which
is the only safe way to check this.  ldb_dn_is_valid() does not actually check, but instead
returns only the result of the previous checks, if there was one.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:24 +02:00
Andrej Gessel
d71c655eec fix mem leak in ldbsearch
Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:24 +02:00
Andrej Gessel
d645546aef fix mem leak in ltdb_index_dn_base_dn and ltdb_search_indexed
Signed-off-by: Andrej Gessel <Andrej.Gessel@janztec.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:24 +02:00
Joe Guo
b9e2a2de24 ldb: no need to call del_transaction in ldb_transaction_commit
No matter commit succeeded or failed, transation will be delete afterwards.
So there is no need to delete it here.

Aganst Samba this causes an `LDAP error 51 LDAP_BUSY` error when the transaction
fails, say while we try to add users to groups in large amount and
the original error is lost.

In Samba, the rootdse module fails early in the del part of the
start/end/del pattern, and in ldb_tdb and ldb_mdb a failed commit
always ends the transaction, even on failure.

Signed-off-by: Joe Guo <joeg@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-15 07:08:23 +02:00
Timur I. Bakeyev
e4f38b0670 ldb tests: fix assertion on wrong pointer
We are allocating msg02, but check in assertion msg01, which makes no
sense here.

Signed-off-by: Timur I. Bakeyev <timur@freebsd.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
2018-08-14 19:08:25 +02:00
Andrew Bartlett
b7f0ee93f5 Release LDB 1.5.0 for CVE-2018-1140
* Security fix for CVE-2018-1140 (NULL pointer de-reference, bug 13374)
* Fix memory leaks and missing error checks (bug 13459, 13471, 13475)

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
2018-08-14 13:57:15 +02:00
Andrew Bartlett
b6b72d0063 CVE-2018-1140 ldb: Add tests for search add and rename with a bad dn= DN
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13374
2018-08-14 13:57:15 +02:00