IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
We should build with lmdb support also if it is not in AD case. The lmdb
backend is also used e.g. by sssd.
If you don't want to build it, you can always specify --without-ldb-lmdb
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15721
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Sep 25 05:36:13 UTC 2024 on atb-devel-224
(cherry picked from commit fdef894d79)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15643
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Sep 27 09:06:43 UTC 2024 on atb-devel-224
(cherry picked from commit 20a3a94e06)
Autobuild-User(v4-21-test): Jule Anger <janger@samba.org>
Autobuild-Date(v4-21-test): Wed Oct 2 09:28:09 UTC 2024 on atb-devel-224
It should be off by default, as this is not needed by default. It
crashes named on startup, if bind is built with jemalloc support.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15643
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
(cherry picked from commit dc6927fdca)
An accidental negation means that ldb_version.h is not installed when
ldb is built as a public library.
This is a regression introduced by commit
625fb48326.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15690
Signed-off-by: Jo Sutton <josutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sun Aug 4 01:35:55 UTC 2024 on atb-devel-224
(cherry picked from commit 5851ae5554)
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Sat Jul 27 23:51:44 UTC 2024 on atb-devel-224
Just in case, but also so ldb_kv_index_transaction_cancel() can use
this and retain the same logic.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We can't just free it, because has the GUID index list as a child, and
these are shared by the new dn list (from the subtransaction we are
committing). But if the dn list is long and the main transaction is
long-lived, we can save a lot of memory by turning this dn list into
an almost empty node in the talloc tree. This returns us to roughly
the situation we had prior to the last commit.
For example, with the repro.sh script on bug 15590 in indexes mode
with 10000 rules, The last 3 commits use this much memory at the end
of an unusually large transaction:
full talloc report on 'struct ldb_context' (total 4012222 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 2405482219 bytes in 90058 blocks)
full talloc report on 'struct ldb_context' (total 4282195 bytes in 90058 blocks)
That is, the last commit increased usage 500 fold, and this commit
brings it back to normal.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15590
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
We don't want to modify the original list, but we can reuse the keys
if we treat them as immutable and don't free them. That makes it a lot
quicker if there are many keys (i.e. where an index is useful) and may
sub-transactions. In particular, it avoids O(n²) talloc_memdups.
A removed comment that says "We have to free the top level index
memory otherwise we would leak", and this will be addressed in the
next commit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15590
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
The strict flag is only read in list intersection, so most of the time
it doesn't matter whether it is set because that path is not used.
Nevertheless seeing it set to all kinds of values is distracting.
The undefined behaviour has likely been hidden from static analysis
because the structure is passed through the in-memory tdb before use.
Incorrect true values will have disabled an optimisation but not
caused the wrong result.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This is done by adding a new API that avoids the problems of
ldb_dn_copy() and makes it clear that a struct ldb_context *
pointer will be stored in the new copy.
Signed-off-by: Jo Sutton <josutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_wrap is a caching mechansim, and it should probably be removed
but for now provide a way to avoid it in specific cases where we
know it is harmful.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jo Sutton <josutton@catalyst.net.nz>
Only accessed through struct ldb_context -> debug_ops, which is already private.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu May 23 00:19:30 UTC 2024 on atb-devel-224
It is only accessed via ldb functions that find it on the already-private
struct ldb_context.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This means ldb-samba/dsdb comparisons will be case-insensitive for
non-ASCII UTF-8 characters (within the bounds of the 16-bit casefold
table). And they will remain transitive.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The default is ASCII only, which is used by SSSD and OpenChange.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Typically in 8-bit character sets, those with the 0x80 bit set are
seen as 288-255, not negative numbers. This will sort them after 'Z',
not before 'A'.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This function is made from the ASCII-only bits of the old
ldb_comparison_fold() -- that is, what you get if you never follow a
`goto utf8str` jump. It comparse the bytes, but collapses spaces and
maps [a-z] to [A-Z].
This does exactly what ldb_comparison_fold_utf8_broken() would do in
situations where ldb_casfold() calls ldb_casefold_default(). That
means SSSD.
The comparison is probably using signed char, so high bytes are
actually low bytes.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
But only if it is set, which it never is (so far).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This replaces ldb_set_utf8_fns(), which will be deprecated really soon.
The reason for this, as shown in surrounding commits, is that without
an explicit case-insensitive comparison we need to rely on the casefold,
and if the casefold can fail (because, e.g. bad utf-8) the comparison
ends up being a bit chaotic. The strings being compared are generally
user controlled, and a malicious user might find ways of hiding values
or perhaps fooling a binary search.
A case-insensitive comparisons that works gradually through the string
without an all-at-once casefold is better placed to deal with problems
where they happen, and we are able to separately specialise for the
ASCII case (used by SSSD) and the UTF-8 case (Samba).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We're going to make this use a configurable pointer.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This isn't used yet, but it will allow library users to select a
case-insensitive comparison function that matches their chosen casefold.
This will allow the comparisons to be consistent when the strings are bad,
whereas currently we kind of guess.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Currently this fails like this:
test_ldb_comparison_fold_default_common: 118 errors out of 256
test_ldb_comparison_fold_default_ascii: 32 errors out of 100
test_ldb_comparison_fold_utf8_common: 40 errors out of 256
test_ldb_comparison_fold_utf8: 28 errors out of 100
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If two strings are invalid UTF-8, the string is first compared with
memcmp(), which compares as unsigned char.
If the strings are of different lengths and one is a substring of the
other, the memcmp() returns 0 and a second comparison is made which
assumes the next character in the shorter string is '\0' -- but this
comparison was done using SIGNED chars (on most systems). That leads
to non-transitive comparisons.
Consider the strings {"a\xff", "a", "ab\xff"} under that system.
"a\xff" < "a", because (char)0xff == -1.
"ab\xff" > "a", because 'b' == 98.
"ab\xff" < "a\xff", because memcmp("ab\xff", "a\xff", 2) avoiding the
signed char tiebreaker.
(Before c49c48afe0, the final character
might br arbitrarily cast into another character -- in latin-1, for
example, the 0xff here would have been seen as 'ÿ', which would be
uppercased to 'Ÿ', which is U+0178, which would be truncated to
'\x78', a positive char.
On the other hand e.g. 0xfe, 'þ', would have mapped to 0xde, 'Þ',
remaining negative).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
When the opaque context blob is not used, we might as well
use a real qsort().
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This didn't fail in the tr_TR locale before recent changes for
https://bugzilla.samba.org/show_bug.cgi?id=15637, because this is a
different casefold codepath. But it could fail if that other path goes
wrong, so we might as well have the test.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This also sorts NULLs after invalid DNs, which matches the comment
above.
CID 1596622.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Error: INTEGER_OVERFLOW (CWE-190):
ldb-2.9.0/common/ldb_ldif.c:84: tainted_data_return: Called function "read(f, buf, size)", and a possible return value may be less than zero.
ldb-2.9.0/common/ldb_ldif.c:84: cast_overflow: An assign that casts to a different type, which might trigger an overflow.
ldb-2.9.0/common/ldb_ldif.c:92: overflow: The expression "size" is considered to have possibly overflowed.
ldb-2.9.0/common/ldb_ldif.c:84: overflow_sink: "size", which might be negative, is passed to "read(f, buf, size)". [Note: The source code implementation of the function has been overridden by a builtin model.]
82| buf = (char *)value->data;
83| while (count < statbuf.st_size) {
84|-> bytes = read(f, buf, size);
85| if (bytes == -1) {
86| talloc_free(value->data);
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Apr 30 15:33:32 UTC 2024 on atb-devel-224
Error: INTEGER_OVERFLOW (CWE-190):
ldb-2.9.0/common/ldb_msg.c:1235: tainted_data_argument: The check "i < msg2->num_elements" contains the tainted expression "i" which causes "msg2->num_elements" to be considered tainted.
ldb-2.9.0/common/ldb_msg.c:1253: overflow: The expression "msg2->num_elements - (i + 1U)" is deemed underflowed because at least one of its arguments has underflowed.
ldb-2.9.0/common/ldb_msg.c:1253: overflow: The expression "32UL * (msg2->num_elements - (i + 1U))" is deemed underflowed because at least one of its arguments has underflowed.
ldb-2.9.0/common/ldb_msg.c:1253: overflow_sink: "32UL * (msg2->num_elements - (i + 1U))", which might have underflowed, is passed to "memmove(el2, el2 + 1, 32UL * (msg2->num_elements - (i + 1U)))". [Note: The source code implementation of the function has been overridden by a builtin model.]
1251| talloc_free(discard_const_p(char, el2->name));
1252| if ((i+1) < msg2->num_elements) {
1253|-> memmove(el2, el2+1, sizeof(struct ldb_message_element) *
1254| (msg2->num_elements - (i+1)));
1255| }
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This declaration is a hold‐over from the Python 2 module initialization
pattern.
Signed-off-by: Jo Sutton <josutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If a non-lowercase ASCII character has an uppercase counterpart in
some locale, toupper() will convert it to an int codepoint. Probably
that codepoint is too big to fit in our char return type, so we would
truncate it to 8 bit. So it becomes an arbitrary mapping.
It would also behave strangely with a byte with the top bit set, say
0xE2. If char is unsigned on this system, that is 'â', which
uppercases to 'Â', with the codepoint 0xC2. That seems fine in
isolation, but remember this is ldb_utf8.c, and that byte was not a
codepoint but a piece of a long utf-8 encoding. In the more likely
case where char is signed, toupper() is being passed a negative
number, the result of which is undefined.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Tue Apr 23 02:37:25 UTC 2024 on atb-devel-224
In a dotless-I locale, we might meet an 'i' before we meet a byte with
the high bit set, in which case we still want the ldb casefold
comparison.
Many ldb operations will do some case-folding before getting here, so
hitting this might be quite rare even in those locales.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15637
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In tr_TR and some other locales where the letter 'i' uppercases to
'İ', which is not ideal for LDB as we need certain strings like 'guid'
to casefold in the ASCII way.
In fixing https://bugzilla.samba.org/show_bug.cgi?id=15248) we solved
this problem in many cases, but for unindexed searches where the 'i'
is not the last character in the string. This test shows that.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15637
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This isn't supposed to be used for sorting, but it is hard to say it
won't be, so we might as well make it sort properly.
Following long-standing behaviour, we try to sort "FALSE" > "TRUE", by
length, then switch to using strncasecmp().
strncasecmp would sort the other way, so we swap the operands. This is
to make e.g. "TRUE\0" sort the same as "TRUE".
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We were returning -1 in all these cases:
ldb_dn_compare(dn, NULL);
ldb_dn_compare(NULL, dn);
ldb_dn_compare(NULL, NULL);
which would give strange results in sort, where this is often used.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We assume no values is unlikely, since we have been dereferencing
->values[0] forever, with no known reports of trouble.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
There are further changes coming here.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15625
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
