IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Reported by Red Hat internal covscan
leaked_storage: Variable "cursor" going out of scope leaks the storage it points to.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
* CVE-2023-0614 Not-secret but access controlled LDAP attributes can be discovered (bug 15270)
* pyldb: Raise an exception if ldb_dn_get_parent() fails
* Implement ldap_whoami in pyldb and add the RFC4532 LDB_EXTENDED_WHOAMI_OID definition
* Documentation and spelling fixes
* Add ldb_val -> bool,uint64,int64 parsing functions
* Split out ldb_val_as_dn() helper function
* add LDB_CHANGETYPE_MODRDN support to ldb_ldif_to_pyobject()
* add LDB_CHANGETYPE_DELETE support to ldb_ldif_to_pyobject()
* let ldb_ldif_parse_modrdn() handle names without 'rdn_name=' prefix
* Don't create error string if there is no error
* Avoid allocation and memcpy() for every wildcard match candidate
* Make ldb_msg_remove_attr O(n)
* pyldb: Throw error on invalid controls
* pyldb: remove py2 ifdefs
* Call tevent_set_max_debug_level(TEVENT_DEBUG_TRACE)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Jule Anger <janger@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
As hinted in f2416493c0 the DCOM and WMI
IDL is now unused. These generate code with PIDL, costing a small
amount of build time but more importantly are fuzzed, which costs an
ongoing amount of CPU time as oss-fuzz tries to find parsing issues.
We do not need to continue this waste, and these can be restored
if this effort is ever to start again.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
MIT Kerberos implements krb5_free_enctypes(), Heimdal is missing it and
offers krb5_xfree() instead.
This introduces a wrapper krb5_free_enctypes() around krb5_xfree() for
Heimdal.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15289
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Fri Jul 21 06:16:30 UTC 2023 on atb-devel-224
The purpose of this is to make it clear which part of the AD DC (in particular)
has faulted without having to deduce it from the stacktrace.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This means ldb_tevent_debug() is only called for TEVENT_DEBUG_TRACE.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This means samba_tevent_debug() is only called when needed.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
- remove py2 ifdefs
- python: Safely clear structure members
- the tevent_thread_call_depth API is updated
in order to allow better tracing.
- add tevent_set_max_debug_level() only and don't
pass TEVENT_DEBUG_TRACE to tevent_debug() callbacks by default.
- Spelling fixes
- Make use of epoll_create1() for epoll backend
- Optimize overhead in the epoll backend
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This is clearer for multiplexed fdes as it means both sides are
already cleared before we call epoll_update_event() again.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
The preparation, function call and cleanup for epoll_check_reopen()
is quite some overhead and not needed most of the time!
So check the pid in the caller avoids most of it.
Review with: git show -w
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
It can be very costly to calculate the arguments passed to
tevent_debug(), just to drop the message within tevent_debug()
or the callback function.
So we add a way to avoid the overhead, it will be used in the
next commits.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Allow an application to decide which log levels it wants to get
in the callback function passed to tevent_set_debug().
By default TEVENT_DEBUG_WARNING is the maximal reported level
and TEVENT_DEBUG_TRACE message no longer reach the callback function
by default.
It seems Samba is the only consumer of tevent_set_debug(), so it
should not be a huge problem, as Samba only reports TEVENT_DEBUG_TRACE
message with log level 50 anyway. And future Samba versions will
call tevent_set_max_debug_level() if needed.
Note the change to tevent-0.14.1.sigs will be reverted
with the release of tevent 0.15.0.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This is similar to the "context" test, but without signal handlers.
It also creates a constant load instead of being time limited,
which makes it useful to analyse using callgrind and other tools.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Note the tevent-0.14.1.sigs changes will be reverted in
the 'tevent 0.15.0' commit.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Note the tevent-0.14.1.sigs changes will be reverted in
the 'tevent 0.15.0' commit.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipensky@samba.org>
Note the tevent-0.14.1.sigs changes will be reverted in
the 'tevent 0.15.0' commit.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipensky@samba.org>
Note the tevent-0.14.1.sigs changes will be reverted in
the 'tevent 0.15.0' commit.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Note the tevent-0.14.1.sigs changes will be reverted in
the 'tevent 0.15.0' commit.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Note the tevent-0.14.1.sigs changes will be reverted in
the 'tevent 0.15.0' commit.
Signed-off-by: Pavel Filipenský <pfilipensky@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Removing the explicit notice about ldb in order to
have the same content in all copies of dlinklist.h
in the next commits.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Prefer epoll_create1(2) over epoll_create(2) and
always require the former to use epoll(7) interface,
thus saving extra fcntl(2) call to set FD_CLOEXEC.
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Prefer epoll_create1(2) over epoll_create(2) and
always require the former to use epoll(7) interface.
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
* Remove remaining, but broken python2 support
* Spelling fixes
* python: Safely clear structure members
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
If this patch is applied, and an environment variable is set, all
access_check calls will be recorded as seeds for
fuzz_security_token_vs_descriptor. See the patch for details.
You probably will never want to apply this patch, but it is here just
in case.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This of course doesn't exercise the object tree or default SID code,
but it still covers a lot to the *_ds access_check functions.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The token and descriptor are stored in NDR format; for this purpose we
add a new IDL struct containing this pair (along with a desired access
mask).
An upcoming commit will show how to collect seeds for this fuzzer.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This fuzzer parses SDDL into a security descriptor and runs an access
check on it using a known security token. This is purely for crash
detection -- we don't know enough to assert whether the check should
succeed or not.
The seed strings used are compatible with those of fuzz_sddl_parse --
anything found by fuzz_sddl_parse is worth trying as a seed here, and
vice versa.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
There are multiple places we make this assumption. For example, in
source3/lib/tldap.c, we have this line
if (ld->msgid == 2147483647) {
where ld->msgid is an int. And in librpc/idl/security.idl we have
several lines like:
const int SEC_MASK_GENERIC = 0xF0000000;
In lib/replace/inet_pton.c and inet_ntop.c we have
/*
* WARNING: Don't even consider trying to compile this on a system where
* sizeof(int) < 4. sizeof(int) > 4 is fine; all the world's not a VAX.
*/
but no attempt to enforce that as far as I can see, until now.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Tue Jul 11 22:44:15 UTC 2023 on atb-devel-224
I can not find the code that required this, even in the history.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Jun 30 14:59:46 UTC 2023 on atb-devel-224
This allows us to remove a lot of conditionally compiled code and so
know with more certainly that our tests are covering our code-paths.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
These will shortly be unused as we will rely on GnuTLS for all AES cryptography
now that we require GnuTLS 3.6.13
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This removes a lot of inline #ifdef and means this feature is always tested.
We can do this as we have chosen GnuTLS 3.6.13 as the new minimum version.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This can be included in logged authentications and authorizations.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This parameter was removed in commit
c37d6be2db.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Such a failure could be caused by situations other than memory errors,
but a simple indication of failure is all that ldb_dn_get_parent() gives
us to work with.
We keep the old behaviour of returning None if the DN has no components,
which an existing test (ldb.python.api.DnTests.test_parent_nonexistent)
expects.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We should not skip all of close_low_fd() just because we
detected valgrind headers at build time.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
json_add_timestamp() is limited to adding a ‘timestamp’ field with the
current time. The new function can add an arbitrary timestamp with an
arbitrary field name.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This replaces a couple of calls to snprintf() in
log_authentication_event_json() and log_successful_authz_event_json()
respectively.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Using Py_CLEAR() ensures that these structures are observed in a
consistent state by any Python code that may run during deconstruction.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Prefer 'size_t' over 'int' in generate_random_buffer(),
generate_secret_buffer() and generate_nonce_buffer() to
match an underlying gnutls_rnd() calls.
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This allows the new unsupported functional levels to be unlocked, but with an smb.conf
option that is easily seen.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Fix the following error observed running samba.test.registry
compiled with clang-17 and UBsan:
lib/ldb/common/ldb_ldif.c:881:9: runtime error: applying non-zero offset 137438953440 to null pointer
#0 0x7faa0eb3932f in ldb_ldif_read lib/ldb/common/ldb_ldif.c:881
#1 0x7faa0eb3aec6 in ldb_ldif_read_string lib/ldb/common/ldb_ldif.c:1004
#2 0x7faa077ed759 in dsdb_set_schema_from_ldif source4/dsdb/schema/schema_set.c:1113
#3 0x7faa068fcbbf in py_dsdb_set_schema_from_ldif source4/dsdb/pydsdb.c:929
#4 0x7faa1d1d4507 in cfunction_call (/lib64/libpython3.11.so.1.0+0x1d4507)
[... a lot of Python calls skipped...]
I.e. number of elements should be checked against zero
before making an attempt to access an element by index.
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
runtime error: left shift of 65535 by 16 places cannot be represented in type 'int'
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Apart from catching crashes in the actual parsing, we abort if the SD
we end up with will not round trip back through SDDL to an identical
SD.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Missed two more places originally when introduced ROLE_IPA_DC.
Signed-off-by: Alexander Bokovoy <ab@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Alexander Bokovoy <ab@samba.org>
Autobuild-Date(master): Tue Apr 25 07:46:36 UTC 2023 on atb-devel-224
This avoids unnecessary attempts to load libgpfs.so when it is not
needed.
Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Fri Apr 14 12:28:23 UTC 2023 on atb-devel-224
If a test fails an assertion, and later calls torture_skip() to skip
part of the test, the TORTURE_SKIP result will overwrite the
TORTURE_FAIL result, and the overall outcome will be successful.
To avoid this, we now arrange possible outcomes in order of priority,
and ensure we always keep the higher priority one.
This reveals some failing tests.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
If we don't check for NULL after each loop iteration, the failure could
be masked in the next iteration by talloc_asprintf_append() allocating
on the NULL context. That could result in values getting lost.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Computing a pointer that points outside of an array, and not to one past
the last element, is undefined behaviour. To avoid this, do our
comparisons in terms of lengths, not pointers.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
If the LDB_UNPACK_DATA_FLAG_NO_ATTRS flag is set, we don't return any
elements, so we should set num_elements accordingly. This ensures
callers don't try to access elements that aren't there.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
That an attribute has been access checked doesn't mean that the user has
the right to view it.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Commit bed9efa6cd introduced
ldb_msg_add_linearized_dn() to replace ldb_msg_add_dn(), but retained
the now-incorrect associated comment. The comment later made its way
into a function added later by commit 'CVE-2022-32746 ldb: Add functions
for appending to an ldb_message'.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15008
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
This comment originally referred, not to a va_copy() call, but to the
use of &c with vsnprintf() rather than passing in NULL with a length of
zero.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr 11 10:08:54 UTC 2023 on atb-devel-224
This avoids sending new or reset passwords in the clear
(integrity protected only) from samba-tool in particular.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15315
Signed-off-by: Rob van der Linde <rob@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 5 03:08:51 UTC 2023 on atb-devel-224
Setting the LDB_HANDLE_FLAG_UNTRUSTED tells the acl_read module to operate on this request.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
The chain for transitive evaluation does consider ACLs, avoiding the disclosure of
confidential information.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Redaction may be expensive if we end up needing to fetch a security
descriptor to verify rights to an attribute. Checking the search scope
is probably cheaper, so do that first.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This makes it less likely that we forget to handle a case.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add a hook, acl_redact_msg_for_filter(), in the aclread module, that
marks inaccessible any message elements used by an LDAP search filter
that the user has no right to access. Make the various ldb_match_*()
functions check whether message elements are accessible, and refuse to
match any that are not. Remaining message elements, not mentioned in the
search filter, are checked in aclread_callback(), and any inaccessible
elements are removed at this point.
Certain attributes, namely objectClass, distinguishedName, name, and
objectGUID, are always present, and hence the presence of said
attributes is always allowed to be checked in a search filter. This
corresponds with the behaviour of Windows.
Further, we unconditionally allow the attributes isDeleted and
isRecycled in a check for presence or equality. Windows is not known to
make this special exception, but it seems mostly harmless, and should
mitigate the performance impact on searches made by the show_deleted
module.
As a result of all these changes, our behaviour regarding confidential
attributes happens to match Windows more closely. For the test in
confidential_attr.py, we can now model our attribute handling with
DC_MODE_RETURN_ALL, which corresponds to the behaviour exhibited by
Windows.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Change all uses of ldb_kv_filter_attrs() to use
ldb_filter_attrs_in_place() instead. This function does less work than
its predecessor, and no longer requires the allocation of a second ldb
message. Some of the work is able to be split out into separate
functions that each accomplish a single task, with a purpose to make the
code clearer.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_filter_attrs() previously did too much. Now its replacement,
ldb_filter_attrs_in_place(), only does the actual filtering, while
taking ownership of each element's values is handled in a separate
function, ldb_msg_elements_take_ownership().
Also, ldb_filter_attrs_in_place() no longer adds the distinguishedName
to the message if it is missing. That is handled in another function,
ldb_msg_add_distinguished_name().
As we're now modifying the original message rather than copying it into
a new one, we no longer need the filtered_msg parameter.
We adapt a test, based on ldb_filter_attrs_test, to exercise the new
function.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
At present this function is an exact duplicate of ldb_filter_attrs(),
but in the next commit we shall modify it to work in place, without the
need for the allocation of a second message.
The test is a near duplicate of the existing test for
ldb_filter_attrs().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Many places in Samba depend upon various components of an ldb message
being talloc allocated, and hence able to be used as talloc contexts.
The elements and values of an unpacked ldb message point to unowned data
inside the memory-mapped database, and this function ensures that such
messages have talloc ownership of said elements and values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add ldb_filter_attrs_test to the list of tests so that it actually gets
run.
Remove a duplicate ldb_msg_test that was accidentally added in commit
5ca90e758a.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If the value of an ldb message element is not zero-terminated, calling
ldb_msg_find_attr_as_string() will cause the function to read off the
end of the buffer in an attempt to verify that the value is
zero-terminated. This can cause unexpected behaviour and make the test
randomly fail.
To avoid this, we must have a terminating null byte that is *not*
counted as part of the length, and so we must calculate the length with
strlen() rather than sizeof.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Best reviewed with: `git show --word-diff`.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr 4 08:30:28 UTC 2023 on atb-devel-224
Best reviewed with: `git show --word-diff`.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Apr 3 04:53:05 UTC 2023 on atb-devel-224
These functions allow us to parse any value of a message element, not
only the first. They also unambiguously indicate whether an error has
occurred.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The "inconvenience function" takes one type, and converts it to another
but the documentation was not updated.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This allows the calculation of the worst case to be shared with callers.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Mar 30 08:08:32 UTC 2023 on atb-devel-224
We don't allow MODRDN and DELETE for now as they
don't work as is anyway. We'll add these in the next steps.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is needed in order to process schema updates.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We should only do this in the LDB_ERR_NO_SUCH_ATTRIBUTE case.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Trying ndr_pull_security_descriptor on SDDL produces just strange
debug messages, which can cause confusion.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This is a change in cmocka to avoid hiding possible errors.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Mar 15 07:53:54 UTC 2023 on atb-devel-224
The value can be quite large, the allocation will take much
longer than the actual match and is repeated per candidate
record.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15331
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Often, on rackspace GitLab CI runners, we get:
UNEXPECTED(failure): samba.unittests.tsocket_tstream.test_tstream_more_tcp_user_timeout_spin(none)
REASON: Exception: Exception: 0xf == 0xf
../../lib/tsocket/tests/test_tstream.c:405: error: Failure!
This allows us more spins before we fail the test.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15328
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Functions like `add_lock_to_json` and `add_profile_item_to_json` pass
some values to `json_add_int` with `intmax_t` types. This may cause
arithmetic overflow when the value grows very fast, such as the
read_bytes profiling data.
Use `json_add_int` instead of `int` to avoid the overflow.
RN: Make json output show intmax_t value properly
Signed-off-by: Li Yuxuan <liyuxuan.darfux@bytedance.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 9 21:33:43 UTC 2023 on atb-devel-224
Show that `json_add_int` can't handle value larger than int32 due to
overflow.
Add knownfail.
Signed-off-by: Li Yuxuan <liyuxuan.darfux@bytedance.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Prefer mallinfo2() with 'size_t' fields over deprecated
mallinfo() (with 'int' fields which may wrap around zero
and so be inaccurate on a 64-bit system) and move relevant
checks to lib/util/wscript_configure because mallinfo()
is not used beyond 'samba-util'.
Suggested-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Also fix an obsolete related comment.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Feb 17 14:52:26 UTC 2023 on atb-devel-224
In file_load()/file_lines_load(), the file's fd is obtained using
open(), and in fd_load() the fd is converted to a FILE* using
fdopen(). However, after fclose(), the fd is closed again using
close().
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15311
Signed-off-by: baixiangcpp baixiangcpp@gmail.com
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Feb 16 12:13:05 UTC 2023 on atb-devel-224
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Feb 15 09:05:56 UTC 2023 on atb-devel-224
Arrays of SIDs are handled not fully consistently throughout the
codebase. Sometimes SIDs in the first and second positions represent a
user and a primary group respectively; other times they don't mean
anything in particular. Using these index constants in situations of the
former sort can help to clarify our intent.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
source3/modules/getdate.c:1192:9: error: variable 'yynerrs' set but not used
[-Werror,-Wunused-but-set-variable]
int yynerrs;
^
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
lib/ldb-samba/ldb_ildap.c:482:47: error: implicit conversion from enumeration
type 'enum ldb_scope' to different enumeration type 'enum ldap_scope'
[-Werror,-Wenum-conversion]
msg->r.SearchRequest.scope = req->op.search.scope;
~ ~~~~~~~~~~~~~~~^~~~~
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This already would not compile with Python 2, because Py_TPFLAGS_HAVE_ITER
is not defined
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Removes:
* waf pydoctor
* waf wafdocs
* make pydoctor
There is no "make wafdocs" it only appears to be in wscript.
The reasoning being is these are broken and appear to not have been run for some time.
Signed-off-by: Rob van der Linde <rob@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb 2 21:15:54 UTC 2023 on atb-devel-224
- Build fix for GNU/Hurd
- Build fix for Solaris, after removal
of ports backend (bug #15298)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb 2 18:27:08 UTC 2023 on atb-devel-224
There is no hardcoded PIPE_BUF pipe limitation on GNU/Hurd, but POSIX
provides a minimum value that we can use.
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Removes need for external stat() code when checking for timechange.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Jan 27 08:30:35 UTC 2023 on atb-devel-224
One of changes is somewhat interesting, it is "tfork waiter proces"
process title in tfork.c. I wonder why no one noticed this before.
There's another similar process title in there, "tfork waiter process(%d)".
Hopefully no one does grep for "proces$" (and there's no reason to).
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rowland Penny <rpenny@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Jan 26 20:46:11 UTC 2023 on atb-devel-224
It is possible that in our test environment one of the config 'include' files
change more than once per second. To avoid missing a file update we use a
higher time resolution than seconds.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jan 18 17:25:51 UTC 2023 on atb-devel-224
* Support python 3.12
* Have python functions operating on DNs raise LdbError
* don't call comparison() directly in LDB_TYPESAFE_QSORT
* Use ldb_ascii_toupper() for case folding to support
tr_TR.UTF-8 and other dotless i locales,
see https://bugzilla.samba.org/show_bug.cgi?id=15248
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
- Support python 3.12
- remove solaris port backend (it's not maintainable)
- make tevent_find_ops_byname() available for callers.
- allow the "standard" backend to be overloaded
- add interface for request/subrequest call depth tracking:
- tevent_thread_call_depth_activate
- tevent_thread_call_depth_deactivate
- tevent_thread_call_depth_start
- tevent_thread_call_depth_stop
- tevent_thread_call_depth_reset_from_req
Note the changes to ABI/tevent-0.13.0.sigs only
revert the temporary changes made there...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
The change to lib/tevent/ABI/tevent-0.13.0.sigs will be reverted
in the commit for the 0.14.0 release...
Signed-off-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This makes it more flexible and allow a caller to overload
a tevent backend. Which will be used by Samba in order to
glue in io_uring support.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
We'll export tevent_find_ops_byname() soon and will allow
the context_init() function of backends to find that standard ops
and hand over to standard_ops->context_init().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
There's no way to verify changes we would have to do tevent_port.c,
as we don't have access to a solaris build machine.
So better use the poll backend instead. In performance critical code
we typically don't deal with a lot of file descriptors so the impact
should be fairly minimal.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Note testsuite.c is only used in Samba's smbtorture as
'smbtorture //a/b local.event'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Tue Jan 17 18:23:18 UTC 2023 on sn-devel-184
Put the division on the correct side of the inequality.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The cleanup phase of tfork_create() saves errno prior to calling
functions that might modify it, with the intention of restoring it
afterwards. However, the value of 'ret' is accidentally overwritten. It
will always be equal to 0, and hence errno will not be restored.
Fix this by introducing a new variable, ret2, for calling functions in
the cleanup phase.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ensuring pointers are always initialised avoids compilation errors with
FORTIFY_SOURCE=2.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
No need to recompile the world when only a few files need this.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
To be used in smbXsrv_open.c, for this we need a lower bound.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The FreeBSD extattr API may return success and truncated
namelist. We need to check for this in bsd_attr_list to
ensure that we don't accidentally read off the end of the
buffer. In the case of a truncated value, the pascal
strings for attr names will reflect the lengths as if
the value were not truncated. For example:
`58DosStrea`
In case of short read we now set error to ERANGE and
fail.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15271
Signed-off-by: Andrew Walker <awalker@ixsystems.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Jan 2 14:27:23 UTC 2023 on sn-devel-184
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15248
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Dec 23 14:17:31 UTC 2022 on sn-devel-184
There are four functions, allowing compression and decompression in
the two formats we support so far. The functions will accept bytes or
unicode strings which are treated as utf-8.
The LZ77+Huffman decompression algorithm requires an exact target
length to decompress, so this is mandatory.
The plain decompression algorithm does not need an exact length, but
you can provide one to help it know how much space to allocate. As
currently written, you can provide a short length and it will often
succeed in decompressing to a different shorter string.
These bindings are intended to make ad-hoc investigation easier, not
for production use. This is reflected in the guesses about output size
that plain_decompress() makes if you don't supply one -- either they
are stupidly wasteful or ridiculously insufficient, depending on
whether or not you were trying to decompress a 20MB string.
>>> a = '12345678'
>>> import compression
>>> b = compression.huffman_compress(a)
>>> b
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 #....
>>> len(b)
262
>>> c = compression.huffman_decompress(b, len(a))
>>> c
b'12345678' # note, c is bytes, a is str
>>> a
'12345678'
>>> d = compression.plain_compress(a)
>>> d
b'\xff\xff\xff\x0012345678'
>>> compression.plain_decompress(d) # no size specified, guesses
b'12345678'
>>> compression.plain_decompress(d,5)
b'12345'
>>> compression.plain_decompress(d,0) # 0 for auto
b'12345678'
>>> compression.plain_decompress(d,1)
b'1'
>>> compression.plain_decompress(a,444)
Traceback (most recent call last):
compression.CompressionError: unable to decompress data into a buffer of 444 bytes.
>>> compression.plain_decompress(b,444)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 #...
That last one decompresses the Huffman compressed file with the plain
compressor; pretty much any string is valid for plain decompression.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We shouldn't get a node with a zero code, and there's probably nothing
to do but stop.
CID 1517261 (#1-2 of 2): Bad bit shift operation
(BAD_SHIFT)11. negative_shift: In expression j >> offset - k,
shifting by a negative amount has undefined behavior. The shift
amount, offset - k, is -3.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec 19 23:29:04 UTC 2022 on sn-devel-184
Because we just wrote the intermediate representation to have no zero
distances, we can be sure it doesn't, but Coverity doesn't know. If
distance is zero, `bitlen_nonzero_16(distance)` would be bad.
CID 1517278 (#1 of 1): Bad bit shift operation
(BAD_SHIFT)41. large_shift: In expression 1 << code_dist, left
shifting by more than 31 bits has undefined behavior. The shift
amount, code_dist, is 65535.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Very long matches would be written instead as very very long matches.
We can't in fact hit this because we have a MAX_MATCH_LENGTH defined
as 64M, but if we could, it might make certain 2GB+ strings impossible
to compress.
CID 1517275 (#1 of 1): Unintended sign extension
(SIGN_EXTENSION)sign_extension: Suspicious implicit sign extension:
intermediate[i + 2UL] with type uint16_t (16 bits, unsigned) is
promoted in intermediate[i + 2UL] << 16 to type int (32 bits, signed),
then sign-extended to type unsigned long (64 bits, unsigned). If
intermediate[i + 2UL] << 16 is greater than 0x7FFFFFFF, the upper bits
of the result will all be 1.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
None of our test vectors are 18446744073709551615 bytes long, which
means we can know an `expected_length == returned_length` check will
catch the case where the compression function returns -1 for error. We
know that, but Coverity doesn't.
It's the same thing over and over again, in two different patterns:
>>> CID 1517301: Memory - corruptions (OVERRUN)
>>> Calling "memcmp" with "original.data" and "original.length" is
suspicious because of the very large index, 18446744073709551615. The index
may be due to a negative parameter being interpreted as unsigned.
393 if (original.length != decomp_written ||
394 memcmp(decompressed.data,
395 original.data,
396 original.length) != 0) {
397 debug_message("\033[1;31mgot %zd, expected %zu\033[0m\n",
398 decomp_written,
*** CID 1517299: Memory - corruptions (OVERRUN)
/lib/compression/tests/test_lzxpress_plain.c: 296 in
test_lzxpress_plain_decompress_more_compressed_files()
290 debug_start_timer();
291 written = lzxpress_decompress(p.compressed.data,
292 p.compressed.length,
293 dest,
294 p.decompressed.length);
295 debug_end_timer("decompress", p.decompressed.length);
>>> CID 1517299: Memory - corruptions (OVERRUN)
>>> Calling "memcmp" with "p.decompressed.data" and
"p.decompressed.length" is suspicious because of the very large index,
18446744073709551615. The index may be due to a negative parameter being
interpreted as unsigned.
296 if (written == p.decompressed.length &&
297 memcmp(dest, p.decompressed.data, p.decompressed.length)
== 0) {
298 debug_message("\033[1;32mdecompressed %s!
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We know that code is non-zero, because it comes from the combination of
the intermediate representation and the symbol tables that were generated
at the same time. But Coverity doesn't know that, and it thinks we could
be doing undefined things in the subsequent shift.
CID 1517302: Integer handling issues (BAD_SHIFT)
In expression "1 << code_bit_len", shifting by a negative amount has
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The struct write_context bit_len attribute is always between 0 and 31,
but if the next patches are applied without this, SUSE GCC -O3 will
worry thusly:
../../lib/compression/lzxpress_huffman.c: In function
‘lzxpress_huffman_compress’:
../../lib/compression/lzxpress_huffman.c:953:5: error: assuming signed
overflow does not occur when simplifying conditional to constant
[-Werror=strict-overflow]
if (wc->bit_len > 16) {
^
cc1: all warnings being treated as errors
Inspection tell us that the invariant holds. Nevertheless, we can
safely use an unsigned type and insist that over- or under- flow is
bad.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The 'compressed' string can be about 9/8 the size of the decompressed
string, but we didn't allow enough memory in the fuzz target for that.
Then when it failed, we didn't check.
Credit to OSSFuzz.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We had
output[output_pos - distance];
where output_pos and distance are size_t and distance can be greater
than output_pos (because it refers to a place in the previous block).
The underflow is defined, leading to a big number, and when
sizeof(size_t) == sizeof(*uint8_t) the subsequent overflow works as
expected. But if size_t is smaller than a pointer, bad things will
happen.
This was found by OSSFuzz with
'UBSAN_OPTIONS=print_stacktrace=1:silence_unsigned_overflow=1'.
Credit to OSSFuzz.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The linker has to relocate the pointers in the array at startup, save
that. I know we have bigger .data blobs, but every bit counts :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
With the proper NULL checks we don't need the stackframe,
use a passed in context instead.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The first call of talloc_asprintf_append() did not have a NULL check.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This adds intermediate NULL checks via talloc_asprintf_addbuf()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
I wanted to use this in debug.c, but this would have meant to pollute
debug's deps with a lot of stuff. Also, looking through uses of
talloc_asprint_append(), very many of those don't do NULL checks
properly and could benefit from the _addbuf() flavor. We can add a
vasprintf variant later if the need shows up.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This allows admins to disable enctypes completely if required.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In order to allow better upgrades we need the default value for smb.conf to the
same even if the effective default value of the software changes in future.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is not squashed in order to allow easier backports...
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We no longer have support for des encryption types in the kerberos
libraries anyway.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
aes encryption types are always supported.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This matches the Windows registry key
HKEY_LOCAL_MACHINE\System\CurrentControlSet\services\KDC\DefaultDomainSupportedEncTypes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15240
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
AES is supported by Windows Server >= 2008R2, Windows (Client) >= 7 and Samba >= 4.0,
so there's no reason to allow md5 clients by default.
However some third party domain members may need it.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15240
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
AES is supported by Windows >= 2008R2 and Samba >= 4.0 so there's no
reason to allow md5 servers by default.
Note the change in netlogon_creds_cli_context_global() is only cosmetic,
but avoids confusion while reading the code. Check with:
git show -U35 libcli/auth/netlogon_creds_cli.c
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15240
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
<sys/stat.h> was missing from compression library tests which resulted
in the following compile time error:
../../lib/compression/tests/test_lzx_huffman.c: In function
‘datablob_from_file’:
../../lib/compression/tests/test_lzx_huffman.c:383:21: error:
storage size of ‘s’ isn’t known
383 | struct stat s;
| ^
../../lib/compression/tests/test_lzx_huffman.c:389:15: warning:
implicit declaration of function ‘fstat’ [-Wimplicit-function-declaration]
389 | ret = fstat(fileno(fh), &s);
| ^~~~~
Signed-off-by: Anoop C S <anoopcs@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Dec 6 11:39:16 UTC 2022 on sn-devel-184
lib/compression/tests/test_lzx_huffman.c: In function ‘test_lzxpress_huffman_overlong_matches’:
lib/compression/tests/test_lzx_huffman.c:1013:35: error: ‘j’ may be used uninitialized [-Werror=maybe-uninitialized]
1013 | assert_int_equal(score, i * j);
| ^
lib/compression/tests/test_lzx_huffman.c:979:19: note: ‘j’ was declared here
979 | size_t i, j;
| ^
lib/compression/tests/test_lzx_huffman.c: In function ‘test_lzxpress_huffman_overlong_matches_abc’:
lib/compression/tests/test_lzx_huffman.c:1059:39: error: ‘k’ may be used uninitialized [-Werror=maybe-uninitialized]
1059 | assert_int_equal(score, i * j * k);
| ^
lib/compression/tests/test_lzx_huffman.c:1020:22: note: ‘k’ was declared here
1020 | size_t i, j, k;
| ^
lib/compression/tests/test_lzx_huffman.c:1059:35: error: ‘j’ may be used uninitialized [-Werror=maybe-uninitialized]
1059 | assert_int_equal(score, i * j * k);
| ^
lib/compression/tests/test_lzx_huffman.c:1020:19: note: ‘j’ was declared here
1020 | size_t i, j, k;
| ^
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Sun Dec 4 09:12:30 UTC 2022 on sn-devel-184
This uses the same hash table method as lzxpress_huffman, though the
code can't be directly reused as the sizes of the offsets is
different, and there is not a block processing step here.
This will worsen the compression ratio compared to the exhaustive
search we previously used, though we still perform better than
Windows. To put numbers on it, the test files used to compress to 0.91
of Windows' compression size, and now they compress to 0.96.
On the other hand this is many orders of magnitude faster. It is
difficult to say exactly how much faster -- while the testsuite time
has only improved 200-fold (from 7 minutes to 2 seconds), most of the
remaining 2 seconds is used in data generation and management, not
compression. OSSFuzz consistently finds new vectors that time out
after a minute; on these we'll see nearly an order of magnitude of
orders of magnitude inprovement.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Joseph Sutton <jsutton@samba.org>
Autobuild-Date(master): Fri Dec 2 00:00:04 UTC 2022 on sn-devel-184
This makes it easier to rework the encoding decision to depend on a
hash table match rather than the current exhaustive search.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This will make it possible to move encoding operations into helper
functions, which will make it easier to restructure the code to use a
hash table for faster matching.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
These are based on (i.e. copied and pasted from) the LZ77 + Huffman
tests.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Everything that is in testdata/compression/lzxpress-huffman/ can also
be used for lzxpress plain tests, which is something we really need.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We are going to change from a slow exact match algorithm to a fast
heuristic search that will not always get the same results as the
exhaustive search.
To be precise, a million zeros will compress to 112 rather than 93 bytes.
We don't insist on an exact size, because that is not an issue here.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Mainly so I can go
make bin/test_lzxpress_plain && bin/test_lzxpress_plain
valgrind bin/test_lzxpress_plain
rr bin/test_lzxpress_plain
rr replay
in a tight loop.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
If compiled on Windows using Cygwin, MSYS2, or similar, this will output
compressed versions of files exactly as specified by MZ-XCA, if the
following conditions are met:
1. The file > 300 bytes.
2. The compressed file is smaller than the decompressed file.
Otherwise it returns the data unchanged. Without warning; that's just
how the API works.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Compression uses a 3 byte hash remember LZ77 matches in a 14-bit table.
This script runs the hash over all 16M combinations, then again over
all ASCII combinations, counting collisions to find hot-spots.
If you think you have a better hash, you are probably right, but you
should try it here -- alter h() -- before committing to it. This one is
literally the first one I thought of.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Huffman tree re-quantisation and perhaps other code paths are only
triggered by pathological data like this.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This compresses some data, decompresses it, and asserts that the
result is identical to the original string.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This differs from fuzz_lzxpress_huffman_round_trip (next commit) in
that the output buffer might be too small for the compressed data, in
which case we want to see an error and not a crash.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Most strings will not successfully decompress, which is OK. What we
care about of course is memory safety.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
With LZXHUFF_DEBUG_VERBOSE set, we measure the compression and
decompression rate relative to the decompressed size.
On reasonably long strings on my laptop, compiled with -O0, it turns
out to between 20 and 500 MB/s, both ways, depending on the complexity
of the string. Very short strings are of course dominated by overhead.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
If you need to see a Huffman tree (and sometimes you do), set
DEBUG_HUFFMAN_TREE to true at the top of lzxpress_huffman.c, and run:
make bin/test_lzx_huffman && bin/test_lzx_huffman
Actually, that will show you hundreds of trees, and you'll be glad of
that if you are ever trying to understand this.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Encoding without LZ77 matches is valid, and it is useful for isolating
bugs.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This compresses files as described in MS-XCA 2.2, and as decompressed
by the decompressor in the previous commit.
As with the decompressor, there are two public functions -- one that
uses a talloc context, and one that uses pre-allocated memory. The
compressor requires a tightly bound amount of auxillary memory
(>220kB) in a few different buffers, which is all gathered together in
the public struct lzxhuff_compressor_mem. An instantiated but not
initialised copy of this struct is required by the non-talloc
function; it can be used over and over again.
Our compression speed is about the same as the decompression speed
(between 20 and 500 MB/s on this laptop, depending on the data), and
our compression ratio is very similar to that of Windows.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This format is described in [MS-XCA] 2.1 and 2.2, with exegesis in
many posts on the cifs-protocol list[1].
The two public functions are:
ssize_t lzxpress_huffman_decompress(const uint8_t *input,
size_t input_size,
uint8_t *output,
size_t output_size);
uint8_t *lzxpress_huffman_decompress_talloc(TALLOC_CTX *mem_ctx,
const uint8_t *input_bytes,
size_t input_size,
size_t output_size);
In both cases the caller needs to know the *exact* decompressed size,
which is essential for decompression. The _talloc version allocates
the buffer for you, and uses the talloc context to allocate a 128k
working buffer. THe non-talloc function will allocate the working
buffer on the stack.
This compression format gives better compression for messages of
several kilobytes than the "plain" LXZPRESS compression, but is
probably a bit slower to decompress and is certainly worse for very
short messages, having a fixed 256 byte overhead for the first Huffman
table.
Experiments show decompression rates between 20 and 500 MB per second,
depending on the compression ratio and data size, on an i5-1135G7 with
no compiler optimisations.
This compression format is used in AD claims and in SMB, but that
doesn't happen with this commit.
I will not try to describe LZ77 or Huffman encoding here. Don't expect
an answer in MS-XCA either; instead read the code and/or Wikipedia.
[1] Much of that starts here:
https://lists.samba.org/archive/cifs-protocol/2022-October/
but there's more earlier, particularly in June/July 2020, when
Aurélien Aptel was working on an implementation that ended up in
Wireshark.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We are going to add more tests for lib/compression, and they can't all
be called "testsuite.c".
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Sometimes (e.g. in lzxpress Huffman encoding, and in some of our
tests: c.f. https://lists.samba.org/archive/samba-technical/2018-March/126010.html)
we want a stable sort algorithm (meaning one that retains the previous
order of items that compare equal).
The GNU libc qsort() is *usually* stable, in that it first tries to
use a mergesort but reverts to quicksort if the necessary allocations
fail. That has led Samba developers to unthinkingly assume qsort() is
stable which is not the case on many platforms, and might not always
be on GNU/Linuxes either.
This adds four functions. stable_sort() sorts an array, and requires
an auxiliary working array of the same size. stable_sort_talloc()
takes a talloc context so it ca create a working array and call
stable_sort(). stable_sort_r() takes an opaque context blob that gets
passed to the compare function, like qsort_r() and ldb_qsort(). And
stable_sort_talloc_r() rounds out the quadrant.
These are LGPL so that the can be used in ldb, which has problems with
unstable sort.
The tests are borrowed and extended from test_ldb_qsort.c.
When sorting non-trivial structs this is roughly as fast as GNU qsort,
but GNU qsort has optimisations for small items, using direct
assignments of rather than memcpy where the size allows the item to be
cast as some kind of int.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We need "struct iovec", which comes in via sys/uio.h, incuded by
system/filesys.h
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
