IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Computing a pointer that points outside of an array, and not to one past
the last element, is undefined behaviour. To avoid this, do our
comparisons in terms of lengths, not pointers.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
If the LDB_UNPACK_DATA_FLAG_NO_ATTRS flag is set, we don't return any
elements, so we should set num_elements accordingly. This ensures
callers don't try to access elements that aren't there.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
That an attribute has been access checked doesn't mean that the user has
the right to view it.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Commit bed9efa6cd introduced
ldb_msg_add_linearized_dn() to replace ldb_msg_add_dn(), but retained
the now-incorrect associated comment. The comment later made its way
into a function added later by commit 'CVE-2022-32746 ldb: Add functions
for appending to an ldb_message'.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15008
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
This comment originally referred, not to a va_copy() call, but to the
use of &c with vsnprintf() rather than passing in NULL with a length of
zero.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr 11 10:08:54 UTC 2023 on atb-devel-224
This avoids sending new or reset passwords in the clear
(integrity protected only) from samba-tool in particular.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15315
Signed-off-by: Rob van der Linde <rob@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Apr 5 03:08:51 UTC 2023 on atb-devel-224
Setting the LDB_HANDLE_FLAG_UNTRUSTED tells the acl_read module to operate on this request.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
The chain for transitive evaluation does consider ACLs, avoiding the disclosure of
confidential information.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Redaction may be expensive if we end up needing to fetch a security
descriptor to verify rights to an attribute. Checking the search scope
is probably cheaper, so do that first.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This makes it less likely that we forget to handle a case.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add a hook, acl_redact_msg_for_filter(), in the aclread module, that
marks inaccessible any message elements used by an LDAP search filter
that the user has no right to access. Make the various ldb_match_*()
functions check whether message elements are accessible, and refuse to
match any that are not. Remaining message elements, not mentioned in the
search filter, are checked in aclread_callback(), and any inaccessible
elements are removed at this point.
Certain attributes, namely objectClass, distinguishedName, name, and
objectGUID, are always present, and hence the presence of said
attributes is always allowed to be checked in a search filter. This
corresponds with the behaviour of Windows.
Further, we unconditionally allow the attributes isDeleted and
isRecycled in a check for presence or equality. Windows is not known to
make this special exception, but it seems mostly harmless, and should
mitigate the performance impact on searches made by the show_deleted
module.
As a result of all these changes, our behaviour regarding confidential
attributes happens to match Windows more closely. For the test in
confidential_attr.py, we can now model our attribute handling with
DC_MODE_RETURN_ALL, which corresponds to the behaviour exhibited by
Windows.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Change all uses of ldb_kv_filter_attrs() to use
ldb_filter_attrs_in_place() instead. This function does less work than
its predecessor, and no longer requires the allocation of a second ldb
message. Some of the work is able to be split out into separate
functions that each accomplish a single task, with a purpose to make the
code clearer.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
ldb_filter_attrs() previously did too much. Now its replacement,
ldb_filter_attrs_in_place(), only does the actual filtering, while
taking ownership of each element's values is handled in a separate
function, ldb_msg_elements_take_ownership().
Also, ldb_filter_attrs_in_place() no longer adds the distinguishedName
to the message if it is missing. That is handled in another function,
ldb_msg_add_distinguished_name().
As we're now modifying the original message rather than copying it into
a new one, we no longer need the filtered_msg parameter.
We adapt a test, based on ldb_filter_attrs_test, to exercise the new
function.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
At present this function is an exact duplicate of ldb_filter_attrs(),
but in the next commit we shall modify it to work in place, without the
need for the allocation of a second message.
The test is a near duplicate of the existing test for
ldb_filter_attrs().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Many places in Samba depend upon various components of an ldb message
being talloc allocated, and hence able to be used as talloc contexts.
The elements and values of an unpacked ldb message point to unowned data
inside the memory-mapped database, and this function ensures that such
messages have talloc ownership of said elements and values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Add ldb_filter_attrs_test to the list of tests so that it actually gets
run.
Remove a duplicate ldb_msg_test that was accidentally added in commit
5ca90e758a.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If the value of an ldb message element is not zero-terminated, calling
ldb_msg_find_attr_as_string() will cause the function to read off the
end of the buffer in an attempt to verify that the value is
zero-terminated. This can cause unexpected behaviour and make the test
randomly fail.
To avoid this, we must have a terminating null byte that is *not*
counted as part of the length, and so we must calculate the length with
strlen() rather than sizeof.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15270
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Best reviewed with: `git show --word-diff`.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Apr 4 08:30:28 UTC 2023 on atb-devel-224
Best reviewed with: `git show --word-diff`.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Mon Apr 3 04:53:05 UTC 2023 on atb-devel-224
These functions allow us to parse any value of a message element, not
only the first. They also unambiguously indicate whether an error has
occurred.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The "inconvenience function" takes one type, and converts it to another
but the documentation was not updated.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This allows the calculation of the worst case to be shared with callers.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Mar 30 08:08:32 UTC 2023 on atb-devel-224
We don't allow MODRDN and DELETE for now as they
don't work as is anyway. We'll add these in the next steps.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is needed in order to process schema updates.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We should only do this in the LDB_ERR_NO_SUCH_ATTRIBUTE case.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Trying ndr_pull_security_descriptor on SDDL produces just strange
debug messages, which can cause confusion.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This is a change in cmocka to avoid hiding possible errors.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Mar 15 07:53:54 UTC 2023 on atb-devel-224
The value can be quite large, the allocation will take much
longer than the actual match and is repeated per candidate
record.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15331
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Often, on rackspace GitLab CI runners, we get:
UNEXPECTED(failure): samba.unittests.tsocket_tstream.test_tstream_more_tcp_user_timeout_spin(none)
REASON: Exception: Exception: 0xf == 0xf
../../lib/tsocket/tests/test_tstream.c:405: error: Failure!
This allows us more spins before we fail the test.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15328
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Functions like `add_lock_to_json` and `add_profile_item_to_json` pass
some values to `json_add_int` with `intmax_t` types. This may cause
arithmetic overflow when the value grows very fast, such as the
read_bytes profiling data.
Use `json_add_int` instead of `int` to avoid the overflow.
RN: Make json output show intmax_t value properly
Signed-off-by: Li Yuxuan <liyuxuan.darfux@bytedance.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Thu Mar 9 21:33:43 UTC 2023 on atb-devel-224
Show that `json_add_int` can't handle value larger than int32 due to
overflow.
Add knownfail.
Signed-off-by: Li Yuxuan <liyuxuan.darfux@bytedance.com>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Prefer mallinfo2() with 'size_t' fields over deprecated
mallinfo() (with 'int' fields which may wrap around zero
and so be inaccurate on a 64-bit system) and move relevant
checks to lib/util/wscript_configure because mallinfo()
is not used beyond 'samba-util'.
Suggested-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Dmitry Antipov <dantipov@cloudlinux.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Also fix an obsolete related comment.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Feb 17 14:52:26 UTC 2023 on atb-devel-224
In file_load()/file_lines_load(), the file's fd is obtained using
open(), and in fd_load() the fd is converted to a FILE* using
fdopen(). However, after fclose(), the fd is closed again using
close().
Bug: https://bugzilla.samba.org/show_bug.cgi?id=15311
Signed-off-by: baixiangcpp baixiangcpp@gmail.com
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Feb 16 12:13:05 UTC 2023 on atb-devel-224
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Feb 15 09:05:56 UTC 2023 on atb-devel-224
Arrays of SIDs are handled not fully consistently throughout the
codebase. Sometimes SIDs in the first and second positions represent a
user and a primary group respectively; other times they don't mean
anything in particular. Using these index constants in situations of the
former sort can help to clarify our intent.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
source3/modules/getdate.c:1192:9: error: variable 'yynerrs' set but not used
[-Werror,-Wunused-but-set-variable]
int yynerrs;
^
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
lib/ldb-samba/ldb_ildap.c:482:47: error: implicit conversion from enumeration
type 'enum ldb_scope' to different enumeration type 'enum ldap_scope'
[-Werror,-Wenum-conversion]
msg->r.SearchRequest.scope = req->op.search.scope;
~ ~~~~~~~~~~~~~~~^~~~~
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This already would not compile with Python 2, because Py_TPFLAGS_HAVE_ITER
is not defined
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Removes:
* waf pydoctor
* waf wafdocs
* make pydoctor
There is no "make wafdocs" it only appears to be in wscript.
The reasoning being is these are broken and appear to not have been run for some time.
Signed-off-by: Rob van der Linde <rob@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb 2 21:15:54 UTC 2023 on atb-devel-224
- Build fix for GNU/Hurd
- Build fix for Solaris, after removal
of ports backend (bug #15298)
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Feb 2 18:27:08 UTC 2023 on atb-devel-224
There is no hardcoded PIPE_BUF pipe limitation on GNU/Hurd, but POSIX
provides a minimum value that we can use.
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Removes need for external stat() code when checking for timechange.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Jan 27 08:30:35 UTC 2023 on atb-devel-224
One of changes is somewhat interesting, it is "tfork waiter proces"
process title in tfork.c. I wonder why no one noticed this before.
There's another similar process title in there, "tfork waiter process(%d)".
Hopefully no one does grep for "proces$" (and there's no reason to).
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Rowland Penny <rpenny@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Jan 26 20:46:11 UTC 2023 on atb-devel-224
It is possible that in our test environment one of the config 'include' files
change more than once per second. To avoid missing a file update we use a
higher time resolution than seconds.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jan 18 17:25:51 UTC 2023 on atb-devel-224
* Support python 3.12
* Have python functions operating on DNs raise LdbError
* don't call comparison() directly in LDB_TYPESAFE_QSORT
* Use ldb_ascii_toupper() for case folding to support
tr_TR.UTF-8 and other dotless i locales,
see https://bugzilla.samba.org/show_bug.cgi?id=15248
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
- Support python 3.12
- remove solaris port backend (it's not maintainable)
- make tevent_find_ops_byname() available for callers.
- allow the "standard" backend to be overloaded
- add interface for request/subrequest call depth tracking:
- tevent_thread_call_depth_activate
- tevent_thread_call_depth_deactivate
- tevent_thread_call_depth_start
- tevent_thread_call_depth_stop
- tevent_thread_call_depth_reset_from_req
Note the changes to ABI/tevent-0.13.0.sigs only
revert the temporary changes made there...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
The change to lib/tevent/ABI/tevent-0.13.0.sigs will be reverted
in the commit for the 0.14.0 release...
Signed-off-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This makes it more flexible and allow a caller to overload
a tevent backend. Which will be used by Samba in order to
glue in io_uring support.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
We'll export tevent_find_ops_byname() soon and will allow
the context_init() function of backends to find that standard ops
and hand over to standard_ops->context_init().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
There's no way to verify changes we would have to do tevent_port.c,
as we don't have access to a solaris build machine.
So better use the poll backend instead. In performance critical code
we typically don't deal with a lot of file descriptors so the impact
should be fairly minimal.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Note testsuite.c is only used in Samba's smbtorture as
'smbtorture //a/b local.event'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Tue Jan 17 18:23:18 UTC 2023 on sn-devel-184
Put the division on the correct side of the inequality.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The cleanup phase of tfork_create() saves errno prior to calling
functions that might modify it, with the intention of restoring it
afterwards. However, the value of 'ret' is accidentally overwritten. It
will always be equal to 0, and hence errno will not be restored.
Fix this by introducing a new variable, ret2, for calling functions in
the cleanup phase.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ensuring pointers are always initialised avoids compilation errors with
FORTIFY_SOURCE=2.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
No need to recompile the world when only a few files need this.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
To be used in smbXsrv_open.c, for this we need a lower bound.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The FreeBSD extattr API may return success and truncated
namelist. We need to check for this in bsd_attr_list to
ensure that we don't accidentally read off the end of the
buffer. In the case of a truncated value, the pascal
strings for attr names will reflect the lengths as if
the value were not truncated. For example:
`58DosStrea`
In case of short read we now set error to ERANGE and
fail.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15271
Signed-off-by: Andrew Walker <awalker@ixsystems.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Jan 2 14:27:23 UTC 2023 on sn-devel-184
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15248
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Dec 23 14:17:31 UTC 2022 on sn-devel-184
There are four functions, allowing compression and decompression in
the two formats we support so far. The functions will accept bytes or
unicode strings which are treated as utf-8.
The LZ77+Huffman decompression algorithm requires an exact target
length to decompress, so this is mandatory.
The plain decompression algorithm does not need an exact length, but
you can provide one to help it know how much space to allocate. As
currently written, you can provide a short length and it will often
succeed in decompressing to a different shorter string.
These bindings are intended to make ad-hoc investigation easier, not
for production use. This is reflected in the guesses about output size
that plain_decompress() makes if you don't supply one -- either they
are stupidly wasteful or ridiculously insufficient, depending on
whether or not you were trying to decompress a 20MB string.
>>> a = '12345678'
>>> import compression
>>> b = compression.huffman_compress(a)
>>> b
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 #....
>>> len(b)
262
>>> c = compression.huffman_decompress(b, len(a))
>>> c
b'12345678' # note, c is bytes, a is str
>>> a
'12345678'
>>> d = compression.plain_compress(a)
>>> d
b'\xff\xff\xff\x0012345678'
>>> compression.plain_decompress(d) # no size specified, guesses
b'12345678'
>>> compression.plain_decompress(d,5)
b'12345'
>>> compression.plain_decompress(d,0) # 0 for auto
b'12345678'
>>> compression.plain_decompress(d,1)
b'1'
>>> compression.plain_decompress(a,444)
Traceback (most recent call last):
compression.CompressionError: unable to decompress data into a buffer of 444 bytes.
>>> compression.plain_decompress(b,444)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 #...
That last one decompresses the Huffman compressed file with the plain
compressor; pretty much any string is valid for plain decompression.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We shouldn't get a node with a zero code, and there's probably nothing
to do but stop.
CID 1517261 (#1-2 of 2): Bad bit shift operation
(BAD_SHIFT)11. negative_shift: In expression j >> offset - k,
shifting by a negative amount has undefined behavior. The shift
amount, offset - k, is -3.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec 19 23:29:04 UTC 2022 on sn-devel-184
Because we just wrote the intermediate representation to have no zero
distances, we can be sure it doesn't, but Coverity doesn't know. If
distance is zero, `bitlen_nonzero_16(distance)` would be bad.
CID 1517278 (#1 of 1): Bad bit shift operation
(BAD_SHIFT)41. large_shift: In expression 1 << code_dist, left
shifting by more than 31 bits has undefined behavior. The shift
amount, code_dist, is 65535.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Very long matches would be written instead as very very long matches.
We can't in fact hit this because we have a MAX_MATCH_LENGTH defined
as 64M, but if we could, it might make certain 2GB+ strings impossible
to compress.
CID 1517275 (#1 of 1): Unintended sign extension
(SIGN_EXTENSION)sign_extension: Suspicious implicit sign extension:
intermediate[i + 2UL] with type uint16_t (16 bits, unsigned) is
promoted in intermediate[i + 2UL] << 16 to type int (32 bits, signed),
then sign-extended to type unsigned long (64 bits, unsigned). If
intermediate[i + 2UL] << 16 is greater than 0x7FFFFFFF, the upper bits
of the result will all be 1.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
None of our test vectors are 18446744073709551615 bytes long, which
means we can know an `expected_length == returned_length` check will
catch the case where the compression function returns -1 for error. We
know that, but Coverity doesn't.
It's the same thing over and over again, in two different patterns:
>>> CID 1517301: Memory - corruptions (OVERRUN)
>>> Calling "memcmp" with "original.data" and "original.length" is
suspicious because of the very large index, 18446744073709551615. The index
may be due to a negative parameter being interpreted as unsigned.
393 if (original.length != decomp_written ||
394 memcmp(decompressed.data,
395 original.data,
396 original.length) != 0) {
397 debug_message("\033[1;31mgot %zd, expected %zu\033[0m\n",
398 decomp_written,
*** CID 1517299: Memory - corruptions (OVERRUN)
/lib/compression/tests/test_lzxpress_plain.c: 296 in
test_lzxpress_plain_decompress_more_compressed_files()
290 debug_start_timer();
291 written = lzxpress_decompress(p.compressed.data,
292 p.compressed.length,
293 dest,
294 p.decompressed.length);
295 debug_end_timer("decompress", p.decompressed.length);
>>> CID 1517299: Memory - corruptions (OVERRUN)
>>> Calling "memcmp" with "p.decompressed.data" and
"p.decompressed.length" is suspicious because of the very large index,
18446744073709551615. The index may be due to a negative parameter being
interpreted as unsigned.
296 if (written == p.decompressed.length &&
297 memcmp(dest, p.decompressed.data, p.decompressed.length)
== 0) {
298 debug_message("\033[1;32mdecompressed %s!
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We know that code is non-zero, because it comes from the combination of
the intermediate representation and the symbol tables that were generated
at the same time. But Coverity doesn't know that, and it thinks we could
be doing undefined things in the subsequent shift.
CID 1517302: Integer handling issues (BAD_SHIFT)
In expression "1 << code_bit_len", shifting by a negative amount has
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The struct write_context bit_len attribute is always between 0 and 31,
but if the next patches are applied without this, SUSE GCC -O3 will
worry thusly:
../../lib/compression/lzxpress_huffman.c: In function
‘lzxpress_huffman_compress’:
../../lib/compression/lzxpress_huffman.c:953:5: error: assuming signed
overflow does not occur when simplifying conditional to constant
[-Werror=strict-overflow]
if (wc->bit_len > 16) {
^
cc1: all warnings being treated as errors
Inspection tell us that the invariant holds. Nevertheless, we can
safely use an unsigned type and insist that over- or under- flow is
bad.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The 'compressed' string can be about 9/8 the size of the decompressed
string, but we didn't allow enough memory in the fuzz target for that.
Then when it failed, we didn't check.
Credit to OSSFuzz.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We had
output[output_pos - distance];
where output_pos and distance are size_t and distance can be greater
than output_pos (because it refers to a place in the previous block).
The underflow is defined, leading to a big number, and when
sizeof(size_t) == sizeof(*uint8_t) the subsequent overflow works as
expected. But if size_t is smaller than a pointer, bad things will
happen.
This was found by OSSFuzz with
'UBSAN_OPTIONS=print_stacktrace=1:silence_unsigned_overflow=1'.
Credit to OSSFuzz.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The linker has to relocate the pointers in the array at startup, save
that. I know we have bigger .data blobs, but every bit counts :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
With the proper NULL checks we don't need the stackframe,
use a passed in context instead.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The first call of talloc_asprintf_append() did not have a NULL check.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
