IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
It is possible that in our test environment one of the config 'include' files
change more than once per second. To avoid missing a file update we use a
higher time resolution than seconds.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Jan 18 17:25:51 UTC 2023 on atb-devel-224
* Support python 3.12
* Have python functions operating on DNs raise LdbError
* don't call comparison() directly in LDB_TYPESAFE_QSORT
* Use ldb_ascii_toupper() for case folding to support
tr_TR.UTF-8 and other dotless i locales,
see https://bugzilla.samba.org/show_bug.cgi?id=15248
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Jule Anger <janger@samba.org>
- Support python 3.12
- remove solaris port backend (it's not maintainable)
- make tevent_find_ops_byname() available for callers.
- allow the "standard" backend to be overloaded
- add interface for request/subrequest call depth tracking:
- tevent_thread_call_depth_activate
- tevent_thread_call_depth_deactivate
- tevent_thread_call_depth_start
- tevent_thread_call_depth_stop
- tevent_thread_call_depth_reset_from_req
Note the changes to ABI/tevent-0.13.0.sigs only
revert the temporary changes made there...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
The change to lib/tevent/ABI/tevent-0.13.0.sigs will be reverted
in the commit for the 0.14.0 release...
Signed-off-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
This makes it more flexible and allow a caller to overload
a tevent backend. Which will be used by Samba in order to
glue in io_uring support.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
We'll export tevent_find_ops_byname() soon and will allow
the context_init() function of backends to find that standard ops
and hand over to standard_ops->context_init().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
There's no way to verify changes we would have to do tevent_port.c,
as we don't have access to a solaris build machine.
So better use the poll backend instead. In performance critical code
we typically don't deal with a lot of file descriptors so the impact
should be fairly minimal.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Note testsuite.c is only used in Samba's smbtorture as
'smbtorture //a/b local.event'
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Pavel Filipenský <pfilipen@redhat.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Björn Baumbach <bb@sernet.de>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Björn Baumbach <bb@sernet.de>
Autobuild-Date(master): Tue Jan 17 18:23:18 UTC 2023 on sn-devel-184
Put the division on the correct side of the inequality.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The cleanup phase of tfork_create() saves errno prior to calling
functions that might modify it, with the intention of restoring it
afterwards. However, the value of 'ret' is accidentally overwritten. It
will always be equal to 0, and hence errno will not be restored.
Fix this by introducing a new variable, ret2, for calling functions in
the cleanup phase.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Ensuring pointers are always initialised avoids compilation errors with
FORTIFY_SOURCE=2.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
No need to recompile the world when only a few files need this.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
To be used in smbXsrv_open.c, for this we need a lower bound.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The FreeBSD extattr API may return success and truncated
namelist. We need to check for this in bsd_attr_list to
ensure that we don't accidentally read off the end of the
buffer. In the case of a truncated value, the pascal
strings for attr names will reflect the lengths as if
the value were not truncated. For example:
`58DosStrea`
In case of short read we now set error to ERANGE and
fail.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15271
Signed-off-by: Andrew Walker <awalker@ixsystems.com>
Reviewed-by: Ralph Boehme <slow@samba.org>
Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Jan 2 14:27:23 UTC 2023 on sn-devel-184
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15248
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Fri Dec 23 14:17:31 UTC 2022 on sn-devel-184
There are four functions, allowing compression and decompression in
the two formats we support so far. The functions will accept bytes or
unicode strings which are treated as utf-8.
The LZ77+Huffman decompression algorithm requires an exact target
length to decompress, so this is mandatory.
The plain decompression algorithm does not need an exact length, but
you can provide one to help it know how much space to allocate. As
currently written, you can provide a short length and it will often
succeed in decompressing to a different shorter string.
These bindings are intended to make ad-hoc investigation easier, not
for production use. This is reflected in the guesses about output size
that plain_decompress() makes if you don't supply one -- either they
are stupidly wasteful or ridiculously insufficient, depending on
whether or not you were trying to decompress a 20MB string.
>>> a = '12345678'
>>> import compression
>>> b = compression.huffman_compress(a)
>>> b
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 #....
>>> len(b)
262
>>> c = compression.huffman_decompress(b, len(a))
>>> c
b'12345678' # note, c is bytes, a is str
>>> a
'12345678'
>>> d = compression.plain_compress(a)
>>> d
b'\xff\xff\xff\x0012345678'
>>> compression.plain_decompress(d) # no size specified, guesses
b'12345678'
>>> compression.plain_decompress(d,5)
b'12345'
>>> compression.plain_decompress(d,0) # 0 for auto
b'12345678'
>>> compression.plain_decompress(d,1)
b'1'
>>> compression.plain_decompress(a,444)
Traceback (most recent call last):
compression.CompressionError: unable to decompress data into a buffer of 444 bytes.
>>> compression.plain_decompress(b,444)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 #...
That last one decompresses the Huffman compressed file with the plain
compressor; pretty much any string is valid for plain decompression.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We shouldn't get a node with a zero code, and there's probably nothing
to do but stop.
CID 1517261 (#1-2 of 2): Bad bit shift operation
(BAD_SHIFT)11. negative_shift: In expression j >> offset - k,
shifting by a negative amount has undefined behavior. The shift
amount, offset - k, is -3.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec 19 23:29:04 UTC 2022 on sn-devel-184
Because we just wrote the intermediate representation to have no zero
distances, we can be sure it doesn't, but Coverity doesn't know. If
distance is zero, `bitlen_nonzero_16(distance)` would be bad.
CID 1517278 (#1 of 1): Bad bit shift operation
(BAD_SHIFT)41. large_shift: In expression 1 << code_dist, left
shifting by more than 31 bits has undefined behavior. The shift
amount, code_dist, is 65535.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Very long matches would be written instead as very very long matches.
We can't in fact hit this because we have a MAX_MATCH_LENGTH defined
as 64M, but if we could, it might make certain 2GB+ strings impossible
to compress.
CID 1517275 (#1 of 1): Unintended sign extension
(SIGN_EXTENSION)sign_extension: Suspicious implicit sign extension:
intermediate[i + 2UL] with type uint16_t (16 bits, unsigned) is
promoted in intermediate[i + 2UL] << 16 to type int (32 bits, signed),
then sign-extended to type unsigned long (64 bits, unsigned). If
intermediate[i + 2UL] << 16 is greater than 0x7FFFFFFF, the upper bits
of the result will all be 1.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
None of our test vectors are 18446744073709551615 bytes long, which
means we can know an `expected_length == returned_length` check will
catch the case where the compression function returns -1 for error. We
know that, but Coverity doesn't.
It's the same thing over and over again, in two different patterns:
>>> CID 1517301: Memory - corruptions (OVERRUN)
>>> Calling "memcmp" with "original.data" and "original.length" is
suspicious because of the very large index, 18446744073709551615. The index
may be due to a negative parameter being interpreted as unsigned.
393 if (original.length != decomp_written ||
394 memcmp(decompressed.data,
395 original.data,
396 original.length) != 0) {
397 debug_message("\033[1;31mgot %zd, expected %zu\033[0m\n",
398 decomp_written,
*** CID 1517299: Memory - corruptions (OVERRUN)
/lib/compression/tests/test_lzxpress_plain.c: 296 in
test_lzxpress_plain_decompress_more_compressed_files()
290 debug_start_timer();
291 written = lzxpress_decompress(p.compressed.data,
292 p.compressed.length,
293 dest,
294 p.decompressed.length);
295 debug_end_timer("decompress", p.decompressed.length);
>>> CID 1517299: Memory - corruptions (OVERRUN)
>>> Calling "memcmp" with "p.decompressed.data" and
"p.decompressed.length" is suspicious because of the very large index,
18446744073709551615. The index may be due to a negative parameter being
interpreted as unsigned.
296 if (written == p.decompressed.length &&
297 memcmp(dest, p.decompressed.data, p.decompressed.length)
== 0) {
298 debug_message("\033[1;32mdecompressed %s!
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We know that code is non-zero, because it comes from the combination of
the intermediate representation and the symbol tables that were generated
at the same time. But Coverity doesn't know that, and it thinks we could
be doing undefined things in the subsequent shift.
CID 1517302: Integer handling issues (BAD_SHIFT)
In expression "1 << code_bit_len", shifting by a negative amount has
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The struct write_context bit_len attribute is always between 0 and 31,
but if the next patches are applied without this, SUSE GCC -O3 will
worry thusly:
../../lib/compression/lzxpress_huffman.c: In function
‘lzxpress_huffman_compress’:
../../lib/compression/lzxpress_huffman.c:953:5: error: assuming signed
overflow does not occur when simplifying conditional to constant
[-Werror=strict-overflow]
if (wc->bit_len > 16) {
^
cc1: all warnings being treated as errors
Inspection tell us that the invariant holds. Nevertheless, we can
safely use an unsigned type and insist that over- or under- flow is
bad.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
The 'compressed' string can be about 9/8 the size of the decompressed
string, but we didn't allow enough memory in the fuzz target for that.
Then when it failed, we didn't check.
Credit to OSSFuzz.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
We had
output[output_pos - distance];
where output_pos and distance are size_t and distance can be greater
than output_pos (because it refers to a place in the previous block).
The underflow is defined, leading to a big number, and when
sizeof(size_t) == sizeof(*uint8_t) the subsequent overflow works as
expected. But if size_t is smaller than a pointer, bad things will
happen.
This was found by OSSFuzz with
'UBSAN_OPTIONS=print_stacktrace=1:silence_unsigned_overflow=1'.
Credit to OSSFuzz.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The linker has to relocate the pointers in the array at startup, save
that. I know we have bigger .data blobs, but every bit counts :-)
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
With the proper NULL checks we don't need the stackframe,
use a passed in context instead.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
The first call of talloc_asprintf_append() did not have a NULL check.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This adds intermediate NULL checks via talloc_asprintf_addbuf()
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
I wanted to use this in debug.c, but this would have meant to pollute
debug's deps with a lot of stuff. Also, looking through uses of
talloc_asprint_append(), very many of those don't do NULL checks
properly and could benefit from the _addbuf() flavor. We can add a
vasprintf variant later if the need shows up.
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This allows admins to disable enctypes completely if required.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
In order to allow better upgrades we need the default value for smb.conf to the
same even if the effective default value of the software changes in future.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This is not squashed in order to allow easier backports...
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We no longer have support for des encryption types in the kerberos
libraries anyway.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
aes encryption types are always supported.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Pair-Programmed-With: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This matches the Windows registry key
HKEY_LOCAL_MACHINE\System\CurrentControlSet\services\KDC\DefaultDomainSupportedEncTypes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15237
Pair-Programmed-With: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15240
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
AES is supported by Windows Server >= 2008R2, Windows (Client) >= 7 and Samba >= 4.0,
so there's no reason to allow md5 clients by default.
However some third party domain members may need it.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15240
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
AES is supported by Windows >= 2008R2 and Samba >= 4.0 so there's no
reason to allow md5 servers by default.
Note the change in netlogon_creds_cli_context_global() is only cosmetic,
but avoids confusion while reading the code. Check with:
git show -U35 libcli/auth/netlogon_creds_cli.c
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15240
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
<sys/stat.h> was missing from compression library tests which resulted
in the following compile time error:
../../lib/compression/tests/test_lzx_huffman.c: In function
‘datablob_from_file’:
../../lib/compression/tests/test_lzx_huffman.c:383:21: error:
storage size of ‘s’ isn’t known
383 | struct stat s;
| ^
../../lib/compression/tests/test_lzx_huffman.c:389:15: warning:
implicit declaration of function ‘fstat’ [-Wimplicit-function-declaration]
389 | ret = fstat(fileno(fh), &s);
| ^~~~~
Signed-off-by: Anoop C S <anoopcs@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Dec 6 11:39:16 UTC 2022 on sn-devel-184
lib/compression/tests/test_lzx_huffman.c: In function ‘test_lzxpress_huffman_overlong_matches’:
lib/compression/tests/test_lzx_huffman.c:1013:35: error: ‘j’ may be used uninitialized [-Werror=maybe-uninitialized]
1013 | assert_int_equal(score, i * j);
| ^
lib/compression/tests/test_lzx_huffman.c:979:19: note: ‘j’ was declared here
979 | size_t i, j;
| ^
lib/compression/tests/test_lzx_huffman.c: In function ‘test_lzxpress_huffman_overlong_matches_abc’:
lib/compression/tests/test_lzx_huffman.c:1059:39: error: ‘k’ may be used uninitialized [-Werror=maybe-uninitialized]
1059 | assert_int_equal(score, i * j * k);
| ^
lib/compression/tests/test_lzx_huffman.c:1020:22: note: ‘k’ was declared here
1020 | size_t i, j, k;
| ^
lib/compression/tests/test_lzx_huffman.c:1059:35: error: ‘j’ may be used uninitialized [-Werror=maybe-uninitialized]
1059 | assert_int_equal(score, i * j * k);
| ^
lib/compression/tests/test_lzx_huffman.c:1020:19: note: ‘j’ was declared here
1020 | size_t i, j, k;
| ^
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Sun Dec 4 09:12:30 UTC 2022 on sn-devel-184
This uses the same hash table method as lzxpress_huffman, though the
code can't be directly reused as the sizes of the offsets is
different, and there is not a block processing step here.
This will worsen the compression ratio compared to the exhaustive
search we previously used, though we still perform better than
Windows. To put numbers on it, the test files used to compress to 0.91
of Windows' compression size, and now they compress to 0.96.
On the other hand this is many orders of magnitude faster. It is
difficult to say exactly how much faster -- while the testsuite time
has only improved 200-fold (from 7 minutes to 2 seconds), most of the
remaining 2 seconds is used in data generation and management, not
compression. OSSFuzz consistently finds new vectors that time out
after a minute; on these we'll see nearly an order of magnitude of
orders of magnitude inprovement.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Autobuild-User(master): Joseph Sutton <jsutton@samba.org>
Autobuild-Date(master): Fri Dec 2 00:00:04 UTC 2022 on sn-devel-184
This makes it easier to rework the encoding decision to depend on a
hash table match rather than the current exhaustive search.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This will make it possible to move encoding operations into helper
functions, which will make it easier to restructure the code to use a
hash table for faster matching.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
These are based on (i.e. copied and pasted from) the LZ77 + Huffman
tests.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Everything that is in testdata/compression/lzxpress-huffman/ can also
be used for lzxpress plain tests, which is something we really need.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We are going to change from a slow exact match algorithm to a fast
heuristic search that will not always get the same results as the
exhaustive search.
To be precise, a million zeros will compress to 112 rather than 93 bytes.
We don't insist on an exact size, because that is not an issue here.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Mainly so I can go
make bin/test_lzxpress_plain && bin/test_lzxpress_plain
valgrind bin/test_lzxpress_plain
rr bin/test_lzxpress_plain
rr replay
in a tight loop.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
If compiled on Windows using Cygwin, MSYS2, or similar, this will output
compressed versions of files exactly as specified by MZ-XCA, if the
following conditions are met:
1. The file > 300 bytes.
2. The compressed file is smaller than the decompressed file.
Otherwise it returns the data unchanged. Without warning; that's just
how the API works.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Compression uses a 3 byte hash remember LZ77 matches in a 14-bit table.
This script runs the hash over all 16M combinations, then again over
all ASCII combinations, counting collisions to find hot-spots.
If you think you have a better hash, you are probably right, but you
should try it here -- alter h() -- before committing to it. This one is
literally the first one I thought of.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Huffman tree re-quantisation and perhaps other code paths are only
triggered by pathological data like this.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This compresses some data, decompresses it, and asserts that the
result is identical to the original string.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This differs from fuzz_lzxpress_huffman_round_trip (next commit) in
that the output buffer might be too small for the compressed data, in
which case we want to see an error and not a crash.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Most strings will not successfully decompress, which is OK. What we
care about of course is memory safety.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
With LZXHUFF_DEBUG_VERBOSE set, we measure the compression and
decompression rate relative to the decompressed size.
On reasonably long strings on my laptop, compiled with -O0, it turns
out to between 20 and 500 MB/s, both ways, depending on the complexity
of the string. Very short strings are of course dominated by overhead.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
If you need to see a Huffman tree (and sometimes you do), set
DEBUG_HUFFMAN_TREE to true at the top of lzxpress_huffman.c, and run:
make bin/test_lzx_huffman && bin/test_lzx_huffman
Actually, that will show you hundreds of trees, and you'll be glad of
that if you are ever trying to understand this.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Encoding without LZ77 matches is valid, and it is useful for isolating
bugs.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This compresses files as described in MS-XCA 2.2, and as decompressed
by the decompressor in the previous commit.
As with the decompressor, there are two public functions -- one that
uses a talloc context, and one that uses pre-allocated memory. The
compressor requires a tightly bound amount of auxillary memory
(>220kB) in a few different buffers, which is all gathered together in
the public struct lzxhuff_compressor_mem. An instantiated but not
initialised copy of this struct is required by the non-talloc
function; it can be used over and over again.
Our compression speed is about the same as the decompression speed
(between 20 and 500 MB/s on this laptop, depending on the data), and
our compression ratio is very similar to that of Windows.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This format is described in [MS-XCA] 2.1 and 2.2, with exegesis in
many posts on the cifs-protocol list[1].
The two public functions are:
ssize_t lzxpress_huffman_decompress(const uint8_t *input,
size_t input_size,
uint8_t *output,
size_t output_size);
uint8_t *lzxpress_huffman_decompress_talloc(TALLOC_CTX *mem_ctx,
const uint8_t *input_bytes,
size_t input_size,
size_t output_size);
In both cases the caller needs to know the *exact* decompressed size,
which is essential for decompression. The _talloc version allocates
the buffer for you, and uses the talloc context to allocate a 128k
working buffer. THe non-talloc function will allocate the working
buffer on the stack.
This compression format gives better compression for messages of
several kilobytes than the "plain" LXZPRESS compression, but is
probably a bit slower to decompress and is certainly worse for very
short messages, having a fixed 256 byte overhead for the first Huffman
table.
Experiments show decompression rates between 20 and 500 MB per second,
depending on the compression ratio and data size, on an i5-1135G7 with
no compiler optimisations.
This compression format is used in AD claims and in SMB, but that
doesn't happen with this commit.
I will not try to describe LZ77 or Huffman encoding here. Don't expect
an answer in MS-XCA either; instead read the code and/or Wikipedia.
[1] Much of that starts here:
https://lists.samba.org/archive/cifs-protocol/2022-October/
but there's more earlier, particularly in June/July 2020, when
Aurélien Aptel was working on an implementation that ended up in
Wireshark.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We are going to add more tests for lib/compression, and they can't all
be called "testsuite.c".
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Sometimes (e.g. in lzxpress Huffman encoding, and in some of our
tests: c.f. https://lists.samba.org/archive/samba-technical/2018-March/126010.html)
we want a stable sort algorithm (meaning one that retains the previous
order of items that compare equal).
The GNU libc qsort() is *usually* stable, in that it first tries to
use a mergesort but reverts to quicksort if the necessary allocations
fail. That has led Samba developers to unthinkingly assume qsort() is
stable which is not the case on many platforms, and might not always
be on GNU/Linuxes either.
This adds four functions. stable_sort() sorts an array, and requires
an auxiliary working array of the same size. stable_sort_talloc()
takes a talloc context so it ca create a working array and call
stable_sort(). stable_sort_r() takes an opaque context blob that gets
passed to the compare function, like qsort_r() and ldb_qsort(). And
stable_sort_talloc_r() rounds out the quadrant.
These are LGPL so that the can be used in ldb, which has problems with
unstable sort.
The tests are borrowed and extended from test_ldb_qsort.c.
When sorting non-trivial structs this is roughly as fast as GNU qsort,
but GNU qsort has optimisations for small items, using direct
assignments of rather than memcpy where the size allows the item to be
cast as some kind of int.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We need "struct iovec", which comes in via sys/uio.h, incuded by
system/filesys.h
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
We can't unconditionally assume (as we did in
third_party/heimdal_build/wscript_configure) that Heimdal has this type,
since we may have an older system Heimdal that lacks it. We must also
check whether krb5_pac_get_buffer() is usable with krb5_const_pac, and
declare krb5_const_pac as a non-const typedef if not.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
https://fedoraproject.org/wiki/Changes/PortingToModernC
We define True to true from stdbool.h and the same for false. So we
don't have to do a cleanup now.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Oct 27 19:11:30 UTC 2022 on sn-devel-184
The result is not used, it is only part of the macro to gain
type-checking.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This allows this type to be used in Samba in the future for
both Kerberos implementations
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
There were some reports that strace output an LDAP server socket is in
CLOSE_WAIT state, returning EAGAIN for writev over and over (after a call to
epoll() each time).
In the tstream_bsd code the problem happens when we have a pending
writev_send, while there's no readv_send pending. In that case
we still ask for TEVENT_FD_READ in order to notice connection errors
early, so we try to call writev even if the socket doesn't report TEVENT_FD_WRITE.
And there are situations where we do that over and over again.
It happens like this with a Linux kernel:
tcp_fin() has this:
struct tcp_sock *tp = tcp_sk(sk);
inet_csk_schedule_ack(sk);
sk->sk_shutdown |= RCV_SHUTDOWN;
sock_set_flag(sk, SOCK_DONE);
switch (sk->sk_state) {
case TCP_SYN_RECV:
case TCP_ESTABLISHED:
/* Move to CLOSE_WAIT */
tcp_set_state(sk, TCP_CLOSE_WAIT);
inet_csk_enter_pingpong_mode(sk);
break;
It means RCV_SHUTDOWN gets set as well as TCP_CLOSE_WAIT, but
sk->sk_err is not changed to indicate an error.
tcp_sendmsg_locked has this:
...
err = -EPIPE;
if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN))
goto do_error;
while (msg_data_left(msg)) {
int copy = 0;
skb = tcp_write_queue_tail(sk);
if (skb)
copy = size_goal - skb->len;
if (copy <= 0 || !tcp_skb_can_collapse_to(skb)) {
bool first_skb;
new_segment:
if (!sk_stream_memory_free(sk))
goto wait_for_space;
...
wait_for_space:
set_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
if (copied)
tcp_push(sk, flags & ~MSG_MORE, mss_now,
TCP_NAGLE_PUSH, size_goal);
err = sk_stream_wait_memory(sk, &timeo);
if (err != 0)
goto do_error;
It means if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) doesn't
hit as we only have RCV_SHUTDOWN and sk_stream_wait_memory returns
-EAGAIN.
tcp_poll has this:
if (sk->sk_shutdown & RCV_SHUTDOWN)
mask |= EPOLLIN | EPOLLRDNORM | EPOLLRDHUP;
So we'll get EPOLLIN | EPOLLRDNORM | EPOLLRDHUP triggering
TEVENT_FD_READ and writev/sendmsg keeps getting EAGAIN.
So we need to always clear TEVENT_FD_READ if we don't
have readable handler in order to avoid burning cpu.
But we turn it on again after a timeout of 1 second
in order to monitor the error state of the connection.
And now that our tsocket_bsd_error() helper checks for POLLRDHUP,
we can check if the socket is in an error state before calling the
writable handler when TEVENT_FD_READ was reported.
Only on error we'll call the writable handler, which will pick
the error without calling writev().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15202
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
If we found that the connection is broken, there's no point
in trying to use it anymore, so just return the first error we detected.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15202
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
This also returns an error if we got TCP_FIN from the peer,
which is only reported by an explicit POLLRDHUP check.
Also on FreeBSD getsockopt(fd, SOL_SOCKET, SO_ERROR) fetches
and resets the error, so a 2nd call no longer returns an error.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15202
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15202
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet@samba.org>
Autobuild-Date(master): Wed Oct 5 05:23:50 UTC 2022 on sn-devel-184
These tests are redeclared later and so are never used. Give them new
names so that they will be run again.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Getting an HMAC too long to fit our array is a programming error. It
should always be 64 bytes exactly.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This matches the other comparisons against krbtgt, kadmin, etc., which
are all case-sensitive.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The return codes of these functions are not often checked. Throwing an
exception ensures we won't continue blindly on if DN manipulation fails.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We presumably meant to clear this bit, rather than clearing all bits
other than it.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Oct 3 21:05:31 UTC 2022 on sn-devel-184
The failure in question would have to be a `talloc_strdup(dn, "")` in
ldb_dn_from_ldb_val().
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Also declared on line 257, exactly the same.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Panic if memset_s() fails.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
These functions allow us to perform key derivation and AES256 encryption
in Python. They will be used in a following commit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14611
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This lets us access MD4, which might not be available in hashlib, from
Python. This function is used in a following commit for hashing a
password to obtain the verifier for a SAMR password change.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14611
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This lets us access single-DES from Python. This function is used in a
following commit for encrypting an NT hash to obtain the verifier for a
SAMR password change.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14611
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
If samba_gnutls_aead_aes_256_cbc_hmac_sha512_encrypt() does not fill the
array completely, we may be comparing uninitialised bytes.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The reason for DISABLE_OPATH is to simulate a non-linux
system, so we should not use openat2() either.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
sys/syscall.h might be older than the runtime kernel.
If the kernel has support for openat2() we should
try to use if anyway.
The callers have to deal with ENOSYS anyway,
so there's no difference if we get that from syscall(__NR_openat2)
or directly from rep_openat2().
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
There's no glibc wrapper for openat2() yet, so we need
to use syscall(__NR_openat2) ourself.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Checking for presence of both netgroup and innetgr. INNETGR is not
defined on libc's such as musl so not checking results in a build error.
Signed-off-by: listout <brahmajit.xyz@gmail.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Mon Aug 8 07:28:31 UTC 2022 on sn-devel-184
This is just that we use the right parameters for gnutls_pbkdf2() and
reach the values from Windows.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
This will explicitly zero data from memory. This is guaranteed to be not
optimized away.
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
In this case it is probably better to crash out.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15103
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andreas Schneider <asn@samba.org>
This aims to minimise usage of the error-prone pattern of searching for
a just-added message element in order to make modifications to it (and
potentially finding the wrong element).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15009
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Currently, there are many places where we use ldb_msg_add_empty() to add
an empty element to a message, and then call ldb_msg_add_value() or
similar to add values to that element. However, this performs an
unnecessary search of the message's elements to locate the new element.
Moreover, if an element with the same attribute name already exists
earlier in the message, the values will be added to that element,
instead of to the intended newly added element.
A similar pattern exists where we add values to a message, and then call
ldb_msg_find_element() to locate that message element and sets its flags
to (e.g.) LDB_FLAG_MOD_REPLACE. This also performs an unnecessary
search, and may locate the wrong message element for setting the flags.
To avoid these problems, add functions for appending a value to a
message, so that a particular value can be added to the end of a message
in a single operation.
For ADD requests, it is important that no two message elements share the
same attribute name, otherwise things will break. (Normally,
ldb_msg_normalize() is called before processing the request to help
ensure this.) Thus, we must be careful not to append an attribute to an
ADD message, unless we are sure (e.g. through ldb_msg_find_element())
that an existing element for that attribute is not present.
These functions will be used in the next commit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15009
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Using the newly added ldb flag, we can now detect when a message has
been shallow-copied so that its elements share their values with the
original message elements. Then when adding values to the copied
message, we now make a copy of the shared values array first.
This should prevent a use-after-free that occurred in LDB modules when
new values were added to a shallow copy of a message by calling
talloc_realloc() on the original values array, invalidating the 'values'
pointer in the original message element. The original values pointer can
later be used in the database audit logging module which logs database
requests, and potentially cause a crash.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15009
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
When making a shallow copy of an ldb message, mark the message elements
of the copy as sharing their values with the message elements in the
original message.
This flag value will be heeded in the next commit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15009
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This avoids wasting getpid() calls in a lot of places...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Our messaging code is very performance critical and
we should note waste time in getpid() syscalls...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
- add tevent_cached_getpid()
Note the changes to ABI/tevent-0.12.1.sigs only
revert the temporary changes made there...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
This avoids a getpid() syscall per tevent_loop_once() iteration.
We provide tevent_cached_getpid() also as helper for external consumers
in order to have the logic only once.
Note the change to ABI/tevent-0.12.1.sigs will be reverted
with the bump to 0.13.0.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Sun Jul 24 00:25:49 UTC 2022 on sn-devel-184
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15073
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Jul 19 12:17:35 UTC 2022 on sn-devel-184
We now allow this to be via the ENCTYPE_AES256_CTS_HMAC_SHA1_96 hash instead
which allows us to decouple Samba from the unsalted NT hash for
organisations that are willing to take this step (for user accounts).
(History checking is limited to the last three passwords only, as
ntPwdHistory is limited to NT hash values, and the PrimaryKerberosCtr4
package only stores three sets of keys.)
Since we don't store a salt per-key, but only a single salt, the check
will fail for a previous password if the account was renamed prior to a
newer password being set.
Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Best reviewed with `git show -b`
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15104
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Jun 22 11:49:23 UTC 2022 on sn-devel-184
Often we go 'torture_assert(tctx, expr, talloc_asprintf(tctx, "foo
%s", foo));' which is just a pain.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15040
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The executables generated from lib/util/tests/test_logging.c are used
by the samba.tests.logfiles tests to test logging with various
smb.confs that assign classes to various files at different levels
etc.
Previously test_logging.c had its own version of the table; now it
shares one with debug.c
We put the table in a sub-directory (lib/util/debug-classes/), because
adding local_include=True to the wscript_build stanza causes the
compiler confusion between <time.h> and lib/util/time.h.
Note: there are still two other lists of the class names, in
python/samba/tests/logfiles.py and
docs-xml/smbdotconf/logging/loglevel.xml.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This can be a useful macro when you are trying to track the behaviour
of one process out of the dozens that samba starts up, and when your
interest is in following it over time, not necessarily in a single
stack.
In DEVELOPER mode, if you call 'debug_developer_enable()' in the
process you're following, then any instances of DBG_DEV() will work
like DBG_ERR(), also adding ":DEV:12345:" where "12345" is the pid of
th current process.
Within debug.c itself, the macro always writes to stderr, because the
debug.c functions are not all reentrant.
When not in DEVELOPER MODE, the macro evaluates to nothing.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
There is one knownfail, where it seems an smb.conf like
log file = foo
log level = 2 tdb:2@baa ldb:3
will send the ldb logs to 'baa' instead of 'foo' (i.e., the last
opened log file, rather than the default log file).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
The 'log level' line in smb.conf allows messages from different log
classes to be sent to different places, but we have not tested that
this works. Now we do, somewhat.
The test involves running a special binary based on a stripped down
source4/samba/server.c that just starts up, parses the command line
and a given smb.conf, then logs messages from multiple classes and
exits.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
CTDB doesn't want this redirection of stderr to the log file. It
expects to be able to capture stderr of subprocesses and log them with
a header. This redirection stops that from happening.
Unfortunately this has to be a negative option (i.e. "no" in the name)
so that the default of 0/false maintains existing behaviour.
Note that the default behaviour is sub-optimal because it causes raw
data (i.e. debug data without a header) to appear in the log.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15090
Signed-off-by: Martin Schwenke <martin@meltin.net>
Reviewed-by: Volker Lendecke <vl@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
gnutls_memcmp() is mostly identical to our own implementation, except
that ours will not break if supplied with 4 GiB or more of data.
However, using an external function permits us to disclaim
responsibility if some CPU/compiler combination happens to invalidate
our constant-time guarantee.
For reference, gnutls_memcmp() implementation:
78d9820de0/lib/safe-memfuncs.c (L41-67)
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Ensure that it gives the correct results for comparing two memory
regions.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Ensure that it gives the correct results for comparing two data blobs.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Since memcmp_const_time() doesn't act as an exact replacement for
memcmp(), and its return value is only ever compared with zero, simplify
it and emphasize the intention of checking equality by returning a bool
instead.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Since data_blob_cmp_const_time() doesn't act as an exact replacement for
data_blob_cmp(), and its return value is only ever compared with zero,
simplify it and emphasize the intention of checking equality by
returning a bool instead.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This helps to avoid timing attacks.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15010
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This allows it to be used in more places without needing to introduce
more dependencies.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Backtraces when Samba is in PANIC state are better with
backtrace_symbols() than with libunwind on Ubuntu 20.04 x86_64
so move libunwind to a off-by-default option, prompted for
if backtrace_symbols() is not available.
Based on a request by Fco Javier Felix <ffelix@inode64.com>
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
We look for backtrace_symbols() in this library, so we should link against
it if we find it.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Previously this would leave static functions unused, which the compiler will
not allow for a developer build.
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
This partly reverts commit 620de975f1.
Drop chunk including system/nis.h, drop wscript_build modifications,
use getdomainname() from glibc instead of yp_get_default_domain() from
libnsl.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15087
Signed-off-by: Samuel Cabrero <scabrero@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
* Fix build problems
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15071
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Wed Jun 8 17:57:53 UTC 2022 on sn-devel-184
The smbconf library defines an enum of error codes that can be returned
from the C calls. The error codes were getting stored in the python
SMBConfError type but it was not easy to access or obvious what the
integer code represented.
This change makes it easier to get the returned error code: via a
`error_code` attribute on the exception value. It also exposes the
integer constants to the module. Simple tests for a few of the more
obvious error codes check that this new error handling correctly
exposes the error code values.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Guenther Deschner <gd@samba.org>
Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Wed Jun 8 13:13:10 UTC 2022 on sn-devel-184
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14765
RN: add new smb.conf parameter "volume serial number" to allow overriding
the generated default value
Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15069
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Christof Schmitt <cs@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Jun 3 21:53:31 UTC 2022 on sn-devel-184
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Mon May 30 11:03:47 UTC 2022 on sn-devel-184
The first time round we maybe didn't know which files we wanted to log to.
Suppose, for example, we had an smb.conf with
log level = 1 dsdb_group_json_audit:5@/var/log/group_json.log
we wouldn't see anything in "/var/log/group_json.log", while the level
5 dsdb_group_json_audit messages would go into the main log.
Note that the named file would still be opened by winbindd and others
that use the s3 code, but would remain empty as they don't have anything
to say about dsdb_group_json_audit.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15076
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
This just prints a warning for:
ms_fnmatch.c:95:51: error: array subscript 0 is outside array bounds of
‘struct max_n[0]’ [-Werror=array-bounds]
95 | if (max_n != NULL && max_n->predot &&
|
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Alexander Bokovoy <ab@samba.org>
A simple degenerate case for our compressor has been a large number of
repeated bytes that will match the maximum length (~64k) at all 8192
search positions, 8191 of which searches are in vain because the
matches are not of greater length than the first one.
Here we recognise the inevitable and reduce runtime proportionately.
Credit to OSS-Fuzz.
REF: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47428
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Autobuild-User(master): Douglas Bagnall <dbagnall@samba.org>
Autobuild-Date(master): Tue May 17 23:11:21 UTC 2022 on sn-devel-184
We get *very* slow when long runs of the bytes are the same. On this
laptop the test takes 18s; with the next commit it will be 0.006s.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We say it is an error to end up at a different result.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
You'll thank me if you're ever debugging these and wondering why
'lzxpress4' calls 'lzxpress2' (or is it the other way round?).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
We set `uncompressed_pos = 0;` unconditionally, just ~10 lines up.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This mirrors the behaviour of lzxpress_compress, which "encodes" an
empty string as an empty string.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Every so often, lzxpress adds a 32-bit block of indicator flags to
help decode the next clump of 32 code words. A naive compressor (such
as we have) might do this at the very end for flags that aren't
actually used because there are no more bytes to decompress. If that
happens we need to stop processing, or we'll come to worse outcome at
the next CHECK_INPUT_BYTES.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Empty strings and trailing flag blocks.
(found with Honggfuzz and a round-trip fuzzer that aborts if the
strings differ).
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This makes the code clearer.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This is more consistent with the compression code.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This makes the code clearer.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
This simplifies the code and makes it clearer.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
If indic_bit == 0, the shift amount of 32 - indic_bit == 32 will equal
the width of a 32-bit integer type, and these shifts will invoke
undefined behaviour, which is likely to cause incorrect output. Fix this
by not shifting a 32-bit integer type by 32 bits or more.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
We can simplify this code using the identity:
byte_left + uncompressed_pos = uncompressed_size
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
This now matches the other use of indic_pos.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
If nibble_index is non-zero, we have already written to it, and so don't
need to check again that it is in bounds.
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
