1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
Commit Graph

131688 Commits

Author SHA1 Message Date
Douglas Bagnall
307aded670 fuzz: add fuzz_lzxpress_huffman_compress
This differs from fuzz_lzxpress_huffman_round_trip (next commit) in
that the output buffer might be too small for the compressed data, in
which case we want to see an error and not a crash.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
cda3c1a227 fuzz: add fuzz_lzxpress_huffman_decompress
Most strings will not successfully decompress, which is OK. What we
care about of course is memory safety.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
e795985067 lib/compression/tests: add lzhuffman timer functions
With LZXHUFF_DEBUG_VERBOSE set, we measure the compression and
decompression rate relative to the decompressed size.

On reasonably long strings on my laptop, compiled with -O0, it turns
out to between 20 and 500 MB/s, both ways, depending on the complexity
of the string. Very short strings are of course dominated by overhead.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
77048aaa61 lib/compression: debug routines for lzxpress-huffman
If you need to see a Huffman tree (and sometimes you do), set
DEBUG_HUFFMAN_TREE to true at the top of lzxpress_huffman.c, and run:

  make bin/test_lzx_huffman && bin/test_lzx_huffman

Actually, that will show you hundreds of trees, and you'll be glad of
that if you are ever trying to understand this.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
955214ef6e lib/compression/lzhuff: add debug flag to skip LZ77
Encoding without LZ77 matches is valid, and it is useful for isolating
bugs.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
d4e3f0c88e lib/compression: LZ77 + Huffman compression
This compresses files as described in MS-XCA 2.2, and as decompressed
by the decompressor in the previous commit.

As with the decompressor, there are two public functions -- one that
uses a talloc context, and one that uses pre-allocated memory. The
compressor requires a tightly bound amount of auxillary memory
(>220kB) in a few different buffers, which is all gathered together in
the public struct lzxhuff_compressor_mem. An instantiated but not
initialised copy of this struct is required by the non-talloc
function; it can be used over and over again.

Our compression speed is about the same as the decompression speed
(between 20 and 500 MB/s on this laptop, depending on the data), and
our compression ratio is very similar to that of Windows.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
f86035c65b lib/compression: add LZ77 + Huffman decompression
This format is described in [MS-XCA] 2.1 and 2.2, with exegesis in
many posts on the cifs-protocol list[1].

The two public functions are:

ssize_t lzxpress_huffman_decompress(const uint8_t *input,
				    size_t input_size,
				    uint8_t *output,
				    size_t output_size);

uint8_t *lzxpress_huffman_decompress_talloc(TALLOC_CTX *mem_ctx,
					    const uint8_t *input_bytes,
					    size_t input_size,
					    size_t output_size);

In both cases the caller needs to know the *exact* decompressed size,
which is essential for decompression. The _talloc version allocates
the buffer for you, and uses the talloc context to allocate a 128k
working buffer. THe non-talloc function will allocate the working
buffer on the stack.

This compression format gives better compression for messages of
several kilobytes than the "plain" LXZPRESS compression, but is
probably a bit slower to decompress and is certainly worse for very
short messages, having a fixed 256 byte overhead for the first Huffman
table.

Experiments show decompression rates between 20 and 500 MB per second,
depending on the compression ratio and data size, on an i5-1135G7 with
no compiler optimisations.

This compression format is used in AD claims and in SMB, but that
doesn't happen with this commit.

I will not try to describe LZ77 or Huffman encoding here. Don't expect
an answer in MS-XCA either; instead read the code and/or Wikipedia.

[1] Much of that starts here:

https://lists.samba.org/archive/cifs-protocol/2022-October/

but there's more earlier, particularly in June/July 2020, when
Aurélien Aptel was working on an implementation that ended up in
Wireshark.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
bd35feaf7e testdata: add test vectors for LZ77+Huffman [de-]compression
Some of the decompressed files were found via fuzzing, some are public
domain texts, and some are designed to test one aspect or another of
the format. For example, some aspects of Huffman tree creation can
only be tested when there is an extreme imbalance in the frequency of
symbols.

See the README for what files are where.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
7cff3ce284 test/source_chars: ignore testdata/compression
We are going to have all kinds of rubbish there.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
f6cda06dfb lib/compression: move lzxpress_plain test into tests/
We are going to add more tests for lib/compression, and they can't all
be called "testsuite.c".

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
e24efb88ef fuzz: add fuzzers for stable_sort
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Douglas Bagnall
4e18e92399 util: add stable sort functions
Sometimes (e.g. in lzxpress Huffman encoding, and in some of our
tests: c.f. https://lists.samba.org/archive/samba-technical/2018-March/126010.html)
we want a stable sort algorithm (meaning one that retains the previous
order of items that compare equal).

The GNU libc qsort() is *usually* stable, in that it first tries to
use a mergesort but reverts to quicksort if the necessary allocations
fail. That has led Samba developers to unthinkingly assume qsort() is
stable which is not the case on many platforms, and might not always
be on GNU/Linuxes either.

This adds four functions. stable_sort() sorts an array, and requires
an auxiliary working array of the same size. stable_sort_talloc()
takes a talloc context so it ca create a working array and call
stable_sort(). stable_sort_r() takes an opaque context blob that gets
passed to the compare function, like qsort_r() and ldb_qsort(). And
stable_sort_talloc_r() rounds out the quadrant.

These are LGPL so that the can be used in ldb, which has problems with
unstable sort.

The tests are borrowed and extended from test_ldb_qsort.c.

When sorting non-trivial structs this is roughly as fast as GNU qsort,
but GNU qsort has optimisations for small items, using direct
assignments of rather than memcpy where the size allows the item to be
cast as some kind of int.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
Jeremy Allison
39df9f4a59 s3: smbd: Fix schedule_smb2_aio_read() to allow the last read in a compound to go async.
Remove knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Thu Dec  1 16:04:07 UTC 2022 on sn-devel-184
2022-12-01 16:04:07 +00:00
Jeremy Allison
0bb4810719 s3: smbd: Fix schedule_aio_smb2_write() to allow the last write in a compound to go async.
Remove knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-12-01 15:04:58 +00:00
Jeremy Allison
088b8a1e3e s4: torture: Add compound_async.read_read test to show we don't go async on the last read in a compound.
Add knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-12-01 15:04:58 +00:00
Jeremy Allison
ffd9b94fe0 s4: torture: Add compound_async.write_write test to show we don't go async on the last write in a compound.
Add knownfail.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-12-01 15:04:58 +00:00
Jeremy Allison
fc6c76e6da s4: torture: Tweak the compound padding streamfile test to send 3 reads instead of 2, and check the middle read padding.
The protocol allows the last read in a related compound to be split
off and possibly go async (and smbd soon will do this). If the
last read is split off, then the padding is different. By sending
3 reads and checking the padding on the 2nd read, we cope with
the smbd change and are still correctly checking the padding
on a compound related read.

Do this for the stream filename compound padding test.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-12-01 15:04:58 +00:00
Jeremy Allison
48b12f11a5 s4: torture: Tweak the compound padding basefile test to send 3 reads instead of 2, and check the middle read padding.
The protocol allows the last read in a related compound to be split
off and possibly go async (and smbd soon will do this). If the
last read is split off, then the padding is different. By sending
3 reads and checking the padding on the 2nd read, we cope with
the smbd change and are still correctly checking the padding
on a compound related read.

Do this for the base filename compound padding test.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-12-01 15:04:58 +00:00
Jeremy Allison
f5b2ae5809 s3: tests: Change smb2.compound_async to run against share aio_delay_inject instead of tmp.
It doesn't hurt the fsync compound async tests, and we need this for
the next commits to ensure smb2_read/smb2_write compound tests take
longer than 500ms so can be sure the last read/write in the compound
will go async.

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-12-01 15:04:58 +00:00
Andreas Schneider
49b40a1334 s4:torture: Fix segfault in multichannel test
The timer for the timeout_cb() handler was created on a memory context
which doesn't get freed, so the timer was still valid when running
the next test and fired there. It was then writing into random memory
leading to segfaults.

Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Thu Dec  1 15:03:19 UTC 2022 on sn-devel-184
2022-12-01 15:03:19 +00:00
Volker Lendecke
357bafe625 smbd: Allow POSIX getinfo levels for smb3 unix extensions
Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: David Mulder <dmulder@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Tue Nov 29 11:23:58 UTC 2022 on sn-devel-184
2022-11-29 11:23:58 +00:00
David Mulder
bbc82a5d42 s3: Test that store_smb2_posix_info hides info for '..'
Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:38 +00:00
David Mulder
d0ad452fc8 s3: smbd: store_smb2_posix_info hide info for '..'
When receiving a query for '..', hide the owner
and group sids, the inode, and the dev id.

Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:38 +00:00
David Mulder
bdb98c8397 smbd: Implement SMB2_FS_POSIX_INFORMATION_INTERNAL
Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
a73d903238 tests/s3: Test file/dir permissions with SMB3 posix
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
09c8426b95 tests/s3: Test case sensitive open with SMB3 posix
Disabled because we don't handle posix paths
correctly yet.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
160173ee06 tests/s3: Test delete on close with SMB3 posix
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
f481cd4a60 libcli: Add client support for SMB2_FILE_POSIX_INFORMATION
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
f0e1137425 tests/s3: Test reserved chars in posix filename
Disabled because we don't handle posix paths
correctly yet.

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
08226d6c2e smbd: Implement SMB2_FILE_POSIX_INFORMATION in smbd_marshall_dir_entry
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
7c2f08d564 tests/s3: Test SMB2_FIND_POSIX_INFORMATION dir query
Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
284787996d libsmb: Allow listing with posix context
Signed-off-by: David Mulder <dmulder@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
David Mulder
99de8d7cfa libsmb: Make info_level configurable in dir listing
This was hard coded to SMB2_FIND_ID_BOTH_DIRECTORY_INFO

Signed-off-by: David Mulder <dmulder@suse.com>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
Jeremy Allison
2c1a02d622 smbd: Plumb SMB2_FIND_POSIX_INFORMATION through the directory reading code.
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
Jeremy Allison
72004f8f94 s3: smbd: Add SMB2_FILE_POSIX_INFORMATION getinfo info level (100 on the wire).
Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-29 10:26:37 +00:00
Ralph Boehme
535a08dfc4 smbd: reject FILE_ATTRIBUTE_TEMPORARY on directories
Cf MS-FSA 2.1.5.14.2

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15252

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Mon Nov 28 10:14:12 UTC 2022 on sn-devel-184
2022-11-28 10:14:12 +00:00
Ralph Boehme
fdb19ce8aa torture: add a test trying to set FILE_ATTRIBUTE_TEMPORARY on a directory
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15252

Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-28 09:19:33 +00:00
Stefan Metzmacher
c8bf9495f4 vfs: fix the build of nfs4acl_xattr_ without rpc/xdr.h support
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>

Autobuild-User(master): Ralph Böhme <slow@samba.org>
Autobuild-Date(master): Fri Nov 25 06:07:32 UTC 2022 on sn-devel-184
2022-11-25 06:07:32 +00:00
Ralph Boehme
3b9ccfa4ac net: use correct printf format, fi3_id is an uint32_t
Signed-off-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Nov 24 16:39:12 UTC 2022 on sn-devel-184
2022-11-24 16:39:12 +00:00
Stefan Metzmacher
95676825ad gitlab-ci: do some basic testing on ubuntu1804-32bit
For now we allow build warnings and only do some basic testing.
We also ignore timestamp related problems, as well as some charset
failures.

Over time we should try to address the situation by not allowing warnings
and verify if expected failures are harmless or not.

But it's already much better then having no 32bit testing at all!

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Thu Nov 24 12:05:26 UTC 2022 on sn-devel-184
2022-11-24 12:05:26 +00:00
Stefan Metzmacher
98c1e357a7 selftest: add --default-ldb-backend option
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
9ba10b97d3 selftest: samba-ktest-mit also needs $ENV{KRB5RCACHETYPE} = "none"
We need to pass --mitkrb5 to selftest.pl in all cases we use
system mit kerberos not only when we also test the kdc.

We can't use a replay cache in selftest verifies the stat.st_uid
against getuid().

BTW: while debugging this on ubuntu 22.04 I exported
KRB5_TRACE="/dev/stderr", which means we get tracing into
the servers log file and into selftest_prefix/subunit for the client...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
dce639f8bd CVE-2022-42898: HEIMDAL: lib/krb5: fix _krb5_get_int64 on systems where 'unsigned long' is just 32-bit
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15203

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
838f620787 third_party: Update socket_wrapper to version 1.3.5
This injects O_LARGEFILE as needed.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
6dddb268df lib/replace: let rep_openat2() inject O_LARGEFILE as needed
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15251

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Ralph Boehme <slow@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
4c2e1d6259 s3:locking: relax __SHARE_MODE_LOCK_SPACE check for 32bit platforms
sizeof(struct share_mode_lock) is only 28 bytes instead of 32 bytes
on 32bit systems...

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
44192d5f2c s4:kdc: make sure reset_bad_password_netlogon() stops subreq before return
We pass the stack variable 'req' to dcerpc_winbind_SendToSam_r_send(),
so we need to make sure the runtime of the subreq in not longer
than the stack variable.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=15253

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
73ec725313 s4:messaging: add irpc_bh_do_ndr_print() in order to debug irpc calls
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15253

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-24 11:01:37 +00:00
Stefan Metzmacher
1414269dcc CVE-2021-20251: s4:auth: fix use after free in authsam_logon_success_accounting()
This fixes a use after free problem introduced by
commit 7b8e32efc3,
which has msg = current; which means the lifetime
of the 'msg' memory is no longer in the scope of th
caller.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=14611
BUG: https://bugzilla.samba.org/show_bug.cgi?id=15253

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2022-11-24 11:01:37 +00:00
Joseph Sutton
2dcd8369fe bootstrap: Remove duplicate line from CentOS 8 powertools install
This was missed in 136ec5bc01 when
we moved to CentOS 8 stream.

Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
2022-11-24 11:01:37 +00:00