1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-22 13:34:15 +03:00
samba-mirror/lib
Douglas Bagnall f86035c65b lib/compression: add LZ77 + Huffman decompression
This format is described in [MS-XCA] 2.1 and 2.2, with exegesis in
many posts on the cifs-protocol list[1].

The two public functions are:

ssize_t lzxpress_huffman_decompress(const uint8_t *input,
				    size_t input_size,
				    uint8_t *output,
				    size_t output_size);

uint8_t *lzxpress_huffman_decompress_talloc(TALLOC_CTX *mem_ctx,
					    const uint8_t *input_bytes,
					    size_t input_size,
					    size_t output_size);

In both cases the caller needs to know the *exact* decompressed size,
which is essential for decompression. The _talloc version allocates
the buffer for you, and uses the talloc context to allocate a 128k
working buffer. THe non-talloc function will allocate the working
buffer on the stack.

This compression format gives better compression for messages of
several kilobytes than the "plain" LXZPRESS compression, but is
probably a bit slower to decompress and is certainly worse for very
short messages, having a fixed 256 byte overhead for the first Huffman
table.

Experiments show decompression rates between 20 and 500 MB per second,
depending on the compression ratio and data size, on an i5-1135G7 with
no compiler optimisations.

This compression format is used in AD claims and in SMB, but that
doesn't happen with this commit.

I will not try to describe LZ77 or Huffman encoding here. Don't expect
an answer in MS-XCA either; instead read the code and/or Wikipedia.

[1] Much of that starts here:

https://lists.samba.org/archive/cifs-protocol/2022-October/

but there's more earlier, particularly in June/July 2020, when
Aurélien Aptel was working on an implementation that ended up in
Wireshark.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
..
addns libaddns: remove duplicate declaration 2022-09-16 05:46:35 +00:00
afs s3:param: make "servicename" a substituted option 2019-11-27 10:25:37 +00:00
async_req lib: Use FIONREAD in wait_for_read_send/recv 2021-03-16 17:09:31 +00:00
audit_logging audit_logging: add method to replace the object for a given key with a new object 2022-08-08 12:56:28 +00:00
cmdline lib:cmdline: Fix error handling of --client-protection=sign|encrypt|off 2022-06-22 11:49:23 +00:00
compression lib/compression: add LZ77 + Huffman decompression 2022-12-01 22:56:39 +00:00
crypto lib:crypto: Change error return to SMB_ASSERT() 2022-10-05 04:23:32 +00:00
dbwrap lib/dbwrap: allow dbwrap_merge_dbufs() to update an existing buffer 2022-09-20 00:34:35 +00:00
fuzzing fuzz: add fuzzers for stable_sort 2022-12-01 22:56:39 +00:00
krb5_wrap krb5: Detect support for krb5_const_pac type 2022-11-08 02:39:37 +00:00
ldb ldb: don't call comparison() directly in LDB_TYPESAFE_QSORT 2022-10-21 03:57:33 +00:00
ldb-samba pyldb: Fix typos in function names 2022-10-05 05:23:50 +00:00
messaging lib/messaging: s/getpid/tevent_cached_getpid 2022-07-25 17:34:33 +00:00
mscat lib;smbd: Fix the -Os build by initializing variables 2021-08-06 17:22:30 +00:00
param dsdb: Allow password history and password changes without an NT hash 2022-06-26 22:10:29 +00:00
printer_driver printing: Align integer types 2021-04-01 19:32:36 +00:00
pthreadpool build: Do not build selftest binaries for builds without --enable-selftest 2019-11-22 11:48:59 +00:00
replace lib/replace: let rep_openat2() inject O_LARGEFILE as needed 2022-11-24 11:01:37 +00:00
smbconf lib/smbconf: expose smbconf error codes to python wrapper 2022-06-08 13:13:10 +00:00
socket lib/socket: autodetect RSS using ETHTOOL_GRXRINGS 2020-05-07 14:44:40 +00:00
talloc talloc: version 2.3.4 2022-06-08 17:02:29 +00:00
tdb tdb: version 1.4.7 2022-06-08 17:57:53 +00:00
tdb_wrap lib: Open tdb files with O_CLOEXEC 2021-06-04 16:47:34 +00:00
tdr lib: Fix 1354521 Unchecked return value 2016-03-01 21:49:44 +01:00
tevent tevent: Fix flag clearing 2022-10-03 21:05:31 +00:00
texpect texpect: don't ignore unknown options 2021-09-10 15:10:30 +00:00
torture torture: add torture_assertf() 2022-06-17 01:28:30 +00:00
tsocket lib/tsocket: avoid endless cpu-spinning in tstream_bsd_fde_handler() 2022-10-19 16:14:36 +00:00
util util: add stable sort functions 2022-12-01 22:56:39 +00:00
README various: Remove references to about to be deleted thirdparty/dnspython 2018-12-11 20:07:18 +01:00
wscript_build

compression - Various compression algorithms (MSZIP, lzxpress)
popt - Command-line option parsing library
replace - Provides replacements for standard (POSIX, C99) functions 
          not provided by the host platform.
subunit - Utilities and bindings for working with the Subunit test result 
          reporting protocol.
talloc - Hierarchical pool based memory allocator 
tdb - Simple but fast key/value database library, supporting multiple writers
torture - Simple unit testing helper library