1
0
mirror of https://github.com/samba-team/samba.git synced 2025-01-25 06:04:04 +03:00
Douglas Bagnall f86035c65b lib/compression: add LZ77 + Huffman decompression
This format is described in [MS-XCA] 2.1 and 2.2, with exegesis in
many posts on the cifs-protocol list[1].

The two public functions are:

ssize_t lzxpress_huffman_decompress(const uint8_t *input,
				    size_t input_size,
				    uint8_t *output,
				    size_t output_size);

uint8_t *lzxpress_huffman_decompress_talloc(TALLOC_CTX *mem_ctx,
					    const uint8_t *input_bytes,
					    size_t input_size,
					    size_t output_size);

In both cases the caller needs to know the *exact* decompressed size,
which is essential for decompression. The _talloc version allocates
the buffer for you, and uses the talloc context to allocate a 128k
working buffer. THe non-talloc function will allocate the working
buffer on the stack.

This compression format gives better compression for messages of
several kilobytes than the "plain" LXZPRESS compression, but is
probably a bit slower to decompress and is certainly worse for very
short messages, having a fixed 256 byte overhead for the first Huffman
table.

Experiments show decompression rates between 20 and 500 MB per second,
depending on the compression ratio and data size, on an i5-1135G7 with
no compiler optimisations.

This compression format is used in AD claims and in SMB, but that
doesn't happen with this commit.

I will not try to describe LZ77 or Huffman encoding here. Don't expect
an answer in MS-XCA either; instead read the code and/or Wikipedia.

[1] Much of that starts here:

https://lists.samba.org/archive/cifs-protocol/2022-October/

but there's more earlier, particularly in June/July 2020, when
Aurélien Aptel was working on an implementation that ended up in
Wireshark.

Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Pair-programmed-with: Joseph Sutton <josephsutton@catalyst.net.nz>
Reviewed-by: Joseph Sutton <josephsutton@catalyst.net.nz>
2022-12-01 22:56:39 +00:00
..
2022-03-03 00:59:34 +00:00
2022-03-03 00:59:34 +00:00
2022-03-03 00:59:34 +00:00
2022-03-03 00:59:34 +00:00
2022-02-02 17:36:35 +00:00
2022-03-03 00:59:34 +00:00
2020-02-04 05:13:39 +00:00
2013-12-11 17:14:21 +01:00

# vim: ft=rst

This directory contains test scripts that are useful for running a
bunch of tests all at once.

There are two parts to this:

 * The test runner (selftest/selftest.pl)
 * The test formatter

selftest.pl simply outputs subunit, which can then be formatted or analyzed
by tools that understand the subunit protocol. One of these tools is
format-subunit, which is used by default as part of "make test".

Available testsuites
====================
The available testsuites are obtained from a script, usually
source{3,4}/selftest/tests.py. This script should for each testsuite output
the name of the test, the command to run and the environment that should be
provided. Use the included "plantest" function to generate the required output.

Testsuite behaviour
===================

Exit code
------------
The testsuites should exit with a non-zero exit code if at least one
test failed. Skipped tests should not influence the exit code.

Output format
-------------
Testsuites can simply use the exit code to indicate whether all of their
tests have succeeded or one or more have failed. It is also possible to
provide more granular information using the Subunit protocol.

This protocol works by writing simple messages to standard output. Any
messages that can not be interpreted by this protocol are considered comments
for the last announced test.

For a full description of the subunit protocol, see the README file in the subunit
repository at http://github.com/testing-cabal/subunit.

The following commands are Samba extensions to Subunit:

start-testsuite
~~~~~~~~~~~~~~~
start-testsuite: name

The testsuite name is used as prefix for all containing tests.

skip-testsuite
~~~~~~~~~~~~~~
skip-testsuite: name

Mark the testsuite with the specified name as skipped.

testsuite-success
~~~~~~~~~~~~~~~~~
testsuite-success: name

Indicate that the testsuite has succeeded successfully.

testsuite-fail
~~~~~~~~~~~~~~
testsuite-fail: name

Indicate that a testsuite has failed.

Environments
============
Tests often need to run against a server with particular things set up,
a "environment". This environment is provided by the test "target": Samba 3,
Samba 4 or Windows.

The environments are currently available include

 - none: No server set up, no variables set.
 - dc,s3dc: Domain controller set up. The following environment variables will
   be set:

     * USERNAME: Administrator user name
     * PASSWORD: Administrator password
     * DOMAIN: Domain name
     * REALM: Realm name
     * SERVER: DC host name
     * SERVER_IP: DC IPv4 address
     * SERVER_IPV6: DC IPv6 address
     * NETBIOSNAME: DC NetBIOS name
     * NETIOSALIAS: DC NetBIOS alias

 - member,s4member,s3member: Domain controller and member server that is joined to it set up. The
   following environment variables will be set:

     * USERNAME: Domain administrator user name
     * PASSWORD: Domain administrator password
     * DOMAIN: Domain name
     * REALM: Realm name
     * SERVER: Name of the member server

See Samba.pm, Samba3.pm and Samba4.pm for the full list.

Running tests
=============

To run all the tests use::

   make test

To run a quicker subset run::

   make quicktest

To run a specific test, use this syntax::

   make test TESTS=testname

for example::

   make test TESTS=samba4.BASE-DELETE