1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-22 13:34:15 +03:00
samba-mirror/selftest
Andreas Schneider 3db299e586 selftest: Add support for setting ENV variables in plansmbtorture4testsuite()
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2021-07-28 06:23:37 +00:00
..
flapping.d selftest/flapping: remove python[23] lines 2021-03-17 07:03:27 +00:00
gnupg
knownfail.d s3:smbd: really support AES-256* in the server 2021-07-20 16:13:28 +00:00
manage-ca selftest/manage-ca: Add certificiate etc. for DC addcsmb1 2020-04-03 15:08:26 +00:00
ns selftest: use 10.53.57.0/8 instead of 127.0.0.1/8 2020-03-27 09:02:38 +00:00
target s3: tests: Our tests for "smbd async dosmode = yes" haven't been working correctly as the parameter has been set incorrectly. 2021-07-15 05:02:30 +00:00
checkpassword_arg1.sh selftest: require SAMBA_CPS_ACCOUNT_NAME in checkpassword_arg1.sh 2019-02-11 07:43:32 +01:00
create_smb1_fail_skipfile.txt selftest: Create instructions for generating skip file entries 2020-04-03 15:08:33 +00:00
devel_env.sh selftest: Add PYTHONPATH for lsp servers to devel_env.sh 2021-07-19 14:38:34 +00:00
filter-subunit python2 reduction: Merge remaining compat code into common 2020-10-02 14:49:36 +00:00
flapping selftest: Remove samba3.blackbox.smbclient_tar from flapping tests 2020-12-01 19:06:45 +00:00
format-subunit PY3: change shebang to python3 in misc dirs 2018-12-14 18:00:40 +01:00
format-subunit-json python: remove all 'from __future__ import print_function' 2021-04-28 03:43:34 +00:00
gdb_backtrace selftest/gdb_backtrace: remove duplicate assignment. 2021-06-10 01:17:52 +00:00
gdb_backtrace_test.c
gdb_run
in_screen testenv: No "mktemp" for in_screen 2020-01-21 14:38:44 +00:00
knownfail s4:torture/krb5/kdc-heimdal: Automatically determine AS-REP enctype to check against 2021-07-01 17:46:31 +00:00
knownfail_heimdal_kdc tests python krb5: MS-KILE client principal look-up 2021-04-12 00:38:26 +00:00
knownfail_mit_kdc s4:torture/krb5/kdc-heimdal: Automatically determine AS-REP enctype to check against 2021-07-01 17:46:31 +00:00
knownfail_mit_krb5_pre_1_18 Fix uxsuccess test with new MIT krb5 library 1.18 2020-03-10 13:02:27 +00:00
no-python-tests.txt selftest: Add basic sanity-check tests for nopython target 2019-02-20 02:10:00 +01:00
perf_tests.py perf_tests: Implicit string concatenation 2021-06-17 04:21:30 +00:00
quick selftest: Run test of how userPassword / crypt() style passwords are stored in quicktest 2020-07-01 13:34:30 +00:00
README
save.env.sh
selftest.pl selftest: use SAMBA_DEPRECATED_SUPPRESS=1 for all tests 2021-07-14 21:57:11 +00:00
selftest.pl.1
selftesthelpers.py selftest: Add support for setting ENV variables in plansmbtorture4testsuite() 2021-07-28 06:23:37 +00:00
skip selftest: remove POSIX test from planned tests for ad_dc_ntvfs environ 2020-10-23 07:56:32 +00:00
skip_mit_kdc selftest: add python S4U2Self tests including unkeyed checksums 2020-05-15 12:25:40 +00:00
skip.no-GSS_KRB5_CRED_NO_CI_FLAGS_X
skip.opath-required CI: add samba-no-opath 2020-12-16 09:08:30 +00:00
slow
slow-none selftest: Move some more tests from the samba-o3 job 2020-10-01 01:18:38 +00:00
SocketWrapper.pm selftest: enable perl warnings 2020-02-04 05:13:39 +00:00
Subunit.pm selftest: enable perl warnings 2020-02-04 05:13:39 +00:00
subunithelper.py python:subunit: Avoid misleading "Test was never started" error message 2021-06-11 08:38:34 +00:00
tap2subunit
tests.py lib: add sys_io_ranges_overlap() 2021-06-30 16:51:29 +00:00
TODO
todo_smb2_tests_to_port.list s4:torture: Convert samba3.raw.mkdir test to smb2 2020-04-28 19:46:32 +00:00
ubsan.supp selftest: enable undefined behaviour sanitizer 2019-05-14 07:20:28 +00:00
valgrind_run
wscript selftest: Disable detection of ODR violations 2021-02-01 21:50:32 +00:00

# vim: ft=rst

This directory contains test scripts that are useful for running a
bunch of tests all at once.

There are two parts to this:

 * The test runner (selftest/selftest.pl)
 * The test formatter

selftest.pl simply outputs subunit, which can then be formatted or analyzed
by tools that understand the subunit protocol. One of these tools is
format-subunit, which is used by default as part of "make test".

Available testsuites
====================
The available testsuites are obtained from a script, usually
source{3,4}/selftest/tests.py. This script should for each testsuite output
the name of the test, the command to run and the environment that should be
provided. Use the included "plantest" function to generate the required output.

Testsuite behaviour
===================

Exit code
------------
The testsuites should exit with a non-zero exit code if at least one
test failed. Skipped tests should not influence the exit code.

Output format
-------------
Testsuites can simply use the exit code to indicate whether all of their
tests have succeeded or one or more have failed. It is also possible to
provide more granular information using the Subunit protocol.

This protocol works by writing simple messages to standard output. Any
messages that can not be interpreted by this protocol are considered comments
for the last announced test.

For a full description of the subunit protocol, see the README file in the subunit
repository at http://github.com/testing-cabal/subunit.

The following commands are Samba extensions to Subunit:

start-testsuite
~~~~~~~~~~~~~~~
start-testsuite: name

The testsuite name is used as prefix for all containing tests.

skip-testsuite
~~~~~~~~~~~~~~
skip-testsuite: name

Mark the testsuite with the specified name as skipped.

testsuite-success
~~~~~~~~~~~~~~~~~
testsuite-success: name

Indicate that the testsuite has succeeded successfully.

testsuite-fail
~~~~~~~~~~~~~~
testsuite-fail: name

Indicate that a testsuite has failed.

Environments
============
Tests often need to run against a server with particular things set up,
a "environment". This environment is provided by the test "target": Samba 3,
Samba 4 or Windows.

The environments are currently available include

 - none: No server set up, no variables set.
 - dc,s3dc: Domain controller set up. The following environment variables will
   be set:

     * USERNAME: Administrator user name
     * PASSWORD: Administrator password
     * DOMAIN: Domain name
     * REALM: Realm name
     * SERVER: DC host name
     * SERVER_IP: DC IPv4 address
     * SERVER_IPV6: DC IPv6 address
     * NETBIOSNAME: DC NetBIOS name
     * NETIOSALIAS: DC NetBIOS alias

 - member,s4member,s3member: Domain controller and member server that is joined to it set up. The
   following environment variables will be set:

     * USERNAME: Domain administrator user name
     * PASSWORD: Domain administrator password
     * DOMAIN: Domain name
     * REALM: Realm name
     * SERVER: Name of the member server

See Samba.pm, Samba3.pm and Samba4.pm for the full list.

Running tests
=============

To run all the tests use::

   make test

To run a quicker subset run::

   make quicktest

To run a specific test, use this syntax::

   make test TESTS=testname

for example::

   make test TESTS=samba4.BASE-DELETE