1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00
samba-mirror/selftest
Christian Ambach be3e479feb selftest: add smb2.rename to testsuite
Autobuild-User: Christian Ambach <ambi@samba.org>
Autobuild-Date: Fri Feb 10 15:02:51 CET 2012 on sn-devel-104
2012-02-10 15:02:51 +01:00
..
target selftest: Remove unused support for --exeext 2012-02-01 02:45:07 +01:00
filter-subunit filter-subunit: Fix default for flapping setting. 2011-12-05 23:11:04 +01:00
flapping selftest: mark posix_s3.rpc.spoolss.printer as flakey test 2012-02-10 10:04:15 +01:00
format-subunit format-subunit: Split out summary file writing. 2010-10-02 18:40:44 +02:00
gdb_backtrace selftest: Cope with no binary argument being supplied to gdb_backtrace 2011-06-24 13:17:19 +10:00
gdb_backtrace_test.c selftest: Fix typo in comment. 2010-06-01 09:45:19 +02:00
gdb_run selftest: move gdb_* and valgrind_* scripts to selftest/ 2008-10-28 12:20:59 +01:00
in_screen fix in_screen so that it works on os that don't chdir to $PWD 2010-11-05 09:27:19 +00:00
knownfail selftest: add smb2.rename to testsuite 2012-02-10 15:02:51 +01:00
quick selftest: Have only one set of selftest knownfail and skip files 2011-10-28 13:10:27 +02:00
README selftest: Update README 2011-05-03 07:37:07 +02:00
selftest.pl selftest: Remove unused support for --exeext 2012-02-01 02:45:07 +01:00
selftesthelpers.py selftest: Remove unused support for --exeext 2012-02-01 02:45:07 +01:00
skip s4-lsarpc Fix segfaults found by the samba4.rpc.lsa.forest test 2011-12-12 12:57:07 +01:00
slow selftest: Have only one set of selftest knownfail and skip files 2011-10-28 13:10:27 +02:00
SocketWrapper.pm selftest: Add copyright headers 2009-03-25 15:40:39 +01:00
Subunit.pm selftest: Let tests add prefix to tests by themselve. 2010-09-22 17:48:23 -07:00
subunithelper.py selftest: Display unexpected successes and expected failures. 2011-12-05 23:11:04 +01:00
TODO Move selftest code to top-level. 2008-09-18 19:51:03 +02:00
valgrind_run selftest: move gdb_* and valgrind_* scripts to selftest/ 2008-10-28 12:20:59 +01:00
wscript build: Add --enable-coverage option to build with gcov support 2012-02-10 16:45:11 +11:00

# vim: ft=rst

This directory contains test scripts that are useful for running a
bunch of tests all at once. 

There are two parts to this: 

 * The test runner (selftest/selftest.pl)
 * The test formatter

selftest.pl simply outputs subunit, which can then be formatted or analyzed 
by tools that understand the subunit protocol. One of these tools is 
format-subunit, which is used by default as part of "make test".

Available testsuites
====================
The available testsuites are obtained from a script, usually 
source{3,4}/selftest/tests.py. This script should for each testsuite output
the name of the test, the command to run and the environment that should be 
provided. Use the included "plantest" function to generate the required output.

Testsuite behaviour
===================

Exit code
------------
The testsuites should exit with a non-zero exit code if at least one 
test failed. Skipped tests should not influence the exit code.

Output format
-------------
Testsuites can simply use the exit code to indicate whether all of their 
tests have succeeded or one or more have failed. It is also possible to 
provide more granular information using the Subunit protocol. 

This protocol works by writing simple messages to standard output. Any 
messages that can not be interpreted by this protocol are considered comments 
for the last announced test.

For a full description of the subunit protocol, see ../lib/subunit/README.

The following commands are Samba extensions to Subunit:

testsuite-count
~~~~~~~~~~~~~~~
testsuite-count: number

Announce the number of tests that is going to be run.

start-testsuite
~~~~~~~~~~~~~~~
start-testsuite: name

The testsuite name is used as prefix for all containing tests.

skip-testsuite
~~~~~~~~~~~~~~
skip-testsuite: name

Mark the testsuite with the specified name as skipped.

testsuite-success
~~~~~~~~~~~~~~~~~
testsuite-success: name

Indicate that the testsuite has succeeded successfully.

testsuite-fail
~~~~~~~~~~~~~~
testsuite-fail: name

Indicate that a testsuite has failed.

Environments
============
Tests often need to run against a server with particular things set up, 
a "environment". This environment is provided by the test "target": Samba 3, 
Samba 4 or Windows.

The environments are currently available include

 - none: No server set up, no variables set.
 - dc,s3dc: Domain controller set up. The following environment variables will
   be set:

     * USERNAME: Administrator user name
	 * PASSWORD: Administrator password
	 * DOMAIN: Domain name
	 * REALM: Realm name
	 * SERVER: DC host name 
	 * SERVER_IP: DC IPv4 address
	 * NETBIOSNAME: DC NetBIOS name
	 * NETIOSALIAS: DC NetBIOS alias

 - member,s4member,s3member: Domain controller and member server that is joined to it set up. The
   following environment variables will be set:

     * USERNAME: Domain administrator user name
	 * PASSWORD: Domain administrator password
	 * DOMAIN: Domain name
	 * REALM: Realm name
	 * SERVER: Name of the member server

See Samba.pm, Samba3.pm and Samba4.pm for the full list.

Running tests
=============

To run all the tests use::

   make test

To run a quicker subset run::

   make quicktest

To run a specific test, use this syntax::

   make test TESTS=testname

for example::

   make test TESTS=samba4.BASE-DELETE