samba-mirror/selftest

# vim: ft=rst

This directory contains test scripts that are useful for running a
bunch of tests all at once. 

Available testsuites
====================
The available testsuites are obtained from a script, usually 
selftest/samba{3,4}_tests.sh. This script should for each testsuite output 
the name of the test, the command to run and the environment that should be 
provided. Use the included "plantest" function to generate the required output.

Testsuite behaviour
===================

Exit code
------------
The testsuites should exit with a non-zero exit code if at least one 
test failed. Skipped tests should not influence the exit code.

Output format
-------------
Testsuites can simply use the exit code to indicate whether all of their 
tests have succeeded or one or more have failed. It is also possible to 
provide more granular information using the Subunit protocol. 

This protocol works by writing simple messages to standard output. Any 
messages that can not be interpreted by this protocol are considered comments 
for the last announced test.

Accepted commands are:

test
~~~~
test: <NAME>

Announce that a new test with the specified name is starting

success
~~~~~~~
success: <NAME> 

Announce that the test with the specified name is done and ran successfully.

failure
~~~~~~~
failure: <NAME> 
failure: <NAME> [ REASON ]

Announce that the test with the specified name failed. Optionally, it is 
possible to specify a reason it failed.

xfail
~~~~~
xfail: <NAME>
xfail: <NAME> [ REASON ]

Announce that the test with the specified name failed but that the failure
was expected, e.g. it's a test for a known bug that hasn't been fixed yet.

skip
~~~~
skip: <NAME>
skip: <NAME> [ REASON ]

Announce that the test with the specified name was skipped. Optionally a 
reason can be specified.

knownfail
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
knownfail: <NAME>
knownfail: <NAME> [ REASON ]

Announce that the test with the specified name was run and failed as expected.
Alternatively it is also possible to simply return "failure:" here but 
specify in the samba4-knownfailures file that it is failing.

Environments
============
Tests often need to run against a server with particular things set up, 
a "environment". This environment is provided by the test "target": Samba 3, 
Samba 4 or Windows.

The following environments are currently available:

 - none: No server set up, no variables set.
 - dc: Domain controller set up. The following environment variables will 
   be set:

     * USERNAME: Administrator user name
	 * PASSWORD: Administrator password
	 * DOMAIN: Domain name
	 * REALM: Realm name
	 * SERVER: DC host name 
	 * SERVER_IP: DC IPv4 address
	 * NETBIOSNAME: DC NetBIOS name
	 * NETIOSALIAS: DC NetBIOS alias

 - member: Domain controller and member server that is joined to it set up. The
   following environment variables will be set:

     * USERNAME: Domain administrator user name
	 * PASSWORD: Domain administrator password
	 * DOMAIN: Domain name
	 * REALM: Realm name
	 * SERVER: Name of the member server


Running tests
=============

To run all the tests use::

   make test

To run a quick subset (aiming for about 1 minute of testing) run::

   make quicktest

To run a specific test, use this syntax::

   make test TESTS=testname

for example::

   make test TESTS=samba4.BASE-DELETE