1
0
mirror of https://github.com/samba-team/samba.git synced 2024-12-23 17:34:34 +03:00

testtools: Fix included testtools, for systems that don't have it.

This commit is contained in:
Jelmer Vernooij 2010-03-31 04:19:36 +02:00
parent 5f3fcf7a77
commit 6897be7475
30 changed files with 679 additions and 1 deletions

139
lib/testtools/HACKING Normal file
View File

@ -0,0 +1,139 @@
===================================
Notes for contributing to testtools
===================================
Coding style
------------
In general, follow PEP 8 <http://www.python.org/dev/peps/pep-0008/>.
For consistency with the standard library's ``unittest`` module, method names
are generally ``camelCase``.
testtools supports Python 2.4 and later, so avoid any 2.5-only features like
the ``with`` statement.
Copyright assignment
--------------------
Part of testtools raison d'etre is to provide Python with improvements to the
testing code it ships. For that reason we require all contributions (that are
non-trivial) to meet one of the following rules:
- be inapplicable for inclusion in Python.
- be able to be included in Python without further contact with the
contributor.
- be copyright assigned to Jonathan M. Lange.
Please pick one of these and specify it when contributing code to testtools.
Licensing
---------
All code that is not copyright assigned to Jonathan M. Lange (see Copyright
Assignment above) needs to be licensed under the MIT license that testtools
uses, so that testtools can ship it.
Testing
-------
Please write tests for every feature. This project ought to be a model
example of well-tested Python code!
Take particular care to make sure the *intent* of each test is clear.
You can run tests with ``make check``, or by running ``./run-tests`` directly.
Source layout
-------------
The top-level directory contains the ``testtools/`` package directory, and
miscellaneous files like README and setup.py.
The ``testtools/`` directory is the Python package itself. It is separated
into submodules for internal clarity, but all public APIs should be “promoted”
into the top-level package by importing them in ``testtools/__init__.py``.
Users of testtools should never import a submodule, they are just
implementation details.
Tests belong in ``testtools/tests/``.
Commiting to trunk
------------------
Testtools is maintained using bzr, with its trunk at lp:testtools. This gives
every contributor the ability to commit their work to their own branches.
However permission must be granted to allow contributors to commit to the trunk
branch.
Commit access to trunk is obtained by joining the testtools-devs Launchpad
team. Membership in this team is contingent on obeying the testtools
contribution policy, including assigning copyright of all the work one creates
and places in trunk to Jonathan Lange.
Code Review
-----------
All code must be reviewed before landing on trunk. The process is to create a
branch in launchpad, and submit it for merging to lp:testtools. It will then
be reviewed before it can be merged to trunk. It will be reviewed by someone:
* not the author
* a committer (member of the testtools-devs team)
As a special exception, while the testtools committers team is small and prone
to blocking, a merge request from a committer that has not been reviewed after
24 hours may be merged by that committer. When the team is larger this policy
will be revisited.
Code reviewers should look for the quality of what is being submitted,
including conformance with this HACKING file.
Changes which all users should be made aware of should be documented in NEWS.
NEWS management
---------------
The file NEWS is structured as a sorted list of releases. Each release can have
a free form description and more or more sections with bullet point items.
Sections in use today are 'Improvements' and 'Changes'. To ease merging between
branches, the bullet points are kept alphabetically sorted. The release NEXT is
permanently present at the top of the list.
Release tasks
-------------
In no particular order:
* Choose a version number.
* Ensure __init__ has that version.
* Add a version number to NEWS immediately below NEXT.
* Possibly write a blurb into NEWS.
* Replace any additional references to NEXT with the version being released.
* Create a source distribution and upload to pypi ('make release').
* Upload to Launchpad as well.
* If a new series has been created (e.g. 0.10.0), make the series on Launchpad.
* Merge or push the release branch to trunk.
* Make a new milestone for the *next release*. We don't really know how we want
to handle these yet, so this is a suggestion not actual practice:
* during release we rename NEXT to $version.
* we call new milestones NEXT.

19
lib/testtools/LICENSE Normal file
View File

@ -0,0 +1,19 @@
Copyright (c) 2008 Jonathan M. Lange <jml@mumak.net> and the testtools authors.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -0,0 +1,9 @@
include LICENSE
include HACKING
include Makefile
include MANIFEST.in
include MANUAL
include NEWS
include README
include run-tests
include .bzrignore

213
lib/testtools/MANUAL Normal file
View File

@ -0,0 +1,213 @@
======
Manual
======
Introduction
------------
This document provides overview of the features provided by testtools. Refer
to the API docs (i.e. docstrings) for full details on a particular feature.
Extensions to TestCase
----------------------
Controlling test execution
~~~~~~~~~~~~~~~~~~~~~~~~~~
Testtools supports two ways to control how tests are executed. The simplest
is to add a new exception to self.exception_handlers::
>>> self.exception_handlers.insert(-1, (ExceptionClass, handler)).
Having done this, if any of setUp, tearDown, or the test method raise
ExceptionClass, handler will be called with the test case, test result and the
raised exception.
Secondly, by overriding __init__ to pass in runTest=RunTestFactory the whole
execution of the test can be altered. The default is testtools.runtest.RunTest
and calls case._run_setup, case._run_test_method and finally
case._run_teardown. Other methods to control what RunTest is used may be
added in future.
TestCase.addCleanup
~~~~~~~~~~~~~~~~~~~
addCleanup is a robust way to arrange for a cleanup function to be called
before tearDown. This is a powerful and simple alternative to putting cleanup
logic in a try/finally block or tearDown method. e.g.::
def test_foo(self):
foo.lock()
self.addCleanup(foo.unlock)
...
TestCase.addOnException
~~~~~~~~~~~~~~~~~~~~~~~
addOnException adds an exception handler that will be called from the test
framework when it detects an exception from your test code. The handler is
given the exc_info for the exception, and can use this opportunity to attach
more data (via the addDetails API) and potentially other uses.
TestCase.skip
~~~~~~~~~~~~~
``skip`` is a simple way to have a test stop running and be reported as a
skipped test, rather than a success/error/failure. This is an alternative to
convoluted logic during test loading, permitting later and more localized
decisions about the appropriateness of running a test. Many reasons exist to
skip a test - for instance when a dependency is missing, or if the test is
expensive and should not be run while on laptop battery power, or if the test
is testing an incomplete feature (this is sometimes called a TODO). Using this
feature when running your test suite with a TestResult object that is missing
the ``addSkip`` method will result in the ``addError`` method being invoked
instead.
New assertion methods
~~~~~~~~~~~~~~~~~~~~~
testtools adds several assertion methods:
* assertIn
* assertNotIn
* assertIs
* assertIsNot
* assertIsInstance
* assertThat
Improved assertRaises
~~~~~~~~~~~~~~~~~~~~~
TestCase.assertRaises returns the caught exception. This is useful for
asserting more things about the exception than just the type::
error = self.assertRaises(UnauthorisedError, thing.frobnicate)
self.assertEqual('bob', error.username)
self.assertEqual('User bob cannot frobnicate', str(error))
TestCase.assertThat
~~~~~~~~~~~~~~~~~~~
assertThat is a clean way to write complex assertions without tying them to
the TestCase inheritance hierarchy (and thus making them easier to reuse).
assertThat takes an object to be matched, and a matcher, and fails if the
matcher does not match the matchee.
See pydoc testtools.Matcher for the protocol that matchers need to implement.
testtools includes some matchers in testtools.matchers.
python -c 'import testtools.matchers; print testtools.matchers.__all__' will
list those matchers.
An example using the DocTestMatches matcher which uses doctests example
matching logic::
def test_foo(self):
self.assertThat([1,2,3,4], DocTestMatches('[1, 2, 3, 4]'))
Creation methods
~~~~~~~~~~~~~~~~
testtools.TestCase implements creation methods called ``getUniqueString`` and
``getUniqueInteger``. See pages 419-423 of *xUnit Test Patterns* by Meszaros
for a detailed discussion of creation methods.
Test renaming
~~~~~~~~~~~~~
``testtools.clone_test_with_new_id`` is a function to copy a test case
instance to one with a new name. This is helpful for implementing test
parameterization.
Extensions to TestResult
------------------------
TestResult.addSkip
~~~~~~~~~~~~~~~~~~
This method is called on result objects when a test skips. The
``testtools.TestResult`` class records skips in its ``skip_reasons`` instance
dict. The can be reported on in much the same way as succesful tests.
TestResult.time
~~~~~~~~~~~~~~~
This method controls the time used by a TestResult, permitting accurate
timing of test results gathered on different machines or in different threads.
See pydoc testtools.TestResult.time for more details.
ThreadsafeForwardingResult
~~~~~~~~~~~~~~~~~~~~~~~~~~
A TestResult which forwards activity to another test result, but synchronises
on a semaphore to ensure that all the activity for a single test arrives in a
batch. This allows simple TestResults which do not expect concurrent test
reporting to be fed the activity from multiple test threads, or processes.
Note that when you provide multiple errors for a single test, the target sees
each error as a distinct complete test.
TextTestResult
~~~~~~~~~~~~~~
A TestResult that provides a text UI very similar to the Python standard
library UI. Key differences are that its supports the extended outcomes and
details API, and is completely encapsulated into the result object, permitting
it to be used without a 'TestRunner' object. Not all the Python 2.7 outcomes
are displayed (yet). It is also a 'quiet' result with no dots or verbose mode.
These limitations will be corrected soon.
Test Doubles
~~~~~~~~~~~~
In testtools.testresult.doubles there are three test doubles that testtools
uses for its own testing: Python26TestResult, Python27TestResult,
ExtendedTestResult. These TestResult objects implement a single variation of
the TestResult API each, and log activity to a list self._events. These are
made available for the convenience of people writing their own extensions.
startTestRun and stopTestRun
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python 2.7 added hooks 'startTestRun' and 'stopTestRun' which are called
before and after the entire test run. 'stopTestRun' is particularly useful for
test results that wish to produce summary output.
testtools.TestResult provides empty startTestRun and stopTestRun methods, and
the default testtools runner will call these methods appropriately.
Extensions to TestSuite
-----------------------
ConcurrentTestSuite
~~~~~~~~~~~~~~~~~~~
A TestSuite for parallel testing. This is used in conjuction with a helper that
runs a single suite in some parallel fashion (for instance, forking, handing
off to a subprocess, to a compute cloud, or simple threads).
ConcurrentTestSuite uses the helper to get a number of separate runnable
objects with a run(result), runs them all in threads using the
ThreadsafeForwardingResult to coalesce their activity.
Running tests
-------------
Testtools provides a convenient way to run a test suite using the testtools
result object: python -m testtools.run testspec [testspec...].

28
lib/testtools/Makefile Normal file
View File

@ -0,0 +1,28 @@
# See README for copyright and licensing details.
PYTHON=python
SOURCES=$(shell find testtools -name "*.py")
check:
PYTHONPATH=$(PWD) $(PYTHON) -m testtools.run testtools.tests.test_suite
TAGS: ${SOURCES}
ctags -e -R testtools/
tags: ${SOURCES}
ctags -R testtools/
clean:
rm -f TAGS tags
find testtools -name "*.pyc" -exec rm '{}' \;
release:
./setup.py sdist upload --sign
apidocs:
pydoctor --make-html --add-package testtools \
--docformat=restructuredtext --project-name=testtools \
--project-url=https://launchpad.net/testtools
.PHONY: check clean release apidocs

191
lib/testtools/NEWS Normal file
View File

@ -0,0 +1,191 @@
testtools NEWS
++++++++++++++
NEXT
~~~~
Improvements
------------
* New matcher "Annotate" that adds a simple string message to another matcher,
much like the option 'message' parameter to standard library assertFoo
methods.
* New matchers "Not" and "MatchesAll". "Not" will invert another matcher, and
"MatchesAll" that needs a successful match for all of its arguments.
* On Python 2.4, where types.FunctionType cannot be deepcopied, testtools will
now monkeypatch copy._deepcopy_dispatch using the same trivial patch that
added such support to Python 2.5. The monkey patch is triggered by the
absence of FunctionType from the dispatch dict rather than a version check.
Bug #498030.
* On windows the test 'test_now_datetime_now' should now work reliably.
* TestCase.getUniqueInteger and TestCase.getUniqueString now have docstrings.
* TestCase.getUniqueString now takes an optional prefix parameter, so you can
now use it in circumstances that forbid strings with '.'s, and such like.
* testtools.testcase.clone_test_with_new_id now uses copy.copy, rather than
copy.deepcopy. Tests that need a deeper copy should use the copy protocol to
control how they are copied. Bug #498869.
* The backtrace test result output tests should now pass on windows and other
systems where os.sep is not '/'.
0.9.2
~~~~~
Python 3 support, more matchers and better consistency with Python 2.7 --
you'd think that would be enough for a point release. Well, we here on the
testtools project think that you deserve more.
We've added a hook so that user code can be called just-in-time whenever there
is an exception, and we've also factored out the "run" logic of test cases so
that new outcomes can be added without fiddling with the actual flow of logic.
It might sound like small potatoes, but it's changes like these that will
bring about the end of test frameworks.
Improvements
------------
* A failure in setUp and tearDown now report as failures not as errors.
* Cleanups now run after tearDown to be consistent with Python 2.7's cleanup
feature.
* ExtendedToOriginalDecorator now passes unrecognised attributes through
to the decorated result object, permitting other extensions to the
TestCase -> TestResult protocol to work.
* It is now possible to trigger code just-in-time after an exception causes
a test outcome such as failure or skip. See the testtools MANUAL or
``pydoc testtools.TestCase.addOnException``. (bug #469092)
* New matcher Equals which performs a simple equality test.
* New matcher MatchesAny which looks for a match of any of its arguments.
* TestCase no longer breaks if a TestSkipped exception is raised with no
parameters.
* TestCase.run now clones test cases before they are run and runs the clone.
This reduces memory footprint in large test runs - state accumulated on
test objects during their setup and execution gets freed when test case
has finished running unless the TestResult object keeps a reference.
NOTE: As test cloning uses deepcopy, this can potentially interfere if
a test suite has shared state (such as the testscenarios or testresources
projects use). Use the __deepcopy__ hook to control the copying of such
objects so that the shared references stay shared.
* Testtools now accepts contributions without copyright assignment under some
circumstances. See HACKING for details.
* Testtools now provides a convenient way to run a test suite using the
testtools result object: python -m testtools.run testspec [testspec...].
* Testtools now works on Python 3, thanks to Benjamin Peterson.
* Test execution now uses a separate class, testtools.RunTest to run single
tests. This can be customised and extended in a more consistent fashion than
the previous run method idiom. See pydoc for more information.
* The test doubles that testtools itself uses are now available as part of
the testtools API in testtols.testresult.doubles.
* TracebackContent now sets utf8 as the charset encoding, rather than not
setting one and encoding with the default encoder.
* With python2.7 testtools.TestSkipped will be the unittest.case.SkipTest
exception class making skips compatible with code that manually raises the
standard library exception. (bug #490109)
Changes
-------
* TestCase.getUniqueInteger is now implemented using itertools.count. Thanks
to Benjamin Peterson for the patch. (bug #490111)
0.9.1
~~~~~
The new matcher API introduced in 0.9.0 had a small flaw where the matchee
would be evaluated twice to get a description of the mismatch. This could lead
to bugs if the act of matching caused side effects to occur in the matchee.
Since having such side effects isn't desirable, we have changed the API now
before it has become widespread.
Changes
-------
* Matcher API changed to avoid evaluating matchee twice. Please consult
the API documentation.
* TestCase.getUniqueString now uses the test id, not the test method name,
which works nicer with parameterised tests.
Improvements
------------
* Python2.4 is now supported again.
0.9.0
~~~~~
This release of testtools is perhaps the most interesting and exciting one
it's ever had. We've continued in bringing together the best practices of unit
testing from across a raft of different Python projects, but we've also
extended our mission to incorporating unit testing concepts from other
languages and from our own research, led by Robert Collins.
We now support skipping and expected failures. We'll make sure that you
up-call setUp and tearDown, avoiding unexpected testing weirdnesses. We're
now compatible with Python 2.5, 2.6 and 2.7 unittest library.
All in all, if you are serious about unit testing and want to get the best
thinking from the whole Python community, you should get this release.
Improvements
------------
* A new TestResult API has been added for attaching details to test outcomes.
This API is currently experimental, but is being prepared with the intent
of becoming an upstream Python API. For more details see pydoc
testtools.TestResult and the TestCase addDetail / getDetails methods.
* assertThat has been added to TestCase. This new assertion supports
a hamcrest-inspired matching protocol. See pydoc testtools.Matcher for
details about writing matchers, and testtools.matchers for the included
matchers. See http://code.google.com/p/hamcrest/.
* Compatible with Python 2.6 and Python 2.7
* Failing to upcall in setUp or tearDown will now cause a test failure.
While the base methods do nothing, failing to upcall is usually a problem
in deeper hierarchies, and checking that the root method is called is a
simple way to catch this common bug.
* New TestResult decorator ExtendedToOriginalDecorator which handles
downgrading extended API calls like addSkip to older result objects that
do not support them. This is used internally to make testtools simpler but
can also be used to simplify other code built on or for use with testtools.
* New TextTestResult supporting the extended APIs that testtools provides.
* Nose will no longer find 'runTest' tests in classes derived from
testtools.testcase.TestCase (bug #312257).
* Supports the Python 2.7/3.1 addUnexpectedSuccess and addExpectedFailure
TestResult methods, with a support function 'knownFailure' to let tests
trigger these outcomes.
* When using the skip feature with TestResult objects that do not support it
a test success will now be reported. Previously an error was reported but
production experience has shown that this is too disruptive for projects that
are using skips: they cannot get a clean run on down-level result objects.

54
lib/testtools/README Normal file
View File

@ -0,0 +1,54 @@
=========
testtools
=========
testtools is a set of extensions to the Python standard library's unit testing
framework.
These extensions have been derived from years of experience with unit testing
in Python and come from many different sources.
Licensing
---------
This project is distributed under the MIT license and copyright is owned by
Jonathan M. Lange. See LICENSE for details.
Dependencies
------------
* Python 2.4+ or 3.0+
Bug reports and patches
-----------------------
Please report bugs using Launchpad at <https://bugs.launchpad.net/testtools>.
Patches can also be submitted via Launchpad, or mailed to the author. You can
mail the author directly at jml@mumak.net.
There's no mailing list for this project yet, however the testing-in-python
mailing list may be a useful resource:
* Address: testing-in-python@lists.idyll.org
* Subscription link: http://lists.idyll.org/listinfo/testing-in-python
History
-------
testtools used to be called 'pyunit3k'. The name was changed to avoid
conflating the library with the Python 3.0 release (commonly referred to as
'py3k').
Thanks
------
* Canonical Ltd
* Bazaar
* Twisted Matrix Labs
* Robert Collins
* Andrew Bennetts
* Benjamin Peterson

25
lib/testtools/setup.py Executable file
View File

@ -0,0 +1,25 @@
#!/usr/bin/env python
"""Distutils installer for testtools."""
from distutils.core import setup
import testtools
version = '.'.join(str(component) for component in testtools.__version__[0:3])
phase = testtools.__version__[3]
if phase != 'final':
import bzrlib.workingtree
t = bzrlib.workingtree.WorkingTree.open_containing(__file__)[0]
if phase == 'alpha':
# No idea what the next version will be
version = 'next-%s' % t.branch.revno()
else:
# Preserve the version number but give it a revno prefix
version = version + '~%s' % t.branch.revno()
setup(name='testtools',
author='Jonathan M. Lange',
author_email='jml+testtools@mumak.net',
url='https://launchpad.net/testtools',
description=('Extensions to the Python standard library unit testing '
'framework'),
version=version,
packages=['testtools', 'testtools.testresult', 'testtools.tests'])

View File

@ -14,6 +14,6 @@ done
echo "Updating testtools..."
bzr export "$WORKDIR/testtools" lp:testtools
rsync -avz --delete "$WORKDIR/testtools/testtools/" "$TARGETDIR/testtools/"
rsync -avz --delete "$WORKDIR/testtools/" "$TARGETDIR/testtools/"
rm -rf "$WORKDIR"