Douglas Bagnall
e908873757
make perftest: for performance testing
...
This runs a selection of subunit tests and reduces the output to only
the time it takes to run each test.
The tests are listed in selftest/perf_tests.py.
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2016-08-31 07:09:26 +02:00
Douglas Bagnall
85b4a3ea61
filter-subunit: default to empty affixes, saving verbose checks
...
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2016-08-31 07:09:26 +02:00
Douglas Bagnall
809f4c718a
subunithelper: use set for efficient inclusion test
...
Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz>
Reviewed-by: Garming Sam <garming@catalyst.net.nz>
2016-08-31 07:09:26 +02:00
Jelmer Vernooij
02da1b3798
Remove another call to addUnexpectedSuccess with too many arguments.
...
Change-Id: I5f1917e74ffd57b5ae228231c27fbdf70de2992f
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:49 +01:00
Jelmer Vernooij
5d4a938c06
TestCase.addUnexpectedSuccess doesn't take an error.
...
Change-Id: I90f7dd225d9ed3cbc515292de9a37a816ac0639f
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:49 +01:00
Jelmer Vernooij
b0cba7950a
Drop support for failfast mode, rather than adding support for it everywhere.
...
Change-Id: I4d6070a0e3b89d5e390f84754dddba9ec17ddf21
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:48 +01:00
Jelmer Vernooij
b2af53db74
Set failfast property for test reporters that need it.
...
Change-Id: Ibd632b9f569c23e60bcd13bcff805e367dd2e71c
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:48 +01:00
Jelmer Vernooij
850b3938e1
Support using third party iso8601 module if system doesn't provide one.
...
Change-Id: I5d035738d244d66d33788636c8ee8b322c227a0e
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:48 +01:00
Jelmer Vernooij
8c6d85b817
subunithelper: Fix progress support.
...
Change-Id: I5dd2ca0f3bc02821d5c9e1dc878bba577667d162
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:47 +01:00
Jelmer Vernooij
80e387adfa
Use iso8601 from the system, rather than the one bundled with subunit.
...
Change-Id: I681bf79eb9ebe45b6972b3783c8e799eb612400b
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:47 +01:00
Jelmer Vernooij
ee0b548e9d
Fix handling of unexpected failures in subunithelper.
...
Change-Id: I0da9fcec4a54c43c171b76bb9015ea84389c9bc6
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:47 +01:00
Jelmer Vernooij
3f88e978a3
Use samba.subunit in selftest.subunithelper, except for iso8601.
...
Change-Id: Iec43e0368db91dbe981b39ac8388b50c1eafee0c
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2015-03-06 04:41:47 +01:00
Jelmer Vernooij
fb4b0ac611
Display total number of tests in format-subunit, not just number of test suites.
...
Change-Id: Iae17df773f380b13d2b3f69e83593139c01c4a6d
Signed-Off-By: Jelmer Vernooij <jelmer@samba.org>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
2014-11-19 02:46:03 +01:00
Michael Adam
5340be1740
subunit: report [X/Y at Zs] instead of [X/Y in Zs]
...
when running test X out of Y after Z secons have passed
Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Matthieu Patou <mat@matws.net>
2014-10-23 15:56:04 +02:00
Stefan Metzmacher
3632c59e25
selftest/subunithelper.py: correctly pass testsuite-uxsuccess to end_testsuite()
...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Mar 13 23:49:36 CET 2014 on sn-devel-104
2014-03-13 23:49:35 +01:00
Stefan Metzmacher
fee156f5b9
selftest/subunithelper.py: correctly handle fail_immediately in end_testsuite of FilterOps
...
This way --fail-immediately also works if a command segfaults.
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-13 21:45:13 +01:00
Stefan Metzmacher
14246953b2
selftest/subunithelper.py: correctly handle unexpected success in FilterOps
...
Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
2014-03-13 21:45:12 +01:00
Andrew Bartlett
0cb0427f13
selftest: Place the test environment at the end of the failure lines
...
This allows a knownfail entry to be added for only one environment, rather than
all environments.
Andrew Bartlett
2012-04-27 14:51:05 +02:00
Jelmer Vernooij
0fee2c01b8
selftest: Display unexpected successes and expected failures.
2011-12-05 23:11:04 +01:00
Jelmer Vernooij
e62654578b
selftest: Add --flapping argument to filter-subunit.
2011-12-05 23:11:04 +01:00
Jelmer Vernooij
bbd77cbbac
subunithelper: Exit with 1 if more than zero testsuites failed or
...
errorred.
Autobuild-User: Jelmer Vernooij <jelmer@samba.org>
Autobuild-Date: Sun Nov 21 21:13:00 CET 2010 on sn-devel-104
2010-11-21 21:13:00 +01:00
Jelmer Vernooij
4f87f8f33e
filter-subunit: Add --fail-on-empty argument.
2010-11-03 17:05:19 +00:00
Kamen Mazdrashki
27f72fb19b
subunithelper: Don't access 'message' attribute directly as it deprecated
...
Signed-off-by: Jelmer Vernooij <jelmer@samba.org>
Autobuild-User: Jelmer Vernooij <jelmer@samba.org>
Autobuild-Date: Sun Oct 10 01:50:09 UTC 2010 on sn-devel-104
2010-10-10 01:50:09 +00:00
Jelmer Vernooij
5cdef70823
subunithelper: Remove accidentally added line.
2010-10-02 23:32:12 +02:00
Jelmer Vernooij
14efa95593
subunithelper: Fix format time.
2010-10-02 22:31:31 +02:00
Jelmer Vernooij
72119de02b
subunithelper: Make filter options optional.
2010-10-02 18:41:14 +02:00
Jelmer Vernooij
4ece674a99
selftest: Move plain text formatter to subunithelper.
2010-10-02 17:02:02 +02:00
Jelmer Vernooij
124d9fc4d1
selftest: Avoid accessing deprecated BaseException.message.
...
Thanks to Andreas for pointing this out.
2010-09-30 10:31:03 +02:00
Jelmer Vernooij
7ac4a71072
Add dedicated exception for immediate failure in filter-subunit, don't raise it on known exceptions.
2010-09-28 09:12:39 +02:00
Andrew Tridgell
71f88c45a7
filter-subunit: added a --fail-immediately option
2010-09-28 09:12:38 +02:00
Jelmer Vernooij
6abe823ce6
subunithelper: Cope with errors outside of testsuites.
2010-09-22 17:48:23 -07:00
Jelmer Vernooij
31d5ac0540
selftest: Let tests add prefix to tests by themselve.
2010-09-22 17:48:23 -07:00
Jelmer Vernooij
adf8ca6ff8
subunit: Re-add outputmsg.
2010-09-15 20:09:25 +02:00
Jelmer Vernooij
dcadb90bd3
subunit: Use RemoteError when passing errors to upstream subunit.
2010-09-14 10:54:28 +02:00
Jelmer Vernooij
d67eeafc4d
subunit: Use standard subunit functions for reproducing subunit streams.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
d2fb9ce472
subunit: Remove unused methods.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
b61cc18972
subunit: Use standard functions for addSuccess, addExpectedFail,
...
addFailure, addSkip.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
c1fe7b498a
subunit: Use standard addError method implementation.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
881f610f06
subunit: Pass TestCase objects to startTest rather than test name strings.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
1626dc2bc9
subunit: Use subunit standard functions for handling time and progress.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
fb1c966652
subunit: Use standard subunit test protocol client, use standard name for startTest.
2010-09-14 10:54:27 +02:00
Jelmer Vernooij
8530d5afde
subunit: Initial work on using the standard TestResult class.
2010-09-14 10:54:26 +02:00
Jelmer Vernooij
474c98fe26
selftest/subunit: Parse timestamps generated by upstream subunit.
2010-09-05 03:14:44 +02:00
Jelmer Vernooij
3c7a7bbb9a
subunit: Make sure "]" in failure reason ends up on its own line, so the
...
next subunit parser in the line parses it correctly.
2010-08-26 04:04:37 +02:00
Jelmer Vernooij
64564f7338
subunithelper: Pass along lines as normal output when ignoring them for not containing commands.
2010-08-26 04:04:37 +02:00
Jelmer Vernooij
598656985a
selftest: list.remove raises ValueError rather than KeyError.
2010-05-30 17:05:49 +02:00
Jelmer Vernooij
7da94cc4a6
subunit: Support formatting compatible with upstream subunit, for consistency.
...
Upstream subunit makes a ":" after commands optional, so I've fixed any
places where we might trigger commands accidently. I've filed a bug
about this in subunit.
2010-04-11 20:57:33 +02:00
Jelmer Vernooij
d0ab7440ac
selftest: Accept 'testing:' to introduce new tests.
2010-04-11 20:57:33 +02:00
Jelmer Vernooij
664eacc53a
subunit: Don't abort when receiving test results from tests that weren't
...
announced.
2010-04-10 22:38:33 +02:00
Jelmer Vernooij
e39ed552b6
selftest: Remove python2.4-isms
2010-04-06 04:01:54 +02:00