mirror of
https://github.com/samba-team/samba.git
synced 2024-12-23 17:34:34 +03:00
selftest: Catch error codes from failing testsuites
Testsuites declared with functions such as plantestsuite() are not run directly, but are piped through filter-subunit. The overall exit code of the executed test command is that returned by the last command in the pipeline (that is, filter-subunit), and thus the actual testsuite return code is lost. A real consequence of this is that an error in setUpClass() in a Python testsuite causes the whole testsuite to be skipped silently. The --fail-on-empty option partially addressed this, but didn't help if the testsuite contained multiple test classes, only one of which contained an error. We now use bash with the pipefail option, which makes the return code of the last failing command into the return code of the entire pipeline. That means that we properly fail if a testsuite returns a non-zero exit code, but doesn't otherwise exhibit any failures in its output. This doesn't help for cases where a testsuite has other failing tests that become xfails due to knownfail entries. In that case, the overall 'testsuite-failure' will be turned into 'testsuite-xfail' by filter-subunit and the silent failures will remain unheeded. Still, this is better than the existing situation. Signed-off-by: Joseph Sutton <josephsutton@catalyst.net.nz> Reviewed-by: Andreas Schneider <asn@samba.org> Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org> Autobuild-Date(master): Wed Apr 12 14:57:55 UTC 2023 on atb-devel-224
This commit is contained in:
parent
62893486c6
commit
2ff55b3da7
@ -129,7 +129,11 @@ sub run_testsuite($$$$$)
|
||||
Subunit::start_testsuite($name);
|
||||
Subunit::progress_push();
|
||||
Subunit::report_time();
|
||||
system($cmd);
|
||||
# Enable pipefail so that we catch failing testsuites that are part of a
|
||||
# pipeline (typically, piped through filter-subunit). This won't catch
|
||||
# any testsuite failures that are turned into testsuite-xfails by
|
||||
# filter-subunit.
|
||||
system("bash", "-o", "pipefail", "-c", $cmd);
|
||||
Subunit::report_time();
|
||||
Subunit::progress_pop();
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user