2006-05-09 19:35:46 +04:00
/*
2014-03-07 17:38:51 +04:00
* testutils . h : test utils
2006-05-09 19:35:46 +04:00
*
2013-02-23 02:42:39 +04:00
* Copyright ( C ) 2005 , 2008 - 2013 Red Hat , Inc .
2006-05-09 19:35:46 +04:00
*
2012-07-27 13:39:53 +04:00
* This library is free software ; you can redistribute it and / or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation ; either
* version 2.1 of the License , or ( at your option ) any later version .
*
* This library is distributed in the hope that it will be useful ,
* but WITHOUT ANY WARRANTY ; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE . See the GNU
* Lesser General Public License for more details .
*
* You should have received a copy of the GNU Lesser General Public
2012-09-21 02:30:55 +04:00
* License along with this library . If not , see
2012-07-27 13:39:53 +04:00
* < http : //www.gnu.org/licenses/>.
2006-05-09 19:35:46 +04:00
*/
2019-06-18 19:13:14 +03:00
# pragma once
2006-05-09 19:35:46 +04:00
2019-06-18 19:13:14 +03:00
# include "viralloc.h"
# include "virfile.h"
# include "virstring.h"
# include "virjson.h"
# include "capabilities.h"
# include "domain_conf.h"
2009-05-12 20:45:14 +04:00
2019-06-18 19:13:14 +03:00
# define EXIT_AM_SKIP 77 /* tell Automake we're skipping a test */
# define EXIT_AM_HARDFAIL 99 /* tell Automake that the framework is broken */
2011-04-29 20:21:20 +04:00
2013-07-29 22:01:19 +04:00
/* Work around lack of gnulib support for fprintf %z */
2019-06-18 19:13:14 +03:00
# ifndef NO_LIBVIRT
# undef fprintf
# define fprintf virFilePrintf
# endif
2013-07-29 22:01:19 +04:00
2011-04-29 20:21:20 +04:00
extern char * progname ;
2013-11-28 01:31:53 +04:00
/* Makefile.am provides these two definitions */
2019-06-18 19:13:14 +03:00
# if !defined(abs_srcdir) || !defined(abs_builddir)
# error Fix Makefile.am
# endif
2009-12-15 11:43:29 +03:00
2016-05-26 18:01:56 +03:00
bool virTestOOMActive ( void ) ;
Introduce new OOM testing support
The previous OOM testing support would re-run the entire "main"
method each iteration, failing a different malloc each time.
When a test suite has 'n' allocations, the number of repeats
requires is (n * (n + 1) ) / 2. This gets very large, very
quickly.
This new OOM testing support instead integrates at the
virtTestRun level, so each individual test case gets repeated,
instead of the entire test suite. This means the values of
'n' are orders of magnitude smaller.
The simple usage is
$ VIR_TEST_OOM=1 ./qemuxml2argvtest
...
29) QEMU XML-2-ARGV clock-utc ... OK
Test OOM for nalloc=36 .................................... OK
30) QEMU XML-2-ARGV clock-localtime ... OK
Test OOM for nalloc=36 .................................... OK
31) QEMU XML-2-ARGV clock-france ... OK
Test OOM for nalloc=38 ...................................... OK
...
the second lines reports how many mallocs have to be failed, and thus
how many repeats of the test will be run.
If it crashes, then running under valgrind will often show the problem
$ VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest
When debugging problems it is also helpful to select an individual
test case
$ VIR_TEST_RANGE=30 VIR_TEST_OOM=1 ../run valgrind ./qemuxml2argvtest
When things get really tricky, it is possible to request that just
specific allocs are failed. eg to fail allocs 5 -> 12, use
$ VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-12 ../run valgrind ./qemuxml2argvtest
In the worse case, you might want to know the stack trace of the
alloc which was failed then VIR_TEST_OOM_TRACE can be set. If it
is set to 1 then it will only print if it thinks a mistake happened.
This is often not reliable, so setting it to 2 will make it print
the stack trace for every alloc that is failed.
$ VIR_TEST_OOM_TRACE=2 VIR_TEST_RANGE=30 VIR_TEST_OOM=1:5-5 ../run valgrind ./qemuxml2argvtest
30) QEMU XML-2-ARGV clock-localtime ... OK
Test OOM for nalloc=36 !virAllocN
/home/berrange/src/virt/libvirt/src/util/viralloc.c:180
virHashCreateFull
/home/berrange/src/virt/libvirt/src/util/virhash.c:144
virDomainDefParseXML
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:11745
virDomainDefParseNode
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12646
virDomainDefParse
/home/berrange/src/virt/libvirt/src/conf/domain_conf.c:12590
testCompareXMLToArgvFiles
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:106
virtTestRun
/home/berrange/src/virt/libvirt/tests/testutils.c:250
mymain
/home/berrange/src/virt/libvirt/tests/qemuxml2argvtest.c:418 (discriminator 2)
virtTestMain
/home/berrange/src/virt/libvirt/tests/testutils.c:750
??
??:0
_start
??:?
FAILED
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2013-09-23 17:21:52 +04:00
2016-05-26 18:01:50 +03:00
int virTestRun ( const char * title ,
int ( * body ) ( const void * data ) ,
const void * data ) ;
2016-05-26 18:01:52 +03:00
int virTestLoadFile ( const char * file , char * * buf ) ;
2017-07-25 16:11:31 +03:00
char * virTestLoadFilePath ( const char * p , . . . )
ATTRIBUTE_SENTINEL ;
2017-07-25 19:40:50 +03:00
virJSONValuePtr virTestLoadFileJSON ( const char * p , . . . )
ATTRIBUTE_SENTINEL ;
2016-05-26 18:02:05 +03:00
int virTestCaptureProgramOutput ( const char * const argv [ ] , char * * buf , int maxlen ) ;
2008-02-05 22:27:37 +03:00
2016-05-26 18:01:55 +03:00
void virTestClearCommandPath ( char * cmdset ) ;
2014-03-20 14:30:44 +04:00
2016-05-26 18:01:51 +03:00
int virTestDifference ( FILE * stream ,
const char * expect ,
const char * actual ) ;
2016-05-26 18:01:54 +03:00
int virTestDifferenceFull ( FILE * stream ,
const char * expect ,
const char * expectName ,
const char * actual ,
const char * actualName ) ;
2016-05-26 18:02:01 +03:00
int virTestDifferenceFullNoRegenerate ( FILE * stream ,
const char * expect ,
const char * expectName ,
const char * actual ,
const char * actualName ) ;
2016-05-26 18:02:04 +03:00
int virTestDifferenceBin ( FILE * stream ,
const char * expect ,
const char * actual ,
size_t length ) ;
2019-02-19 20:15:23 +03:00
int virTestCompareToFile ( const char * actual ,
2016-05-26 18:01:53 +03:00
const char * filename ) ;
2019-02-19 20:15:23 +03:00
int virTestCompareToString ( const char * expect ,
const char * actual ) ;
int virTestCompareToULL ( unsigned long long expect ,
unsigned long long actual ) ;
2006-05-09 19:35:46 +04:00
2009-11-30 22:01:31 +03:00
unsigned int virTestGetDebug ( void ) ;
unsigned int virTestGetVerbose ( void ) ;
2013-08-03 01:43:07 +04:00
unsigned int virTestGetExpensive ( void ) ;
2015-12-10 15:30:37 +03:00
unsigned int virTestGetRegenerate ( void ) ;
2009-10-16 19:37:36 +04:00
2019-06-18 19:13:14 +03:00
# define VIR_TEST_DEBUG(...) \
2017-11-03 15:09:47 +03:00
do { \
if ( virTestGetDebug ( ) ) \
fprintf ( stderr , __VA_ARGS__ ) ; \
2015-04-23 20:38:00 +03:00
} while ( 0 )
2019-06-18 19:13:14 +03:00
# define VIR_TEST_VERBOSE(...) \
2017-11-03 15:09:47 +03:00
do { \
if ( virTestGetVerbose ( ) ) \
fprintf ( stderr , __VA_ARGS__ ) ; \
2015-04-23 20:38:00 +03:00
} while ( 0 )
2016-05-26 18:01:58 +03:00
char * virTestLogContentAndReset ( void ) ;
tests: silence qemuargv2xmltest noise
Before this patch, the testsuite was noisy:
TEST: qemuargv2xmltest
........................................ 40
................20:41:28.046: warning : qemuParseCommandLine:6565 : unknown QEMU argument '-unknown', adding to the qemu namespace
20:41:28.046: warning : qemuParseCommandLine:6565 : unknown QEMU argument 'parameter', adding to the qemu namespace
. 57 OK
PASS: qemuargv2xmltest
It's not a real failure (which is why the test was completing
successfully), so much as an intentional warning to the user that use
of the qemu namespace has the potential for undefined effects that
leaked through the default logging behavior. After this patch series,
all tests can access any logged data, and this particular test can
explicitly check for the presence or absence of the warning, such that
the test output becomes:
TEST: qemuargv2xmltest
........................................ 40
................. 57 OK
PASS: qemuargv2xmltest
* tests/testutils.h (virtTestLogContentAndReset): New prototype.
* tests/testutils.c (struct virtTestLogData): New struct.
(virtTestLogOutput, virtTestLogClose, virtTestLogContentAndReset):
New functions.
(virtTestMain): Always capture log data emitted during tests.
* tests/qemuargv2xmltest.c (testCompareXMLToArgvHelper, mymain):
Use flag to mark which tests expect noisy stderr.
(testCompareXMLToArgvFiles): Add parameter to test whether stderr
was appropriately silent.
2010-09-10 20:25:49 +04:00
2016-05-26 18:02:00 +03:00
void virTestQuiesceLibvirtErrors ( bool always ) ;
2013-09-17 17:20:24 +04:00
2016-05-26 18:01:57 +03:00
void virTestCounterReset ( const char * prefix ) ;
2016-05-26 18:02:06 +03:00
const char * virTestCounterNext ( void ) ;
2015-01-15 13:44:58 +03:00
2016-05-26 18:02:08 +03:00
int virTestMain ( int argc ,
char * * argv ,
int ( * func ) ( void ) ,
. . . ) ;
2008-05-29 19:21:45 +04:00
2011-04-29 20:21:20 +04:00
/* Setup, then call func() */
2019-06-18 19:13:14 +03:00
# define VIR_TEST_MAIN(func) \
2017-11-03 15:09:47 +03:00
int main ( int argc , char * * argv ) { \
return virTestMain ( argc , argv , func , NULL ) ; \
2008-05-29 19:21:45 +04:00
}
2019-08-21 19:13:15 +03:00
# ifdef __APPLE__
# define PRELOAD_VAR "DYLD_INSERT_LIBRARIES"
2019-08-21 19:13:17 +03:00
# define MOCK_EXT ".dylib"
2019-08-21 19:13:15 +03:00
# else
# define PRELOAD_VAR "LD_PRELOAD"
2019-08-21 19:13:17 +03:00
# define MOCK_EXT ".so"
2019-08-21 19:13:15 +03:00
# endif
2019-06-18 19:13:14 +03:00
# define VIR_TEST_PRELOAD(lib) \
2017-11-03 15:09:47 +03:00
do { \
2019-08-21 19:13:15 +03:00
const char * preload = getenv ( PRELOAD_VAR ) ; \
2017-11-03 15:09:47 +03:00
if ( preload = = NULL | | strstr ( preload , lib ) = = NULL ) { \
char * newenv ; \
if ( ! virFileIsExecutable ( lib ) ) { \
perror ( lib ) ; \
return EXIT_FAILURE ; \
} \
if ( ! preload ) { \
newenv = ( char * ) lib ; \
} else if ( virAsprintf ( & newenv , " %s:%s " , lib , preload ) < 0 ) { \
perror ( " virAsprintf " ) ; \
return EXIT_FAILURE ; \
} \
2019-08-21 19:13:15 +03:00
setenv ( PRELOAD_VAR , newenv , 1 ) ; \
2017-11-03 15:09:47 +03:00
execv ( argv [ 0 ] , argv ) ; \
} \
2016-04-26 16:00:24 +03:00
} while ( 0 )
2019-06-18 19:13:14 +03:00
# define VIR_TEST_MAIN_PRELOAD(func, ...) \
2017-11-03 15:09:47 +03:00
int main ( int argc , char * * argv ) { \
return virTestMain ( argc , argv , func , __VA_ARGS__ , NULL ) ; \
2012-08-10 17:31:14 +04:00
}
2019-08-21 19:13:16 +03:00
# define VIR_TEST_MOCK(mock) (abs_builddir " / .libs / lib" mock "mock" MOCK_EXT)
2013-11-12 15:57:56 +04:00
virCapsPtr virTestGenericCapsInit ( void ) ;
tests: Create full host NUMA topology in more cases
vircapstest has code to add a full host NUMA topology, that
is, one that includes all information about nodes and CPUs
including IDs; testQemuCapsInit(), which is used to create a
mock virCapsPtr for QEMU tests, however, just fakes it by
setting nnumaCell_max to some number.
While the latter approach has served us well so far, we're
going to need all the information to be filled in soon. In
order to do that, we can just move the existing code from
vircapstest to testutils and, with some renaming and
trivial tweaking, use it as-is.
Interestingly, the NUMA topology generated by the function
is rigged up so that the NUMA nodes aren't (necessarily)
numbered starting from 0, which is a nice way to spot
mistaken assumptions in our codebase.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-04-10 18:42:14 +03:00
int virTestCapsBuildNUMATopology ( virCapsPtr caps ,
int seq ) ;
2013-11-12 15:57:56 +04:00
virDomainXMLOptionPtr virTestGenericDomainXMLConfInit ( void ) ;
2016-04-08 19:04:10 +03:00
typedef enum {
TEST_COMPARE_DOM_XML2XML_RESULT_SUCCESS ,
TEST_COMPARE_DOM_XML2XML_RESULT_FAIL_PARSE ,
TEST_COMPARE_DOM_XML2XML_RESULT_FAIL_STABILITY ,
TEST_COMPARE_DOM_XML2XML_RESULT_FAIL_FORMAT ,
TEST_COMPARE_DOM_XML2XML_RESULT_FAIL_COMPARE ,
} testCompareDomXML2XMLResult ;
2016-01-08 23:55:44 +03:00
int testCompareDomXML2XMLFiles ( virCapsPtr caps ,
virDomainXMLOptionPtr xmlopt ,
const char * inxml ,
const char * outfile ,
2016-01-27 23:55:01 +03:00
bool live ,
2016-04-08 19:04:10 +03:00
unsigned int parseFlags ,
testCompareDomXML2XMLResult expectResult ) ;