1.1 --- a/gmyth/configure.ac Thu Apr 26 22:33:00 2007 +0100
1.2 +++ b/gmyth/configure.ac Thu Apr 26 22:34:37 2007 +0100
1.3 @@ -112,6 +112,25 @@
1.4 LDFLAGS="$LDFLAGS -lgcov"
1.5 fi
1.6
1.7 +AC_ARG_ENABLE(mudflap-debug,
1.8 + AS_HELP_STRING([--enable-mudflap-debug],
1.9 + [enable libmudflap debug (default = no)]),
1.10 + [
1.11 + if test $enableval = no; then
1.12 + USE_MUDFLAP_DEBUG=no;
1.13 + else
1.14 + USE_MUDFLAP_DEBUG=yes;
1.15 + fi
1.16 + ],[
1.17 + USE_MUDFLAP_DEBUG=no;
1.18 + ]
1.19 +)
1.20 +
1.21 +if [ test $USE_MUDFLAP_DEBUG = yes ]; then
1.22 + CFLAGS="$CFLAGS -fmudflap -fmudflapth -O0"
1.23 + LDFLAGS="$LDFLAGS -lmudflapth"
1.24 +fi
1.25 +
1.26 # Check for pkgconfig
1.27 AC_CHECK_PROG(HAVE_PKGCONFIG, pkg-config, yes, no)
1.28 # Give error and exit if we don't have pkgconfig
2.1 --- a/gmyth/tests/Makefile.am Thu Apr 26 22:33:00 2007 +0100
2.2 +++ b/gmyth/tests/Makefile.am Thu Apr 26 22:34:37 2007 +0100
2.3 @@ -1,3 +1,4 @@
2.4 +INCLUDES = -I$(top_srcdir)/src -I$(top_builddir)/src
2.5
2.6 LDADD = \
2.7 $(top_srcdir)/src/.libs/libgmyth.la
3.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000
3.2 +++ b/gmyth/tests/README Thu Apr 26 22:34:37 2007 +0100
3.3 @@ -0,0 +1,166 @@
3.4 +=== Regression Testing Scripts How-To ===
3.5 +
3.6 +Authors:
3.7 + Rafael Espindola
3.8 + Rosfran Borges
3.9 +
3.10 +Requirements:
3.11 +
3.12 +The only additional dependency for running the tests is python 2.3 or
3.13 +newer. Other then this, the tests can be run on any system that can
3.14 +compile gmyth.
3.15 +
3.16 +The test system has some optional features that require some additional
3.17 +tools:
3.18 +
3.19 +To use mudflap during the test, its headers must be installed. Package
3.20 +libmudflap0-dev provides them on Debian.
3.21 +
3.22 +To test with valgrind, the valgrind package must be installed. It might
3.23 +be necessary to tweak the suppression file.
3.24 +
3.25 +To compute line coverage, lcov must be installed. Unfortunately, the
3.26 +current release of lcov has some problems finding the source code when
3.27 +libtool is used. So it is necessary to download the cvs version
3.28 +(http://ltp.cvs.sourceforge.net/ltp/utils/analysis/lcov/bin/lcov).
3.29 +
3.30 +Running the Tests:
3.31 +
3.32 +The tests can be run by the check target. I.E., by running "make check"
3.33 +after compiling gmyth. The way in which the tests are run depends on
3.34 +the options passed to configure:
3.35 +
3.36 +If the --enable-coverage option is used, the library and test programs
3.37 +are compiled with support for generating line coverage
3.38 +information. After running the tests, a summary will be available in
3.39 +tests/regression/covere/index.html.
3.40 +
3.41 +If --enable-mudflap-debug is used, libmudflap is linked with
3.42 +gmyth. Libmudflap will test for and report some common memory
3.43 +management errors.
3.44 +
3.45 +If none of these options are used, valgrind will be used if it is
3.46 +available. Otherwise, the tests will be runs without any special
3.47 +tools.
3.48 +
3.49 +Basics of the test system:
3.50 +
3.51 +Each test consists of a small C program that links with gmyth and a
3.52 +python script. The script runs the C program and interacts with it. By
3.53 +doing so, it can control which code paths are executed.
3.54 +
3.55 +With this separation it is possible to reuse the C program in many
3.56 +tests and use a higher level language for building the tests.
3.57 +
3.58 +If a script returns 0, the test is considered successful. If it
3.59 +returns 77, the test is ignored. If it returns any other value, the
3.60 +test is a failure. After running all tests, Automake reports how many
3.61 +of them were successful.
3.62 +
3.63 +Declaring test programs and test scripts:
3.64 +
3.65 +The test programs are declared in the check_PROGRAMS variable in
3.66 +Makefile.am and the test scripts in the TESTS variable.
3.67 +
3.68 +For example, gmyth_test_livetv is small program that contains just one control
3.69 +point. test_livetv.py and test_epg.py are two scripts
3.70 +that test different actions (tests) on the gmyth modules. To declare them,
3.71 +the Makefile.am should contain:
3.72 +
3.73 +--------------------------------------
3.74 +check_PROGRAMS = gmyth_test_livetv
3.75 +TESTS = test_livetv.py test_epg.py
3.76 +--------------------------------------
3.77 +
3.78 +Synchronizing the Tested Program and the Test Script:
3.79 +
3.80 +Automake will only run the test script. It is up to the script to run
3.81 +the program to be tested. Since it takes some time for the test
3.82 +program to start up, the test script must be informed when the test
3.83 +program is ready. The test script must also be able to instruct the
3.84 +program to exit
3.85 +
3.86 +In gmyth, the following technique is used.
3.87 + the script starts the server "inside" a pipe
3.88 + the test script tries to read a line from the pipe and blocks
3.89 + after startup, the server prints "===SYNC===" to stdout
3.90 + the read in the script succeeds
3.91 + the server tries to read a line from stdin and blocks
3.92 + the test script does the testing
3.93 + the test script writes an empty line to the pipe
3.94 + the read in the server succeeds
3.95 + the test script waits for the server to exit
3.96 +
3.97 +In the test program this is implemented by the following template:
3.98 +
3.99 +--------------------------------------------------
3.100 +#include "common.h"
3.101 +...
3.102 +start the server
3.103 +printf(SYNC_STRING);
3.104 +fflush(NULL);
3.105 +getchar();
3.106 +stop the server
3.107 +--------------------------------------------------
3.108 +
3.109 +In the test script the synchronization is done automatically inside
3.110 +the start function:
3.111 +
3.112 +-------------------------------------------------------
3.113 +p = start('gmyth_test_livetv')
3.114 +-------------------------------------------------------
3.115 +
3.116 +The API
3.117 +
3.118 +Some utility function are provided that abstract some low level stuff
3.119 +like were to find the libraries, the kind of test and error checking.
3.120 +
3.121 +
3.122 +testing_tools.start(exe, ignore):
3.123 +
3.124 +Start the executable "exe" and ignore at most "ignore" libmudflap errors.
3.125 +
3.126 +The "ignore" argument exists because libmudflap doesn't has a builtin suppression
3.127 +mechanism.
3.128 +
3.129 +When start return the synchronization described above is already
3.130 +done.
3.131 +
3.132 +Start returns an object that represent the process. This object has the
3.133 +following attributes:
3.134 +
3.135 + stdin: A pipe to the standard input of the process
3.136 + stdout: A pipe to the standard output of the process
3.137 + stderr: A pipe to the standard error of the process
3.138 +
3.139 +
3.140 +testing_tools.finish(p):
3.141 +
3.142 +Stop the process "p"
3.143 +
3.144 +This function tries to stop the process nicely by using the protocol
3.145 +described is the synchronization section. If the process fails to quit,
3.146 +it is killed.
3.147 +
3.148 +Error checking is dependent of how the tests are being run:
3.149 +
3.150 + coverage or without any tools: check if the program returned 0
3.151 + mudflap: similar coverage, but also check that libmudflap didn't print any errors
3.152 + valgrind: similar to coverage, but also check that valgrind didn't
3.153 + find any errors or memory leaks.
3.154 +
3.155 +if no errors are found, finish returns 0. If any error is found, it returns 1.
3.156 +
3.157 +Example:
3.158 +The following script just starts and stops the 'gmyth_test_livetv' program.
3.159 +
3.160 +--------------------------------------
3.161 +#!/usr/bin/python
3.162 +
3.163 +from test import start
3.164 +import sys
3.165 +
3.166 +p = start('gmyth_test_epg', 2)
3.167 +sys.exit(finish(p))
3.168 +----------------------------------
3.169 +
4.1 --- a/gmyth/tests/coverage.py Thu Apr 26 22:33:00 2007 +0100
4.2 +++ b/gmyth/tests/coverage.py Thu Apr 26 22:34:37 2007 +0100
4.3 @@ -21,7 +21,7 @@
4.4 os.system('genhtml gmyth.info')
4.5
4.6 exe = 'test'
4.7 -exe = '.libs/' + exe
4.8 +exe = [ '.libs/' + exe ]
4.9 if is_coverage(exe):
4.10 bdir = build_dir(exe)
4.11 gen_cov_info(bdir)