| 1 | /*
|
|---|
| 2 | * Project: MoleCuilder
|
|---|
| 3 | * Description: creates and alters molecular systems
|
|---|
| 4 | * Copyright (C) 2010 University of Bonn. All rights reserved.
|
|---|
| 5 | * Please see the LICENSE file or "Copyright notice" in builder.cpp for details.
|
|---|
| 6 | */
|
|---|
| 7 |
|
|---|
| 8 | /**
|
|---|
| 9 | * \file tests.dox
|
|---|
| 10 | *
|
|---|
| 11 | * Created on: Oct 28, 2011
|
|---|
| 12 | * Author: heber
|
|---|
| 13 | */
|
|---|
| 14 |
|
|---|
| 15 | /**
|
|---|
| 16 | * \page tests Automated Tests
|
|---|
| 17 | *
|
|---|
| 18 | * There are three kinds of tests:
|
|---|
| 19 | *
|
|---|
| 20 | * \li \subpage codetest "Code tests"
|
|---|
| 21 | * \li \subpage regressiontest "Regression tests"
|
|---|
| 22 | * \li \subpage unittest "Unit test"
|
|---|
| 23 | *
|
|---|
| 24 | * These behave more or less like top-down and bottom-up approaches to automated
|
|---|
| 25 | * testing. Tests are regarded here as a kind of contract. The code itself is
|
|---|
| 26 | * just one hand in two hands shaking, the other hand is resembled by the tests
|
|---|
| 27 | * that check whether the code does exactly what it's supposed to do. Without
|
|---|
| 28 | * testing a larger project is impossible because it cannot evolve: The old
|
|---|
| 29 | * addage and compromises grow and grow. With increasing size, a project must be
|
|---|
| 30 | * refactored (http://en.wikipedia.org/wiki/Code_refactoring) such that new code
|
|---|
| 31 | * does not have to wiggle itself around the same old issues that are
|
|---|
| 32 | * inadvertently and unavoidably present from the start. Before one starts
|
|---|
| 33 | * refactoring, it must be assured by some means that the code before and after
|
|---|
| 34 | * behaves the same with respect to its intended functionality. These means are
|
|---|
| 35 | * the tests.
|
|---|
| 36 | *
|
|---|
| 37 | * Unit tests (http://en.wikipedia.org/wiki/Unit_testing) check on single
|
|---|
| 38 | * components (e.g. classes), dependencies of the components on other classes
|
|---|
| 39 | * with the test frame are often just mimicked via so-called stubs (sort of
|
|---|
| 40 | * dummy components that don't calculate or do anything but return an expected
|
|---|
| 41 | * result suitable for this test only, see http://en.wikipedia.org/wiki/Test_stubs).
|
|---|
| 42 | * These test whether a component always behaves as desired.
|
|---|
| 43 | *
|
|---|
| 44 | * Regression test (http://en.wikipedia.org/wiki/Regression_testing) on the other
|
|---|
| 45 | * hand test some specific functionality of the code from the top-most scope,
|
|---|
| 46 | * i.e. they are integration test that check whether the code eventually does
|
|---|
| 47 | * what's wanted. Lateron, they also check whether added or altered functionality
|
|---|
| 48 | * has not changed the outcome of older functions.
|
|---|
| 49 | *
|
|---|
| 50 | * \section tests-launch-all Launching all tests
|
|---|
| 51 | *
|
|---|
| 52 | * Note that all tests can be launched via
|
|---|
| 53 | * \code make check \endcode
|
|---|
| 54 | * in the top build directory.
|
|---|
| 55 | *
|
|---|
| 56 | * \section tests-policy Policy on launching tests
|
|---|
| 57 | *
|
|---|
| 58 | * We have the the following list of checks on every new version:
|
|---|
| 59 | *
|
|---|
| 60 | * -# make check on every commit between old and new version in history
|
|---|
| 61 | * for both --enable-debug and --disable-debug
|
|---|
| 62 | * -# make distcheck on last version (where version is updated)
|
|---|
| 63 | * -# make check with --enable-valgrind on last version
|
|---|
| 64 | * -# make check on last version with every possible --enable/--disable
|
|---|
| 65 | * combination (excluding ecut, valgrind)
|
|---|
| 66 | *
|
|---|
| 67 | * \subsection tests-policy-check make check
|
|---|
| 68 | *
|
|---|
| 69 | * Note that the call
|
|---|
| 70 | * \code make check \endcode
|
|---|
| 71 | * should \e pass for \e each \e and \e every \a single \e test for each and
|
|---|
| 72 | * every single commit in the code history before it is pushed to the central
|
|---|
| 73 | * repository (git has ample means via \a rebase for correcting later found
|
|---|
| 74 | * errors), both for \a --disable-debug and \a --enable-debug.
|
|---|
| 75 | *
|
|---|
| 76 | * \subsection tests-policy-distcheck make distcheck
|
|---|
| 77 | *
|
|---|
| 78 | * Before a version tag is given (e.g. v1.1.3) it is to be made sure that also
|
|---|
| 79 | * \code make distcheck \endcode
|
|---|
| 80 | * runs fine and produces a distributable archive.
|
|---|
| 81 | *
|
|---|
| 82 | * \subsection tests-policy-enablevalgrind make check with --enable-valgrind
|
|---|
| 83 | *
|
|---|
| 84 | * Since v1.4.8 valgrind's memcheck tool runs through as well (on all regression
|
|---|
| 85 | * tests) on make check if configure has been executed with
|
|---|
| 86 | * \code ./configure .. -enable-valgrind \endcode
|
|---|
| 87 | * There are three glitches, namely regression tests 28-30 which load
|
|---|
| 88 | * python scripts and cause many \a PyObject_Free and other errors. These are
|
|---|
| 89 | * ignored. A python suppression file was tried but to no success.
|
|---|
| 90 | *
|
|---|
| 91 | * In summary, a valgrind check prior to giving a version tag is to be made
|
|---|
| 92 | * starting from this version to prevent memory leaks.
|
|---|
| 93 | *
|
|---|
| 94 | *
|
|---|
| 95 | * \date 2015-07-31
|
|---|
| 96 | *
|
|---|
| 97 | */
|
|---|
