/*
 * Project: MoleCuilder
 * Description: creates and alters molecular systems
 * Copyright (C)  2010 University of Bonn. All rights reserved.
 * Please see the LICENSE file or "Copyright notice" in builder.cpp for details.
 */

/**
 * \file regression-tests.dox
 *
 * This file contains explanation how to write, launch and use regression tests.
 *
 * Created on: Oct 11, 2011
 *    Author: heber
 */


/**
 * \page regressiontest "Regression tests"
 *
 * Regression test with this project are used to check on the functionality of
 * all Actions. The launch the MoleCuilder from the command line with a given
 * action and option and basically diff the output against a stored file with
 * the desired result. This should be the behavior of more than 90% of all
 * regression tests.
 *
 * \section Directory structure
 *
 * They are contained in the source folder \b tests/regression. There are
 * categories containing a distinct folder for each action. The folder for the
 * test of a specific Action has the following structure:
 *  \li a folder \b pre contains all files required as input to the test.
 *  \li a folder \b post contains all output files against which we compare all
 *  or a number of resulting files.
 *  \li a test script \b testsuite-....at which is included in a similarly-
 *  named test script one directory level higher.
 *
 *  \section Adding a new test
 *
 *  Adding a new regression tests consists of the following items:
 *  -# Create a new folder in a matching category
 *  -# Create therein subfolders \b pre and \b post
 *  -# Create a test script where the name begins with \b testsuite- and contains
 *  category and the action token. See other test scripts to get an idea on how
 *  to write these and also look here (http://www.gnu.org/s/hello/manual/autoconf/Using-Autotest.html#Using-Autotest).
 *  -# In the command \a AT_KEYWORD which must be present you should given the
 *  token of the Action as it would be called from the command line (see below for
 *  the reason).
 *  -# Include your testscript in the one present one directory-level up.
 *  -# Also include your script in \b tests/scripts/Makefile.am such that the
 *  testsuite is automatically re-compiled when one of the test files has changed.
 *
 *  \section Launching all tests
 *
 *  Launching all regression tests is as simple as:
 *  -# Enter the build directory
 *  -# There, enter \b tests/regression
 *  -# Run
 *  \code make check \endcode
 *  (at you liberty with option \a -j8 or similar for running the tests in parallel.
 *
 *  \section Launching a specific tests
 *
 *  Launching a single or just some of the tests is only a little bit more complicated.
 *  There are two options: either by the test number which however changes when new
 *  tests are added, and by keywords.
 *
 *  Then proceed as follows:
 *  -# Enter the build directory
 *  -# There, enter \b tests/regression
 *  -# Run
 *  \code ../../../tests/regression/testsuite <option> AUTOTEST_PATH="<buildpath>/src" \endcode,
 *  where \a <option> is explained in the subsections below and \a <buildpath> is the build
 *  path (i.e. the variable \a AUTOTEST_PATH should contain the path to the executable).
 *
 *  \subsection ... by number
 *
 *    Tests can be launched by specifying their test number, e.g. then \a <option>
 *    might be \a 283 or \a 283-284 or \a 283,285-286 or alike
 *
 *  \subsection .. by keyword
 *
 *    Tests may as well be launched by some keywords, e.g. each Action has a specific
 *    token which is also one of its keyword (see above for the policy). I.e. we may
 *    launch the test on fill-void via the \a <option> \a -k \a fill-void. Also
 *    multiple keywords may be given.
 *
 * \section Inspecting test results
 *
 *  The testsuite can be launched with the additional option of \a -d which leaves the
 *  directory of the test present even though the test has passed for inspection.
 *
 *  If a test fails, in whatever way it was launched, will leave in the build directory
 *  a folder \b tests/regression/testsuite.dir/<nr> where \a <nr> is the number of the
 *  test (padded maybe with some zeros).
 *
 */