Introduction
+Preparation
+Execution
+Adding a new test
+Requirements
+History
Boost's internal regression test suite produces the +compiler status tables.
Although not ordinarily run by Boost library users, it is documented here for the benefit of Boost developers, and for Boost users porting to a new platform.
-Boost is transitioning to a new version of the regression tests. During -the transition, both versions are available.
- -- - -- -
-You should be able to compile and link this file using whatever C++ -compiler is at your disposition. However, you may need to configure -both the compiler and the standard library to use "strict" ISO -compliance mode. Also, you need to extend the search path for include -files with the main boost directory so that the header file -boost/config.hpp -can be found. This header file is required to work around compiler -deficiencies. -
- -You can then start the resulting executable to run regression tests. -By default, the regression test program reads the file "compiler.cfg" -in the current directory to determine the list of compilers and their -invocation syntax. After that, it reads the file "regression.cfg" in -the current directory to determine the regression tests to run. The -results of the regression tests are written in an HTML formatted text -file. This file is by default named "cs-OS.html" in the current -directory, where "OS" is a placeholder for the name of the operating -system (e.g., "linux" or "win32"). -
- -To generate the -compiler status tables, -boost uses the files -status/compiler.cfg -and -status/regression.cfg. -
- -The regression test program accepts some command-line options to alter -its behavior. -
-
| -h or --help | -prints a help message - |
| --config file | -Use file instead of "compiler.cfg" as the compiler -configuration file. This allows for private compiler setups. | -
| --tests file | -Use file instead of "regression.cfg" as the tests -configuration file. This allows individual libraries to specify -additional tests not to be published in the main -compiler status tables. - | -
| --boost path | -Use path as the filesystem path to the main boost -directory. The default is "..", i.e. the parent directory. | -
| --output file -o file |
-Write the HTML output to file instead of the default -"cs-OS.html". | -
| --compiler name | -Run the tests only with compiler name. The name -must be defined in the second line of an applicable compiler -configuration (see below). The default is to run the tests with all -compilers suitable for the platform. | -
| --diff | -Read the HTML output file before writing it. In the HTML output, -highlight differences in test outcomes compared to the previous -run. | -
| test | -Run only the named test. The syntax is the same as in the -configuration file (see below). | -
-When running only a selected test, you must also provide an alternate -HTML output filename with "--output" so that the full test output is -not accidentally overwritten. -
-
-You should redirect the output (std::cout) and error
-(std::cerr) channels to a suitable log file for later
-inspection.
-
-
-
std::cout prior to running a test with that compiler.
-- -The following test types are available: -
main
-function must be present in the test file.
-get_host() and return a unique string identifying the
-platform (this string is used to filter the compile configuration
-file).
-get_system_configuration() is
-appropriately defined for your platform. For a Unix platform, it most
-likely is. The function may return a verbose HTML string describing
-the platform.
-You may need to add an entry for the compiler to boost/config.hpp, -but only if entries for the compiler is not already present, and the compiler -doesn't fully conform to the ISO C++ Standard.
-Version 3 of the Boost regression testing framework is based on Boost.Build, and uses bjam to actually run the tests. Because Boost.Build does dependency analysis, only tests for which some dependency has change are rerun.
-The reporting of test results as HTML files is accomplished by separate C++ -programs which process the residue and log files from the bjam run.
-The reporting of test results as HTML files is accomplished by separate +regression reporting +programs which process the residue and log files from the bjam run.
+Install the following programs on your system, in some location suitable for -program executables. Normally that location will be a directory which is part of +program executables. Normally that location must be in a directory which is part of your search path for executables.
These examples assume several environment variables have been set:
cd %BOOST_ROOT%\status md bin 2>nul -bjam test >bin\regr.log 2>&1 -start notepad bin\regr.log -process_jam_log <bin\regr.log +bjam --dump-tests test >bin\bjam.log 2>&1 +start notepad bin\bjam.log +process_jam_log <bin\bjam.log compiler_status %BOOST_ROOT% cs-win32.html rem Specify links file, even though it will be overwritten, so report html includes links rem This works because the generated bookmark names are the same regardless of other settings @@ -321,9 +98,12 @@ rather than "test". For example, to debug configurations, it mig useful to just run the config_info test, with a switch to force even up-to-date programs be rebuilt:--bjam -a config_info+bjam -a --dump-tests config_info >bin\bjam.log 2>&1 +process_jam_log <bin\bjam.log +compiler_status %BOOST_ROOT% cs-win32.html +Adding a new test
+Adding a new test
Overall Boost regression tests
Adding tests is as simple as adding a single line to the boost-root/status/Jamfile:
@@ -341,7 +121,22 @@ suites show more of the power of Jam based testing.A library can have its own private set of tests by creating a Jamfile in one of the library's own sub-directories. For an example of this, see boost-root/libs/test/test/Jamfile.
-History
+ +Requirements
+ +The test suite has been designed to meet to the following requirements.
++
+ +- Doesn't depend on an external toolchain. This requirement is met by +supplying all tools in the regular Boost distribution, except +for a C++ compiler.
- Tools are written in C++; it is the only language that +all Boost developers and users are comfortable with.
- Supports tests which expect an error +
- Configuration is independent of the target platform or compiler. +
- HTML output +
These requirements rule out any script-based approach such as dejagnu +(requires Tcl and expect) or even shell scripts.
+History
The version 3 testing.jam and status/Jamfile foundation was contributed by Dave Abrahams. The post-bjam processing programs were contributed by Beman Dawes.
@@ -350,8 +145,11 @@ Dawes. by Jens Maurer, generalizing and improving an earlier version 1 program by Beman Dawes.
-2001-01-30
-Jens Maurer +Revised +07 January, 2003+Original author: +Jens Maurer
+Updates: Beman Dawes