Eiffeltest
This page specifies a tool that has not been written yet.
eiffeltest is a tool that runs a suite of tests. This tool allows to validate, with the provided test-suite, the compiler and the libraries. Users can use it to run a test-suite of their own, which is a usefull tool for their project robustness.
Synopsis
eiffeltest directory
eiffeltest source_file.e [source_file2.e...]
File handling
The tools recursively iterates over the directory given as a command-line parameter, looking for test files or over the given test file names.
Test files are Eiffel source files with special names:
- test_*.e: valid source file that should be compilable and runnable without causing an error
- bad_*.e: invalid source file that should trigger a given compiler error message
For each test file, there can be optional files that have the same name but a different extension. These optional files can be used to provide:
- output that the compiler is expected to provide when compiling bad_*.e files. This file mandatory file for invalid tests has the file name extension .msg
- input data to be fed to the program. This optional file (allowed only for valid tests) has the file name extension .in
- output that the program is expected to provide when run. This optional file (allowed only for valid tests) has the file name extension .out
Open issues
- How can we use .in and .out files on platforms that do not allow redirecting the standard input and output of programs?
Easy: just finish the "exec" cluster ;-) --Cyril 08:16, 10 nov 2005 (CET)
- Which compilation modes are used for testing? There are many options and running all possibilities is probably too much.
Flags I can remember from: -no_gc, -flat_check, -no_split, (-debug_check | -all_check | -loop_check | -invariant_check | -ensure_check | -require_check | -no_check | -boost) This means compiling and running about 64 times (minus incompatibilities between -flat_check and boost/no_check).
- How to test some specific capabilities: options -profile, -no_main, -c_mode, -sedb, -cecil, or testing with c2j?
- What about tests specific to some compiler mode? For example GC tests should not be run without GC, optimizer tests are invalid in modes other than -boost...