Eiffeltest

This page specifies a tool that has not been written yet.

eiffeltest is a tool that runs a suite of tests. This tool allows to validate, with the provided test-suite, the compiler and the libraries. Users can use it to run a test-suite of their own, which is a usefull tool for their project robustness.

Synopsis

• eiffeltest directory

• eiffeltest source_file.e [source_file2.e...]

File handling

The tools recursively iterates over the directory given as a command-line parameter, looking for test files or over the given test file names.

Test files are Eiffel source files with special names:

• test_*.e: valid source file that should be compilable and runnable without causing an error
• bad_*.e: invalid source file that should trigger a given compiler error message

For each test file, there can be optional files that have the same name but a different extension. These optional files can be used to provide:

• output that the compiler is expected to provide when compiling bad_*.e files. This file mandatory file for invalid tests has the file name extension .msg
• input data to be fed to the program. This optional file (allowed only for valid tests) has the file name extension .in
• output that the program is expected to provide when run. This optional file (allowed only for valid tests) has the file name extension .out

Open issues

• How can we use .in and .out files on platforms that do not allow redirecting the standard input and output of programs?

Easy: just finish the "exec" cluster ;-) --Cyril 08:16, 10 nov 2005 (CET)

• Which compilation modes are used for testing? There are many options and running all possibilities is probably too much.

Flags I can remember from: -no_gc, -flat_check, -no_split, (-debug_check | -all_check | -loop_check | -invariant_check | -ensure_check | -require_check | -no_check | -boost) This means compiling and running about 64 times (minus incompatibilities between -flat_check and boost/no_check).

• How to test some specific capabilities: options -profile, -no_main, -c_mode, -sedb, -cecil, or testing with c2j?

• What about tests specific to some compiler mode? For example GC tests should not be run without GC, optimizer tests are invalid in modes other than -boost...