Eiffeltest
This page specifies a tool that has not been written yet.
eiffeltest is a tool that runs a suite of tests to validate the compiler
Synopsis
eiffeltest directory
eiffeltest source_file.e [source_file2.e...]
File handling
The tools recursively iterates over the directory given as a command-line parameter, looking for test files or over the given test file names.
Test files are Eiffel source files with special names:
- test_*.e: valid source file that should be compilable and runnable without causing an error
- bad_*.e: invalid source file that should trigger a given compiler error message
For each test file, there can be optional files that have the same name but a different extension. These optional files can be used to provide:
- output that the compiler is expected to provide when compiling bad_*.e files. This file mandatory file for invalid tests has the file name extension .msg
- input data to be fed to the program. This optional file (allowed only for valid tests) has the file name extension .in
- output that the program is expected to provide when run. This optional file (allowed only for valid tests) has the file name extension .out
Open issues
How can we use .in and .out files on platforms that do not allow redirecting the standard input and output of programs?
Easy: just finish the "exec" cluster ;-) --Cyril 08:16, 10 nov 2005 (CET)
Which compilation modes are used for testing? There are many options and running all possibilities is probably too much. Flags I can remember from: -no_gc, -flat_check, -no_split, (-debug_check | -all_check | -loop_check | -invariant_check | -ensure_check | -require_check | -no_check | -boost) This means compiling and running about 64 times (minus incompatibilities between -flat_check and boost/no_check).
How to test some specific capabilities: options -profile, -no_main, -c_mode, -sedb, -cecil, or testing with c2j?
What about tests specific to some compiler mode? For example GC tests should not be run without GC, optimizer tests are invalid in modes other than -boost...