Simple Conformance Test
The archive below contains a few simple programs to test the
conformance of your solver to the requirements of the PB06 evaluation
concerning the solver output. These tests are far from being complete
and will only detect obvious violations of the evaluation rules. The
most important point is that these test are mostly concerned with the
syntax output by your solver. No
serious test of the correctness of your solver is performed.
To use these tests, download the archive, unpack it and change to
tar xvzf PB06ConformanceTest.tar.gz
Edit the file PB06ConformanceTest and change the commandLine
variable. It must contain the command line that you must use to run
your solver. BENCHNAME and RANDOMSEED are placeholders for the instance
file name and the random seed, respectively. They will be replaced
appropriately by the test program.
$commandLine="/path/yourProgramName BENCHNAME RANDOMSEED";
Type make to compile the auxilliary programs and run the test.
The tests create a few files
- /tmp/solverfile: the
output of your solver
- /tmp/verifierfile: the
output of the program that checks the model output by your solver
- /tmp/watcherfile: the
output of the killer program
- verifier: a program to
verify that the model output by your solver satisfies all constraints
(and to compute the value of the objective constraint). This is a
simplified version of the program used during the evaluation. You may
use this program to check your solutions.
- killer: a program that
stops your solver after x
seconds (sends SIGTERM then SIGKILL). This emulates the way runsolver
may stop your solver during the evaluation.
To clean all the files that were created, type