This site gives some information based on the answers provided by the different solvers. Not all answers given by a solver can be checked efficiently. Therefore, some answers given by a solver may be wrong but this may be unnoticed by the evaluation environment.

Here are the checks made for each answer.

- SATISFIABLE answers:

The model provided by the solver is given to a verifier program which checks that all constraints are actually satisfied. These answers are the only ones that are guaranteed to be true. - UNSATISFIABLE answers:

The only check that is made is to verify that no other solver found a solution to this instance. An UNSATISFIABLE answer is not reliable. Some solvers may wrongly indicate UNSATISFIABLE but unless another solver finds a model, this will remain undetected. - OPTIMUM FOUND answers:

Beyond verifying the model and the value of the objective function, the only check that is made is to verify that no other solver found a better solution to this instance. An OPTIMUM FOUND answer is not reliable. Some solvers may wrongly indicate OPTIMUM FOUND but unless another solver finds a better model, this will remain undetected.

- Information about solvers (command line, supported categories,...)
- Information about selected instances (no solver result)

- CPU Ranking of solvers
- WC Ranking of solvers
- All solvers (tabular format, summary)
- All solvers (tabular format, detailed results)

- AbsCon-basic
*2017-06-11*(complete) (list format) - BTD
*2017-07-25*(complete) (list format) - BTD
*2017-08-10*(complete) (list format) - choco-solver
*4.0.5 par (2017-08-09)*(complete) (list format) - choco-solver
*4.0.5 par (2017-08-18)*(complete) (list format) - choco-solver
*4.0.5 seq (2017-08-09)*(complete) (list format) - choco-solver
*4.0.5 seq (2017-08-18)*(complete) (list format) - choco-solver
*5a (2017-07-26)*(complete) (list format) - choco-solver
*5a (2017-08-18)*(complete) (list format) - Concrete
*3.4*(complete) (list format) - cosoco
*1.12*(complete) (list format) - cosoco-sat
*1.12*(complete) (list format) - Mistral-2.0
*2017-07-28*(complete) (list format) - OscaR - ALNS
*2017-07-26*(complete) (list format) - OscaR - Conflict Ordering
*2017-07-26*(complete) (list format) - OscaR - Hybrid
*2017-07-26*(complete) (list format) - OscaR - Parallel with EPS
*2017-07-26*(complete) (list format) - OscaR - Parallel with EPS
*2017-08-22*(complete) (list format) - R1: AbsCon-basic
*2017-06-11*(complete) (list format) - R1: BTD
*2017-06-30*(complete) (list format) - R1: choco-solver
*4.0.5 (2017-07-05)*(complete) (list format) - R1: choco-solver
*5a (2017-07-05)*(complete) (list format) - R1: Concrete
*3.4*(complete) (list format) - R1: cosoco
*1.1*(complete) (list format) - R1: cosoco-sat
*1.1*(complete) (list format) - R1: Mistral-2.0
*2017-06-28*(complete) (list format) - R1: OscaR - ALNS
*2017-06-30*(complete) (list format) - R1: OscaR - Conflict Ordering
*2017-06-30*(complete) (list format) - R1: OscaR - Hybrid
*2017-06-30*(complete) (list format) - R1: sat4j-CSP
*2017-07-05*(complete) (list format) - sat4j-CSP
*2017-07-05*(complete) (list format)

- Number of instances solved in a given amount of time
- Number of instances solved within a given amount of memory
- Memory versus time for each solvers
- Evolution of the solution quality for solver AbsCon-basic
*2017-06-11*(complete) - Evolution of the solution quality for solver BTD
*2017-07-25*(complete) - Evolution of the solution quality for solver BTD
*2017-08-10*(complete) - Evolution of the solution quality for solver choco-solver
*4.0.5 par (2017-08-09)*(complete) - Evolution of the solution quality for solver choco-solver
*4.0.5 par (2017-08-18)*(complete) - Evolution of the solution quality for solver choco-solver
*4.0.5 seq (2017-08-09)*(complete) - Evolution of the solution quality for solver choco-solver
*4.0.5 seq (2017-08-18)*(complete) - Evolution of the solution quality for solver choco-solver
*5a (2017-07-26)*(complete) - Evolution of the solution quality for solver choco-solver
*5a (2017-08-18)*(complete) - Evolution of the solution quality for solver Concrete
*3.4*(complete) - Evolution of the solution quality for solver cosoco
*1.12*(complete) - Evolution of the solution quality for solver cosoco-sat
*1.12*(complete) - Evolution of the solution quality for solver Mistral-2.0
*2017-07-28*(complete) - Evolution of the solution quality for solver OscaR - ALNS
*2017-07-26*(complete) - Evolution of the solution quality for solver OscaR - Conflict Ordering
*2017-07-26*(complete) - Evolution of the solution quality for solver OscaR - Hybrid
*2017-07-26*(complete) - Evolution of the solution quality for solver OscaR - Parallel with EPS
*2017-07-26*(complete) - Evolution of the solution quality for solver OscaR - Parallel with EPS
*2017-08-22*(complete) - Evolution of the solution quality for solver R1: AbsCon-basic
*2017-06-11*(complete) - Evolution of the solution quality for solver R1: BTD
*2017-06-30*(complete) - Evolution of the solution quality for solver R1: choco-solver
*4.0.5 (2017-07-05)*(complete) - Evolution of the solution quality for solver R1: choco-solver
*5a (2017-07-05)*(complete) - Evolution of the solution quality for solver R1: Concrete
*3.4*(complete) - Evolution of the solution quality for solver R1: cosoco
*1.1*(complete) - Evolution of the solution quality for solver R1: cosoco-sat
*1.1*(complete) - Evolution of the solution quality for solver R1: Mistral-2.0
*2017-06-28*(complete) - Evolution of the solution quality for solver R1: OscaR - ALNS
*2017-06-30*(complete) - Evolution of the solution quality for solver R1: OscaR - Conflict Ordering
*2017-06-30*(complete) - Evolution of the solution quality for solver R1: OscaR - Hybrid
*2017-06-30*(complete) - Evolution of the solution quality for solver R1: sat4j-CSP
*2017-07-05*(complete) - Evolution of the solution quality for solver sat4j-CSP
*2017-07-05*(complete)

- Graphical comparison of two solvers based on CPU time
- Graphical comparison of two solvers based on memory

- Results for category COP (optimization problem, 438 instances)
- Results for category CSP (decision problem, 510 instances)
- Results for all categories