PB'11 competition: WBO track: solvers page (tabular format, summary)

Some explanations

#Benchs: Number of benchs for this solver
Solved: Total number of solved instances (this is the sum of the SAT and UNSAT columns)
UNSAT: solver proved unsatitisfiability ("s UNSATISFIABLE" was output)
MOPT: Total number of OPT answers
SAT: Total number of SAT answers
MSAT: Total number of MSAT answers
UNKNOWN: Total number of UNKNOWN answers
Incomplete Answers: Total number of incomplete answers
Bad Answers: Total number of bad answers

Category "both soft and hard constraints, big integers, linear constraints" (PARTIAL-BIGINT-LIN)

Solver Name	#benchs	Solved	UNSAT	MOPT	SAT	MSAT	UNKNOWN	Incomplete Answers	Bad Answers
Sat4j Resolution 2011-06-08 (complete)	263	115 (43.73%)	0 (0%)	115 (43.73%)	0 (0%)	126 (47.91%)	22 (8.37%)	0 (0%)	0 (0%)

Solver Name

Progress

Sat4j Resolution
2011-06-08
(complete)

done
263

MOPT
115

MSAT
126

?
22

Category "both soft and hard constraints, small integers, linear constraints" (PARTIAL-SMALLINT-LIN)

Solver Name	#benchs	Solved	UNSAT	MOPT	MSAT	UNKNOWN
clasp 2.0-RC2 (complete)	536	455 (84.89%)	1 (0.19%)	454 (84.7%)	81 (15.11%)	0 (0%)
Sat4j Resolution 2011-06-08 (complete)	536	448 (83.58%)	1 (0.19%)	447 (83.4%)	87 (16.23%)	1 (0.19%)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 (complete)	536	383 (71.46%)	1 (0.19%)	382 (71.27%)	128 (23.88%)	25 (4.66%)
wbo 1.6 (complete)	536	373 (69.59%)	1 (0.19%)	372 (69.4%)	25 (4.66%)	138 (25.75%)

Solver Name

Progress

clasp
2.0-RC2
(complete)

done
536

UNSAT
1

MOPT
454

MSAT
81

Sat4j Resolution
2011-06-08
(complete)

done
536

UNSAT
1

MOPT
447

MSAT
87

?
1

SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4
(complete)

done
536

UNSAT
1

MOPT
382

MSAT
128

?
25

wbo
1.6
(complete)

done
536

UNSAT
1

MOPT
372

MSAT
25

?
138

Category "only soft constraints, big integers, linear constraints" (SOFT-BIGINT-LIN)

Solver Name	#benchs	Solved	UNSAT	MOPT	SAT	MSAT	UNKNOWN	Incomplete Answers	Bad Answers
Sat4j Resolution 2011-06-08 (complete)	46	33 (71.74%)	0 (0%)	33 (71.74%)	0 (0%)	13 (28.26%)	0 (0%)	0 (0%)	0 (0%)

Solver Name

Progress

Sat4j Resolution
2011-06-08
(complete)

done
46

MOPT
33

MSAT
13

Category "only soft constraints, small integers, linear constraints" (SOFT-SMALLINT-LIN)

Solver Name	#benchs	Solved	MOPT	MSAT	UNKNOWN
clasp 2.0-RC2 (complete)	201	163 (81.09%)	163 (81.09%)	38 (18.91%)	0 (0%)
Sat4j Resolution 2011-06-08 (complete)	201	162 (80.6%)	162 (80.6%)	39 (19.4%)	0 (0%)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 (complete)	201	120 (59.7%)	120 (59.7%)	47 (23.38%)	34 (16.92%)
wbo 1.6 (complete)	201	160 (79.6%)	160 (79.6%)	0 (0%)	41 (20.4%)

Solver Name

Progress

clasp
2.0-RC2
(complete)

done
201

MOPT
163

MSAT
38

Sat4j Resolution
2011-06-08
(complete)

done
201

MOPT
162

MSAT
39

SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4
(complete)

done
201

MOPT
120

MSAT
47

?
34

wbo
1.6
(complete)

done
201

MOPT
160

?
41

All categories for solvers registered in all categories

Solver Name	#benchs	Solved	UNSAT	MOPT	SAT	MSAT	UNKNOWN	Incomplete Answers	Bad Answers
Sat4j Resolution 2011-06-08 (complete)	1046	758 (72.47%)	1 (0.1%)	757 (72.37%)	0 (0%)	265 (25.33%)	23 (2.2%)	0 (0%)	0 (0%)

Solver Name

Progress

Sat4j Resolution
2011-06-08
(complete)

done
1046

UNSAT
1

MOPT
757

MSAT
265

?
23

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name	#benchs	Solved	UNSAT	MOPT	MSAT	UNKNOWN
clasp 2.0-RC2 (complete)	536	455 (84.89%)	1 (0.19%)	454 (84.7%)	81 (15.11%)	0 (0%)
clasp 2.0-RC2 (complete)	201	163 (81.09%)	0 (0%)	163 (81.09%)	38 (18.91%)	0 (0%)
Sat4j Resolution 2011-06-08 (complete)	1046	758 (72.47%)	1 (0.1%)	757 (72.37%)	265 (25.33%)	23 (2.2%)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 (complete)	737	503 (68.25%)	1 (0.14%)	502 (68.11%)	175 (23.74%)	59 (8.01%)
wbo 1.6 (complete)	737	533 (72.32%)	1 (0.14%)	532 (72.18%)	25 (3.39%)	179 (24.29%)