MAX-CSP 2008 Competition: solvers page (tabular format, summary)

Some explanations

Category "binary constraints in extension" (2-ARY-EXT)

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
AbsconMax 112
pc-d		 53486
(16.1%)		 86
(16.1%)		 0
(0%)		 448
(83.9%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
AbsconMax 112
pc-w		 534121
(22.66%)		 121
(22.66%)		 0
(0%)		 413
(77.34%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - MCRW Engine
2008-05-30		 53432
(5.99%)		 32
(5.99%)		 0
(0%)		 497
(93.07%)		 5
(0.94%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 53432
(5.99%)		 32
(5.99%)		 0
(0%)		 497
(93.07%)		 5
(0.94%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 534240
(44.94%)		 240
(44.94%)		 0
(0%)		 216
(40.45%)		 78
(14.61%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 534229
(42.88%)		 229
(42.88%)		 0
(0%)		 215
(40.26%)		 88
(16.48%)		 2
(0.37%)		 0
(0%)		 0
toulbar2
2008-06-27		 534412
(77.15%)		 412
(77.15%)		 0
(0%)		 0
(0%)		 122
(22.85%)		 0
(0%)		 0
(0%)		 0
toulbar2/BTD
2008-06-27		 534411
(76.97%)		 411
(76.97%)		 0
(0%)		 0
(0%)		 121
(22.66%)		 0
(0%)		 2
(0.37%)		 0

Solver NameProgress
AbsconMax 112
pc-d
done
534
MOPT
86		MSAT
448
AbsconMax 112
pc-w
done
534
MOPT
121		MSAT
413
Concrete + CSP4J - MCRW Engine
2008-05-30
done
534
MOPT
32		MSAT
497		?
5
Concrete + CSP4J - Tabu Engine
2008-05-30
done
534
MOPT
32		MSAT
497		?
5
Sugar
v1.13+minisat
done
534
MOPT
240		MSAT
216		?
78
Sugar++
v1.13+minisat-inc
done
534
MOPT
229		MSAT
215		?
88		Inc. Answer
2
toulbar2
2008-06-27
done
534
MOPT
412		?
122
toulbar2/BTD
2008-06-27
done
534
MOPT
411		?
121		ERR
2

Category "binary constraints in intension" (2-ARY-INT)

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
AbsconMax 112
pc-d		 27630
(10.87%)		 30
(10.87%)		 0
(0%)		 168
(60.87%)		 76
(27.54%)		 2
(0.72%)		 0
(0%)		 0
AbsconMax 112
pc-w		 27632
(11.59%)		 32
(11.59%)		 0
(0%)		 166
(60.14%)		 76
(27.54%)		 2
(0.72%)		 0
(0%)		 0
Concrete + CSP4J - MCRW Engine
2008-05-30		 2760
(0%)		 0
(0%)		 0
(0%)		 270
(97.83%)		 6
(2.17%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 2760
(0%)		 0
(0%)		 0
(0%)		 269
(97.46%)		 7
(2.54%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 276101
(36.59%)		 101
(36.59%)		 0
(0%)		 71
(25.72%)		 104
(37.68%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 27699
(35.87%)		 99
(35.87%)		 0
(0%)		 53
(19.2%)		 111
(40.22%)		 13
(4.71%)		 0
(0%)		 0

Solver NameProgress
AbsconMax 112
pc-d
done
276
MOPT
30		MSAT
168		?
76		Inc. Answer
2
AbsconMax 112
pc-w
done
276
MOPT
32		MSAT
166		?
76		Inc. Answer
2
Concrete + CSP4J - MCRW Engine
2008-05-30
done
276
MSAT
270		?
6
Concrete + CSP4J - Tabu Engine
2008-05-30
done
276
MSAT
269		?
7
Sugar
v1.13+minisat
done
276
MOPT
101		MSAT
71		?
104
Sugar++
v1.13+minisat-inc
done
276
MOPT
99		MSAT
53		?
111		Inc. Answer
13

Category "global constraints" (GLOBAL)

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Sugar
v1.13+minisat		 9865
(66.33%)		 65
(66.33%)		 0
(0%)		 21
(21.43%)		 12
(12.24%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 9850
(51.02%)		 50
(51.02%)		 0
(0%)		 21
(21.43%)		 27
(27.55%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Sugar
v1.13+minisat
done
98
MOPT
65		MSAT
21		?
12
Sugar++
v1.13+minisat-inc
done
98
MOPT
50		MSAT
21		?
27

Category "n-ary constraints in extension" (N-ARY-EXT)

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
AbsconMax 112
pc-d		 27871
(25.54%)		 71
(25.54%)		 0
(0%)		 205
(73.74%)		 2
(0.72%)		 0
(0%)		 0
(0%)		 0
AbsconMax 112
pc-w		 278103
(37.05%)		 103
(37.05%)		 0
(0%)		 172
(61.87%)		 3
(1.08%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - MCRW Engine
2008-05-30		 2784
(1.44%)		 4
(1.44%)		 0
(0%)		 227
(81.65%)		 47
(16.91%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 2784
(1.44%)		 4
(1.44%)		 0
(0%)		 227
(81.65%)		 47
(16.91%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 278118
(42.45%)		 118
(42.45%)		 0
(0%)		 72
(25.9%)		 88
(31.65%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 278118
(42.45%)		 118
(42.45%)		 0
(0%)		 73
(26.26%)		 87
(31.29%)		 0
(0%)		 0
(0%)		 0
toulbar2
2008-06-27		 278147
(52.88%)		 147
(52.88%)		 0
(0%)		 0
(0%)		 130
(46.76%)		 0
(0%)		 1
(0.36%)		 0
toulbar2/BTD
2008-06-27		 278153
(55.04%)		 153
(55.04%)		 0
(0%)		 0
(0%)		 125
(44.96%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
AbsconMax 112
pc-d
done
278
MOPT
71		MSAT
205		?
2
AbsconMax 112
pc-w
done
278
MOPT
103		MSAT
172		?
3
Concrete + CSP4J - MCRW Engine
2008-05-30
done
278
MOPT
4		MSAT
227		?
47
Concrete + CSP4J - Tabu Engine
2008-05-30
done
278
MOPT
4		MSAT
227		?
47
Sugar
v1.13+minisat
done
278
MOPT
118		MSAT
72		?
88
Sugar++
v1.13+minisat-inc
done
278
MOPT
118		MSAT
73		?
87
toulbar2
2008-06-27
done
278
MOPT
147		?
130		ERR
1
toulbar2/BTD
2008-06-27
done
278
MOPT
153		?
125

Category "n-ary constraints in intension" (N-ARY-INT)

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
AbsconMax 112
pc-d		 10910
(9.17%)		 10
(9.17%)		 0
(0%)		 70
(64.22%)		 29
(26.61%)		 0
(0%)		 0
(0%)		 0
AbsconMax 112
pc-w		 10914
(12.84%)		 14
(12.84%)		 0
(0%)		 64
(58.72%)		 31
(28.44%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - MCRW Engine
2008-05-30		 1090
(0%)		 0
(0%)		 0
(0%)		 84
(77.06%)		 25
(22.94%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 1090
(0%)		 0
(0%)		 0
(0%)		 82
(75.23%)		 27
(24.77%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 10939
(35.78%)		 39
(35.78%)		 0
(0%)		 29
(26.61%)		 41
(37.61%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 10939
(35.78%)		 39
(35.78%)		 0
(0%)		 29
(26.61%)		 40
(36.7%)		 1
(0.92%)		 0
(0%)		 0

Solver NameProgress
AbsconMax 112
pc-d
done
109
MOPT
10		MSAT
70		?
29
AbsconMax 112
pc-w
done
109
MOPT
14		MSAT
64		?
31
Concrete + CSP4J - MCRW Engine
2008-05-30
done
109
MSAT
84		?
25
Concrete + CSP4J - Tabu Engine
2008-05-30
done
109
MSAT
82		?
27
Sugar
v1.13+minisat
done
109
MOPT
39		MSAT
29		?
41
Sugar++
v1.13+minisat-inc
done
109
MOPT
39		MSAT
29		?
40		Inc. Answer
1

All categories for solvers registered in all categories

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Sugar
v1.13+minisat		 1295563
(43.47%)		 563
(43.47%)		 0
(0%)		 409
(31.58%)		 323
(24.94%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 1295535
(41.31%)		 535
(41.31%)		 0
(0%)		 391
(30.19%)		 353
(27.26%)		 16
(1.24%)		 0
(0%)		 0

Solver NameProgress
Sugar
v1.13+minisat
done
1295
MOPT
563		MSAT
409		?
323
Sugar++
v1.13+minisat-inc
done
1295
MOPT
535		MSAT
391		?
353		Inc. Answer
16

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedMOPTSATMSATUNKNOWNIncomplete AnswersBad AnswersMisc.
AbsconMax 112
pc-d		 1197197
(16.46%)		 197
(16.46%)		 0
(0%)		 891
(74.44%)		 107
(8.94%)		 2
(0.17%)		 0
(0%)		 0
AbsconMax 112
pc-w		 1197270
(22.56%)		 270
(22.56%)		 0
(0%)		 815
(68.09%)		 110
(9.19%)		 2
(0.17%)		 0
(0%)		 0
Concrete + CSP4J - MCRW Engine
2008-05-30		 119736
(3.01%)		 36
(3.01%)		 0
(0%)		 1078
(90.06%)		 83
(6.93%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 119736
(3.01%)		 36
(3.01%)		 0
(0%)		 1075
(89.81%)		 86
(7.18%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 1295563
(43.47%)		 563
(43.47%)		 0
(0%)		 409
(31.58%)		 323
(24.94%)		 0
(0%)		 0
(0%)		 0
Sugar++
v1.13+minisat-inc		 1295535
(41.31%)		 535
(41.31%)		 0
(0%)		 391
(30.19%)		 353
(27.26%)		 16
(1.24%)		 0
(0%)		 0
toulbar2
2008-06-27		 812559
(68.84%)		 559
(68.84%)		 0
(0%)		 0
(0%)		 252
(31.03%)		 0
(0%)		 1
(0.12%)		 0
toulbar2/BTD
2008-06-27		 812564
(69.46%)		 564
(69.46%)		 0
(0%)		 0
(0%)		 246
(30.3%)		 0
(0%)		 2
(0.25%)		 0

Solver NameProgress
AbsconMax 112
pc-d
done
1197
MOPT
197		MSAT
891		?
107		Inc. Answer
2
AbsconMax 112
pc-w
done
1197
MOPT
270		MSAT
815		?
110		Inc. Answer
2
Concrete + CSP4J - MCRW Engine
2008-05-30
done
1197
MOPT
36		MSAT
1078		?
83
Concrete + CSP4J - Tabu Engine
2008-05-30
done
1197
MOPT
36		MSAT
1075		?
86
Sugar
v1.13+minisat
done
1295
MOPT
563		MSAT
409		?
323
Sugar++
v1.13+minisat-inc
done
1295
MOPT
535		MSAT
391		?
353		Inc. Answer
16
toulbar2
2008-06-27
done
812
MOPT
559		?
252		ERR
1
toulbar2/BTD
2008-06-27
done
812
MOPT
564		?
246		ERR
2