CSP 2009 Competition: solvers page (tabular format, summary)

Some explanations

Category "binary constraints in extension" (2-ARY-EXT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 635551
(86.77%)		 237
(37.32%)		 314
(49.45%)		 84
(13.23%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 635547
(86.14%)		 237
(37.32%)		 310
(48.82%)		 88
(13.86%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 635416
(65.51%)		 163
(25.67%)		 253
(39.84%)		 219
(34.49%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 635547
(86.14%)		 238
(37.48%)		 309
(48.66%)		 88
(13.86%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 635556
(87.56%)		 242
(38.11%)		 314
(49.45%)		 79
(12.44%)		 0
(0%)		 0
(0%)
Concrete
2009-07-14
(complete)		 635503
(79.21%)		 206
(32.44%)		 297
(46.77%)		 132
(20.79%)		 0
(0%)		 0
(0%)
Concrete DC
2009-07-14
(complete)		 635504
(79.37%)		 207
(32.6%)		 297
(46.77%)		 131
(20.63%)		 0
(0%)		 0
(0%)
Conquer
2009-07-10
(complete)		 635423
(66.61%)		 165
(25.98%)		 258
(40.63%)		 206
(32.44%)		 0
(0%)		 6
(0.94%)
Mistral
1.545
(complete)		 635570
(89.76%)		 244
(38.43%)		 326
(51.34%)		 65
(10.24%)		 0
(0%)		 0
(0%)
pcs
0.3.2
(complete)		 635393
(61.89%)		 155
(24.41%)		 238
(37.48%)		 242
(38.11%)		 0
(0%)		 0
(0%)
pcs-restart
0.3.2
(complete)		 635394
(62.05%)		 153
(24.09%)		 241
(37.95%)		 241
(37.95%)		 0
(0%)		 0
(0%)
SAT4J CSP
2.1.1
(complete)		 635421
(66.3%)		 162
(25.51%)		 259
(40.79%)		 214
(33.7%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 635466
(73.39%)		 192
(30.24%)		 274
(43.15%)		 169
(26.61%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 635438
(68.98%)		 168
(26.46%)		 270
(42.52%)		 197
(31.02%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
635
UNSAT
237		SAT
314		?
84
Abscon 112v4
ESAC
(complete)
done
635
UNSAT
237		SAT
310		?
88
bpsolver
09
(complete)
done
635
UNSAT
163		SAT
253		?
219
Choco2.1.1
2009-06-10
(complete)
done
635
UNSAT
238		SAT
309		?
88
Choco2.1.1b
2009-07-16
(complete)
done
635
UNSAT
242		SAT
314		?
79
Concrete
2009-07-14
(complete)
done
635
UNSAT
206		SAT
297		?
132
Concrete DC
2009-07-14
(complete)
done
635
UNSAT
207		SAT
297		?
131
Conquer
2009-07-10
(complete)
done
635
UNSAT
165		SAT
258		?
206		ERR
6
Mistral
1.545
(complete)
done
635
UNSAT
244		SAT
326		?
65
pcs
0.3.2
(complete)
done
635
UNSAT
155		SAT
238		?
242
pcs-restart
0.3.2
(complete)
done
635
UNSAT
153		SAT
241		?
241
SAT4J CSP
2.1.1
(complete)
done
635
UNSAT
162		SAT
259		?
214
Sugar
v1.14.6+minisat
(complete)
done
635
UNSAT
192		SAT
274		?
169
Sugar
v1.14.6+picosat
(complete)
done
635
UNSAT
168		SAT
270		?
197

Category "binary constraints in intension" (2-ARY-INT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 686513
(74.78%)		 255
(37.17%)		 258
(37.61%)		 173
(25.22%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 686517
(75.36%)		 258
(37.61%)		 259
(37.76%)		 169
(24.64%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 686349
(50.87%)		 171
(24.93%)		 178
(25.95%)		 337
(49.13%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 686508
(74.05%)		 272
(39.65%)		 236
(34.4%)		 178
(25.95%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 686510
(74.34%)		 270
(39.36%)		 240
(34.99%)		 176
(25.66%)		 0
(0%)		 0
(0%)
Concrete
2009-07-14
(complete)		 686428
(62.39%)		 191
(27.84%)		 237
(34.55%)		 258
(37.61%)		 0
(0%)		 0
(0%)
Concrete DC
2009-07-14
(complete)		 686372
(54.23%)		 193
(28.13%)		 179
(26.09%)		 314
(45.77%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 686511
(74.49%)		 269
(39.21%)		 242
(35.28%)		 175
(25.51%)		 0
(0%)		 0
(0%)
pcs
0.3.2
(complete)		 686419
(61.08%)		 209
(30.47%)		 210
(30.61%)		 267
(38.92%)		 0
(0%)		 0
(0%)
pcs-restart
0.3.2
(complete)		 686419
(61.08%)		 208
(30.32%)		 211
(30.76%)		 267
(38.92%)		 0
(0%)		 0
(0%)
SAT4J CSP
2.1.1
(complete)		 686306
(44.61%)		 129
(18.8%)		 177
(25.8%)		 380
(55.39%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 686470
(68.51%)		 216
(31.49%)		 254
(37.03%)		 216
(31.49%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 686479
(69.83%)		 216
(31.49%)		 263
(38.34%)		 207
(30.17%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
686
UNSAT
255		SAT
258		?
173
Abscon 112v4
ESAC
(complete)
done
686
UNSAT
258		SAT
259		?
169
bpsolver
09
(complete)
done
686
UNSAT
171		SAT
178		?
337
Choco2.1.1
2009-06-10
(complete)
done
686
UNSAT
272		SAT
236		?
178
Choco2.1.1b
2009-07-16
(complete)
done
686
UNSAT
270		SAT
240		?
176
Concrete
2009-07-14
(complete)
done
686
UNSAT
191		SAT
237		?
258
Concrete DC
2009-07-14
(complete)
done
686
UNSAT
193		SAT
179		?
314
Mistral
1.545
(complete)
done
686
UNSAT
269		SAT
242		?
175
pcs
0.3.2
(complete)
done
686
UNSAT
209		SAT
210		?
267
pcs-restart
0.3.2
(complete)
done
686
UNSAT
208		SAT
211		?
267
SAT4J CSP
2.1.1
(complete)
done
686
UNSAT
129		SAT
177		?
380
Sugar
v1.14.6+minisat
(complete)
done
686
UNSAT
216		SAT
254		?
216
Sugar
v1.14.6+picosat
(complete)
done
686
UNSAT
216		SAT
263		?
207

Category "alldiff" (Alldiff)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 11877
(65.25%)		 24
(20.34%)		 53
(44.92%)		 41
(34.75%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 11878
(66.1%)		 24
(20.34%)		 54
(45.76%)		 40
(33.9%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 11860
(50.85%)		 23
(19.49%)		 37
(31.36%)		 58
(49.15%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 11872
(61.02%)		 23
(19.49%)		 49
(41.53%)		 46
(38.98%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 11871
(60.17%)		 23
(19.49%)		 48
(40.68%)		 47
(39.83%)		 0
(0%)		 0
(0%)
Concrete
2009-07-14
(complete)		 11873
(61.86%)		 23
(19.49%)		 50
(42.37%)		 45
(38.14%)		 0
(0%)		 0
(0%)
Concrete DC
2009-07-14
(complete)		 11868
(57.63%)		 23
(19.49%)		 45
(38.14%)		 50
(42.37%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 11898
(83.05%)		 23
(19.49%)		 75
(63.56%)		 20
(16.95%)		 0
(0%)		 0
(0%)
SAT4J CSP
2.1.1
(complete)		 11865
(55.08%)		 12
(10.17%)		 53
(44.92%)		 53
(44.92%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 11888
(74.58%)		 27
(22.88%)		 61
(51.69%)		 30
(25.42%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 118104
(88.14%)		 27
(22.88%)		 77
(65.25%)		 14
(11.86%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
118
UNSAT
24		SAT
53		?
41
Abscon 112v4
ESAC
(complete)
done
118
UNSAT
24		SAT
54		?
40
bpsolver
09
(complete)
done
118
UNSAT
23		SAT
37		?
58
Choco2.1.1
2009-06-10
(complete)
done
118
UNSAT
23		SAT
49		?
46
Choco2.1.1b
2009-07-16
(complete)
done
118
UNSAT
23		SAT
48		?
47
Concrete
2009-07-14
(complete)
done
118
UNSAT
23		SAT
50		?
45
Concrete DC
2009-07-14
(complete)
done
118
UNSAT
23		SAT
45		?
50
Mistral
1.545
(complete)
done
118
UNSAT
23		SAT
75		?
20
SAT4J CSP
2.1.1
(complete)
done
118
UNSAT
12		SAT
53		?
53
Sugar
v1.14.6+minisat
(complete)
done
118
UNSAT
27		SAT
61		?
30
Sugar
v1.14.6+picosat
(complete)
done
118
UNSAT
27		SAT
77		?
14

Category "alldiff,cumulative,element,weightedsum" (Alldiff+Cumul+Elt+WSum)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
bpsolver
09
(complete)		 162111
(68.52%)		 53
(32.72%)		 58
(35.8%)		 51
(31.48%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 162125
(77.16%)		 66
(40.74%)		 59
(36.42%)		 37
(22.84%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 162122
(75.31%)		 65
(40.12%)		 57
(35.19%)		 40
(24.69%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 162120
(74.07%)		 58
(35.8%)		 62
(38.27%)		 42
(25.93%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 162136
(83.95%)		 62
(38.27%)		 74
(45.68%)		 26
(16.05%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 162135
(83.33%)		 62
(38.27%)		 73
(45.06%)		 27
(16.67%)		 0
(0%)		 0
(0%)

Solver NameProgress
bpsolver
09
(complete)
done
162
UNSAT
53		SAT
58		?
51
Choco2.1.1
2009-06-10
(complete)
done
162
UNSAT
66		SAT
59		?
37
Choco2.1.1b
2009-07-16
(complete)
done
162
UNSAT
65		SAT
57		?
40
Mistral
1.545
(complete)
done
162
UNSAT
58		SAT
62		?
42
Sugar
v1.14.6+minisat
(complete)
done
162
UNSAT
62		SAT
74		?
26
Sugar
v1.14.6+picosat
(complete)
done
162
UNSAT
62		SAT
73		?
27

Category "alldiff,element,weightedsum" (Alldiff+Elt+WSum)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 276113
(40.94%)		 44
(15.94%)		 69
(25%)		 163
(59.06%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 27695
(34.42%)		 41
(14.86%)		 54
(19.57%)		 181
(65.58%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 276186
(67.39%)		 77
(27.9%)		 109
(39.49%)		 90
(32.61%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 276194
(70.29%)		 65
(23.55%)		 129
(46.74%)		 82
(29.71%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 276193
(69.93%)		 64
(23.19%)		 129
(46.74%)		 83
(30.07%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 276217
(78.62%)		 69
(25%)		 148
(53.62%)		 59
(21.38%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 276222
(80.43%)		 85
(30.8%)		 137
(49.64%)		 54
(19.57%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 276229
(82.97%)		 83
(30.07%)		 146
(52.9%)		 47
(17.03%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
276
UNSAT
44		SAT
69		?
163
Abscon 112v4
ESAC
(complete)
done
276
UNSAT
41		SAT
54		?
181
bpsolver
09
(complete)
done
276
UNSAT
77		SAT
109		?
90
Choco2.1.1
2009-06-10
(complete)
done
276
UNSAT
65		SAT
129		?
82
Choco2.1.1b
2009-07-16
(complete)
done
276
UNSAT
64		SAT
129		?
83
Mistral
1.545
(complete)
done
276
UNSAT
69		SAT
148		?
59
Sugar
v1.14.6+minisat
(complete)
done
276
UNSAT
85		SAT
137		?
54
Sugar
v1.14.6+picosat
(complete)
done
276
UNSAT
83		SAT
146		?
47

Category "n-ary constraints in extension" (N-ARY-EXT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 699545
(77.97%)		 295
(42.2%)		 250
(35.77%)		 154
(22.03%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 699543
(77.68%)		 294
(42.06%)		 249
(35.62%)		 156
(22.32%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 699394
(56.37%)		 198
(28.33%)		 196
(28.04%)		 305
(43.63%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 699532
(76.11%)		 265
(37.91%)		 267
(38.2%)		 167
(23.89%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 699528
(75.54%)		 266
(38.05%)		 262
(37.48%)		 171
(24.46%)		 0
(0%)		 0
(0%)
Concrete
2009-07-14
(complete)		 699544
(77.83%)		 277
(39.63%)		 267
(38.2%)		 155
(22.17%)		 0
(0%)		 0
(0%)
Concrete DC
2009-07-14
(complete)		 699532
(76.11%)		 267
(38.2%)		 265
(37.91%)		 167
(23.89%)		 0
(0%)		 0
(0%)
Conquer
2009-07-10
(complete)		 699532
(76.11%)		 271
(38.77%)		 261
(37.34%)		 167
(23.89%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 699585
(83.69%)		 292
(41.77%)		 293
(41.92%)		 114
(16.31%)		 0
(0%)		 0
(0%)
pcs
0.3.2
(complete)		 699497
(71.1%)		 272
(38.91%)		 225
(32.19%)		 202
(28.9%)		 0
(0%)		 0
(0%)
pcs-restart
0.3.2
(complete)		 699496
(70.96%)		 273
(39.06%)		 223
(31.9%)		 203
(29.04%)		 0
(0%)		 0
(0%)
SAT4J CSP
2.1.1
(complete)		 699209
(29.9%)		 90
(12.88%)		 119
(17.02%)		 490
(70.1%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 699374
(53.51%)		 190
(27.18%)		 184
(26.32%)		 325
(46.49%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 699350
(50.07%)		 167
(23.89%)		 183
(26.18%)		 349
(49.93%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
699
UNSAT
295		SAT
250		?
154
Abscon 112v4
ESAC
(complete)
done
699
UNSAT
294		SAT
249		?
156
bpsolver
09
(complete)
done
699
UNSAT
198		SAT
196		?
305
Choco2.1.1
2009-06-10
(complete)
done
699
UNSAT
265		SAT
267		?
167
Choco2.1.1b
2009-07-16
(complete)
done
699
UNSAT
266		SAT
262		?
171
Concrete
2009-07-14
(complete)
done
699
UNSAT
277		SAT
267		?
155
Concrete DC
2009-07-14
(complete)
done
699
UNSAT
267		SAT
265		?
167
Conquer
2009-07-10
(complete)
done
699
UNSAT
271		SAT
261		?
167
Mistral
1.545
(complete)
done
699
UNSAT
292		SAT
293		?
114
pcs
0.3.2
(complete)
done
699
UNSAT
272		SAT
225		?
202
pcs-restart
0.3.2
(complete)
done
699
UNSAT
273		SAT
223		?
203
SAT4J CSP
2.1.1
(complete)
done
699
UNSAT
90		SAT
119		?
490
Sugar
v1.14.6+minisat
(complete)
done
699
UNSAT
190		SAT
184		?
325
Sugar
v1.14.6+picosat
(complete)
done
699
UNSAT
167		SAT
183		?
349

Category "n-ary constraints in intension" (N-ARY-INT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 709481
(67.84%)		 63
(8.89%)		 418
(58.96%)		 228
(32.16%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 709489
(68.97%)		 63
(8.89%)		 426
(60.08%)		 220
(31.03%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 709513
(72.36%)		 71
(10.01%)		 442
(62.34%)		 196
(27.64%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 709560
(78.98%)		 78
(11%)		 482
(67.98%)		 149
(21.02%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 709548
(77.29%)		 79
(11.14%)		 469
(66.15%)		 161
(22.71%)		 0
(0%)		 0
(0%)
Concrete
2009-07-14
(complete)		 709439
(61.92%)		 60
(8.46%)		 379
(53.46%)		 270
(38.08%)		 0
(0%)		 0
(0%)
Concrete DC
2009-07-14
(complete)		 709342
(48.24%)		 52
(7.33%)		 290
(40.9%)		 367
(51.76%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 709572
(80.68%)		 78
(11%)		 494
(69.68%)		 137
(19.32%)		 0
(0%)		 0
(0%)
pcs
0.3.2
(complete)		 709542
(76.45%)		 85
(11.99%)		 457
(64.46%)		 167
(23.55%)		 0
(0%)		 0
(0%)
pcs-restart
0.3.2
(complete)		 709546
(77.01%)		 85
(11.99%)		 461
(65.02%)		 163
(22.99%)		 0
(0%)		 0
(0%)
SAT4J CSP
2.1.1
(complete)		 709171
(24.12%)		 38
(5.36%)		 133
(18.76%)		 538
(75.88%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 709481
(67.84%)		 84
(11.85%)		 397
(55.99%)		 228
(32.16%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 709478
(67.42%)		 82
(11.57%)		 396
(55.85%)		 231
(32.58%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
709
UNSAT
63		SAT
418		?
228
Abscon 112v4
ESAC
(complete)
done
709
UNSAT
63		SAT
426		?
220
bpsolver
09
(complete)
done
709
UNSAT
71		SAT
442		?
196
Choco2.1.1
2009-06-10
(complete)
done
709
UNSAT
78		SAT
482		?
149
Choco2.1.1b
2009-07-16
(complete)
done
709
UNSAT
79		SAT
469		?
161
Concrete
2009-07-14
(complete)
done
709
UNSAT
60		SAT
379		?
270
Concrete DC
2009-07-14
(complete)
done
709
UNSAT
52		SAT
290		?
367
Mistral
1.545
(complete)
done
709
UNSAT
78		SAT
494		?
137
pcs
0.3.2
(complete)
done
709
UNSAT
85		SAT
457		?
167
pcs-restart
0.3.2
(complete)
done
709
UNSAT
85		SAT
461		?
163
SAT4J CSP
2.1.1
(complete)
done
709
UNSAT
38		SAT
133		?
538
Sugar
v1.14.6+minisat
(complete)
done
709
UNSAT
84		SAT
397		?
228
Sugar
v1.14.6+picosat
(complete)
done
709
UNSAT
82		SAT
396		?
231

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
bpsolver
09
(complete)		 32852029
(61.77%)		 756
(23.01%)		 1273
(38.75%)		 1256
(38.23%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 32852538
(77.26%)		 1007
(30.65%)		 1531
(46.61%)		 747
(22.74%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 32852528
(76.96%)		 1009
(30.72%)		 1519
(46.24%)		 757
(23.04%)		 0
(0%)		 0
(0%)
Mistral
1.545
(complete)		 32852673
(81.37%)		 1033
(31.45%)		 1640
(49.92%)		 612
(18.63%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 32852237
(68.1%)		 856
(26.06%)		 1381
(42.04%)		 1048
(31.9%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 32852213
(67.37%)		 805
(24.51%)		 1408
(42.86%)		 1072
(32.63%)		 0
(0%)		 0
(0%)

Solver NameProgress
bpsolver
09
(complete)
done
3285
UNSAT
756		SAT
1273		?
1256
Choco2.1.1
2009-06-10
(complete)
done
3285
UNSAT
1007		SAT
1531		?
747
Choco2.1.1b
2009-07-16
(complete)
done
3285
UNSAT
1009		SAT
1519		?
757
Mistral
1.545
(complete)
done
3285
UNSAT
1033		SAT
1640		?
612
Sugar
v1.14.6+minisat
(complete)
done
3285
UNSAT
856		SAT
1381		?
1048
Sugar
v1.14.6+picosat
(complete)
done
3285
UNSAT
805		SAT
1408		?
1072

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad Answers
Abscon 112v4
AC
(complete)		 31232280
(73.01%)		 918
(29.39%)		 1362
(43.61%)		 843
(26.99%)		 0
(0%)		 0
(0%)
Abscon 112v4
ESAC
(complete)		 31232269
(72.65%)		 917
(29.36%)		 1352
(43.29%)		 854
(27.35%)		 0
(0%)		 0
(0%)
bpsolver
09
(complete)		 32852029
(61.77%)		 756
(23.01%)		 1273
(38.75%)		 1256
(38.23%)		 0
(0%)		 0
(0%)
Choco2.1.1
2009-06-10
(complete)		 32852538
(77.26%)		 1007
(30.65%)		 1531
(46.61%)		 747
(22.74%)		 0
(0%)		 0
(0%)
Choco2.1.1b
2009-07-16
(complete)		 32852528
(76.96%)		 1009
(30.72%)		 1519
(46.24%)		 757
(23.04%)		 0
(0%)		 0
(0%)
Concrete
2009-07-14
(complete)		 28471987
(69.79%)		 757
(26.59%)		 1230
(43.2%)		 860
(30.21%)		 0
(0%)		 0
(0%)
Concrete DC
2009-07-14
(complete)		 28471818
(63.86%)		 742
(26.06%)		 1076
(37.79%)		 1029
(36.14%)		 0
(0%)		 0
(0%)
Conquer
2009-07-10
(complete)		 1334955
(71.59%)		 436
(32.68%)		 519
(38.91%)		 373
(27.96%)		 0
(0%)		 6
(0.45%)
Mistral
1.545
(complete)		 32852673
(81.37%)		 1033
(31.45%)		 1640
(49.92%)		 612
(18.63%)		 0
(0%)		 0
(0%)
pcs
0.3.2
(complete)		 27291851
(67.83%)		 721
(26.42%)		 1130
(41.41%)		 878
(32.17%)		 0
(0%)		 0
(0%)
pcs-restart
0.3.2
(complete)		 27291855
(67.97%)		 719
(26.35%)		 1136
(41.63%)		 874
(32.03%)		 0
(0%)		 0
(0%)
SAT4J CSP
2.1.1
(complete)		 28471172
(41.17%)		 431
(15.14%)		 741
(26.03%)		 1675
(58.83%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+minisat
(complete)		 32852237
(68.1%)		 856
(26.06%)		 1381
(42.04%)		 1048
(31.9%)		 0
(0%)		 0
(0%)
Sugar
v1.14.6+picosat
(complete)		 32852213
(67.37%)		 805
(24.51%)		 1408
(42.86%)		 1072
(32.63%)		 0
(0%)		 0
(0%)

Solver NameProgress
Abscon 112v4
AC
(complete)
done
3123
UNSAT
918		SAT
1362		?
843
Abscon 112v4
ESAC
(complete)
done
3123
UNSAT
917		SAT
1352		?
854
bpsolver
09
(complete)
done
3285
UNSAT
756		SAT
1273		?
1256
Choco2.1.1
2009-06-10
(complete)
done
3285
UNSAT
1007		SAT
1531		?
747
Choco2.1.1b
2009-07-16
(complete)
done
3285
UNSAT
1009		SAT
1519		?
757
Concrete
2009-07-14
(complete)
done
2847
UNSAT
757		SAT
1230		?
860
Concrete DC
2009-07-14
(complete)
done
2847
UNSAT
742		SAT
1076		?
1029
Conquer
2009-07-10
(complete)
done
1334
UNSAT
436		SAT
519		?
373		ERR
6
Mistral
1.545
(complete)
done
3285
UNSAT
1033		SAT
1640		?
612
pcs
0.3.2
(complete)
done
2729
UNSAT
721		SAT
1130		?
878
pcs-restart
0.3.2
(complete)
done
2729
UNSAT
719		SAT
1136		?
874
SAT4J CSP
2.1.1
(complete)
done
2847
UNSAT
431		SAT
741		?
1675
Sugar
v1.14.6+minisat
(complete)
done
3285
UNSAT
856		SAT
1381		?
1048
Sugar
v1.14.6+picosat
(complete)
done
3285
UNSAT
805		SAT
1408		?
1072