CSP 2008 Competition: solvers page (tabular format, summary)

Some explanations

Category "binary constraints in extension" (2-ARY-EXT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 635546
(85.98%)		 229
(36.06%)		 317
(49.92%)		 89
(14.02%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 635546
(85.98%)		 229
(36.06%)		 317
(49.92%)		 89
(14.02%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 635495
(77.95%)		 214
(33.7%)		 281
(44.25%)		 140
(22.05%)		 0
(0%)		 0
(0%)		 0
casper
zao		 635415
(65.35%)		 160
(25.2%)		 255
(40.16%)		 220
(34.65%)		 0
(0%)		 0
(0%)		 0
casper
zito		 635433
(68.19%)		 169
(26.61%)		 264
(41.57%)		 202
(31.81%)		 0
(0%)		 0
(0%)		 0
choco2_dwdeg
2008-06-26		 635554
(87.24%)		 230
(36.22%)		 324
(51.02%)		 81
(12.76%)		 0
(0%)		 0
(0%)		 0
choco2_impwdeg
2008-06-26		 635550
(86.61%)		 230
(36.22%)		 320
(50.39%)		 85
(13.39%)		 0
(0%)		 0
(0%)		 0
Concrete + CPS4J
2008-05-30		 635498
(78.43%)		 201
(31.65%)		 297
(46.77%)		 137
(21.57%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 635348
(54.8%)		 0
(0%)		 348
(54.8%)		 287
(45.2%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - WMC Engine
2008-05-30		 635116
(18.27%)		 0
(0%)		 116
(18.27%)		 518
(81.57%)		 0
(0%)		 1
(0.16%)		 0
cpHydra
k_10		 635574
(90.39%)		 242
(38.11%)		 332
(52.28%)		 61
(9.61%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 635567
(89.29%)		 236
(37.17%)		 331
(52.13%)		 68
(10.71%)		 0
(0%)		 0
(0%)		 0
galac
E		 635425
(66.93%)		 163
(25.67%)		 262
(41.26%)		 210
(33.07%)		 0
(0%)		 0
(0%)		 0
galac
M		 635422
(66.46%)		 162
(25.51%)		 260
(40.94%)		 213
(33.54%)		 0
(0%)		 0
(0%)		 0
MDG-noprobe
2008-06-27		 635558
(87.87%)		 235
(37.01%)		 323
(50.87%)		 77
(12.13%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 635561
(88.35%)		 237
(37.32%)		 324
(51.02%)		 74
(11.65%)		 0
(0%)		 0
(0%)		 0
Minion/Tailor
2008-07-04		 635414
(65.2%)		 166
(26.14%)		 248
(39.06%)		 221
(34.8%)		 0
(0%)		 0
(0%)		 0
Mistral-option
1.314		 635559
(88.03%)		 235
(37.01%)		 324
(51.02%)		 76
(11.97%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 635556
(87.56%)		 235
(37.01%)		 321
(50.55%)		 79
(12.44%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 635426
(67.09%)		 158
(24.88%)		 268
(42.2%)		 209
(32.91%)		 0
(0%)		 0
(0%)		 0
spider
2008-06-14		 635472
(74.33%)		 186
(29.29%)		 286
(45.04%)		 163
(25.67%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 635470
(74.02%)		 192
(30.24%)		 278
(43.78%)		 165
(25.98%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 635443
(69.76%)		 163
(25.67%)		 280
(44.09%)		 192
(30.24%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
635
UNSAT
229		SAT
317		?
89
Abscon 112v4
ESAC
done
635
UNSAT
229		SAT
317		?
89
bpsolver
2008-06-27
done
635
UNSAT
214		SAT
281		?
140
casper
zao
done
635
UNSAT
160		SAT
255		?
220
casper
zito
done
635
UNSAT
169		SAT
264		?
202
choco2_dwdeg
2008-06-26
done
635
UNSAT
230		SAT
324		?
81
choco2_impwdeg
2008-06-26
done
635
UNSAT
230		SAT
320		?
85
Concrete + CPS4J
2008-05-30
done
635
UNSAT
201		SAT
297		?
137
Concrete + CSP4J - Tabu Engine
2008-05-30
done
635
SAT
348		?
287
Concrete + CSP4J - WMC Engine
2008-05-30
done
635
SAT
116		?
518		ERR
1
cpHydra
k_10
done
635
UNSAT
242		SAT
332		?
61
cpHydra
k_40
done
635
UNSAT
236		SAT
331		?
68
galac
E
done
635
UNSAT
163		SAT
262		?
210
galac
M
done
635
UNSAT
162		SAT
260		?
213
MDG-noprobe
2008-06-27
done
635
UNSAT
235		SAT
323		?
77
MDG-probe
2008-06-27
done
635
UNSAT
237		SAT
324		?
74
Minion/Tailor
2008-07-04
done
635
UNSAT
166		SAT
248		?
221
Mistral-option
1.314
done
635
UNSAT
235		SAT
324		?
76
Mistral-prime
1.313
done
635
UNSAT
235		SAT
321		?
79
SAT4J CSP
2008-06-13
done
635
UNSAT
158		SAT
268		?
209
spider
2008-06-14
done
635
UNSAT
186		SAT
286		?
163
Sugar
v1.13+minisat
done
635
UNSAT
192		SAT
278		?
165
Sugar
v1.13+picosat
done
635
UNSAT
163		SAT
280		?
192

Category "binary constraints in intension" (2-ARY-INT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 696513
(73.71%)		 265
(38.07%)		 248
(35.63%)		 183
(26.29%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 696520
(74.71%)		 273
(39.22%)		 247
(35.49%)		 176
(25.29%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 696375
(53.88%)		 205
(29.45%)		 170
(24.43%)		 321
(46.12%)		 0
(0%)		 0
(0%)		 0
casper
zao		 696332
(47.7%)		 157
(22.56%)		 175
(25.14%)		 360
(51.72%)		 0
(0%)		 4
(0.57%)		 0
casper
zito		 696325
(46.7%)		 151
(21.7%)		 174
(25%)		 367
(52.73%)		 0
(0%)		 4
(0.57%)		 0
choco2_dwdeg
2008-06-26		 696523
(75.14%)		 285
(40.95%)		 238
(34.2%)		 173
(24.86%)		 0
(0%)		 0
(0%)		 0
choco2_impwdeg
2008-06-26		 696521
(74.86%)		 286
(41.09%)		 235
(33.76%)		 175
(25.14%)		 0
(0%)		 0
(0%)		 0
Concrete + CPS4J
2008-05-30		 696415
(59.63%)		 190
(27.3%)		 225
(32.33%)		 281
(40.37%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 696183
(26.29%)		 35
(5.03%)		 148
(21.26%)		 513
(73.71%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - WMC Engine
2008-05-30		 696222
(31.9%)		 33
(4.74%)		 189
(27.16%)		 474
(68.1%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_10		 696596
(85.63%)		 312
(44.83%)		 284
(40.8%)		 100
(14.37%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 696597
(85.78%)		 314
(45.11%)		 283
(40.66%)		 99
(14.22%)		 0
(0%)		 0
(0%)		 0
galac
E		 696290
(41.67%)		 133
(19.11%)		 157
(22.56%)		 405
(58.19%)		 0
(0%)		 1
(0.14%)		 0
galac
M		 696307
(44.11%)		 142
(20.4%)		 165
(23.71%)		 388
(55.75%)		 0
(0%)		 1
(0.14%)		 0
MDG-noprobe
2008-06-27		 696499
(71.7%)		 274
(39.37%)		 225
(32.33%)		 197
(28.3%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 696515
(73.99%)		 280
(40.23%)		 235
(33.76%)		 181
(26.01%)		 0
(0%)		 0
(0%)		 0
Minion/Tailor
2008-07-04		 696305
(43.82%)		 168
(24.14%)		 137
(19.68%)		 364
(52.3%)		 0
(0%)		 27
(3.88%)		 0
Mistral-option
1.314		 696451
(64.8%)		 237
(34.05%)		 214
(30.75%)		 245
(35.2%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 696522
(75%)		 285
(40.95%)		 237
(34.05%)		 174
(25%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 696309
(44.4%)		 138
(19.83%)		 171
(24.57%)		 387
(55.6%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 696484
(69.54%)		 227
(32.61%)		 257
(36.93%)		 212
(30.46%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 696486
(69.83%)		 225
(32.33%)		 261
(37.5%)		 210
(30.17%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
696
UNSAT
265		SAT
248		?
183
Abscon 112v4
ESAC
done
696
UNSAT
273		SAT
247		?
176
bpsolver
2008-06-27
done
696
UNSAT
205		SAT
170		?
321
casper
zao
done
696
UNSAT
157		SAT
175		?
360		ERR
4
casper
zito
done
696
UNSAT
151		SAT
174		?
367		ERR
4
choco2_dwdeg
2008-06-26
done
696
UNSAT
285		SAT
238		?
173
choco2_impwdeg
2008-06-26
done
696
UNSAT
286		SAT
235		?
175
Concrete + CPS4J
2008-05-30
done
696
UNSAT
190		SAT
225		?
281
Concrete + CSP4J - Tabu Engine
2008-05-30
done
696
UNSAT
35		SAT
148		?
513
Concrete + CSP4J - WMC Engine
2008-05-30
done
696
UNSAT
33		SAT
189		?
474
cpHydra
k_10
done
696
UNSAT
312		SAT
284		?
100
cpHydra
k_40
done
696
UNSAT
314		SAT
283		?
99
galac
E
done
696
UNSAT
133		SAT
157		?
405		ERR
1
galac
M
done
696
UNSAT
142		SAT
165		?
388		ERR
1
MDG-noprobe
2008-06-27
done
696
UNSAT
274		SAT
225		?
197
MDG-probe
2008-06-27
done
696
UNSAT
280		SAT
235		?
181
Minion/Tailor
2008-07-04
done
696
UNSAT
168		SAT
137		?
364		ERR
27
Mistral-option
1.314
done
696
UNSAT
237		SAT
214		?
245
Mistral-prime
1.313
done
696
UNSAT
285		SAT
237		?
174
SAT4J CSP
2008-06-13
done
696
UNSAT
138		SAT
171		?
387
Sugar
v1.13+minisat
done
696
UNSAT
227		SAT
257		?
212
Sugar
v1.13+picosat
done
696
UNSAT
225		SAT
261		?
210

Category "global constraints" (GLOBAL)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 556184
(33.09%)		 64
(11.51%)		 120
(21.58%)		 372
(66.91%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 556173
(31.12%)		 63
(11.33%)		 110
(19.78%)		 383
(68.88%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 556347
(62.41%)		 151
(27.16%)		 196
(35.25%)		 209
(37.59%)		 0
(0%)		 0
(0%)		 0
casper
zao		 556390
(70.14%)		 172
(30.94%)		 218
(39.21%)		 166
(29.86%)		 0
(0%)		 0
(0%)		 0
casper
zito		 556397
(71.4%)		 173
(31.12%)		 224
(40.29%)		 159
(28.6%)		 0
(0%)		 0
(0%)		 0
choco2_dwdeg
2008-06-26		 556358
(64.39%)		 145
(26.08%)		 213
(38.31%)		 198
(35.61%)		 0
(0%)		 0
(0%)		 0
choco2_impwdeg
2008-06-26		 556347
(62.41%)		 144
(25.9%)		 203
(36.51%)		 209
(37.59%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_10		 556419
(75.36%)		 149
(26.8%)		 270
(48.56%)		 137
(24.64%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 556420
(75.54%)		 149
(26.8%)		 271
(48.74%)		 136
(24.46%)		 0
(0%)		 0
(0%)		 0
MDG-noprobe
2008-06-27		 556353
(63.49%)		 139
(25%)		 214
(38.49%)		 203
(36.51%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 556337
(60.61%)		 138
(24.82%)		 199
(35.79%)		 219
(39.39%)		 0
(0%)		 0
(0%)		 0
Minion/Tailor
2008-07-04		 556216
(38.85%)		 86
(15.47%)		 130
(23.38%)		 340
(61.15%)		 0
(0%)		 0
(0%)		 0
Mistral-option
1.314		 556383
(68.88%)		 135
(24.28%)		 248
(44.6%)		 173
(31.12%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 556403
(72.48%)		 136
(24.46%)		 267
(48.02%)		 153
(27.52%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 55663
(11.33%)		 11
(1.98%)		 52
(9.35%)		 493
(88.67%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 556405
(72.84%)		 153
(27.52%)		 252
(45.32%)		 151
(27.16%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 556424
(76.26%)		 151
(27.16%)		 273
(49.1%)		 132
(23.74%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
556
UNSAT
64		SAT
120		?
372
Abscon 112v4
ESAC
done
556
UNSAT
63		SAT
110		?
383
bpsolver
2008-06-27
done
556
UNSAT
151		SAT
196		?
209
casper
zao
done
556
UNSAT
172		SAT
218		?
166
casper
zito
done
556
UNSAT
173		SAT
224		?
159
choco2_dwdeg
2008-06-26
done
556
UNSAT
145		SAT
213		?
198
choco2_impwdeg
2008-06-26
done
556
UNSAT
144		SAT
203		?
209
cpHydra
k_10
done
556
UNSAT
149		SAT
270		?
137
cpHydra
k_40
done
556
UNSAT
149		SAT
271		?
136
MDG-noprobe
2008-06-27
done
556
UNSAT
139		SAT
214		?
203
MDG-probe
2008-06-27
done
556
UNSAT
138		SAT
199		?
219
Minion/Tailor
2008-07-04
done
556
UNSAT
86		SAT
130		?
340
Mistral-option
1.314
done
556
UNSAT
135		SAT
248		?
173
Mistral-prime
1.313
done
556
UNSAT
136		SAT
267		?
153
SAT4J CSP
2008-06-13
done
556
UNSAT
11		SAT
52		?
493
Sugar
v1.13+minisat
done
556
UNSAT
153		SAT
252		?
151
Sugar
v1.13+picosat
done
556
UNSAT
151		SAT
273		?
132

Category "n-ary constraints in extension" (N-ARY-EXT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 704545
(77.41%)		 295
(41.9%)		 250
(35.51%)		 159
(22.59%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 704541
(76.85%)		 293
(41.62%)		 248
(35.23%)		 163
(23.15%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 704322
(45.74%)		 131
(18.61%)		 191
(27.13%)		 382
(54.26%)		 0
(0%)		 0
(0%)		 0
casper
zao		 704386
(54.83%)		 204
(28.98%)		 182
(25.85%)		 318
(45.17%)		 0
(0%)		 0
(0%)		 0
casper
zito		 704391
(55.54%)		 208
(29.55%)		 183
(25.99%)		 313
(44.46%)		 0
(0%)		 0
(0%)		 0
choco2_dwdeg
2008-06-26		 704477
(67.76%)		 247
(35.09%)		 230
(32.67%)		 227
(32.24%)		 0
(0%)		 0
(0%)		 0
choco2_impwdeg
2008-06-26		 704441
(62.64%)		 231
(32.81%)		 210
(29.83%)		 263
(37.36%)		 0
(0%)		 0
(0%)		 0
Concrete + CPS4J
2008-05-30		 704544
(77.27%)		 277
(39.35%)		 267
(37.93%)		 160
(22.73%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 704210
(29.83%)		 30
(4.26%)		 180
(25.57%)		 494
(70.17%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - WMC Engine
2008-05-30		 704135
(19.18%)		 30
(4.26%)		 105
(14.91%)		 569
(80.82%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_10		 704586
(83.24%)		 303
(43.04%)		 283
(40.2%)		 118
(16.76%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 704587
(83.38%)		 301
(42.76%)		 286
(40.63%)		 117
(16.62%)		 0
(0%)		 0
(0%)		 0
galac
E		 704323
(45.88%)		 196
(27.84%)		 127
(18.04%)		 381
(54.12%)		 0
(0%)		 0
(0%)		 0
galac
M		 704326
(46.31%)		 196
(27.84%)		 130
(18.47%)		 378
(53.69%)		 0
(0%)		 0
(0%)		 0
mddc-solv
2008-06-09		 704576
(81.82%)		 290
(41.19%)		 286
(40.63%)		 128
(18.18%)		 0
(0%)		 0
(0%)		 0
MDG-noprobe
2008-06-27		 704570
(80.97%)		 283
(40.2%)		 287
(40.77%)		 134
(19.03%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 704564
(80.11%)		 281
(39.91%)		 283
(40.2%)		 139
(19.74%)		 0
(0%)		 1
(0.14%)		 0
Minion/Tailor
2008-07-04		 704406
(57.67%)		 213
(30.26%)		 193
(27.41%)		 298
(42.33%)		 0
(0%)		 0
(0%)		 0
Mistral-option
1.314		 704565
(80.26%)		 282
(40.06%)		 283
(40.2%)		 139
(19.74%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 704569
(80.82%)		 283
(40.2%)		 286
(40.63%)		 135
(19.18%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 704218
(30.97%)		 93
(13.21%)		 125
(17.76%)		 486
(69.03%)		 0
(0%)		 0
(0%)		 0
spider
2008-06-14		 704421
(59.8%)		 184
(26.14%)		 237
(33.66%)		 282
(40.06%)		 0
(0%)		 1
(0.14%)		 0
Sugar
v1.13+minisat		 704370
(52.56%)		 191
(27.13%)		 179
(25.43%)		 334
(47.44%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 704347
(49.29%)		 169
(24.01%)		 178
(25.28%)		 357
(50.71%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
704
UNSAT
295		SAT
250		?
159
Abscon 112v4
ESAC
done
704
UNSAT
293		SAT
248		?
163
bpsolver
2008-06-27
done
704
UNSAT
131		SAT
191		?
382
casper
zao
done
704
UNSAT
204		SAT
182		?
318
casper
zito
done
704
UNSAT
208		SAT
183		?
313
choco2_dwdeg
2008-06-26
done
704
UNSAT
247		SAT
230		?
227
choco2_impwdeg
2008-06-26
done
704
UNSAT
231		SAT
210		?
263
Concrete + CPS4J
2008-05-30
done
704
UNSAT
277		SAT
267		?
160
Concrete + CSP4J - Tabu Engine
2008-05-30
done
704
UNSAT
30		SAT
180		?
494
Concrete + CSP4J - WMC Engine
2008-05-30
done
704
UNSAT
30		SAT
105		?
569
cpHydra
k_10
done
704
UNSAT
303		SAT
283		?
118
cpHydra
k_40
done
704
UNSAT
301		SAT
286		?
117
galac
E
done
704
UNSAT
196		SAT
127		?
381
galac
M
done
704
UNSAT
196		SAT
130		?
378
mddc-solv
2008-06-09
done
704
UNSAT
290		SAT
286		?
128
MDG-noprobe
2008-06-27
done
704
UNSAT
283		SAT
287		?
134
MDG-probe
2008-06-27
done
704
UNSAT
281		SAT
283		?
139		ERR
1
Minion/Tailor
2008-07-04
done
704
UNSAT
213		SAT
193		?
298
Mistral-option
1.314
done
704
UNSAT
282		SAT
283		?
139
Mistral-prime
1.313
done
704
UNSAT
283		SAT
286		?
135
SAT4J CSP
2008-06-13
done
704
UNSAT
93		SAT
125		?
486
spider
2008-06-14
done
704
UNSAT
184		SAT
237		?
282		ERR
1
Sugar
v1.13+minisat
done
704
UNSAT
191		SAT
179		?
334
Sugar
v1.13+picosat
done
704
UNSAT
169		SAT
178		?
357

Category "n-ary constraints in intension" (N-ARY-INT)

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 716486
(67.88%)		 67
(9.36%)		 419
(58.52%)		 230
(32.12%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 716488
(68.16%)		 65
(9.08%)		 423
(59.08%)		 228
(31.84%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 716459
(64.11%)		 74
(10.34%)		 385
(53.77%)		 257
(35.89%)		 0
(0%)		 0
(0%)		 0
casper
zao		 716562
(78.49%)		 99
(13.83%)		 463
(64.66%)		 154
(21.51%)		 0
(0%)		 0
(0%)		 0
casper
zito		 716555
(77.51%)		 91
(12.71%)		 464
(64.8%)		 161
(22.49%)		 0
(0%)		 0
(0%)		 0
choco2_dwdeg
2008-06-26		 716543
(75.84%)		 69
(9.64%)		 474
(66.2%)		 172
(24.02%)		 0
(0%)		 1
(0.14%)		 0
choco2_impwdeg
2008-06-26		 716518
(72.35%)		 66
(9.22%)		 452
(63.13%)		 198
(27.65%)		 0
(0%)		 0
(0%)		 0
Concrete + CPS4J
2008-05-30		 716456
(63.69%)		 58
(8.1%)		 398
(55.59%)		 260
(36.31%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 716199
(27.79%)		 21
(2.93%)		 178
(24.86%)		 517
(72.21%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - WMC Engine
2008-05-30		 716174
(24.3%)		 21
(2.93%)		 153
(21.37%)		 542
(75.7%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_10		 716569
(79.47%)		 88
(12.29%)		 481
(67.18%)		 147
(20.53%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 716569
(79.47%)		 86
(12.01%)		 483
(67.46%)		 147
(20.53%)		 0
(0%)		 0
(0%)		 0
galac
E		 716189
(26.4%)		 37
(5.17%)		 152
(21.23%)		 488
(68.16%)		 0
(0%)		 39
(5.45%)		 0
galac
M		 716183
(25.56%)		 37
(5.17%)		 146
(20.39%)		 491
(68.58%)		 0
(0%)		 42
(5.87%)		 0
MDG-noprobe
2008-06-27		 716530
(74.02%)		 86
(12.01%)		 444
(62.01%)		 186
(25.98%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 716560
(78.21%)		 91
(12.71%)		 469
(65.5%)		 156
(21.79%)		 0
(0%)		 0
(0%)		 0
Minion/Tailor
2008-07-04		 716403
(56.28%)		 51
(7.12%)		 352
(49.16%)		 310
(43.3%)		 0
(0%)		 3
(0.42%)		 0
Mistral-option
1.314		 716554
(77.37%)		 83
(11.59%)		 471
(65.78%)		 162
(22.63%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 716560
(78.21%)		 88
(12.29%)		 472
(65.92%)		 156
(21.79%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 716174
(24.3%)		 40
(5.59%)		 134
(18.72%)		 542
(75.7%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 716486
(67.88%)		 87
(12.15%)		 399
(55.73%)		 230
(32.12%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 716481
(67.18%)		 88
(12.29%)		 393
(54.89%)		 235
(32.82%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
716
UNSAT
67		SAT
419		?
230
Abscon 112v4
ESAC
done
716
UNSAT
65		SAT
423		?
228
bpsolver
2008-06-27
done
716
UNSAT
74		SAT
385		?
257
casper
zao
done
716
UNSAT
99		SAT
463		?
154
casper
zito
done
716
UNSAT
91		SAT
464		?
161
choco2_dwdeg
2008-06-26
done
716
UNSAT
69		SAT
474		?
172		ERR
1
choco2_impwdeg
2008-06-26
done
716
UNSAT
66		SAT
452		?
198
Concrete + CPS4J
2008-05-30
done
716
UNSAT
58		SAT
398		?
260
Concrete + CSP4J - Tabu Engine
2008-05-30
done
716
UNSAT
21		SAT
178		?
517
Concrete + CSP4J - WMC Engine
2008-05-30
done
716
UNSAT
21		SAT
153		?
542
cpHydra
k_10
done
716
UNSAT
88		SAT
481		?
147
cpHydra
k_40
done
716
UNSAT
86		SAT
483		?
147
galac
E
done
716
UNSAT
37		SAT
152		?
488		ERR
39
galac
M
done
716
UNSAT
37		SAT
146		?
491		ERR
42
MDG-noprobe
2008-06-27
done
716
UNSAT
86		SAT
444		?
186
MDG-probe
2008-06-27
done
716
UNSAT
91		SAT
469		?
156
Minion/Tailor
2008-07-04
done
716
UNSAT
51		SAT
352		?
310		ERR
3
Mistral-option
1.314
done
716
UNSAT
83		SAT
471		?
162
Mistral-prime
1.313
done
716
UNSAT
88		SAT
472		?
156
SAT4J CSP
2008-06-13
done
716
UNSAT
40		SAT
134		?
542
Sugar
v1.13+minisat
done
716
UNSAT
87		SAT
399		?
230
Sugar
v1.13+picosat
done
716
UNSAT
88		SAT
393		?
235

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 33072274
(68.76%)		 920
(27.82%)		 1354
(40.94%)		 1033
(31.24%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 33072268
(68.58%)		 923
(27.91%)		 1345
(40.67%)		 1039
(31.42%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 33071998
(60.42%)		 775
(23.44%)		 1223
(36.98%)		 1309
(39.58%)		 0
(0%)		 0
(0%)		 0
casper
zao		 33072085
(63.05%)		 792
(23.95%)		 1293
(39.1%)		 1218
(36.83%)		 0
(0%)		 4
(0.12%)		 0
casper
zito		 33072101
(63.53%)		 792
(23.95%)		 1309
(39.58%)		 1202
(36.35%)		 0
(0%)		 4
(0.12%)		 0
choco2_dwdeg
2008-06-26		 33072455
(74.24%)		 976
(29.51%)		 1479
(44.72%)		 851
(25.73%)		 0
(0%)		 1
(0.03%)		 0
choco2_impwdeg
2008-06-26		 33072377
(71.88%)		 957
(28.94%)		 1420
(42.94%)		 930
(28.12%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_10		 33072744
(82.98%)		 1094
(33.08%)		 1650
(49.89%)		 563
(17.02%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 33072740
(82.85%)		 1086
(32.84%)		 1654
(50.02%)		 567
(17.15%)		 0
(0%)		 0
(0%)		 0
MDG-noprobe
2008-06-27		 33072510
(75.9%)		 1017
(30.75%)		 1493
(45.15%)		 797
(24.1%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 33072537
(76.72%)		 1027
(31.06%)		 1510
(45.66%)		 769
(23.25%)		 0
(0%)		 1
(0.03%)		 0
Minion/Tailor
2008-07-04		 33071744
(52.74%)		 684
(20.68%)		 1060
(32.05%)		 1533
(46.36%)		 0
(0%)		 30
(0.91%)		 0
Mistral-option
1.314		 33072512
(75.96%)		 972
(29.39%)		 1540
(46.57%)		 795
(24.04%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 33072610
(78.92%)		 1027
(31.06%)		 1583
(47.87%)		 697
(21.08%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 33071190
(35.98%)		 440
(13.31%)		 750
(22.68%)		 2117
(64.02%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+minisat		 33072215
(66.98%)		 850
(25.7%)		 1365
(41.28%)		 1092
(33.02%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 33072181
(65.95%)		 796
(24.07%)		 1385
(41.88%)		 1126
(34.05%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
3307
UNSAT
920		SAT
1354		?
1033
Abscon 112v4
ESAC
done
3307
UNSAT
923		SAT
1345		?
1039
bpsolver
2008-06-27
done
3307
UNSAT
775		SAT
1223		?
1309
casper
zao
done
3307
UNSAT
792		SAT
1293		?
1218		ERR
4
casper
zito
done
3307
UNSAT
792		SAT
1309		?
1202		ERR
4
choco2_dwdeg
2008-06-26
done
3307
UNSAT
976		SAT
1479		?
851		ERR
1
choco2_impwdeg
2008-06-26
done
3307
UNSAT
957		SAT
1420		?
930
cpHydra
k_10
done
3307
UNSAT
1094		SAT
1650		?
563
cpHydra
k_40
done
3307
UNSAT
1086		SAT
1654		?
567
MDG-noprobe
2008-06-27
done
3307
UNSAT
1017		SAT
1493		?
797
MDG-probe
2008-06-27
done
3307
UNSAT
1027		SAT
1510		?
769		ERR
1
Minion/Tailor
2008-07-04
done
3307
UNSAT
684		SAT
1060		?
1533		ERR
30
Mistral-option
1.314
done
3307
UNSAT
972		SAT
1540		?
795
Mistral-prime
1.313
done
3307
UNSAT
1027		SAT
1583		?
697
SAT4J CSP
2008-06-13
done
3307
UNSAT
440		SAT
750		?
2117
Sugar
v1.13+minisat
done
3307
UNSAT
850		SAT
1365		?
1092
Sugar
v1.13+picosat
done
3307
UNSAT
796		SAT
1385		?
1126

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATSATUNKNOWNIncomplete AnswersBad AnswersMisc.
Abscon 112v4
AC		 33072274
(68.76%)		 920
(27.82%)		 1354
(40.94%)		 1033
(31.24%)		 0
(0%)		 0
(0%)		 0
Abscon 112v4
ESAC		 33072268
(68.58%)		 923
(27.91%)		 1345
(40.67%)		 1039
(31.42%)		 0
(0%)		 0
(0%)		 0
bpsolver
2008-06-27		 33071998
(60.42%)		 775
(23.44%)		 1223
(36.98%)		 1309
(39.58%)		 0
(0%)		 0
(0%)		 0
casper
zao		 33072085
(63.05%)		 792
(23.95%)		 1293
(39.1%)		 1218
(36.83%)		 0
(0%)		 4
(0.12%)		 0
casper
zito		 33072101
(63.53%)		 792
(23.95%)		 1309
(39.58%)		 1202
(36.35%)		 0
(0%)		 4
(0.12%)		 0
choco2_dwdeg
2008-06-26		 33072455
(74.24%)		 976
(29.51%)		 1479
(44.72%)		 851
(25.73%)		 0
(0%)		 1
(0.03%)		 0
choco2_impwdeg
2008-06-26		 33072377
(71.88%)		 957
(28.94%)		 1420
(42.94%)		 930
(28.12%)		 0
(0%)		 0
(0%)		 0
Concrete + CPS4J
2008-05-30		 27511913
(69.54%)		 726
(26.39%)		 1187
(43.15%)		 838
(30.46%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - Tabu Engine
2008-05-30		 2751940
(34.17%)		 86
(3.13%)		 854
(31.04%)		 1811
(65.83%)		 0
(0%)		 0
(0%)		 0
Concrete + CSP4J - WMC Engine
2008-05-30		 2751647
(23.52%)		 84
(3.05%)		 563
(20.47%)		 2103
(76.44%)		 0
(0%)		 1
(0.04%)		 0
cpHydra
k_10		 33072744
(82.98%)		 1094
(33.08%)		 1650
(49.89%)		 563
(17.02%)		 0
(0%)		 0
(0%)		 0
cpHydra
k_40		 33072740
(82.85%)		 1086
(32.84%)		 1654
(50.02%)		 567
(17.15%)		 0
(0%)		 0
(0%)		 0
galac
E		 27511227
(44.6%)		 529
(19.23%)		 698
(25.37%)		 1484
(53.94%)		 0
(0%)		 40
(1.45%)		 0
galac
M		 27511238
(45%)		 537
(19.52%)		 701
(25.48%)		 1470
(53.44%)		 0
(0%)		 43
(1.56%)		 0
mddc-solv
2008-06-09		 704576
(81.82%)		 290
(41.19%)		 286
(40.63%)		 128
(18.18%)		 0
(0%)		 0
(0%)		 0
MDG-noprobe
2008-06-27		 33072510
(75.9%)		 1017
(30.75%)		 1493
(45.15%)		 797
(24.1%)		 0
(0%)		 0
(0%)		 0
MDG-probe
2008-06-27		 33072537
(76.72%)		 1027
(31.06%)		 1510
(45.66%)		 769
(23.25%)		 0
(0%)		 1
(0.03%)		 0
Minion/Tailor
2008-07-04		 33071744
(52.74%)		 684
(20.68%)		 1060
(32.05%)		 1533
(46.36%)		 0
(0%)		 30
(0.91%)		 0
Mistral-option
1.314		 33072512
(75.96%)		 972
(29.39%)		 1540
(46.57%)		 795
(24.04%)		 0
(0%)		 0
(0%)		 0
Mistral-prime
1.313		 33072610
(78.92%)		 1027
(31.06%)		 1583
(47.87%)		 697
(21.08%)		 0
(0%)		 0
(0%)		 0
SAT4J CSP
2008-06-13		 33071190
(35.98%)		 440
(13.31%)		 750
(22.68%)		 2117
(64.02%)		 0
(0%)		 0
(0%)		 0
spider
2008-06-14		 1339893
(66.69%)		 370
(27.63%)		 523
(39.06%)		 445
(33.23%)		 0
(0%)		 1
(0.07%)		 0
Sugar
v1.13+minisat		 33072215
(66.98%)		 850
(25.7%)		 1365
(41.28%)		 1092
(33.02%)		 0
(0%)		 0
(0%)		 0
Sugar
v1.13+picosat		 33072181
(65.95%)		 796
(24.07%)		 1385
(41.88%)		 1126
(34.05%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
Abscon 112v4
AC
done
3307
UNSAT
920		SAT
1354		?
1033
Abscon 112v4
ESAC
done
3307
UNSAT
923		SAT
1345		?
1039
bpsolver
2008-06-27
done
3307
UNSAT
775		SAT
1223		?
1309
casper
zao
done
3307
UNSAT
792		SAT
1293		?
1218		ERR
4
casper
zito
done
3307
UNSAT
792		SAT
1309		?
1202		ERR
4
choco2_dwdeg
2008-06-26
done
3307
UNSAT
976		SAT
1479		?
851		ERR
1
choco2_impwdeg
2008-06-26
done
3307
UNSAT
957		SAT
1420		?
930
Concrete + CPS4J
2008-05-30
done
2751
UNSAT
726		SAT
1187		?
838
Concrete + CSP4J - Tabu Engine
2008-05-30
done
2751
UNSAT
86		SAT
854		?
1811
Concrete + CSP4J - WMC Engine
2008-05-30
done
2751
UNSAT
84		SAT
563		?
2103		ERR
1
cpHydra
k_10
done
3307
UNSAT
1094		SAT
1650		?
563
cpHydra
k_40
done
3307
UNSAT
1086		SAT
1654		?
567
galac
E
done
2751
UNSAT
529		SAT
698		?
1484		ERR
40
galac
M
done
2751
UNSAT
537		SAT
701		?
1470		ERR
43
mddc-solv
2008-06-09
done
704
UNSAT
290		SAT
286		?
128
MDG-noprobe
2008-06-27
done
3307
UNSAT
1017		SAT
1493		?
797
MDG-probe
2008-06-27
done
3307
UNSAT
1027		SAT
1510		?
769		ERR
1
Minion/Tailor
2008-07-04
done
3307
UNSAT
684		SAT
1060		?
1533		ERR
30
Mistral-option
1.314
done
3307
UNSAT
972		SAT
1540		?
795
Mistral-prime
1.313
done
3307
UNSAT
1027		SAT
1583		?
697
SAT4J CSP
2008-06-13
done
3307
UNSAT
440		SAT
750		?
2117
spider
2008-06-14
done
1339
UNSAT
370		SAT
523		?
445		ERR
1
Sugar
v1.13+minisat
done
3307
UNSAT
850		SAT
1365		?
1092
Sugar
v1.13+picosat
done
3307
UNSAT
796		SAT
1385		?
1126