CSP 2008 Competition: solvers page (tabular format, detailed results)

Some explanations

Category "binary constraints in extension" (2-ARY-EXT)

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
635546
(85.98%)
229
(36.06%)
317
(49.92%)
0
(0%)
0
(0%)
89
(14.02%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
635546
(85.98%)
229
(36.06%)
317
(49.92%)
0
(0%)
0
(0%)
89
(14.02%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
635495
(77.95%)
214
(33.7%)
281
(44.25%)
0
(0%)
0
(0%)
120
(18.9%)
0
(0%)
0
(0%)
0
(0%)
20
(3.15%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
635415
(65.35%)
160
(25.2%)
255
(40.16%)
0
(0%)
9
(1.42%)
206
(32.44%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
5
(0.79%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zito
635433
(68.19%)
169
(26.61%)
264
(41.57%)
0
(0%)
9
(1.42%)
187
(29.45%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
6
(0.94%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_dwdeg
2008-06-26
635554
(87.24%)
230
(36.22%)
324
(51.02%)
0
(0%)
0
(0%)
75
(11.81%)
0
(0%)
0
(0%)
0
(0%)
6
(0.94%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_impwdeg
2008-06-26
635550
(86.61%)
230
(36.22%)
320
(50.39%)
0
(0%)
0
(0%)
79
(12.44%)
0
(0%)
0
(0%)
0
(0%)
6
(0.94%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CPS4J
2008-05-30
635498
(78.43%)
201
(31.65%)
297
(46.77%)
0
(0%)
0
(0%)
137
(21.57%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - Tabu Engine
2008-05-30
635348
(54.8%)
0
(0%)
348
(54.8%)
0
(0%)
0
(0%)
287
(45.2%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - WMC Engine
2008-05-30
635116
(18.27%)
0
(0%)
116
(18.27%)
0
(0%)
0
(0%)
518
(81.57%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.16%)
0
(0%)
0
cpHydra
k_10
635574
(90.39%)
242
(38.11%)
332
(52.28%)
0
(0%)
0
(0%)
61
(9.61%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
635567
(89.29%)
236
(37.17%)
331
(52.13%)
0
(0%)
0
(0%)
68
(10.71%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
E
635425
(66.93%)
163
(25.67%)
262
(41.26%)
0
(0%)
0
(0%)
192
(30.24%)
0
(0%)
0
(0%)
0
(0%)
18
(2.83%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
M
635422
(66.46%)
162
(25.51%)
260
(40.94%)
0
(0%)
0
(0%)
191
(30.08%)
4
(0.63%)
0
(0%)
0
(0%)
18
(2.83%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-noprobe
2008-06-27
635558
(87.87%)
235
(37.01%)
323
(50.87%)
0
(0%)
0
(0%)
77
(12.13%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
635561
(88.35%)
237
(37.32%)
324
(51.02%)
0
(0%)
0
(0%)
74
(11.65%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Minion/Tailor
2008-07-04
635414
(65.2%)
166
(26.14%)
248
(39.06%)
0
(0%)
0
(0%)
198
(31.18%)
0
(0%)
20
(3.15%)
0
(0%)
3
(0.47%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-option
1.314
635559
(88.03%)
235
(37.01%)
324
(51.02%)
0
(0%)
0
(0%)
76
(11.97%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
635556
(87.56%)
235
(37.01%)
321
(50.55%)
0
(0%)
0
(0%)
79
(12.44%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
635426
(67.09%)
158
(24.88%)
268
(42.2%)
0
(0%)
3
(0.47%)
206
(32.44%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
spider
2008-06-14
635472
(74.33%)
186
(29.29%)
286
(45.04%)
0
(0%)
0
(0%)
163
(25.67%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+minisat
635470
(74.02%)
192
(30.24%)
278
(43.78%)
0
(0%)
3
(0.47%)
145
(22.83%)
17
(2.68%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
635443
(69.76%)
163
(25.67%)
280
(44.09%)
0
(0%)
0
(0%)
172
(27.09%)
20
(3.15%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
635
UNSAT.
229
SAT.
317
? (TO)
89
Abscon 112v4
ESAC
done
635
UNSAT.
229
SAT.
317
? (TO)
89
bpsolver
2008-06-27
done
635
UNSAT.
214
SAT.
281
? (TO)
120
? (?)
20
casper
zao
done
635
UNSAT.
160
SAT.
255
?
9
? (TO)
206
? (SIG)
5
casper
zito
done
635
UNSAT.
169
SAT.
264
?
9
? (TO)
187
? (SIG)
6
choco2_dwdeg
2008-06-26
done
635
UNSAT.
230
SAT.
324
? (TO)
75
? (?)
6
choco2_impwdeg
2008-06-26
done
635
UNSAT.
230
SAT.
320
? (TO)
79
? (?)
6
Concrete + CPS4J
2008-05-30
done
635
UNSAT.
201
SAT.
297
? (TO)
137
Concrete + CSP4J - Tabu Engine
2008-05-30
done
635
SAT.
348
? (TO)
287
Concrete + CSP4J - WMC Engine
2008-05-30
done
635
SAT.
116
? (TO)
518
ERR CERT
1
cpHydra
k_10
done
635
UNSAT.
242
SAT.
332
? (TO)
61
cpHydra
k_40
done
635
UNSAT.
236
SAT.
331
? (TO)
68
galac
E
done
635
UNSAT.
163
SAT.
262
? (TO)
192
? (?)
18
galac
M
done
635
UNSAT.
162
SAT.
260
? (TO)
191
? (MO)
4
? (?)
18
MDG-noprobe
2008-06-27
done
635
UNSAT.
235
SAT.
323
? (TO)
77
MDG-probe
2008-06-27
done
635
UNSAT.
237
SAT.
324
? (TO)
74
Minion/Tailor
2008-07-04
done
635
UNSAT.
166
SAT.
248
? (TO)
198
? (EC)
20
? (?)
3
Mistral-option
1.314
done
635
UNSAT.
235
SAT.
324
? (TO)
76
Mistral-prime
1.313
done
635
UNSAT.
235
SAT.
321
? (TO)
79
SAT4J CSP
2008-06-13
done
635
UNSAT.
158
SAT.
268
?
3
? (TO)
206
spider
2008-06-14
done
635
UNSAT.
186
SAT.
286
? (TO)
163
Sugar
v1.13+minisat
done
635
UNSAT.
192
SAT.
278
?
3
? (TO)
145
? (MO)
17
Sugar
v1.13+picosat
done
635
UNSAT.
163
SAT.
280
? (TO)
172
? (MO)
20

Category "binary constraints in intension" (2-ARY-INT)

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
696513
(73.71%)
265
(38.07%)
248
(35.63%)
0
(0%)
0
(0%)
179
(25.72%)
4
(0.57%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
696520
(74.71%)
273
(39.22%)
247
(35.49%)
0
(0%)
0
(0%)
173
(24.86%)
3
(0.43%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
696375
(53.88%)
205
(29.45%)
170
(24.43%)
0
(0%)
0
(0%)
320
(45.98%)
0
(0%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
696332
(47.7%)
157
(22.56%)
175
(25.14%)
0
(0%)
2
(0.29%)
357
(51.29%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
(0%)
4
(0.57%)
0
casper
zito
696325
(46.7%)
151
(21.7%)
174
(25%)
0
(0%)
2
(0.29%)
365
(52.44%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
4
(0.57%)
0
choco2_dwdeg
2008-06-26
696523
(75.14%)
285
(40.95%)
238
(34.2%)
0
(0%)
0
(0%)
163
(23.42%)
1
(0.14%)
0
(0%)
0
(0%)
9
(1.29%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_impwdeg
2008-06-26
696521
(74.86%)
286
(41.09%)
235
(33.76%)
0
(0%)
0
(0%)
165
(23.71%)
1
(0.14%)
0
(0%)
0
(0%)
9
(1.29%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CPS4J
2008-05-30
696415
(59.63%)
190
(27.3%)
225
(32.33%)
0
(0%)
16
(2.3%)
265
(38.07%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - Tabu Engine
2008-05-30
696183
(26.29%)
35
(5.03%)
148
(21.26%)
0
(0%)
0
(0%)
513
(73.71%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - WMC Engine
2008-05-30
696222
(31.9%)
33
(4.74%)
189
(27.16%)
0
(0%)
0
(0%)
474
(68.1%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_10
696596
(85.63%)
312
(44.83%)
284
(40.8%)
0
(0%)
0
(0%)
100
(14.37%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
696597
(85.78%)
314
(45.11%)
283
(40.66%)
0
(0%)
0
(0%)
98
(14.08%)
0
(0%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
E
696290
(41.67%)
133
(19.11%)
157
(22.56%)
0
(0%)
0
(0%)
137
(19.68%)
31
(4.45%)
0
(0%)
0
(0%)
237
(34.05%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
galac
M
696307
(44.11%)
142
(20.4%)
165
(23.71%)
0
(0%)
0
(0%)
104
(14.94%)
50
(7.18%)
0
(0%)
0
(0%)
234
(33.62%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
MDG-noprobe
2008-06-27
696499
(71.7%)
274
(39.37%)
225
(32.33%)
0
(0%)
0
(0%)
197
(28.3%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
696515
(73.99%)
280
(40.23%)
235
(33.76%)
0
(0%)
0
(0%)
181
(26.01%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Minion/Tailor
2008-07-04
696305
(43.82%)
168
(24.14%)
137
(19.68%)
0
(0%)
0
(0%)
358
(51.44%)
0
(0%)
6
(0.86%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
27
(3.88%)
0
Mistral-option
1.314
696451
(64.8%)
237
(34.05%)
214
(30.75%)
0
(0%)
0
(0%)
245
(35.2%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
696522
(75%)
285
(40.95%)
237
(34.05%)
0
(0%)
0
(0%)
174
(25%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
696309
(44.4%)
138
(19.83%)
171
(24.57%)
0
(0%)
238
(34.2%)
118
(16.95%)
0
(0%)
31
(4.45%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+minisat
696484
(69.54%)
227
(32.61%)
257
(36.93%)
0
(0%)
60
(8.62%)
122
(17.53%)
30
(4.31%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
696486
(69.83%)
225
(32.33%)
261
(37.5%)
0
(0%)
61
(8.76%)
126
(18.1%)
23
(3.3%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
696
UNSAT.
265
SAT.
248
? (TO)
179
? (MO)
4
Abscon 112v4
ESAC
done
696
UNSAT.
273
SAT.
247
? (TO)
173
? (MO)
3
bpsolver
2008-06-27
done
696
UNSAT.
205
SAT.
170
? (TO)
320
? (?)
1
casper
zao
done
696
UNSAT.
157
SAT.
175
?
2
? (TO)
357
? (SIG)
1
WRONG UNSAT.
4
casper
zito
done
696
UNSAT.
151
SAT.
174
?
2
? (TO)
365
WRONG UNSAT.
4
choco2_dwdeg
2008-06-26
done
696
UNSAT.
285
SAT.
238
? (TO)
163
? (MO)
1
? (?)
9
choco2_impwdeg
2008-06-26
done
696
UNSAT.
286
SAT.
235
? (TO)
165
? (MO)
1
? (?)
9
Concrete + CPS4J
2008-05-30
done
696
UNSAT.
190
SAT.
225
?
16
? (TO)
265
Concrete + CSP4J - Tabu Engine
2008-05-30
done
696
UNSAT.
35
SAT.
148
? (TO)
513
Concrete + CSP4J - WMC Engine
2008-05-30
done
696
UNSAT.
33
SAT.
189
? (TO)
474
cpHydra
k_10
done
696
UNSAT.
312
SAT.
284
? (TO)
100
cpHydra
k_40
done
696
UNSAT.
314
SAT.
283
? (TO)
98
? (?)
1
galac
E
done
696
UNSAT.
133
SAT.
157
? (TO)
137
? (MO)
31
? (?)
237
ERR CERT
1
galac
M
done
696
UNSAT.
142
SAT.
165
? (TO)
104
? (MO)
50
? (?)
234
ERR CERT
1
MDG-noprobe
2008-06-27
done
696
UNSAT.
274
SAT.
225
? (TO)
197
MDG-probe
2008-06-27
done
696
UNSAT.
280
SAT.
235
? (TO)
181
Minion/Tailor
2008-07-04
done
696
UNSAT.
168
SAT.
137
? (TO)
358
? (EC)
6
WRONG UNSAT.
27
Mistral-option
1.314
done
696
UNSAT.
237
SAT.
214
? (TO)
245
Mistral-prime
1.313
done
696
UNSAT.
285
SAT.
237
? (TO)
174
SAT4J CSP
2008-06-13
done
696
UNSAT.
138
SAT.
171
?
238
? (TO)
118
? (EC)
31
Sugar
v1.13+minisat
done
696
UNSAT.
227
SAT.
257
?
60
? (TO)
122
? (MO)
30
Sugar
v1.13+picosat
done
696
UNSAT.
225
SAT.
261
?
61
? (TO)
126
? (MO)
23

Category "global constraints" (GLOBAL)

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
556184
(33.09%)
64
(11.51%)
120
(21.58%)
0
(0%)
169
(30.4%)
196
(35.25%)
7
(1.26%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
556173
(31.12%)
63
(11.33%)
110
(19.78%)
0
(0%)
169
(30.4%)
207
(37.23%)
7
(1.26%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
556347
(62.41%)
151
(27.16%)
196
(35.25%)
0
(0%)
0
(0%)
209
(37.59%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
556390
(70.14%)
172
(30.94%)
218
(39.21%)
0
(0%)
3
(0.54%)
163
(29.32%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zito
556397
(71.4%)
173
(31.12%)
224
(40.29%)
0
(0%)
3
(0.54%)
155
(27.88%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.18%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_dwdeg
2008-06-26
556358
(64.39%)
145
(26.08%)
213
(38.31%)
0
(0%)
0
(0%)
195
(35.07%)
1
(0.18%)
0
(0%)
0
(0%)
2
(0.36%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_impwdeg
2008-06-26
556347
(62.41%)
144
(25.9%)
203
(36.51%)
0
(0%)
0
(0%)
206
(37.05%)
2
(0.36%)
0
(0%)
0
(0%)
1
(0.18%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_10
556419
(75.36%)
149
(26.8%)
270
(48.56%)
0
(0%)
0
(0%)
132
(23.74%)
1
(0.18%)
0
(0%)
0
(0%)
4
(0.72%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
556420
(75.54%)
149
(26.8%)
271
(48.74%)
0
(0%)
0
(0%)
133
(23.92%)
1
(0.18%)
0
(0%)
0
(0%)
2
(0.36%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-noprobe
2008-06-27
556353
(63.49%)
139
(25%)
214
(38.49%)
0
(0%)
0
(0%)
197
(35.43%)
0
(0%)
0
(0%)
0
(0%)
6
(1.08%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
556337
(60.61%)
138
(24.82%)
199
(35.79%)
0
(0%)
0
(0%)
213
(38.31%)
0
(0%)
0
(0%)
0
(0%)
6
(1.08%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Minion/Tailor
2008-07-04
556216
(38.85%)
86
(15.47%)
130
(23.38%)
0
(0%)
0
(0%)
126
(22.66%)
1
(0.18%)
206
(37.05%)
0
(0%)
7
(1.26%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-option
1.314
556383
(68.88%)
135
(24.28%)
248
(44.6%)
0
(0%)
0
(0%)
168
(30.22%)
0
(0%)
0
(0%)
0
(0%)
5
(0.9%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
556403
(72.48%)
136
(24.46%)
267
(48.02%)
0
(0%)
0
(0%)
147
(26.44%)
0
(0%)
0
(0%)
0
(0%)
6
(1.08%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
55663
(11.33%)
11
(1.98%)
52
(9.35%)
0
(0%)
428
(76.98%)
55
(9.89%)
10
(1.8%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+minisat
556405
(72.84%)
153
(27.52%)
252
(45.32%)
0
(0%)
55
(9.89%)
90
(16.19%)
6
(1.08%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
556424
(76.26%)
151
(27.16%)
273
(49.1%)
0
(0%)
55
(9.89%)
72
(12.95%)
5
(0.9%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
556
UNSAT.
64
SAT.
120
?
169
? (TO)
196
? (MO)
7
Abscon 112v4
ESAC
done
556
UNSAT.
63
SAT.
110
?
169
? (TO)
207
? (MO)
7
bpsolver
2008-06-27
done
556
UNSAT.
151
SAT.
196
? (TO)
209
casper
zao
done
556
UNSAT.
172
SAT.
218
?
3
? (TO)
163
casper
zito
done
556
UNSAT.
173
SAT.
224
?
3
? (TO)
155
? (SIG)
1
choco2_dwdeg
2008-06-26
done
556
UNSAT.
145
SAT.
213
? (TO)
195
? (MO)
1
? (?)
2
choco2_impwdeg
2008-06-26
done
556
UNSAT.
144
SAT.
203
? (TO)
206
? (MO)
2
? (?)
1
cpHydra
k_10
done
556
UNSAT.
149
SAT.
270
? (TO)
132
? (MO)
1
? (?)
4
cpHydra
k_40
done
556
UNSAT.
149
SAT.
271
? (TO)
133
? (MO)
1
? (?)
2
MDG-noprobe
2008-06-27
done
556
UNSAT.
139
SAT.
214
? (TO)
197
? (?)
6
MDG-probe
2008-06-27
done
556
UNSAT.
138
SAT.
199
? (TO)
213
? (?)
6
Minion/Tailor
2008-07-04
done
556
UNSAT.
86
SAT.
130
? (TO)
126
? (MO)
1
? (EC)
206
? (?)
7
Mistral-option
1.314
done
556
UNSAT.
135
SAT.
248
? (TO)
168
? (?)
5
Mistral-prime
1.313
done
556
UNSAT.
136
SAT.
267
? (TO)
147
? (?)
6
SAT4J CSP
2008-06-13
done
556
UNSAT.
11
SAT.
52
?
428
? (TO)
55
? (MO)
10
Sugar
v1.13+minisat
done
556
UNSAT.
153
SAT.
252
?
55
? (TO)
90
? (MO)
6
Sugar
v1.13+picosat
done
556
UNSAT.
151
SAT.
273
?
55
? (TO)
72
? (MO)
5

Category "n-ary constraints in extension" (N-ARY-EXT)

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
704545
(77.41%)
295
(41.9%)
250
(35.51%)
0
(0%)
0
(0%)
159
(22.59%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
704541
(76.85%)
293
(41.62%)
248
(35.23%)
0
(0%)
0
(0%)
160
(22.73%)
3
(0.43%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
704322
(45.74%)
131
(18.61%)
191
(27.13%)
0
(0%)
0
(0%)
297
(42.19%)
0
(0%)
0
(0%)
0
(0%)
85
(12.07%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
704386
(54.83%)
204
(28.98%)
182
(25.85%)
0
(0%)
0
(0%)
318
(45.17%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zito
704391
(55.54%)
208
(29.55%)
183
(25.99%)
0
(0%)
0
(0%)
313
(44.46%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_dwdeg
2008-06-26
704477
(67.76%)
247
(35.09%)
230
(32.67%)
0
(0%)
0
(0%)
227
(32.24%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_impwdeg
2008-06-26
704441
(62.64%)
231
(32.81%)
210
(29.83%)
0
(0%)
0
(0%)
263
(37.36%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CPS4J
2008-05-30
704544
(77.27%)
277
(39.35%)
267
(37.93%)
0
(0%)
0
(0%)
160
(22.73%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - Tabu Engine
2008-05-30
704210
(29.83%)
30
(4.26%)
180
(25.57%)
0
(0%)
0
(0%)
494
(70.17%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - WMC Engine
2008-05-30
704135
(19.18%)
30
(4.26%)
105
(14.91%)
0
(0%)
0
(0%)
569
(80.82%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_10
704586
(83.24%)
303
(43.04%)
283
(40.2%)
0
(0%)
0
(0%)
118
(16.76%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
704587
(83.38%)
301
(42.76%)
286
(40.63%)
0
(0%)
0
(0%)
117
(16.62%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
E
704323
(45.88%)
196
(27.84%)
127
(18.04%)
0
(0%)
0
(0%)
308
(43.75%)
58
(8.24%)
0
(0%)
0
(0%)
15
(2.13%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
M
704326
(46.31%)
196
(27.84%)
130
(18.47%)
0
(0%)
0
(0%)
305
(43.32%)
58
(8.24%)
0
(0%)
0
(0%)
15
(2.13%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
mddc-solv
2008-06-09
704576
(81.82%)
290
(41.19%)
286
(40.63%)
0
(0%)
0
(0%)
128
(18.18%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-noprobe
2008-06-27
704570
(80.97%)
283
(40.2%)
287
(40.77%)
0
(0%)
0
(0%)
134
(19.03%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
704564
(80.11%)
281
(39.91%)
283
(40.2%)
0
(0%)
0
(0%)
139
(19.74%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
Minion/Tailor
2008-07-04
704406
(57.67%)
213
(30.26%)
193
(27.41%)
0
(0%)
0
(0%)
141
(20.03%)
112
(15.91%)
18
(2.56%)
0
(0%)
27
(3.84%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-option
1.314
704565
(80.26%)
282
(40.06%)
283
(40.2%)
0
(0%)
0
(0%)
139
(19.74%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
704569
(80.82%)
283
(40.2%)
286
(40.63%)
0
(0%)
0
(0%)
135
(19.18%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
704218
(30.97%)
93
(13.21%)
125
(17.76%)
0
(0%)
341
(48.44%)
121
(17.19%)
13
(1.85%)
11
(1.56%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
spider
2008-06-14
704421
(59.8%)
184
(26.14%)
237
(33.66%)
0
(0%)
0
(0%)
281
(39.91%)
1
(0.14%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.14%)
0
Sugar
v1.13+minisat
704370
(52.56%)
191
(27.13%)
179
(25.43%)
0
(0%)
70
(9.94%)
154
(21.88%)
110
(15.63%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
704347
(49.29%)
169
(24.01%)
178
(25.28%)
0
(0%)
70
(9.94%)
178
(25.28%)
109
(15.48%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
704
UNSAT.
295
SAT.
250
? (TO)
159
Abscon 112v4
ESAC
done
704
UNSAT.
293
SAT.
248
? (TO)
160
? (MO)
3
bpsolver
2008-06-27
done
704
UNSAT.
131
SAT.
191
? (TO)
297
? (?)
85
casper
zao
done
704
UNSAT.
204
SAT.
182
? (TO)
318
casper
zito
done
704
UNSAT.
208
SAT.
183
? (TO)
313
choco2_dwdeg
2008-06-26
done
704
UNSAT.
247
SAT.
230
? (TO)
227
choco2_impwdeg
2008-06-26
done
704
UNSAT.
231
SAT.
210
? (TO)
263
Concrete + CPS4J
2008-05-30
done
704
UNSAT.
277
SAT.
267
? (TO)
160
Concrete + CSP4J - Tabu Engine
2008-05-30
done
704
UNSAT.
30
SAT.
180
? (TO)
494
Concrete + CSP4J - WMC Engine
2008-05-30
done
704
UNSAT.
30
SAT.
105
? (TO)
569
cpHydra
k_10
done
704
UNSAT.
303
SAT.
283
? (TO)
118
cpHydra
k_40
done
704
UNSAT.
301
SAT.
286
? (TO)
117
galac
E
done
704
UNSAT.
196
SAT.
127
? (TO)
308
? (MO)
58
? (?)
15
galac
M
done
704
UNSAT.
196
SAT.
130
? (TO)
305
? (MO)
58
? (?)
15
mddc-solv
2008-06-09
done
704
UNSAT.
290
SAT.
286
? (TO)
128
MDG-noprobe
2008-06-27
done
704
UNSAT.
283
SAT.
287
? (TO)
134
MDG-probe
2008-06-27
done
704
UNSAT.
281
SAT.
283
? (TO)
139
ERR CERT
1
Minion/Tailor
2008-07-04
done
704
UNSAT.
213
SAT.
193
? (TO)
141
? (MO)
112
? (EC)
18
? (?)
27
Mistral-option
1.314
done
704
UNSAT.
282
SAT.
283
? (TO)
139
Mistral-prime
1.313
done
704
UNSAT.
283
SAT.
286
? (TO)
135
SAT4J CSP
2008-06-13
done
704
UNSAT.
93
SAT.
125
?
341
? (TO)
121
? (MO)
13
? (EC)
11
spider
2008-06-14
done
704
UNSAT.
184
SAT.
237
? (TO)
281
? (MO)
1
WRONG UNSAT.
1
Sugar
v1.13+minisat
done
704
UNSAT.
191
SAT.
179
?
70
? (TO)
154
? (MO)
110
Sugar
v1.13+picosat
done
704
UNSAT.
169
SAT.
178
?
70
? (TO)
178
? (MO)
109

Category "n-ary constraints in intension" (N-ARY-INT)

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
716486
(67.88%)
67
(9.36%)
419
(58.52%)
0
(0%)
35
(4.89%)
191
(26.68%)
4
(0.56%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
716488
(68.16%)
65
(9.08%)
423
(59.08%)
0
(0%)
35
(4.89%)
191
(26.68%)
2
(0.28%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
716459
(64.11%)
74
(10.34%)
385
(53.77%)
0
(0%)
0
(0%)
252
(35.2%)
0
(0%)
0
(0%)
0
(0%)
5
(0.7%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
716562
(78.49%)
99
(13.83%)
463
(64.66%)
0
(0%)
0
(0%)
148
(20.67%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
6
(0.84%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zito
716555
(77.51%)
91
(12.71%)
464
(64.8%)
0
(0%)
0
(0%)
156
(21.79%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
5
(0.7%)
0
(0%)
0
(0%)
0
(0%)
0
choco2_dwdeg
2008-06-26
716543
(75.84%)
69
(9.64%)
474
(66.2%)
0
(0%)
0
(0%)
165
(23.04%)
0
(0%)
0
(0%)
0
(0%)
7
(0.98%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
choco2_impwdeg
2008-06-26
716518
(72.35%)
66
(9.22%)
452
(63.13%)
0
(0%)
0
(0%)
188
(26.26%)
1
(0.14%)
0
(0%)
0
(0%)
9
(1.26%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CPS4J
2008-05-30
716456
(63.69%)
58
(8.1%)
398
(55.59%)
0
(0%)
41
(5.73%)
200
(27.93%)
19
(2.65%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - Tabu Engine
2008-05-30
716199
(27.79%)
21
(2.93%)
178
(24.86%)
0
(0%)
27
(3.77%)
473
(66.06%)
17
(2.37%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - WMC Engine
2008-05-30
716174
(24.3%)
21
(2.93%)
153
(21.37%)
0
(0%)
25
(3.49%)
498
(69.55%)
19
(2.65%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_10
716569
(79.47%)
88
(12.29%)
481
(67.18%)
0
(0%)
0
(0%)
143
(19.97%)
4
(0.56%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
716569
(79.47%)
86
(12.01%)
483
(67.46%)
0
(0%)
0
(0%)
143
(19.97%)
4
(0.56%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
E
716189
(26.4%)
37
(5.17%)
152
(21.23%)
0
(0%)
0
(0%)
361
(50.42%)
30
(4.19%)
0
(0%)
0
(0%)
97
(13.55%)
0
(0%)
0
(0%)
33
(4.61%)
6
(0.84%)
0
galac
M
716183
(25.56%)
37
(5.17%)
146
(20.39%)
0
(0%)
0
(0%)
360
(50.28%)
33
(4.61%)
0
(0%)
0
(0%)
98
(13.69%)
0
(0%)
0
(0%)
36
(5.03%)
6
(0.84%)
0
MDG-noprobe
2008-06-27
716530
(74.02%)
86
(12.01%)
444
(62.01%)
0
(0%)
0
(0%)
182
(25.42%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
4
(0.56%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
716560
(78.21%)
91
(12.71%)
469
(65.5%)
0
(0%)
0
(0%)
152
(21.23%)
1
(0.14%)
0
(0%)
0
(0%)
0
(0%)
3
(0.42%)
0
(0%)
0
(0%)
0
(0%)
0
Minion/Tailor
2008-07-04
716403
(56.28%)
51
(7.12%)
352
(49.16%)
0
(0%)
85
(11.87%)
187
(26.12%)
16
(2.23%)
22
(3.07%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
3
(0.42%)
0
(0%)
0
Mistral-option
1.314
716554
(77.37%)
83
(11.59%)
471
(65.78%)
0
(0%)
0
(0%)
161
(22.49%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.14%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
716560
(78.21%)
88
(12.29%)
472
(65.92%)
0
(0%)
0
(0%)
152
(21.23%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
4
(0.56%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
716174
(24.3%)
40
(5.59%)
134
(18.72%)
0
(0%)
253
(35.34%)
173
(24.16%)
115
(16.06%)
1
(0.14%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+minisat
716486
(67.88%)
87
(12.15%)
399
(55.73%)
0
(0%)
55
(7.68%)
115
(16.06%)
60
(8.38%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
716481
(67.18%)
88
(12.29%)
393
(54.89%)
0
(0%)
55
(7.68%)
121
(16.9%)
59
(8.24%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
716
UNSAT.
67
SAT.
419
?
35
? (TO)
191
? (MO)
4
Abscon 112v4
ESAC
done
716
UNSAT.
65
SAT.
423
?
35
? (TO)
191
? (MO)
2
bpsolver
2008-06-27
done
716
UNSAT.
74
SAT.
385
? (TO)
252
? (?)
5
casper
zao
done
716
UNSAT.
99
SAT.
463
? (TO)
148
? (SIG)
6
casper
zito
done
716
UNSAT.
91
SAT.
464
? (TO)
156
? (SIG)
5
choco2_dwdeg
2008-06-26
done
716
UNSAT.
69
SAT.
474
? (TO)
165
? (?)
7
ERR CERT
1
choco2_impwdeg
2008-06-26
done
716
UNSAT.
66
SAT.
452
? (TO)
188
? (MO)
1
? (?)
9
Concrete + CPS4J
2008-05-30
done
716
UNSAT.
58
SAT.
398
?
41
? (TO)
200
? (MO)
19
Concrete + CSP4J - Tabu Engine
2008-05-30
done
716
UNSAT.
21
SAT.
178
?
27
? (TO)
473
? (MO)
17
Concrete + CSP4J - WMC Engine
2008-05-30
done
716
UNSAT.
21
SAT.
153
?
25
? (TO)
498
? (MO)
19
cpHydra
k_10
done
716
UNSAT.
88
SAT.
481
? (TO)
143
? (MO)
4
cpHydra
k_40
done
716
UNSAT.
86
SAT.
483
? (TO)
143
? (MO)
4
galac
E
done
716
UNSAT.
37
SAT.
152
? (TO)
361
? (MO)
30
? (?)
97
ERR CERT
33
WRONG UNSAT.
6
galac
M
done
716
UNSAT.
37
SAT.
146
? (TO)
360
? (MO)
33
? (?)
98
ERR CERT
36
WRONG UNSAT.
6
MDG-noprobe
2008-06-27
done
716
UNSAT.
86
SAT.
444
? (TO)
182
? (SIG)
4
MDG-probe
2008-06-27
done
716
UNSAT.
91
SAT.
469
? (TO)
152
? (MO)
1
? (SIG)
3
Minion/Tailor
2008-07-04
done
716
UNSAT.
51
SAT.
352
?
85
? (TO)
187
? (MO)
16
? (EC)
22
ERR CERT
3
Mistral-option
1.314
done
716
UNSAT.
83
SAT.
471
? (TO)
161
? (SIG)
1
Mistral-prime
1.313
done
716
UNSAT.
88
SAT.
472
? (TO)
152
? (SIG)
4
SAT4J CSP
2008-06-13
done
716
UNSAT.
40
SAT.
134
?
253
? (TO)
173
? (MO)
115
? (EC)
1
Sugar
v1.13+minisat
done
716
UNSAT.
87
SAT.
399
?
55
? (TO)
115
? (MO)
60
Sugar
v1.13+picosat
done
716
UNSAT.
88
SAT.
393
?
55
? (TO)
121
? (MO)
59

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
33072274
(68.76%)
920
(27.82%)
1354
(40.94%)
0
(0%)
204
(6.17%)
814
(24.61%)
15
(0.45%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
33072268
(68.58%)
923
(27.91%)
1345
(40.67%)
0
(0%)
204
(6.17%)
820
(24.8%)
15
(0.45%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
33071998
(60.42%)
775
(23.44%)
1223
(36.98%)
0
(0%)
0
(0%)
1198
(36.23%)
0
(0%)
0
(0%)
0
(0%)
111
(3.36%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
33072085
(63.05%)
792
(23.95%)
1293
(39.1%)
0
(0%)
14
(0.42%)
1192
(36.04%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
12
(0.36%)
0
(0%)
0
(0%)
4
(0.12%)
0
casper
zito
33072101
(63.53%)
792
(23.95%)
1309
(39.58%)
0
(0%)
14
(0.42%)
1176
(35.56%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
12
(0.36%)
0
(0%)
0
(0%)
4
(0.12%)
0
choco2_dwdeg
2008-06-26
33072455
(74.24%)
976
(29.51%)
1479
(44.72%)
0
(0%)
0
(0%)
825
(24.95%)
2
(0.06%)
0
(0%)
0
(0%)
24
(0.73%)
0
(0%)
0
(0%)
1
(0.03%)
0
(0%)
0
choco2_impwdeg
2008-06-26
33072377
(71.88%)
957
(28.94%)
1420
(42.94%)
0
(0%)
0
(0%)
901
(27.25%)
4
(0.12%)
0
(0%)
0
(0%)
25
(0.76%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_10
33072744
(82.98%)
1094
(33.08%)
1650
(49.89%)
0
(0%)
0
(0%)
554
(16.75%)
5
(0.15%)
0
(0%)
0
(0%)
4
(0.12%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
33072740
(82.85%)
1086
(32.84%)
1654
(50.02%)
0
(0%)
0
(0%)
559
(16.9%)
5
(0.15%)
0
(0%)
0
(0%)
3
(0.09%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-noprobe
2008-06-27
33072510
(75.9%)
1017
(30.75%)
1493
(45.15%)
0
(0%)
0
(0%)
787
(23.8%)
0
(0%)
0
(0%)
0
(0%)
6
(0.18%)
4
(0.12%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
33072537
(76.72%)
1027
(31.06%)
1510
(45.66%)
0
(0%)
0
(0%)
759
(22.95%)
1
(0.03%)
0
(0%)
0
(0%)
6
(0.18%)
3
(0.09%)
0
(0%)
1
(0.03%)
0
(0%)
0
Minion/Tailor
2008-07-04
33071744
(52.74%)
684
(20.68%)
1060
(32.05%)
0
(0%)
85
(2.57%)
1010
(30.54%)
129
(3.9%)
272
(8.22%)
0
(0%)
37
(1.12%)
0
(0%)
0
(0%)
3
(0.09%)
27
(0.82%)
0
Mistral-option
1.314
33072512
(75.96%)
972
(29.39%)
1540
(46.57%)
0
(0%)
0
(0%)
789
(23.86%)
0
(0%)
0
(0%)
0
(0%)
5
(0.15%)
1
(0.03%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
33072610
(78.92%)
1027
(31.06%)
1583
(47.87%)
0
(0%)
0
(0%)
687
(20.77%)
0
(0%)
0
(0%)
0
(0%)
6
(0.18%)
4
(0.12%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
33071190
(35.98%)
440
(13.31%)
750
(22.68%)
0
(0%)
1263
(38.19%)
673
(20.35%)
138
(4.17%)
43
(1.3%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+minisat
33072215
(66.98%)
850
(25.7%)
1365
(41.28%)
0
(0%)
243
(7.35%)
626
(18.93%)
223
(6.74%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
33072181
(65.95%)
796
(24.07%)
1385
(41.88%)
0
(0%)
241
(7.29%)
669
(20.23%)
216
(6.53%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
3307
UNSAT.
920
SAT.
1354
?
204
? (TO)
814
? (MO)
15
Abscon 112v4
ESAC
done
3307
UNSAT.
923
SAT.
1345
?
204
? (TO)
820
? (MO)
15
bpsolver
2008-06-27
done
3307
UNSAT.
775
SAT.
1223
? (TO)
1198
? (?)
111
casper
zao
done
3307
UNSAT.
792
SAT.
1293
?
14
? (TO)
1192
? (SIG)
12
WRONG UNSAT.
4
casper
zito
done
3307
UNSAT.
792
SAT.
1309
?
14
? (TO)
1176
? (SIG)
12
WRONG UNSAT.
4
choco2_dwdeg
2008-06-26
done
3307
UNSAT.
976
SAT.
1479
? (TO)
825
? (MO)
2
? (?)
24
ERR CERT
1
choco2_impwdeg
2008-06-26
done
3307
UNSAT.
957
SAT.
1420
? (TO)
901
? (MO)
4
? (?)
25
cpHydra
k_10
done
3307
UNSAT.
1094
SAT.
1650
? (TO)
554
? (MO)
5
? (?)
4
cpHydra
k_40
done
3307
UNSAT.
1086
SAT.
1654
? (TO)
559
? (MO)
5
? (?)
3
MDG-noprobe
2008-06-27
done
3307
UNSAT.
1017
SAT.
1493
? (TO)
787
? (?)
6
? (SIG)
4
MDG-probe
2008-06-27
done
3307
UNSAT.
1027
SAT.
1510
? (TO)
759
? (MO)
1
? (?)
6
? (SIG)
3
ERR CERT
1
Minion/Tailor
2008-07-04
done
3307
UNSAT.
684
SAT.
1060
?
85
? (TO)
1010
? (MO)
129
? (EC)
272
? (?)
37
ERR CERT
3
WRONG UNSAT.
27
Mistral-option
1.314
done
3307
UNSAT.
972
SAT.
1540
? (TO)
789
? (?)
5
? (SIG)
1
Mistral-prime
1.313
done
3307
UNSAT.
1027
SAT.
1583
? (TO)
687
? (?)
6
? (SIG)
4
SAT4J CSP
2008-06-13
done
3307
UNSAT.
440
SAT.
750
?
1263
? (TO)
673
? (MO)
138
? (EC)
43
Sugar
v1.13+minisat
done
3307
UNSAT.
850
SAT.
1365
?
243
? (TO)
626
? (MO)
223
Sugar
v1.13+picosat
done
3307
UNSAT.
796
SAT.
1385
?
241
? (TO)
669
? (MO)
216

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSAT.SAT.SAT (timeout)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (conv. script)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG UNSAT.Misc.
Abscon 112v4
AC
33072274
(68.76%)
920
(27.82%)
1354
(40.94%)
0
(0%)
204
(6.17%)
814
(24.61%)
15
(0.45%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Abscon 112v4
ESAC
33072268
(68.58%)
923
(27.91%)
1345
(40.67%)
0
(0%)
204
(6.17%)
820
(24.8%)
15
(0.45%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
bpsolver
2008-06-27
33071998
(60.42%)
775
(23.44%)
1223
(36.98%)
0
(0%)
0
(0%)
1198
(36.23%)
0
(0%)
0
(0%)
0
(0%)
111
(3.36%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
casper
zao
33072085
(63.05%)
792
(23.95%)
1293
(39.1%)
0
(0%)
14
(0.42%)
1192
(36.04%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
12
(0.36%)
0
(0%)
0
(0%)
4
(0.12%)
0
casper
zito
33072101
(63.53%)
792
(23.95%)
1309
(39.58%)
0
(0%)
14
(0.42%)
1176
(35.56%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
12
(0.36%)
0
(0%)
0
(0%)
4
(0.12%)
0
choco2_dwdeg
2008-06-26
33072455
(74.24%)
976
(29.51%)
1479
(44.72%)
0
(0%)
0
(0%)
825
(24.95%)
2
(0.06%)
0
(0%)
0
(0%)
24
(0.73%)
0
(0%)
0
(0%)
1
(0.03%)
0
(0%)
0
choco2_impwdeg
2008-06-26
33072377
(71.88%)
957
(28.94%)
1420
(42.94%)
0
(0%)
0
(0%)
901
(27.25%)
4
(0.12%)
0
(0%)
0
(0%)
25
(0.76%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CPS4J
2008-05-30
27511913
(69.54%)
726
(26.39%)
1187
(43.15%)
0
(0%)
57
(2.07%)
762
(27.7%)
19
(0.69%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - Tabu Engine
2008-05-30
2751940
(34.17%)
86
(3.13%)
854
(31.04%)
0
(0%)
27
(0.98%)
1767
(64.23%)
17
(0.62%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Concrete + CSP4J - WMC Engine
2008-05-30
2751647
(23.52%)
84
(3.05%)
563
(20.47%)
0
(0%)
25
(0.91%)
2059
(74.85%)
19
(0.69%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.04%)
0
(0%)
0
cpHydra
k_10
33072744
(82.98%)
1094
(33.08%)
1650
(49.89%)
0
(0%)
0
(0%)
554
(16.75%)
5
(0.15%)
0
(0%)
0
(0%)
4
(0.12%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
cpHydra
k_40
33072740
(82.85%)
1086
(32.84%)
1654
(50.02%)
0
(0%)
0
(0%)
559
(16.9%)
5
(0.15%)
0
(0%)
0
(0%)
3
(0.09%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
galac
E
27511227
(44.6%)
529
(19.23%)
698
(25.37%)
0
(0%)
0
(0%)
998
(36.28%)
119
(4.33%)
0
(0%)
0
(0%)
367
(13.34%)
0
(0%)
0
(0%)
34
(1.24%)
6
(0.22%)
0
galac
M
27511238
(45%)
537
(19.52%)
701
(25.48%)
0
(0%)
0
(0%)
960
(34.9%)
145
(5.27%)
0
(0%)
0
(0%)
365
(13.27%)
0
(0%)
0
(0%)
37
(1.34%)
6
(0.22%)
0
mddc-solv
2008-06-09
704576
(81.82%)
290
(41.19%)
286
(40.63%)
0
(0%)
0
(0%)
128
(18.18%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-noprobe
2008-06-27
33072510
(75.9%)
1017
(30.75%)
1493
(45.15%)
0
(0%)
0
(0%)
787
(23.8%)
0
(0%)
0
(0%)
0
(0%)
6
(0.18%)
4
(0.12%)
0
(0%)
0
(0%)
0
(0%)
0
MDG-probe
2008-06-27
33072537
(76.72%)
1027
(31.06%)
1510
(45.66%)
0
(0%)
0
(0%)
759
(22.95%)
1
(0.03%)
0
(0%)
0
(0%)
6
(0.18%)
3
(0.09%)
0
(0%)
1
(0.03%)
0
(0%)
0
Minion/Tailor
2008-07-04
33071744
(52.74%)
684
(20.68%)
1060
(32.05%)
0
(0%)
85
(2.57%)
1010
(30.54%)
129
(3.9%)
272
(8.22%)
0
(0%)
37
(1.12%)
0
(0%)
0
(0%)
3
(0.09%)
27
(0.82%)
0
Mistral-option
1.314
33072512
(75.96%)
972
(29.39%)
1540
(46.57%)
0
(0%)
0
(0%)
789
(23.86%)
0
(0%)
0
(0%)
0
(0%)
5
(0.15%)
1
(0.03%)
0
(0%)
0
(0%)
0
(0%)
0
Mistral-prime
1.313
33072610
(78.92%)
1027
(31.06%)
1583
(47.87%)
0
(0%)
0
(0%)
687
(20.77%)
0
(0%)
0
(0%)
0
(0%)
6
(0.18%)
4
(0.12%)
0
(0%)
0
(0%)
0
(0%)
0
SAT4J CSP
2008-06-13
33071190
(35.98%)
440
(13.31%)
750
(22.68%)
0
(0%)
1263
(38.19%)
673
(20.35%)
138
(4.17%)
43
(1.3%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
spider
2008-06-14
1339893
(66.69%)
370
(27.63%)
523
(39.06%)
0
(0%)
0
(0%)
444
(33.16%)
1
(0.07%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
1
(0.07%)
0
Sugar
v1.13+minisat
33072215
(66.98%)
850
(25.7%)
1365
(41.28%)
0
(0%)
243
(7.35%)
626
(18.93%)
223
(6.74%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
Sugar
v1.13+picosat
33072181
(65.95%)
796
(24.07%)
1385
(41.88%)
0
(0%)
241
(7.29%)
669
(20.23%)
216
(6.53%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
0

Solver NameProgress
Abscon 112v4
AC
done
3307
UNSAT.
920
SAT.
1354
?
204
? (TO)
814
? (MO)
15
Abscon 112v4
ESAC
done
3307
UNSAT.
923
SAT.
1345
?
204
? (TO)
820
? (MO)
15
bpsolver
2008-06-27
done
3307
UNSAT.
775
SAT.
1223
? (TO)
1198
? (?)
111
casper
zao
done
3307
UNSAT.
792
SAT.
1293
?
14
? (TO)
1192
? (SIG)
12
WRONG UNSAT.
4
casper
zito
done
3307
UNSAT.
792
SAT.
1309
?
14
? (TO)
1176
? (SIG)
12
WRONG UNSAT.
4
choco2_dwdeg
2008-06-26
done
3307
UNSAT.
976
SAT.
1479
? (TO)
825
? (MO)
2
? (?)
24
ERR CERT
1
choco2_impwdeg
2008-06-26
done
3307
UNSAT.
957
SAT.
1420
? (TO)
901
? (MO)
4
? (?)
25
Concrete + CPS4J
2008-05-30
done
2751
UNSAT.
726
SAT.
1187
?
57
? (TO)
762
? (MO)
19
Concrete + CSP4J - Tabu Engine
2008-05-30
done
2751
UNSAT.
86
SAT.
854
?
27
? (TO)
1767
? (MO)
17
Concrete + CSP4J - WMC Engine
2008-05-30
done
2751
UNSAT.
84
SAT.
563
?
25
? (TO)
2059
? (MO)
19
ERR CERT
1
cpHydra
k_10
done
3307
UNSAT.
1094
SAT.
1650
? (TO)
554
? (MO)
5
? (?)
4
cpHydra
k_40
done
3307
UNSAT.
1086
SAT.
1654
? (TO)
559
? (MO)
5
? (?)
3
galac
E
done
2751
UNSAT.
529
SAT.
698
? (TO)
998
? (MO)
119
? (?)
367
ERR CERT
34
WRONG UNSAT.
6
galac
M
done
2751
UNSAT.
537
SAT.
701
? (TO)
960
? (MO)
145
? (?)
365
ERR CERT
37
WRONG UNSAT.
6
mddc-solv
2008-06-09
done
704
UNSAT.
290
SAT.
286
? (TO)
128
MDG-noprobe
2008-06-27
done
3307
UNSAT.
1017
SAT.
1493
? (TO)
787
? (?)
6
? (SIG)
4
MDG-probe
2008-06-27
done
3307
UNSAT.
1027
SAT.
1510
? (TO)
759
? (MO)
1
? (?)
6
? (SIG)
3
ERR CERT
1
Minion/Tailor
2008-07-04
done
3307
UNSAT.
684
SAT.
1060
?
85
? (TO)
1010
? (MO)
129
? (EC)
272
? (?)
37
ERR CERT
3
WRONG UNSAT.
27
Mistral-option
1.314
done
3307
UNSAT.
972
SAT.
1540
? (TO)
789
? (?)
5
? (SIG)
1
Mistral-prime
1.313
done
3307
UNSAT.
1027
SAT.
1583
? (TO)
687
? (?)
6
? (SIG)
4
SAT4J CSP
2008-06-13
done
3307
UNSAT.
440
SAT.
750
?
1263
? (TO)
673
? (MO)
138
? (EC)
43
spider
2008-06-14
done
1339
UNSAT.
370
SAT.
523
? (TO)
444
? (MO)
1
WRONG UNSAT.
1
Sugar
v1.13+minisat
done
3307
UNSAT.
850
SAT.
1365
?
243
? (TO)
626
? (MO)
223
Sugar
v1.13+picosat
done
3307
UNSAT.
796
SAT.
1385
?
241
? (TO)
669
? (MO)
216