2017 XCSP3 competition: main track (CSP and COP, sequential and parallel solvers): solvers page (tabular format, summary)

Some explanations

Category "optimization problem" (COP)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
AbsCon-basic
2017-06-11
(complete)		 438117
(26.71%)		 2
(0.46%)		 115
(26.26%)		 210
(47.95%)		 111
(25.34%)		 0
(0%)
choco-solver
4.0.5 par (2017-08-09)
(complete)		 438147
(33.56%)		 2
(0.46%)		 145
(33.11%)		 215
(49.09%)		 76
(17.35%)		 0
(0%)
choco-solver
4.0.5 par (2017-08-18)
(complete)		 438145
(33.11%)		 2
(0.46%)		 143
(32.65%)		 218
(49.77%)		 75
(17.12%)		 0
(0%)
choco-solver
4.0.5 seq (2017-08-09)
(complete)		 438115
(26.26%)		 2
(0.46%)		 113
(25.8%)		 187
(42.69%)		 135
(30.82%)		 1
(0.23%)
choco-solver
4.0.5 seq (2017-08-18)
(complete)		 438115
(26.26%)		 2
(0.46%)		 113
(25.8%)		 193
(44.06%)		 130
(29.68%)		 0
(0%)
choco-solver
5a (2017-07-26)
(complete)		 438106
(24.2%)		 2
(0.46%)		 104
(23.74%)		 197
(44.98%)		 134
(30.59%)		 1
(0.23%)
choco-solver
5a (2017-08-18)
(complete)		 438107
(24.43%)		 2
(0.46%)		 105
(23.97%)		 197
(44.98%)		 134
(30.59%)		 0
(0%)
Concrete
3.4
(complete)		 438103
(23.52%)		 2
(0.46%)		 101
(23.06%)		 200
(45.66%)		 135
(30.82%)		 0
(0%)
cosoco
1.12
(complete)		 43898
(22.37%)		 2
(0.46%)		 96
(21.92%)		 180
(41.1%)		 160
(36.53%)		 0
(0%)
cosoco-sat
1.12
(complete)		 43898
(22.37%)		 2
(0.46%)		 96
(21.92%)		 152
(34.7%)		 188
(42.92%)		 0
(0%)
Mistral-2.0
2017-07-28
(complete)		 438133
(30.37%)		 2
(0.46%)		 131
(29.91%)		 216
(49.32%)		 89
(20.32%)		 0
(0%)
OscaR - ALNS
2017-07-26
(complete)		 4382
(0.46%)		 2
(0.46%)		 0
(0%)		 335
(76.48%)		 101
(23.06%)		 0
(0%)
OscaR - Conflict Ordering
2017-07-26
(complete)		 438125
(28.54%)		 2
(0.46%)		 123
(28.08%)		 209
(47.72%)		 104
(23.74%)		 0
(0%)
OscaR - Hybrid
2017-07-26
(complete)		 438109
(24.89%)		 2
(0.46%)		 107
(24.43%)		 224
(51.14%)		 105
(23.97%)		 0
(0%)
OscaR - Parallel with EPS
2017-07-26
(complete)		 438130
(29.68%)		 2
(0.46%)		 128
(29.22%)		 200
(45.66%)		 108
(24.66%)		 0
(0%)
OscaR - Parallel with EPS
2017-08-22
(complete)		 438135
(30.82%)		 2
(0.46%)		 133
(30.37%)		 201
(45.89%)		 102
(23.29%)		 0
(0%)
R1: AbsCon-basic
2017-06-11
(complete)		 438117
(26.71%)		 2
(0.46%)		 115
(26.26%)		 210
(47.95%)		 111
(25.34%)		 0
(0%)
R1: choco-solver
4.0.5 (2017-07-05)
(complete)		 438130
(29.68%)		 2
(0.46%)		 128
(29.22%)		 211
(48.17%)		 85
(19.41%)		 12
(2.74%)
R1: choco-solver
5a (2017-07-05)
(complete)		 438128
(29.22%)		 2
(0.46%)		 126
(28.77%)		 212
(48.4%)		 86
(19.63%)		 12
(2.74%)
R1: Concrete
3.4
(complete)		 438103
(23.52%)		 2
(0.46%)		 101
(23.06%)		 199
(45.43%)		 136
(31.05%)		 0
(0%)
R1: cosoco
1.1
(complete)		 43898
(22.37%)		 2
(0.46%)		 96
(21.92%)		 11
(2.51%)		 329
(75.11%)		 0
(0%)
R1: cosoco-sat
1.1
(complete)		 43898
(22.37%)		 2
(0.46%)		 96
(21.92%)		 12
(2.74%)		 328
(74.89%)		 0
(0%)
R1: Mistral-2.0
2017-06-28
(complete)		 438132
(30.14%)		 2
(0.46%)		 130
(29.68%)		 216
(49.32%)		 90
(20.55%)		 0
(0%)
R1: OscaR - ALNS
2017-06-30
(complete)		 4382
(0.46%)		 2
(0.46%)		 0
(0%)		 336
(76.71%)		 100
(22.83%)		 0
(0%)
R1: OscaR - Conflict Ordering
2017-06-30
(complete)		 438125
(28.54%)		 2
(0.46%)		 123
(28.08%)		 209
(47.72%)		 104
(23.74%)		 0
(0%)
R1: OscaR - Hybrid
2017-06-30
(complete)		 438109
(24.89%)		 2
(0.46%)		 107
(24.43%)		 225
(51.37%)		 104
(23.74%)		 0
(0%)
R1: sat4j-CSP
2017-07-05
(complete)		 43876
(17.35%)		 2
(0.46%)		 74
(16.89%)		 91
(20.78%)		 271
(61.87%)		 0
(0%)
sat4j-CSP
2017-07-05
(complete)		 43877
(17.58%)		 2
(0.46%)		 75
(17.12%)		 87
(19.86%)		 274
(62.56%)		 0
(0%)

Solver NameProgress
AbsCon-basic
2017-06-11
(complete)
done
438
UNSAT
2		OPT/SAT
115		SAT (TO/MO)
210		?
111
choco-solver
4.0.5 par (2017-08-09)
(complete)
done
438
UNSAT
2		OPT/SAT
145		SAT (TO/MO)
215		?
76
choco-solver
4.0.5 par (2017-08-18)
(complete)
done
438
UNSAT
2		OPT/SAT
143		SAT (TO/MO)
218		?
75
choco-solver
4.0.5 seq (2017-08-09)
(complete)
done
438
UNSAT
2		OPT/SAT
113		SAT (TO/MO)
187		?
135		ERR
1
choco-solver
4.0.5 seq (2017-08-18)
(complete)
done
438
UNSAT
2		OPT/SAT
113		SAT (TO/MO)
193		?
130
choco-solver
5a (2017-07-26)
(complete)
done
438
UNSAT
2		OPT/SAT
104		SAT (TO/MO)
197		?
134		ERR
1
choco-solver
5a (2017-08-18)
(complete)
done
438
UNSAT
2		OPT/SAT
105		SAT (TO/MO)
197		?
134
Concrete
3.4
(complete)
done
438
UNSAT
2		OPT/SAT
101		SAT (TO/MO)
200		?
135
cosoco
1.12
(complete)
done
438
UNSAT
2		OPT/SAT
96		SAT (TO/MO)
180		?
160
cosoco-sat
1.12
(complete)
done
438
UNSAT
2		OPT/SAT
96		SAT (TO/MO)
152		?
188
Mistral-2.0
2017-07-28
(complete)
done
438
UNSAT
2		OPT/SAT
131		SAT (TO/MO)
216		?
89
OscaR - ALNS
2017-07-26
(complete)
done
438
UNSAT
2		SAT (TO/MO)
335		?
101
OscaR - Conflict Ordering
2017-07-26
(complete)
done
438
UNSAT
2		OPT/SAT
123		SAT (TO/MO)
209		?
104
OscaR - Hybrid
2017-07-26
(complete)
done
438
UNSAT
2		OPT/SAT
107		SAT (TO/MO)
224		?
105
OscaR - Parallel with EPS
2017-07-26
(complete)
done
438
UNSAT
2		OPT/SAT
128		SAT (TO/MO)
200		?
108
OscaR - Parallel with EPS
2017-08-22
(complete)
done
438
UNSAT
2		OPT/SAT
133		SAT (TO/MO)
201		?
102
R1: AbsCon-basic
2017-06-11
(complete)
done
438
UNSAT
2		OPT/SAT
115		SAT (TO/MO)
210		?
111
R1: choco-solver
4.0.5 (2017-07-05)
(complete)
done
438
UNSAT
2		OPT/SAT
128		SAT (TO/MO)
211		?
85		ERR
12
R1: choco-solver
5a (2017-07-05)
(complete)
done
438
UNSAT
2		OPT/SAT
126		SAT (TO/MO)
212		?
86		ERR
12
R1: Concrete
3.4
(complete)
done
438
UNSAT
2		OPT/SAT
101		SAT (TO/MO)
199		?
136
R1: cosoco
1.1
(complete)
done
438
UNSAT
2		OPT/SAT
96		SAT (TO/MO)
11		?
329
R1: cosoco-sat
1.1
(complete)
done
438
UNSAT
2		OPT/SAT
96		SAT (TO/MO)
12		?
328
R1: Mistral-2.0
2017-06-28
(complete)
done
438
UNSAT
2		OPT/SAT
130		SAT (TO/MO)
216		?
90
R1: OscaR - ALNS
2017-06-30
(complete)
done
438
UNSAT
2		SAT (TO/MO)
336		?
100
R1: OscaR - Conflict Ordering
2017-06-30
(complete)
done
438
UNSAT
2		OPT/SAT
123		SAT (TO/MO)
209		?
104
R1: OscaR - Hybrid
2017-06-30
(complete)
done
438
UNSAT
2		OPT/SAT
107		SAT (TO/MO)
225		?
104
R1: sat4j-CSP
2017-07-05
(complete)
done
438
UNSAT
2		OPT/SAT
74		SAT (TO/MO)
91		?
271
sat4j-CSP
2017-07-05
(complete)
done
438
UNSAT
2		OPT/SAT
75		SAT (TO/MO)
87		?
274

Category "decision problem" (CSP)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
AbsCon-basic
2017-06-11
(complete)		 510368
(72.16%)		 102
(20%)		 266
(52.16%)		 0
(0%)		 142
(27.84%)		 0
(0%)
BTD
2017-07-25
(complete)		 510240
(47.06%)		 82
(16.08%)		 158
(30.98%)		 0
(0%)		 269
(52.75%)		 1
(0.2%)
BTD
2017-08-10
(complete)		 510241
(47.25%)		 82
(16.08%)		 159
(31.18%)		 0
(0%)		 269
(52.75%)		 0
(0%)
choco-solver
4.0.5 par (2017-08-09)
(complete)		 510393
(77.06%)		 111
(21.76%)		 282
(55.29%)		 0
(0%)		 115
(22.55%)		 2
(0.39%)
choco-solver
4.0.5 par (2017-08-18)
(complete)		 510396
(77.65%)		 109
(21.37%)		 287
(56.27%)		 0
(0%)		 114
(22.35%)		 0
(0%)
choco-solver
4.0.5 seq (2017-08-09)
(complete)		 510372
(72.94%)		 110
(21.57%)		 262
(51.37%)		 0
(0%)		 137
(26.86%)		 1
(0.2%)
choco-solver
4.0.5 seq (2017-08-18)
(complete)		 510372
(72.94%)		 108
(21.18%)		 264
(51.76%)		 0
(0%)		 138
(27.06%)		 0
(0%)
choco-solver
5a (2017-07-26)
(complete)		 510373
(73.14%)		 114
(22.35%)		 259
(50.78%)		 0
(0%)		 135
(26.47%)		 2
(0.39%)
choco-solver
5a (2017-08-18)
(complete)		 510371
(72.75%)		 109
(21.37%)		 262
(51.37%)		 0
(0%)		 139
(27.25%)		 0
(0%)
Concrete
3.4
(complete)		 510331
(64.9%)		 93
(18.24%)		 238
(46.67%)		 0
(0%)		 179
(35.1%)		 0
(0%)
cosoco
1.12
(complete)		 510335
(65.69%)		 97
(19.02%)		 238
(46.67%)		 0
(0%)		 175
(34.31%)		 0
(0%)
cosoco-sat
1.12
(complete)		 510322
(63.14%)		 94
(18.43%)		 228
(44.71%)		 0
(0%)		 187
(36.67%)		 1
(0.2%)
Mistral-2.0
2017-07-28
(complete)		 510365
(71.57%)		 91
(17.84%)		 274
(53.73%)		 0
(0%)		 145
(28.43%)		 0
(0%)
OscaR - Conflict Ordering
2017-07-26
(complete)		 510337
(66.08%)		 102
(20%)		 235
(46.08%)		 0
(0%)		 173
(33.92%)		 0
(0%)
OscaR - Parallel with EPS
2017-07-26
(complete)		 510335
(65.69%)		 101
(19.8%)		 234
(45.88%)		 0
(0%)		 175
(34.31%)		 0
(0%)
OscaR - Parallel with EPS
2017-08-22
(complete)		 510340
(66.67%)		 101
(19.8%)		 239
(46.86%)		 0
(0%)		 170
(33.33%)		 0
(0%)
R1: AbsCon-basic
2017-06-11
(complete)		 510368
(72.16%)		 102
(20%)		 266
(52.16%)		 0
(0%)		 142
(27.84%)		 0
(0%)
R1: BTD
2017-06-30
(complete)		 510239
(46.86%)		 82
(16.08%)		 157
(30.78%)		 0
(0%)		 269
(52.75%)		 2
(0.39%)
R1: choco-solver
4.0.5 (2017-07-05)
(complete)		 510374
(73.33%)		 103
(20.2%)		 271
(53.14%)		 0
(0%)		 134
(26.27%)		 2
(0.39%)
R1: choco-solver
5a (2017-07-05)
(complete)		 510374
(73.33%)		 103
(20.2%)		 271
(53.14%)		 0
(0%)		 135
(26.47%)		 1
(0.2%)
R1: Concrete
3.4
(complete)		 510330
(64.71%)		 93
(18.24%)		 237
(46.47%)		 0
(0%)		 180
(35.29%)		 0
(0%)
R1: cosoco
1.1
(complete)		 510334
(65.49%)		 97
(19.02%)		 237
(46.47%)		 0
(0%)		 175
(34.31%)		 1
(0.2%)
R1: cosoco-sat
1.1
(complete)		 510319
(62.55%)		 93
(18.24%)		 226
(44.31%)		 0
(0%)		 189
(37.06%)		 2
(0.39%)
R1: Mistral-2.0
2017-06-28
(complete)		 510365
(71.57%)		 91
(17.84%)		 274
(53.73%)		 0
(0%)		 143
(28.04%)		 2
(0.39%)
R1: OscaR - Conflict Ordering
2017-06-30
(complete)		 510335
(65.69%)		 102
(20%)		 233
(45.69%)		 0
(0%)		 175
(34.31%)		 0
(0%)
R1: sat4j-CSP
2017-07-05
(complete)		 510236
(46.27%)		 68
(13.33%)		 168
(32.94%)		 0
(0%)		 274
(53.73%)		 0
(0%)
sat4j-CSP
2017-07-05
(complete)		 510235
(46.08%)		 67
(13.14%)		 168
(32.94%)		 0
(0%)		 275
(53.92%)		 0
(0%)

Solver NameProgress
AbsCon-basic
2017-06-11
(complete)
done
510
UNSAT
102		OPT/SAT
266		?
142
BTD
2017-07-25
(complete)
done
510
UNSAT
82		OPT/SAT
158		?
269		ERR
1
BTD
2017-08-10
(complete)
done
510
UNSAT
82		OPT/SAT
159		?
269
choco-solver
4.0.5 par (2017-08-09)
(complete)
done
510
UNSAT
111		OPT/SAT
282		?
115		ERR
2
choco-solver
4.0.5 par (2017-08-18)
(complete)
done
510
UNSAT
109		OPT/SAT
287		?
114
choco-solver
4.0.5 seq (2017-08-09)
(complete)
done
510
UNSAT
110		OPT/SAT
262		?
137		ERR
1
choco-solver
4.0.5 seq (2017-08-18)
(complete)
done
510
UNSAT
108		OPT/SAT
264		?
138
choco-solver
5a (2017-07-26)
(complete)
done
510
UNSAT
114		OPT/SAT
259		?
135		ERR
2
choco-solver
5a (2017-08-18)
(complete)
done
510
UNSAT
109		OPT/SAT
262		?
139
Concrete
3.4
(complete)
done
510
UNSAT
93		OPT/SAT
238		?
179
cosoco
1.12
(complete)
done
510
UNSAT
97		OPT/SAT
238		?
175
cosoco-sat
1.12
(complete)
done
510
UNSAT
94		OPT/SAT
228		?
187		ERR
1
Mistral-2.0
2017-07-28
(complete)
done
510
UNSAT
91		OPT/SAT
274		?
145
OscaR - Conflict Ordering
2017-07-26
(complete)
done
510
UNSAT
102		OPT/SAT
235		?
173
OscaR - Parallel with EPS
2017-07-26
(complete)
done
510
UNSAT
101		OPT/SAT
234		?
175
OscaR - Parallel with EPS
2017-08-22
(complete)
done
510
UNSAT
101		OPT/SAT
239		?
170
R1: AbsCon-basic
2017-06-11
(complete)
done
510
UNSAT
102		OPT/SAT
266		?
142
R1: BTD
2017-06-30
(complete)
done
510
UNSAT
82		OPT/SAT
157		?
269		ERR
2
R1: choco-solver
4.0.5 (2017-07-05)
(complete)
done
510
UNSAT
103		OPT/SAT
271		?
134		ERR
2
R1: choco-solver
5a (2017-07-05)
(complete)
done
510
UNSAT
103		OPT/SAT
271		?
135		ERR
1
R1: Concrete
3.4
(complete)
done
510
UNSAT
93		OPT/SAT
237		?
180
R1: cosoco
1.1
(complete)
done
510
UNSAT
97		OPT/SAT
237		?
175		ERR
1
R1: cosoco-sat
1.1
(complete)
done
510
UNSAT
93		OPT/SAT
226		?
189		ERR
2
R1: Mistral-2.0
2017-06-28
(complete)
done
510
UNSAT
91		OPT/SAT
274		?
143		ERR
2
R1: OscaR - Conflict Ordering
2017-06-30
(complete)
done
510
UNSAT
102		OPT/SAT
233		?
175
R1: sat4j-CSP
2017-07-05
(complete)
done
510
UNSAT
68		OPT/SAT
168		?
274
sat4j-CSP
2017-07-05
(complete)
done
510
UNSAT
67		OPT/SAT
168		?
275

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
AbsCon-basic
2017-06-11
(complete)		 948485
(51.16%)		 104
(10.97%)		 381
(40.19%)		 210
(22.15%)		 253
(26.69%)		 0
(0%)
choco-solver
4.0.5 par (2017-08-09)
(complete)		 948540
(56.96%)		 113
(11.92%)		 427
(45.04%)		 215
(22.68%)		 191
(20.15%)		 2
(0.21%)
choco-solver
4.0.5 par (2017-08-18)
(complete)		 948541
(57.07%)		 111
(11.71%)		 430
(45.36%)		 218
(23%)		 189
(19.94%)		 0
(0%)
choco-solver
4.0.5 seq (2017-08-09)
(complete)		 948487
(51.37%)		 112
(11.81%)		 375
(39.56%)		 187
(19.73%)		 272
(28.69%)		 2
(0.21%)
choco-solver
4.0.5 seq (2017-08-18)
(complete)		 948487
(51.37%)		 110
(11.6%)		 377
(39.77%)		 193
(20.36%)		 268
(28.27%)		 0
(0%)
choco-solver
5a (2017-07-26)
(complete)		 948479
(50.53%)		 116
(12.24%)		 363
(38.29%)		 197
(20.78%)		 269
(28.38%)		 3
(0.32%)
choco-solver
5a (2017-08-18)
(complete)		 948478
(50.42%)		 111
(11.71%)		 367
(38.71%)		 197
(20.78%)		 273
(28.8%)		 0
(0%)
Concrete
3.4
(complete)		 948434
(45.78%)		 95
(10.02%)		 339
(35.76%)		 200
(21.1%)		 314
(33.12%)		 0
(0%)
cosoco
1.12
(complete)		 948433
(45.68%)		 99
(10.44%)		 334
(35.23%)		 180
(18.99%)		 335
(35.34%)		 0
(0%)
cosoco-sat
1.12
(complete)		 948420
(44.3%)		 96
(10.13%)		 324
(34.18%)		 152
(16.03%)		 375
(39.56%)		 1
(0.11%)
Mistral-2.0
2017-07-28
(complete)		 948498
(52.53%)		 93
(9.81%)		 405
(42.72%)		 216
(22.78%)		 234
(24.68%)		 0
(0%)
OscaR - Conflict Ordering
2017-07-26
(complete)		 948462
(48.73%)		 104
(10.97%)		 358
(37.76%)		 209
(22.05%)		 277
(29.22%)		 0
(0%)
OscaR - Parallel with EPS
2017-07-26
(complete)		 948465
(49.05%)		 103
(10.86%)		 362
(38.19%)		 200
(21.1%)		 283
(29.85%)		 0
(0%)
OscaR - Parallel with EPS
2017-08-22
(complete)		 948475
(50.11%)		 103
(10.86%)		 372
(39.24%)		 201
(21.2%)		 272
(28.69%)		 0
(0%)
R1: AbsCon-basic
2017-06-11
(complete)		 948485
(51.16%)		 104
(10.97%)		 381
(40.19%)		 210
(22.15%)		 253
(26.69%)		 0
(0%)
R1: choco-solver
4.0.5 (2017-07-05)
(complete)		 948504
(53.16%)		 105
(11.08%)		 399
(42.09%)		 211
(22.26%)		 219
(23.1%)		 14
(1.48%)
R1: choco-solver
5a (2017-07-05)
(complete)		 948502
(52.95%)		 105
(11.08%)		 397
(41.88%)		 212
(22.36%)		 221
(23.31%)		 13
(1.37%)
R1: Concrete
3.4
(complete)		 948433
(45.68%)		 95
(10.02%)		 338
(35.65%)		 199
(20.99%)		 316
(33.33%)		 0
(0%)
R1: cosoco
1.1
(complete)		 948432
(45.57%)		 99
(10.44%)		 333
(35.13%)		 11
(1.16%)		 504
(53.16%)		 1
(0.11%)
R1: cosoco-sat
1.1
(complete)		 948417
(43.99%)		 95
(10.02%)		 322
(33.97%)		 12
(1.27%)		 517
(54.54%)		 2
(0.21%)
R1: Mistral-2.0
2017-06-28
(complete)		 948497
(52.43%)		 93
(9.81%)		 404
(42.62%)		 216
(22.78%)		 233
(24.58%)		 2
(0.21%)
R1: OscaR - Conflict Ordering
2017-06-30
(complete)		 948460
(48.52%)		 104
(10.97%)		 356
(37.55%)		 209
(22.05%)		 279
(29.43%)		 0
(0%)
R1: sat4j-CSP
2017-07-05
(complete)		 948312
(32.91%)		 70
(7.38%)		 242
(25.53%)		 91
(9.6%)		 545
(57.49%)		 0
(0%)
sat4j-CSP
2017-07-05
(complete)		 948312
(32.91%)		 69
(7.28%)		 243
(25.63%)		 87
(9.18%)		 549
(57.91%)		 0
(0%)

Solver NameProgress
AbsCon-basic
2017-06-11
(complete)
done
948
UNSAT
104		OPT/SAT
381		SAT (TO/MO)
210		?
253
choco-solver
4.0.5 par (2017-08-09)
(complete)
done
948
UNSAT
113		OPT/SAT
427		SAT (TO/MO)
215		?
191		ERR
2
choco-solver
4.0.5 par (2017-08-18)
(complete)
done
948
UNSAT
111		OPT/SAT
430		SAT (TO/MO)
218		?
189
choco-solver
4.0.5 seq (2017-08-09)
(complete)
done
948
UNSAT
112		OPT/SAT
375		SAT (TO/MO)
187		?
272		ERR
2
choco-solver
4.0.5 seq (2017-08-18)
(complete)
done
948
UNSAT
110		OPT/SAT
377		SAT (TO/MO)
193		?
268
choco-solver
5a (2017-07-26)
(complete)
done
948
UNSAT
116		OPT/SAT
363		SAT (TO/MO)
197		?
269		ERR
3
choco-solver
5a (2017-08-18)
(complete)
done
948
UNSAT
111		OPT/SAT
367		SAT (TO/MO)
197		?
273
Concrete
3.4
(complete)
done
948
UNSAT
95		OPT/SAT
339		SAT (TO/MO)
200		?
314
cosoco
1.12
(complete)
done
948
UNSAT
99		OPT/SAT
334		SAT (TO/MO)
180		?
335
cosoco-sat
1.12
(complete)
done
948
UNSAT
96		OPT/SAT
324		SAT (TO/MO)
152		?
375		ERR
1
Mistral-2.0
2017-07-28
(complete)
done
948
UNSAT
93		OPT/SAT
405		SAT (TO/MO)
216		?
234
OscaR - Conflict Ordering
2017-07-26
(complete)
done
948
UNSAT
104		OPT/SAT
358		SAT (TO/MO)
209		?
277
OscaR - Parallel with EPS
2017-07-26
(complete)
done
948
UNSAT
103		OPT/SAT
362		SAT (TO/MO)
200		?
283
OscaR - Parallel with EPS
2017-08-22
(complete)
done
948
UNSAT
103		OPT/SAT
372		SAT (TO/MO)
201		?
272
R1: AbsCon-basic
2017-06-11
(complete)
done
948
UNSAT
104		OPT/SAT
381		SAT (TO/MO)
210		?
253
R1: choco-solver
4.0.5 (2017-07-05)
(complete)
done
948
UNSAT
105		OPT/SAT
399		SAT (TO/MO)
211		?
219		ERR
14
R1: choco-solver
5a (2017-07-05)
(complete)
done
948
UNSAT
105		OPT/SAT
397		SAT (TO/MO)
212		?
221		ERR
13
R1: Concrete
3.4
(complete)
done
948
UNSAT
95		OPT/SAT
338		SAT (TO/MO)
199		?
316
R1: cosoco
1.1
(complete)
done
948
UNSAT
99		OPT/SAT
333		SAT (TO/MO)
11		?
504		ERR
1
R1: cosoco-sat
1.1
(complete)
done
948
UNSAT
95		OPT/SAT
322		SAT (TO/MO)
12		?
517		ERR
2
R1: Mistral-2.0
2017-06-28
(complete)
done
948
UNSAT
93		OPT/SAT
404		SAT (TO/MO)
216		?
233		ERR
2
R1: OscaR - Conflict Ordering
2017-06-30
(complete)
done
948
UNSAT
104		OPT/SAT
356		SAT (TO/MO)
209		?
279
R1: sat4j-CSP
2017-07-05
(complete)
done
948
UNSAT
70		OPT/SAT
242		SAT (TO/MO)
91		?
545
sat4j-CSP
2017-07-05
(complete)
done
948
UNSAT
69		OPT/SAT
243		SAT (TO/MO)
87		?
549

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
AbsCon-basic
2017-06-11
(complete)		 948485
(51.16%)		 104
(10.97%)		 381
(40.19%)		 210
(22.15%)		 253
(26.69%)		 0
(0%)
BTD
2017-07-25
(complete)		 510240
(47.06%)		 82
(16.08%)		 158
(30.98%)		 0
(0%)		 269
(52.75%)		 1
(0.2%)
BTD
2017-08-10
(complete)		 510241
(47.25%)		 82
(16.08%)		 159
(31.18%)		 0
(0%)		 269
(52.75%)		 0
(0%)
choco-solver
4.0.5 par (2017-08-09)
(complete)		 948540
(56.96%)		 113
(11.92%)		 427
(45.04%)		 215
(22.68%)		 191
(20.15%)		 2
(0.21%)
choco-solver
4.0.5 par (2017-08-18)
(complete)		 948541
(57.07%)		 111
(11.71%)		 430
(45.36%)		 218
(23%)		 189
(19.94%)		 0
(0%)
choco-solver
4.0.5 seq (2017-08-09)
(complete)		 948487
(51.37%)		 112
(11.81%)		 375
(39.56%)		 187
(19.73%)		 272
(28.69%)		 2
(0.21%)
choco-solver
4.0.5 seq (2017-08-18)
(complete)		 948487
(51.37%)		 110
(11.6%)		 377
(39.77%)		 193
(20.36%)		 268
(28.27%)		 0
(0%)
choco-solver
5a (2017-07-26)
(complete)		 948479
(50.53%)		 116
(12.24%)		 363
(38.29%)		 197
(20.78%)		 269
(28.38%)		 3
(0.32%)
choco-solver
5a (2017-08-18)
(complete)		 948478
(50.42%)		 111
(11.71%)		 367
(38.71%)		 197
(20.78%)		 273
(28.8%)		 0
(0%)
Concrete
3.4
(complete)		 948434
(45.78%)		 95
(10.02%)		 339
(35.76%)		 200
(21.1%)		 314
(33.12%)		 0
(0%)
cosoco
1.12
(complete)		 948433
(45.68%)		 99
(10.44%)		 334
(35.23%)		 180
(18.99%)		 335
(35.34%)		 0
(0%)
cosoco-sat
1.12
(complete)		 948420
(44.3%)		 96
(10.13%)		 324
(34.18%)		 152
(16.03%)		 375
(39.56%)		 1
(0.11%)
Mistral-2.0
2017-07-28
(complete)		 948498
(52.53%)		 93
(9.81%)		 405
(42.72%)		 216
(22.78%)		 234
(24.68%)		 0
(0%)
OscaR - ALNS
2017-07-26
(complete)		 4382
(0.46%)		 2
(0.46%)		 0
(0%)		 335
(76.48%)		 101
(23.06%)		 0
(0%)
OscaR - Conflict Ordering
2017-07-26
(complete)		 948462
(48.73%)		 104
(10.97%)		 358
(37.76%)		 209
(22.05%)		 277
(29.22%)		 0
(0%)
OscaR - Hybrid
2017-07-26
(complete)		 438109
(24.89%)		 2
(0.46%)		 107
(24.43%)		 224
(51.14%)		 105
(23.97%)		 0
(0%)
OscaR - Parallel with EPS
2017-07-26
(complete)		 948465
(49.05%)		 103
(10.86%)		 362
(38.19%)		 200
(21.1%)		 283
(29.85%)		 0
(0%)
OscaR - Parallel with EPS
2017-08-22
(complete)		 948475
(50.11%)		 103
(10.86%)		 372
(39.24%)		 201
(21.2%)		 272
(28.69%)		 0
(0%)
R1: AbsCon-basic
2017-06-11
(complete)		 948485
(51.16%)		 104
(10.97%)		 381
(40.19%)		 210
(22.15%)		 253
(26.69%)		 0
(0%)
R1: BTD
2017-06-30
(complete)		 510239
(46.86%)		 82
(16.08%)		 157
(30.78%)		 0
(0%)		 269
(52.75%)		 2
(0.39%)
R1: choco-solver
4.0.5 (2017-07-05)
(complete)		 948504
(53.16%)		 105
(11.08%)		 399
(42.09%)		 211
(22.26%)		 219
(23.1%)		 14
(1.48%)
R1: choco-solver
5a (2017-07-05)
(complete)		 948502
(52.95%)		 105
(11.08%)		 397
(41.88%)		 212
(22.36%)		 221
(23.31%)		 13
(1.37%)
R1: Concrete
3.4
(complete)		 948433
(45.68%)		 95
(10.02%)		 338
(35.65%)		 199
(20.99%)		 316
(33.33%)		 0
(0%)
R1: cosoco
1.1
(complete)		 948432
(45.57%)		 99
(10.44%)		 333
(35.13%)		 11
(1.16%)		 504
(53.16%)		 1
(0.11%)
R1: cosoco-sat
1.1
(complete)		 948417
(43.99%)		 95
(10.02%)		 322
(33.97%)		 12
(1.27%)		 517
(54.54%)		 2
(0.21%)
R1: Mistral-2.0
2017-06-28
(complete)		 948497
(52.43%)		 93
(9.81%)		 404
(42.62%)		 216
(22.78%)		 233
(24.58%)		 2
(0.21%)
R1: OscaR - ALNS
2017-06-30
(complete)		 4382
(0.46%)		 2
(0.46%)		 0
(0%)		 336
(76.71%)		 100
(22.83%)		 0
(0%)
R1: OscaR - Conflict Ordering
2017-06-30
(complete)		 948460
(48.52%)		 104
(10.97%)		 356
(37.55%)		 209
(22.05%)		 279
(29.43%)		 0
(0%)
R1: OscaR - Hybrid
2017-06-30
(complete)		 438109
(24.89%)		 2
(0.46%)		 107
(24.43%)		 225
(51.37%)		 104
(23.74%)		 0
(0%)
R1: sat4j-CSP
2017-07-05
(complete)		 948312
(32.91%)		 70
(7.38%)		 242
(25.53%)		 91
(9.6%)		 545
(57.49%)		 0
(0%)
sat4j-CSP
2017-07-05
(complete)		 948312
(32.91%)		 69
(7.28%)		 243
(25.63%)		 87
(9.18%)		 549
(57.91%)		 0
(0%)

Solver NameProgress
AbsCon-basic
2017-06-11
(complete)
done
948
UNSAT
104		OPT/SAT
381		SAT (TO/MO)
210		?
253
BTD
2017-07-25
(complete)
done
510
UNSAT
82		OPT/SAT
158		?
269		ERR
1
BTD
2017-08-10
(complete)
done
510
UNSAT
82		OPT/SAT
159		?
269
choco-solver
4.0.5 par (2017-08-09)
(complete)
done
948
UNSAT
113		OPT/SAT
427		SAT (TO/MO)
215		?
191		ERR
2
choco-solver
4.0.5 par (2017-08-18)
(complete)
done
948
UNSAT
111		OPT/SAT
430		SAT (TO/MO)
218		?
189
choco-solver
4.0.5 seq (2017-08-09)
(complete)
done
948
UNSAT
112		OPT/SAT
375		SAT (TO/MO)
187		?
272		ERR
2
choco-solver
4.0.5 seq (2017-08-18)
(complete)
done
948
UNSAT
110		OPT/SAT
377		SAT (TO/MO)
193		?
268
choco-solver
5a (2017-07-26)
(complete)
done
948
UNSAT
116		OPT/SAT
363		SAT (TO/MO)
197		?
269		ERR
3
choco-solver
5a (2017-08-18)
(complete)
done
948
UNSAT
111		OPT/SAT
367		SAT (TO/MO)
197		?
273
Concrete
3.4
(complete)
done
948
UNSAT
95		OPT/SAT
339		SAT (TO/MO)
200		?
314
cosoco
1.12
(complete)
done
948
UNSAT
99		OPT/SAT
334		SAT (TO/MO)
180		?
335
cosoco-sat
1.12
(complete)
done
948
UNSAT
96		OPT/SAT
324		SAT (TO/MO)
152		?
375		ERR
1
Mistral-2.0
2017-07-28
(complete)
done
948
UNSAT
93		OPT/SAT
405		SAT (TO/MO)
216		?
234
OscaR - ALNS
2017-07-26
(complete)
done
438
UNSAT
2		SAT (TO/MO)
335		?
101
OscaR - Conflict Ordering
2017-07-26
(complete)
done
948
UNSAT
104		OPT/SAT
358		SAT (TO/MO)
209		?
277
OscaR - Hybrid
2017-07-26
(complete)
done
438
UNSAT
2		OPT/SAT
107		SAT (TO/MO)
224		?
105
OscaR - Parallel with EPS
2017-07-26
(complete)
done
948
UNSAT
103		OPT/SAT
362		SAT (TO/MO)
200		?
283
OscaR - Parallel with EPS
2017-08-22
(complete)
done
948
UNSAT
103		OPT/SAT
372		SAT (TO/MO)
201		?
272
R1: AbsCon-basic
2017-06-11
(complete)
done
948
UNSAT
104		OPT/SAT
381		SAT (TO/MO)
210		?
253
R1: BTD
2017-06-30
(complete)
done
510
UNSAT
82		OPT/SAT
157		?
269		ERR
2
R1: choco-solver
4.0.5 (2017-07-05)
(complete)
done
948
UNSAT
105		OPT/SAT
399		SAT (TO/MO)
211		?
219		ERR
14
R1: choco-solver
5a (2017-07-05)
(complete)
done
948
UNSAT
105		OPT/SAT
397		SAT (TO/MO)
212		?
221		ERR
13
R1: Concrete
3.4
(complete)
done
948
UNSAT
95		OPT/SAT
338		SAT (TO/MO)
199		?
316
R1: cosoco
1.1
(complete)
done
948
UNSAT
99		OPT/SAT
333		SAT (TO/MO)
11		?
504		ERR
1
R1: cosoco-sat
1.1
(complete)
done
948
UNSAT
95		OPT/SAT
322		SAT (TO/MO)
12		?
517		ERR
2
R1: Mistral-2.0
2017-06-28
(complete)
done
948
UNSAT
93		OPT/SAT
404		SAT (TO/MO)
216		?
233		ERR
2
R1: OscaR - ALNS
2017-06-30
(complete)
done
438
UNSAT
2		SAT (TO/MO)
336		?
100
R1: OscaR - Conflict Ordering
2017-06-30
(complete)
done
948
UNSAT
104		OPT/SAT
356		SAT (TO/MO)
209		?
279
R1: OscaR - Hybrid
2017-06-30
(complete)
done
438
UNSAT
2		OPT/SAT
107		SAT (TO/MO)
225		?
104
R1: sat4j-CSP
2017-07-05
(complete)
done
948
UNSAT
70		OPT/SAT
242		SAT (TO/MO)
91		?
545
sat4j-CSP
2017-07-05
(complete)
done
948
UNSAT
69		OPT/SAT
243		SAT (TO/MO)
87		?
549