2018 XCSP3 competition: sequential solvers tracks: solvers page (tabular format, summary)

Some explanations

Category "optimization problem" (COP)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
Choco-solver
4.0.7 seq (493a269)
(complete)		 34699
(28.61%)		 0
(0%)		 99
(28.61%)		 201
(58.09%)		 37
(10.69%)		 9
(2.6%)
Choco-solver
4.0.7b seq (e747e1e)
(complete)		 346102
(29.48%)		 0
(0%)		 102
(29.48%)		 206
(59.54%)		 38
(10.98%)		 0
(0%)
Concrete
3.9.2
(complete)		 346105
(30.35%)		 0
(0%)		 105
(30.35%)		 193
(55.78%)		 48
(13.87%)		 0
(0%)
Concrete
3.9.2-SuperNG
(complete)		 34699
(28.61%)		 0
(0%)		 99
(28.61%)		 198
(57.23%)		 49
(14.16%)		 0
(0%)
Concrete 3.8
2018-06-13
(complete)		 346103
(29.77%)		 0
(0%)		 103
(29.77%)		 189
(54.62%)		 50
(14.45%)		 4
(1.16%)
Concrete 3.8-SuperNG
2018-06-13
(complete)		 34699
(28.61%)		 0
(0%)		 99
(28.61%)		 190
(54.91%)		 53
(15.32%)		 4
(1.16%)
cosoco
1.12
(complete)		 34664
(18.5%)		 0
(0%)		 64
(18.5%)		 194
(56.07%)		 88
(25.43%)		 0
(0%)
Mistral-2.0
2018-06-15
(complete)		 346105
(30.35%)		 0
(0%)		 105
(30.35%)		 169
(48.84%)		 60
(17.34%)		 12
(3.47%)
Mistral-2.0
2018-08-01
(complete)		 346106
(30.64%)		 0
(0%)		 106
(30.64%)		 178
(51.45%)		 61
(17.63%)		 1
(0.29%)
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)		 3460
(0%)		 0
(0%)		 0
(0%)		 271
(78.32%)		 69
(19.94%)		 6
(1.73%)
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)		 3460
(0%)		 0
(0%)		 0
(0%)		 281
(81.21%)		 65
(18.79%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)		 34699
(28.61%)		 0
(0%)		 99
(28.61%)		 178
(51.45%)		 69
(19.94%)		 0
(0%)
OscaR - Hybrid
2018-07-02
(complete)		 34663
(18.21%)		 0
(0%)		 63
(18.21%)		 199
(57.51%)		 66
(19.08%)		 18
(5.2%)
OscaR - Hybrid
2018-08-14
(complete)		 34661
(17.63%)		 0
(0%)		 61
(17.63%)		 215
(62.14%)		 70
(20.23%)		 0
(0%)
PicatSAT
2018-06-15
(complete)		 346130
(37.57%)		 0
(0%)		 130
(37.57%)		 0
(0%)		 214
(61.85%)		 2
(0.58%)
PicatSAT
2018-08-02
(complete)		 346132
(38.15%)		 0
(0%)		 132
(38.15%)		 0
(0%)		 213
(61.56%)		 1
(0.29%)
PicatSAT
2018-08-14
(complete)		 346132
(38.15%)		 0
(0%)		 132
(38.15%)		 0
(0%)		 214
(61.85%)		 0
(0%)
Sat4j-CSP
2018-07-11
(complete)		 34654
(15.61%)		 0
(0%)		 54
(15.61%)		 111
(32.08%)		 181
(52.31%)		 0
(0%)

Solver NameProgress
Choco-solver
4.0.7 seq (493a269)
(complete)
done
346
OPT/SAT
99		SAT (TO/MO)
201		?
37		ERR
9
Choco-solver
4.0.7b seq (e747e1e)
(complete)
done
346
OPT/SAT
102		SAT (TO/MO)
206		?
38
Concrete
3.9.2
(complete)
done
346
OPT/SAT
105		SAT (TO/MO)
193		?
48
Concrete
3.9.2-SuperNG
(complete)
done
346
OPT/SAT
99		SAT (TO/MO)
198		?
49
Concrete 3.8
2018-06-13
(complete)
done
346
OPT/SAT
103		SAT (TO/MO)
189		?
50		ERR
4
Concrete 3.8-SuperNG
2018-06-13
(complete)
done
346
OPT/SAT
99		SAT (TO/MO)
190		?
53		ERR
4
cosoco
1.12
(complete)
done
346
OPT/SAT
64		SAT (TO/MO)
194		?
88
Mistral-2.0
2018-06-15
(complete)
done
346
OPT/SAT
105		SAT (TO/MO)
169		?
60		ERR
12
Mistral-2.0
2018-08-01
(complete)
done
346
OPT/SAT
106		SAT (TO/MO)
178		?
61		ERR
1
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)
done
346
SAT (TO/MO)
271		?
69		ERR
6
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)
done
346
SAT (TO/MO)
281		?
65
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)
done
346
OPT/SAT
99		SAT (TO/MO)
178		?
69
OscaR - Hybrid
2018-07-02
(complete)
done
346
OPT/SAT
63		SAT (TO/MO)
199		?
66		ERR
18
OscaR - Hybrid
2018-08-14
(complete)
done
346
OPT/SAT
61		SAT (TO/MO)
215		?
70
PicatSAT
2018-06-15
(complete)
done
346
OPT/SAT
130		?
214		ERR
2
PicatSAT
2018-08-02
(complete)
done
346
OPT/SAT
132		?
213		ERR
1
PicatSAT
2018-08-14
(complete)
done
346
OPT/SAT
132		?
214
Sat4j-CSP
2018-07-11
(complete)
done
346
OPT/SAT
54		SAT (TO/MO)
111		?
181

Category "decision problem" (CSP)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
BTD
2018.06.11_3
(complete)		 23680
(33.9%)		 50
(21.19%)		 30
(12.71%)		 0
(0%)		 155
(65.68%)		 1
(0.42%)
BTD
2018.07.27_3
(complete)		 23676
(32.2%)		 45
(19.07%)		 31
(13.14%)		 0
(0%)		 160
(67.8%)		 0
(0%)
BTD_12
2018-06-11_12
(complete)		 23680
(33.9%)		 49
(20.76%)		 31
(13.14%)		 0
(0%)		 155
(65.68%)		 1
(0.42%)
BTD_12
2018.07.27_12
(complete)		 23677
(32.63%)		 45
(19.07%)		 32
(13.56%)		 0
(0%)		 159
(67.37%)		 0
(0%)
Choco-solver
4.0.7 seq (493a269)
(complete)		 236115
(48.73%)		 38
(16.1%)		 77
(32.63%)		 0
(0%)		 121
(51.27%)		 0
(0%)
Choco-solver
4.0.7b seq (e747e1e)
(complete)		 236115
(48.73%)		 38
(16.1%)		 77
(32.63%)		 0
(0%)		 121
(51.27%)		 0
(0%)
Concrete
3.9.2
(complete)		 23692
(38.98%)		 28
(11.86%)		 64
(27.12%)		 0
(0%)		 144
(61.02%)		 0
(0%)
Concrete
3.9.2-SuperNG
(complete)		 23684
(35.59%)		 29
(12.29%)		 55
(23.31%)		 0
(0%)		 152
(64.41%)		 0
(0%)
Concrete 3.8
2018-06-13
(complete)		 23695
(40.25%)		 30
(12.71%)		 65
(27.54%)		 0
(0%)		 141
(59.75%)		 0
(0%)
Concrete 3.8-SuperNG
2018-06-13
(complete)		 23686
(36.44%)		 28
(11.86%)		 58
(24.58%)		 0
(0%)		 150
(63.56%)		 0
(0%)
cosoco
1.12
(complete)		 23679
(33.47%)		 26
(11.02%)		 53
(22.46%)		 0
(0%)		 157
(66.53%)		 0
(0%)
macht
2018.06.11
(complete)		 23670
(29.66%)		 38
(16.1%)		 32
(13.56%)		 0
(0%)		 165
(69.92%)		 1
(0.42%)
macht
2018.07.27
(complete)		 23666
(27.97%)		 33
(13.98%)		 33
(13.98%)		 0
(0%)		 170
(72.03%)		 0
(0%)
Mistral-2.0
2018-06-15
(complete)		 236115
(48.73%)		 37
(15.68%)		 78
(33.05%)		 0
(0%)		 121
(51.27%)		 0
(0%)
Mistral-2.0
2018-08-01
(complete)		 236116
(49.15%)		 36
(15.25%)		 80
(33.9%)		 0
(0%)		 120
(50.85%)		 0
(0%)
OscaR - Conflict Ordering
2018-07-02
(complete)		 23681
(34.32%)		 30
(12.71%)		 51
(21.61%)		 0
(0%)		 155
(65.68%)		 0
(0%)
OscaR - Conflict Ordering
2018-08-14
(complete)		 23682
(34.75%)		 30
(12.71%)		 52
(22.03%)		 0
(0%)		 154
(65.25%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)		 23660
(25.42%)		 0
(0%)		 60
(25.42%)		 0
(0%)		 176
(74.58%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)		 23660
(25.42%)		 0
(0%)		 60
(25.42%)		 0
(0%)		 176
(74.58%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)		 23690
(38.14%)		 28
(11.86%)		 62
(26.27%)		 0
(0%)		 146
(61.86%)		 0
(0%)
PicatSAT
2018-06-15
(complete)		 236137
(58.05%)		 53
(22.46%)		 84
(35.59%)		 0
(0%)		 99
(41.95%)		 0
(0%)
PicatSAT
2018-08-02
(complete)		 236137
(58.05%)		 53
(22.46%)		 84
(35.59%)		 0
(0%)		 99
(41.95%)		 0
(0%)
PicatSAT
2018-08-14
(complete)		 236138
(58.47%)		 53
(22.46%)		 85
(36.02%)		 0
(0%)		 98
(41.53%)		 0
(0%)
Sat4j-CSP
2018-07-11
(complete)		 23683
(35.17%)		 43
(18.22%)		 40
(16.95%)		 0
(0%)		 153
(64.83%)		 0
(0%)
scop
both+MapleCOMSPS (2018-06-13)
(complete)		 236123
(52.12%)		 52
(22.03%)		 71
(30.08%)		 0
(0%)		 113
(47.88%)		 0
(0%)
scop
both+MapleCOMSPS (2018-07-31)
(complete)		 236141
(59.75%)		 53
(22.46%)		 88
(37.29%)		 0
(0%)		 95
(40.25%)		 0
(0%)
scop
order+MapleCOMSPS (2018-06-13)
(complete)		 236123
(52.12%)		 53
(22.46%)		 70
(29.66%)		 0
(0%)		 113
(47.88%)		 0
(0%)
scop
order+MapleCOMSPS (2018-07-31)
(complete)		 236146
(61.86%)		 54
(22.88%)		 92
(38.98%)		 0
(0%)		 90
(38.14%)		 0
(0%)

Solver NameProgress
BTD
2018.06.11_3
(complete)
done
236
UNSAT
50		OPT/SAT
30		?
155		ERR
1
BTD
2018.07.27_3
(complete)
done
236
UNSAT
45		OPT/SAT
31		?
160
BTD_12
2018-06-11_12
(complete)
done
236
UNSAT
49		OPT/SAT
31		?
155		ERR
1
BTD_12
2018.07.27_12
(complete)
done
236
UNSAT
45		OPT/SAT
32		?
159
Choco-solver
4.0.7 seq (493a269)
(complete)
done
236
UNSAT
38		OPT/SAT
77		?
121
Choco-solver
4.0.7b seq (e747e1e)
(complete)
done
236
UNSAT
38		OPT/SAT
77		?
121
Concrete
3.9.2
(complete)
done
236
UNSAT
28		OPT/SAT
64		?
144
Concrete
3.9.2-SuperNG
(complete)
done
236
UNSAT
29		OPT/SAT
55		?
152
Concrete 3.8
2018-06-13
(complete)
done
236
UNSAT
30		OPT/SAT
65		?
141
Concrete 3.8-SuperNG
2018-06-13
(complete)
done
236
UNSAT
28		OPT/SAT
58		?
150
cosoco
1.12
(complete)
done
236
UNSAT
26		OPT/SAT
53		?
157
macht
2018.06.11
(complete)
done
236
UNSAT
38		OPT/SAT
32		?
165		ERR
1
macht
2018.07.27
(complete)
done
236
UNSAT
33		OPT/SAT
33		?
170
Mistral-2.0
2018-06-15
(complete)
done
236
UNSAT
37		OPT/SAT
78		?
121
Mistral-2.0
2018-08-01
(complete)
done
236
UNSAT
36		OPT/SAT
80		?
120
OscaR - Conflict Ordering
2018-07-02
(complete)
done
236
UNSAT
30		OPT/SAT
51		?
155
OscaR - Conflict Ordering
2018-08-14
(complete)
done
236
UNSAT
30		OPT/SAT
52		?
154
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)
done
236
OPT/SAT
60		?
176
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)
done
236
OPT/SAT
60		?
176
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)
done
236
UNSAT
28		OPT/SAT
62		?
146
PicatSAT
2018-06-15
(complete)
done
236
UNSAT
53		OPT/SAT
84		?
99
PicatSAT
2018-08-02
(complete)
done
236
UNSAT
53		OPT/SAT
84		?
99
PicatSAT
2018-08-14
(complete)
done
236
UNSAT
53		OPT/SAT
85		?
98
Sat4j-CSP
2018-07-11
(complete)
done
236
UNSAT
43		OPT/SAT
40		?
153
scop
both+MapleCOMSPS (2018-06-13)
(complete)
done
236
UNSAT
52		OPT/SAT
71		?
113
scop
both+MapleCOMSPS (2018-07-31)
(complete)
done
236
UNSAT
53		OPT/SAT
88		?
95
scop
order+MapleCOMSPS (2018-06-13)
(complete)
done
236
UNSAT
53		OPT/SAT
70		?
113
scop
order+MapleCOMSPS (2018-07-31)
(complete)
done
236
UNSAT
54		OPT/SAT
92		?
90

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
Choco-solver
4.0.7 seq (493a269)
(complete)		 582214
(36.77%)		 38
(6.53%)		 176
(30.24%)		 201
(34.54%)		 158
(27.15%)		 9
(1.55%)
Choco-solver
4.0.7b seq (e747e1e)
(complete)		 582217
(37.29%)		 38
(6.53%)		 179
(30.76%)		 206
(35.4%)		 159
(27.32%)		 0
(0%)
Concrete
3.9.2
(complete)		 582197
(33.85%)		 28
(4.81%)		 169
(29.04%)		 193
(33.16%)		 192
(32.99%)		 0
(0%)
Concrete
3.9.2-SuperNG
(complete)		 582183
(31.44%)		 29
(4.98%)		 154
(26.46%)		 198
(34.02%)		 201
(34.54%)		 0
(0%)
Concrete 3.8
2018-06-13
(complete)		 582198
(34.02%)		 30
(5.15%)		 168
(28.87%)		 189
(32.47%)		 191
(32.82%)		 4
(0.69%)
Concrete 3.8-SuperNG
2018-06-13
(complete)		 582185
(31.79%)		 28
(4.81%)		 157
(26.98%)		 190
(32.65%)		 203
(34.88%)		 4
(0.69%)
cosoco
1.12
(complete)		 582143
(24.57%)		 26
(4.47%)		 117
(20.1%)		 194
(33.33%)		 245
(42.1%)		 0
(0%)
Mistral-2.0
2018-06-15
(complete)		 582220
(37.8%)		 37
(6.36%)		 183
(31.44%)		 169
(29.04%)		 181
(31.1%)		 12
(2.06%)
Mistral-2.0
2018-08-01
(complete)		 582222
(38.14%)		 36
(6.19%)		 186
(31.96%)		 178
(30.58%)		 181
(31.1%)		 1
(0.17%)
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)		 58260
(10.31%)		 0
(0%)		 60
(10.31%)		 271
(46.56%)		 245
(42.1%)		 6
(1.03%)
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)		 58260
(10.31%)		 0
(0%)		 60
(10.31%)		 281
(48.28%)		 241
(41.41%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)		 582189
(32.47%)		 28
(4.81%)		 161
(27.66%)		 178
(30.58%)		 215
(36.94%)		 0
(0%)
PicatSAT
2018-06-15
(complete)		 582267
(45.88%)		 53
(9.11%)		 214
(36.77%)		 0
(0%)		 313
(53.78%)		 2
(0.34%)
PicatSAT
2018-08-02
(complete)		 582269
(46.22%)		 53
(9.11%)		 216
(37.11%)		 0
(0%)		 312
(53.61%)		 1
(0.17%)
PicatSAT
2018-08-14
(complete)		 582270
(46.39%)		 53
(9.11%)		 217
(37.29%)		 0
(0%)		 312
(53.61%)		 0
(0%)
Sat4j-CSP
2018-07-11
(complete)		 582137
(23.54%)		 43
(7.39%)		 94
(16.15%)		 111
(19.07%)		 334
(57.39%)		 0
(0%)

Solver NameProgress
Choco-solver
4.0.7 seq (493a269)
(complete)
done
582
UNSAT
38		OPT/SAT
176		SAT (TO/MO)
201		?
158		ERR
9
Choco-solver
4.0.7b seq (e747e1e)
(complete)
done
582
UNSAT
38		OPT/SAT
179		SAT (TO/MO)
206		?
159
Concrete
3.9.2
(complete)
done
582
UNSAT
28		OPT/SAT
169		SAT (TO/MO)
193		?
192
Concrete
3.9.2-SuperNG
(complete)
done
582
UNSAT
29		OPT/SAT
154		SAT (TO/MO)
198		?
201
Concrete 3.8
2018-06-13
(complete)
done
582
UNSAT
30		OPT/SAT
168		SAT (TO/MO)
189		?
191		ERR
4
Concrete 3.8-SuperNG
2018-06-13
(complete)
done
582
UNSAT
28		OPT/SAT
157		SAT (TO/MO)
190		?
203		ERR
4
cosoco
1.12
(complete)
done
582
UNSAT
26		OPT/SAT
117		SAT (TO/MO)
194		?
245
Mistral-2.0
2018-06-15
(complete)
done
582
UNSAT
37		OPT/SAT
183		SAT (TO/MO)
169		?
181		ERR
12
Mistral-2.0
2018-08-01
(complete)
done
582
UNSAT
36		OPT/SAT
186		SAT (TO/MO)
178		?
181		ERR
1
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)
done
582
OPT/SAT
60		SAT (TO/MO)
271		?
245		ERR
6
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)
done
582
OPT/SAT
60		SAT (TO/MO)
281		?
241
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)
done
582
UNSAT
28		OPT/SAT
161		SAT (TO/MO)
178		?
215
PicatSAT
2018-06-15
(complete)
done
582
UNSAT
53		OPT/SAT
214		?
313		ERR
2
PicatSAT
2018-08-02
(complete)
done
582
UNSAT
53		OPT/SAT
216		?
312		ERR
1
PicatSAT
2018-08-14
(complete)
done
582
UNSAT
53		OPT/SAT
217		?
312
Sat4j-CSP
2018-07-11
(complete)
done
582
UNSAT
43		OPT/SAT
94		SAT (TO/MO)
111		?
334

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
BTD
2018.06.11_3
(complete)		 23680
(33.9%)		 50
(21.19%)		 30
(12.71%)		 0
(0%)		 155
(65.68%)		 1
(0.42%)
BTD
2018.07.27_3
(complete)		 23676
(32.2%)		 45
(19.07%)		 31
(13.14%)		 0
(0%)		 160
(67.8%)		 0
(0%)
BTD_12
2018-06-11_12
(complete)		 23680
(33.9%)		 49
(20.76%)		 31
(13.14%)		 0
(0%)		 155
(65.68%)		 1
(0.42%)
BTD_12
2018.07.27_12
(complete)		 23677
(32.63%)		 45
(19.07%)		 32
(13.56%)		 0
(0%)		 159
(67.37%)		 0
(0%)
Choco-solver
4.0.7 seq (493a269)
(complete)		 582214
(36.77%)		 38
(6.53%)		 176
(30.24%)		 201
(34.54%)		 158
(27.15%)		 9
(1.55%)
Choco-solver
4.0.7b seq (e747e1e)
(complete)		 582217
(37.29%)		 38
(6.53%)		 179
(30.76%)		 206
(35.4%)		 159
(27.32%)		 0
(0%)
Concrete
3.9.2
(complete)		 582197
(33.85%)		 28
(4.81%)		 169
(29.04%)		 193
(33.16%)		 192
(32.99%)		 0
(0%)
Concrete
3.9.2-SuperNG
(complete)		 582183
(31.44%)		 29
(4.98%)		 154
(26.46%)		 198
(34.02%)		 201
(34.54%)		 0
(0%)
Concrete 3.8
2018-06-13
(complete)		 582198
(34.02%)		 30
(5.15%)		 168
(28.87%)		 189
(32.47%)		 191
(32.82%)		 4
(0.69%)
Concrete 3.8-SuperNG
2018-06-13
(complete)		 582185
(31.79%)		 28
(4.81%)		 157
(26.98%)		 190
(32.65%)		 203
(34.88%)		 4
(0.69%)
cosoco
1.12
(complete)		 582143
(24.57%)		 26
(4.47%)		 117
(20.1%)		 194
(33.33%)		 245
(42.1%)		 0
(0%)
macht
2018.06.11
(complete)		 23670
(29.66%)		 38
(16.1%)		 32
(13.56%)		 0
(0%)		 165
(69.92%)		 1
(0.42%)
macht
2018.07.27
(complete)		 23666
(27.97%)		 33
(13.98%)		 33
(13.98%)		 0
(0%)		 170
(72.03%)		 0
(0%)
Mistral-2.0
2018-06-15
(complete)		 582220
(37.8%)		 37
(6.36%)		 183
(31.44%)		 169
(29.04%)		 181
(31.1%)		 12
(2.06%)
Mistral-2.0
2018-08-01
(complete)		 582222
(38.14%)		 36
(6.19%)		 186
(31.96%)		 178
(30.58%)		 181
(31.1%)		 1
(0.17%)
OscaR - Conflict Ordering
2018-07-02
(complete)		 23681
(34.32%)		 30
(12.71%)		 51
(21.61%)		 0
(0%)		 155
(65.68%)		 0
(0%)
OscaR - Conflict Ordering
2018-08-14
(complete)		 23682
(34.75%)		 30
(12.71%)		 52
(22.03%)		 0
(0%)		 154
(65.25%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)		 58260
(10.31%)		 0
(0%)		 60
(10.31%)		 271
(46.56%)		 245
(42.1%)		 6
(1.03%)
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)		 58260
(10.31%)		 0
(0%)		 60
(10.31%)		 281
(48.28%)		 241
(41.41%)		 0
(0%)
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)		 582189
(32.47%)		 28
(4.81%)		 161
(27.66%)		 178
(30.58%)		 215
(36.94%)		 0
(0%)
OscaR - Hybrid
2018-07-02
(complete)		 34663
(18.21%)		 0
(0%)		 63
(18.21%)		 199
(57.51%)		 66
(19.08%)		 18
(5.2%)
OscaR - Hybrid
2018-08-14
(complete)		 34661
(17.63%)		 0
(0%)		 61
(17.63%)		 215
(62.14%)		 70
(20.23%)		 0
(0%)
PicatSAT
2018-06-15
(complete)		 582267
(45.88%)		 53
(9.11%)		 214
(36.77%)		 0
(0%)		 313
(53.78%)		 2
(0.34%)
PicatSAT
2018-08-02
(complete)		 582269
(46.22%)		 53
(9.11%)		 216
(37.11%)		 0
(0%)		 312
(53.61%)		 1
(0.17%)
PicatSAT
2018-08-14
(complete)		 582270
(46.39%)		 53
(9.11%)		 217
(37.29%)		 0
(0%)		 312
(53.61%)		 0
(0%)
Sat4j-CSP
2018-07-11
(complete)		 582137
(23.54%)		 43
(7.39%)		 94
(16.15%)		 111
(19.07%)		 334
(57.39%)		 0
(0%)
scop
both+MapleCOMSPS (2018-06-13)
(complete)		 236123
(52.12%)		 52
(22.03%)		 71
(30.08%)		 0
(0%)		 113
(47.88%)		 0
(0%)
scop
both+MapleCOMSPS (2018-07-31)
(complete)		 236141
(59.75%)		 53
(22.46%)		 88
(37.29%)		 0
(0%)		 95
(40.25%)		 0
(0%)
scop
order+MapleCOMSPS (2018-06-13)
(complete)		 236123
(52.12%)		 53
(22.46%)		 70
(29.66%)		 0
(0%)		 113
(47.88%)		 0
(0%)
scop
order+MapleCOMSPS (2018-07-31)
(complete)		 236146
(61.86%)		 54
(22.88%)		 92
(38.98%)		 0
(0%)		 90
(38.14%)		 0
(0%)

Solver NameProgress
BTD
2018.06.11_3
(complete)
done
236
UNSAT
50		OPT/SAT
30		?
155		ERR
1
BTD
2018.07.27_3
(complete)
done
236
UNSAT
45		OPT/SAT
31		?
160
BTD_12
2018-06-11_12
(complete)
done
236
UNSAT
49		OPT/SAT
31		?
155		ERR
1
BTD_12
2018.07.27_12
(complete)
done
236
UNSAT
45		OPT/SAT
32		?
159
Choco-solver
4.0.7 seq (493a269)
(complete)
done
582
UNSAT
38		OPT/SAT
176		SAT (TO/MO)
201		?
158		ERR
9
Choco-solver
4.0.7b seq (e747e1e)
(complete)
done
582
UNSAT
38		OPT/SAT
179		SAT (TO/MO)
206		?
159
Concrete
3.9.2
(complete)
done
582
UNSAT
28		OPT/SAT
169		SAT (TO/MO)
193		?
192
Concrete
3.9.2-SuperNG
(complete)
done
582
UNSAT
29		OPT/SAT
154		SAT (TO/MO)
198		?
201
Concrete 3.8
2018-06-13
(complete)
done
582
UNSAT
30		OPT/SAT
168		SAT (TO/MO)
189		?
191		ERR
4
Concrete 3.8-SuperNG
2018-06-13
(complete)
done
582
UNSAT
28		OPT/SAT
157		SAT (TO/MO)
190		?
203		ERR
4
cosoco
1.12
(complete)
done
582
UNSAT
26		OPT/SAT
117		SAT (TO/MO)
194		?
245
macht
2018.06.11
(complete)
done
236
UNSAT
38		OPT/SAT
32		?
165		ERR
1
macht
2018.07.27
(complete)
done
236
UNSAT
33		OPT/SAT
33		?
170
Mistral-2.0
2018-06-15
(complete)
done
582
UNSAT
37		OPT/SAT
183		SAT (TO/MO)
169		?
181		ERR
12
Mistral-2.0
2018-08-01
(complete)
done
582
UNSAT
36		OPT/SAT
186		SAT (TO/MO)
178		?
181		ERR
1
OscaR - Conflict Ordering
2018-07-02
(complete)
done
236
UNSAT
30		OPT/SAT
51		?
155
OscaR - Conflict Ordering
2018-08-14
(complete)
done
236
UNSAT
30		OPT/SAT
52		?
154
OscaR - Conflict Ordering with restarts
2018-07-02
(complete)
done
582
OPT/SAT
60		SAT (TO/MO)
271		?
245		ERR
6
OscaR - Conflict Ordering with restarts
2018-08-14
(complete)
done
582
OPT/SAT
60		SAT (TO/MO)
281		?
241
OscaR - Conflict Ordering with restarts
2018-08-17
(complete)
done
582
UNSAT
28		OPT/SAT
161		SAT (TO/MO)
178		?
215
OscaR - Hybrid
2018-07-02
(complete)
done
346
OPT/SAT
63		SAT (TO/MO)
199		?
66		ERR
18
OscaR - Hybrid
2018-08-14
(complete)
done
346
OPT/SAT
61		SAT (TO/MO)
215		?
70
PicatSAT
2018-06-15
(complete)
done
582
UNSAT
53		OPT/SAT
214		?
313		ERR
2
PicatSAT
2018-08-02
(complete)
done
582
UNSAT
53		OPT/SAT
216		?
312		ERR
1
PicatSAT
2018-08-14
(complete)
done
582
UNSAT
53		OPT/SAT
217		?
312
Sat4j-CSP
2018-07-11
(complete)
done
582
UNSAT
43		OPT/SAT
94		SAT (TO/MO)
111		?
334
scop
both+MapleCOMSPS (2018-06-13)
(complete)
done
236
UNSAT
52		OPT/SAT
71		?
113
scop
both+MapleCOMSPS (2018-07-31)
(complete)
done
236
UNSAT
53		OPT/SAT
88		?
95
scop
order+MapleCOMSPS (2018-06-13)
(complete)
done
236
UNSAT
53		OPT/SAT
70		?
113
scop
order+MapleCOMSPS (2018-07-31)
(complete)
done
236
UNSAT
54		OPT/SAT
92		?
90