2018 XCSP3 competition: parallel solvers tracks: solvers page (tabular format, summary)

Some explanations

Category "optimization problem" (COP)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
Choco-solver
4.0.7b par (e747e1e)
(complete)		 346126
(36.42%)		 0
(0%)		 126
(36.42%)		 198
(57.23%)		 22
(6.36%)		 0
(0%)
OscaR - Parallel with EPS
2018-07-02
(complete)		 346110
(31.79%)		 0
(0%)		 110
(31.79%)		 138
(39.88%)		 76
(21.97%)		 22
(6.36%)
OscaR - Parallel with EPS
2018-08-14
(complete)		 346113
(32.66%)		 0
(0%)		 113
(32.66%)		 145
(41.91%)		 85
(24.57%)		 3
(0.87%)

Solver NameProgress
Choco-solver
4.0.7b par (e747e1e)
(complete)
done
346
OPT/SAT
126		SAT (TO/MO)
198		?
22
OscaR - Parallel with EPS
2018-07-02
(complete)
done
346
OPT/SAT
110		SAT (TO/MO)
138		?
76		ERR
22
OscaR - Parallel with EPS
2018-08-14
(complete)
done
346
OPT/SAT
113		SAT (TO/MO)
145		?
85		ERR
3

Category "decision problem" (CSP)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
Choco-solver
4.0.7b par (e747e1e)
(complete)		 236134
(56.78%)		 46
(19.49%)		 88
(37.29%)		 0
(0%)		 102
(43.22%)		 0
(0%)
OscaR - Parallel with EPS
2018-07-02
(complete)		 23694
(39.83%)		 33
(13.98%)		 61
(25.85%)		 0
(0%)		 142
(60.17%)		 0
(0%)
OscaR - Parallel with EPS
2018-08-14
(complete)		 23695
(40.25%)		 33
(13.98%)		 62
(26.27%)		 0
(0%)		 141
(59.75%)		 0
(0%)
scop
both+glucose-syrup (2018-07-07)
(complete)		 236140
(59.32%)		 55
(23.31%)		 85
(36.02%)		 0
(0%)		 96
(40.68%)		 0
(0%)
scop
both+glucose-syrup (2018-07-31)
(complete)		 236139
(58.9%)		 57
(24.15%)		 82
(34.75%)		 0
(0%)		 97
(41.1%)		 0
(0%)
scop
order+glucose-syrup (2018-07-07)
(complete)		 236153
(64.83%)		 57
(24.15%)		 96
(40.68%)		 0
(0%)		 83
(35.17%)		 0
(0%)
scop
order+glucose-syrup (2018-07-31)
(complete)		 236152
(64.41%)		 56
(23.73%)		 96
(40.68%)		 0
(0%)		 84
(35.59%)		 0
(0%)

Solver NameProgress
Choco-solver
4.0.7b par (e747e1e)
(complete)
done
236
UNSAT
46		OPT/SAT
88		?
102
OscaR - Parallel with EPS
2018-07-02
(complete)
done
236
UNSAT
33		OPT/SAT
61		?
142
OscaR - Parallel with EPS
2018-08-14
(complete)
done
236
UNSAT
33		OPT/SAT
62		?
141
scop
both+glucose-syrup (2018-07-07)
(complete)
done
236
UNSAT
55		OPT/SAT
85		?
96
scop
both+glucose-syrup (2018-07-31)
(complete)
done
236
UNSAT
57		OPT/SAT
82		?
97
scop
order+glucose-syrup (2018-07-07)
(complete)
done
236
UNSAT
57		OPT/SAT
96		?
83
scop
order+glucose-syrup (2018-07-31)
(complete)
done
236
UNSAT
56		OPT/SAT
96		?
84

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
Choco-solver
4.0.7b par (e747e1e)
(complete)		 582260
(44.67%)		 46
(7.9%)		 214
(36.77%)		 198
(34.02%)		 124
(21.31%)		 0
(0%)
OscaR - Parallel with EPS
2018-07-02
(complete)		 582204
(35.05%)		 33
(5.67%)		 171
(29.38%)		 138
(23.71%)		 218
(37.46%)		 22
(3.78%)
OscaR - Parallel with EPS
2018-08-14
(complete)		 582208
(35.74%)		 33
(5.67%)		 175
(30.07%)		 145
(24.91%)		 226
(38.83%)		 3
(0.52%)

Solver NameProgress
Choco-solver
4.0.7b par (e747e1e)
(complete)
done
582
UNSAT
46		OPT/SAT
214		SAT (TO/MO)
198		?
124
OscaR - Parallel with EPS
2018-07-02
(complete)
done
582
UNSAT
33		OPT/SAT
171		SAT (TO/MO)
138		?
218		ERR
22
OscaR - Parallel with EPS
2018-08-14
(complete)
done
582
UNSAT
33		OPT/SAT
175		SAT (TO/MO)
145		?
226		ERR
3

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
Choco-solver
4.0.7b par (e747e1e)
(complete)		 582260
(44.67%)		 46
(7.9%)		 214
(36.77%)		 198
(34.02%)		 124
(21.31%)		 0
(0%)
OscaR - Parallel with EPS
2018-07-02
(complete)		 582204
(35.05%)		 33
(5.67%)		 171
(29.38%)		 138
(23.71%)		 218
(37.46%)		 22
(3.78%)
OscaR - Parallel with EPS
2018-08-14
(complete)		 582208
(35.74%)		 33
(5.67%)		 175
(30.07%)		 145
(24.91%)		 226
(38.83%)		 3
(0.52%)
scop
both+glucose-syrup (2018-07-07)
(complete)		 236140
(59.32%)		 55
(23.31%)		 85
(36.02%)		 0
(0%)		 96
(40.68%)		 0
(0%)
scop
both+glucose-syrup (2018-07-31)
(complete)		 236139
(58.9%)		 57
(24.15%)		 82
(34.75%)		 0
(0%)		 97
(41.1%)		 0
(0%)
scop
order+glucose-syrup (2018-07-07)
(complete)		 236153
(64.83%)		 57
(24.15%)		 96
(40.68%)		 0
(0%)		 83
(35.17%)		 0
(0%)
scop
order+glucose-syrup (2018-07-31)
(complete)		 236152
(64.41%)		 56
(23.73%)		 96
(40.68%)		 0
(0%)		 84
(35.59%)		 0
(0%)

Solver NameProgress
Choco-solver
4.0.7b par (e747e1e)
(complete)
done
582
UNSAT
46		OPT/SAT
214		SAT (TO/MO)
198		?
124
OscaR - Parallel with EPS
2018-07-02
(complete)
done
582
UNSAT
33		OPT/SAT
171		SAT (TO/MO)
138		?
218		ERR
22
OscaR - Parallel with EPS
2018-08-14
(complete)
done
582
UNSAT
33		OPT/SAT
175		SAT (TO/MO)
145		?
226		ERR
3
scop
both+glucose-syrup (2018-07-07)
(complete)
done
236
UNSAT
55		OPT/SAT
85		?
96
scop
both+glucose-syrup (2018-07-31)
(complete)
done
236
UNSAT
57		OPT/SAT
82		?
97
scop
order+glucose-syrup (2018-07-07)
(complete)
done
236
UNSAT
57		OPT/SAT
96		?
83
scop
order+glucose-syrup (2018-07-31)
(complete)
done
236
UNSAT
56		OPT/SAT
96		?
84