2018 XCSP3 competition: mini-solvers track: solvers page (tabular format, summary)

Some explanations

#Benchs: Number of benchs for this solver
Solved: Total number of solved instances (this is the sum of the SAT and UNSAT columns)
UNSAT: Total number of UNSAT answers
OPT/SAT: solver found an optimum solution to an optimization problem (the provided implicant satisfies each constraint and no solver gave a better solution) or solver found a solution to a SAT/UNSAT instance
SAT (TO/MO): Solver found a solution but this may not be the optimal solution
UNKNOWN: Total number of UNKNOWN answers
Bad Answers: Total number of bad answers
solver a "light red" background denotes a solver which was disqualified because it gave an incorrect answer, and which will not appear in the ranking
solver a "light yellow" background denotes a solver which was replaced by another version, and which will not appear in the ranking

Category "optimization problem" (COP)

Solver Name	#benchs	Solved	OPT/SAT	SAT (TO/MO)	UNKNOWN	Bad Answers
cosoco 1.12 (complete)	188	46 (24.47%)	46 (24.47%)	127 (67.55%)	15 (7.98%)	0 (0%)
GG's minicp 2018-04-29 (complete)	188	3 (1.6%)	3 (1.6%)	118 (62.77%)	67 (35.64%)	0 (0%)
MiniCPFever 2018-04-29 (complete)	188	0 (0%)	0 (0%)	120 (63.83%)	62 (32.98%)	6 (3.19%)
slowpoke 2018-04-29 (incomplete)	188	0 (0%)	0 (0%)	125 (66.49%)	63 (33.51%)	0 (0%)
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	188	35 (18.62%)	35 (18.62%)	88 (46.81%)	65 (34.57%)	0 (0%)
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	188	0 (0%)	0 (0%)	122 (64.89%)	66 (35.11%)	0 (0%)
The dodo solver 2018-04-29 (complete)	188	0 (0%)	0 (0%)	125 (66.49%)	63 (33.51%)	0 (0%)

Solver Name

Progress

cosoco
1.12
(complete)

done
188

OPT/SAT
46

SAT (TO/MO)
127

?
15

GG's minicp
2018-04-29
(complete)

done
188

OPT/SAT
3

SAT (TO/MO)
118

?
67

MiniCPFever
2018-04-29
(complete)

done
188

SAT (TO/MO)
120

?
62

ERR
6

slowpoke
2018-04-29
(incomplete)

done
188

SAT (TO/MO)
125

?
63

Solver of Xavier Schul & Yvhan Smal
2018-04-28
(incomplete)

done
188

OPT/SAT
35

SAT (TO/MO)
88

?
65

SuperSolver_Macq_Stevenart
2018-04-27
(incomplete)

done
188

SAT (TO/MO)
122

?
66

The dodo solver
2018-04-29
(complete)

done
188

SAT (TO/MO)
125

?
63

Category "decision problem" (CSP)

Solver Name	#benchs	Solved	UNSAT	OPT/SAT	UNKNOWN	Bad Answers
cosoco 1.12 (complete)	176	72 (40.91%)	30 (17.05%)	42 (23.86%)	104 (59.09%)	0 (0%)
GG's minicp 2018-04-29 (complete)	176	56 (31.82%)	19 (10.8%)	37 (21.02%)	120 (68.18%)	0 (0%)
miniBTD 2018.07.27_3 (complete)	176	77 (43.75%)	44 (25%)	33 (18.75%)	99 (56.25%)	0 (0%)
miniBTD_12 2018.07.27_12 (complete)	176	79 (44.89%)	43 (24.43%)	36 (20.45%)	97 (55.11%)	0 (0%)
MiniCPFever 2018-04-29 (complete)	176	54 (30.68%)	20 (11.36%)	34 (19.32%)	122 (69.32%)	0 (0%)
minimacht 2018.07.27 (complete)	176	69 (39.2%)	32 (18.18%)	37 (21.02%)	107 (60.8%)	0 (0%)
NACRE 1.0.4 (complete)	176	86 (48.86%)	43 (24.43%)	43 (24.43%)	90 (51.14%)	0 (0%)
slowpoke 2018-04-29 (incomplete)	176	38 (21.59%)	0 (0%)	38 (21.59%)	138 (78.41%)	0 (0%)
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	176	54 (30.68%)	31 (17.61%)	23 (13.07%)	113 (64.2%)	9 (5.11%)
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	176	31 (17.61%)	0 (0%)	31 (17.61%)	141 (80.11%)	4 (2.27%)
The dodo solver 2018-04-29 (complete)	176	58 (32.95%)	25 (14.2%)	33 (18.75%)	118 (67.05%)	0 (0%)

Solver Name

Progress

cosoco
1.12
(complete)

done
176

UNSAT
30

OPT/SAT
42

?
104

GG's minicp
2018-04-29
(complete)

done
176

UNSAT
19

OPT/SAT
37

?
120

miniBTD
2018.07.27_3
(complete)

done
176

UNSAT
44

OPT/SAT
33

?
99

miniBTD_12
2018.07.27_12
(complete)

done
176

UNSAT
43

OPT/SAT
36

?
97

MiniCPFever
2018-04-29
(complete)

done
176

UNSAT
20

OPT/SAT
34

?
122

minimacht
2018.07.27
(complete)

done
176

UNSAT
32

OPT/SAT
37

?
107

NACRE
1.0.4
(complete)

done
176

UNSAT
43

OPT/SAT
43

?
90

slowpoke
2018-04-29
(incomplete)

done
176

OPT/SAT
38

?
138

Solver of Xavier Schul & Yvhan Smal
2018-04-28
(incomplete)

done
176

UNSAT
31

OPT/SAT
23

?
113

ERR
9

SuperSolver_Macq_Stevenart
2018-04-27
(incomplete)

done
176

OPT/SAT
31

?
141

ERR
4

The dodo solver
2018-04-29
(complete)

done
176

UNSAT
25

OPT/SAT
33

?
118

All categories for solvers registered in all categories

Solver Name	#benchs	Solved	UNSAT	OPT/SAT	SAT (TO/MO)	UNKNOWN	Bad Answers
cosoco 1.12 (complete)	364	118 (32.42%)	30 (8.24%)	88 (24.18%)	127 (34.89%)	119 (32.69%)	0 (0%)
GG's minicp 2018-04-29 (complete)	364	59 (16.21%)	19 (5.22%)	40 (10.99%)	118 (32.42%)	187 (51.37%)	0 (0%)
MiniCPFever 2018-04-29 (complete)	364	54 (14.84%)	20 (5.49%)	34 (9.34%)	120 (32.97%)	184 (50.55%)	6 (1.65%)
slowpoke 2018-04-29 (incomplete)	364	38 (10.44%)	0 (0%)	38 (10.44%)	125 (34.34%)	201 (55.22%)	0 (0%)
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	364	89 (24.45%)	31 (8.52%)	58 (15.93%)	88 (24.18%)	178 (48.9%)	9 (2.47%)
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	364	31 (8.52%)	0 (0%)	31 (8.52%)	122 (33.52%)	207 (56.87%)	4 (1.1%)
The dodo solver 2018-04-29 (complete)	364	58 (15.93%)	25 (6.87%)	33 (9.07%)	125 (34.34%)	181 (49.73%)	0 (0%)

Solver Name

Progress

cosoco
1.12
(complete)

done
364

UNSAT
30

OPT/SAT
88

SAT (TO/MO)
127

?
119

GG's minicp
2018-04-29
(complete)

done
364

UNSAT
19

OPT/SAT
40

SAT (TO/MO)
118

?
187

MiniCPFever
2018-04-29
(complete)

done
364

UNSAT
20

OPT/SAT
34

SAT (TO/MO)
120

?
184

ERR
6

slowpoke
2018-04-29
(incomplete)

done
364

OPT/SAT
38

SAT (TO/MO)
125

?
201

Solver of Xavier Schul & Yvhan Smal
2018-04-28
(incomplete)

done
364

UNSAT
31

OPT/SAT
58

SAT (TO/MO)
88

?
178

ERR
9

SuperSolver_Macq_Stevenart
2018-04-27
(incomplete)

done
364

OPT/SAT
31

SAT (TO/MO)
122

?
207

ERR
4

The dodo solver
2018-04-29
(complete)

done
364

UNSAT
25

OPT/SAT
33

SAT (TO/MO)
125

?
181

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name	#benchs	Solved	UNSAT	OPT/SAT	SAT (TO/MO)	UNKNOWN	Bad Answers
cosoco 1.12 (complete)	364	118 (32.42%)	30 (8.24%)	88 (24.18%)	127 (34.89%)	119 (32.69%)	0 (0%)
GG's minicp 2018-04-29 (complete)	364	59 (16.21%)	19 (5.22%)	40 (10.99%)	118 (32.42%)	187 (51.37%)	0 (0%)
miniBTD 2018.07.27_3 (complete)	176	77 (43.75%)	44 (25%)	33 (18.75%)	0 (0%)	99 (56.25%)	0 (0%)
miniBTD_12 2018.07.27_12 (complete)	176	79 (44.89%)	43 (24.43%)	36 (20.45%)	0 (0%)	97 (55.11%)	0 (0%)
MiniCPFever 2018-04-29 (complete)	364	54 (14.84%)	20 (5.49%)	34 (9.34%)	120 (32.97%)	184 (50.55%)	6 (1.65%)
minimacht 2018.07.27 (complete)	176	69 (39.2%)	32 (18.18%)	37 (21.02%)	0 (0%)	107 (60.8%)	0 (0%)
NACRE 1.0.4 (complete)	176	86 (48.86%)	43 (24.43%)	43 (24.43%)	0 (0%)	90 (51.14%)	0 (0%)
slowpoke 2018-04-29 (incomplete)	364	38 (10.44%)	0 (0%)	38 (10.44%)	125 (34.34%)	201 (55.22%)	0 (0%)
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	364	89 (24.45%)	31 (8.52%)	58 (15.93%)	88 (24.18%)	178 (48.9%)	9 (2.47%)
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	364	31 (8.52%)	0 (0%)	31 (8.52%)	122 (33.52%)	207 (56.87%)	4 (1.1%)
The dodo solver 2018-04-29 (complete)	364	58 (15.93%)	25 (6.87%)	33 (9.07%)	125 (34.34%)	181 (49.73%)	0 (0%)