solver | a "light red" background denotes a solver which was disqualified because it gave an incorrect answer, and which will not appear in the ranking |
solver | a "light yellow" background denotes a solver which was replaced by another version, and which will not appear in the ranking |
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
cosoco 1.12 (complete) |
188 | 46 (24.47%) |
0 (0%) |
46 (24.47%) |
0 (0%) |
0 (0%) |
127 (67.55%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (2.13%) |
1 (0.53%) |
10 (5.32%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
GG's minicp 2018-04-29 (complete) |
188 | 3 (1.6%) |
0 (0%) |
3 (1.6%) |
0 (0%) |
0 (0%) |
118 (62.77%) |
0 (0%) |
34 (18.09%) |
10 (5.32%) |
23 (12.23%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
MiniCPFever 2018-04-29 (complete) |
188 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
120 (63.83%) |
0 (0%) |
32 (17.02%) |
10 (5.32%) |
20 (10.64%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
6 (3.19%) |
0 (0%) |
slowpoke 2018-04-29 (incomplete) |
188 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
125 (66.49%) |
0 (0%) |
33 (17.55%) |
10 (5.32%) |
20 (10.64%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
188 | 35 (18.62%) |
0 (0%) |
35 (18.62%) |
0 (0%) |
0 (0%) |
88 (46.81%) |
0 (0%) |
31 (16.49%) |
10 (5.32%) |
24 (12.77%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
188 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
122 (64.89%) |
0 (0%) |
0 (0%) |
32 (17.02%) |
10 (5.32%) |
24 (12.77%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
The dodo solver 2018-04-29 (complete) |
188 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
125 (66.49%) |
0 (0%) |
32 (17.02%) |
22 (11.7%) |
9 (4.79%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | cosoco 1.12 (complete) |
|
GG's minicp 2018-04-29 (complete) |
|
MiniCPFever 2018-04-29 (complete) |
|
slowpoke 2018-04-29 (incomplete) |
|
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
|
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
|
The dodo solver 2018-04-29 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
cosoco 1.12 (complete) |
176 | 72 (40.91%) |
30 (17.05%) |
0 (0%) |
42 (23.86%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
104 (59.09%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
GG's minicp 2018-04-29 (complete) |
176 | 56 (31.82%) |
19 (10.8%) |
0 (0%) |
37 (21.02%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.57%) |
119 (67.61%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
miniBTD 2018.07.27_3 (complete) |
176 | 77 (43.75%) |
44 (25%) |
0 (0%) |
33 (18.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
31 (17.61%) |
0 (0%) |
68 (38.64%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
miniBTD_12 2018.07.27_12 (complete) |
176 | 79 (44.89%) |
43 (24.43%) |
0 (0%) |
36 (20.45%) |
0 (0%) |
0 (0%) |
0 (0%) |
19 (10.8%) |
0 (0%) |
78 (44.32%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
MiniCPFever 2018-04-29 (complete) |
176 | 54 (30.68%) |
20 (11.36%) |
0 (0%) |
34 (19.32%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
122 (69.32%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
minimacht 2018.07.27 (complete) |
176 | 69 (39.2%) |
32 (18.18%) |
0 (0%) |
37 (21.02%) |
0 (0%) |
0 (0%) |
0 (0%) |
16 (9.09%) |
0 (0%) |
91 (51.7%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NACRE 1.0.4 (complete) |
176 | 86 (48.86%) |
43 (24.43%) |
0 (0%) |
43 (24.43%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
90 (51.14%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
slowpoke 2018-04-29 (incomplete) |
176 | 38 (21.59%) |
0 (0%) |
0 (0%) |
38 (21.59%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.57%) |
1 (0.57%) |
136 (77.27%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
176 | 54 (30.68%) |
31 (17.61%) |
0 (0%) |
23 (13.07%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
113 (64.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
9 (5.11%) |
0 (0%) |
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
176 | 31 (17.61%) |
0 (0%) |
0 (0%) |
31 (17.61%) |
0 (0%) |
0 (0%) |
0 (0%) |
16 (9.09%) |
0 (0%) |
125 (71.02%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (2.27%) |
0 (0%) |
0 (0%) |
0 (0%) |
The dodo solver 2018-04-29 (complete) |
176 | 25 (14.2%) |
25 (14.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
33 (18.75%) |
0 (0%) |
0 (0%) |
1 (0.57%) |
117 (66.48%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | cosoco 1.12 (complete) |
|
GG's minicp 2018-04-29 (complete) |
|
miniBTD 2018.07.27_3 (complete) |
|
miniBTD_12 2018.07.27_12 (complete) |
|
MiniCPFever 2018-04-29 (complete) |
|
minimacht 2018.07.27 (complete) |
|
NACRE 1.0.4 (complete) |
|
slowpoke 2018-04-29 (incomplete) |
|
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
|
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
|
The dodo solver 2018-04-29 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
cosoco 1.12 (complete) |
364 | 118 (32.42%) |
30 (8.24%) |
46 (12.64%) |
42 (11.54%) |
0 (0%) |
127 (34.89%) |
0 (0%) |
0 (0%) |
0 (0%) |
108 (29.67%) |
1 (0.27%) |
10 (2.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
GG's minicp 2018-04-29 (complete) |
364 | 59 (16.21%) |
19 (5.22%) |
3 (0.82%) |
37 (10.16%) |
0 (0%) |
118 (32.42%) |
0 (0%) |
34 (9.34%) |
11 (3.02%) |
142 (39.01%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
MiniCPFever 2018-04-29 (complete) |
364 | 54 (14.84%) |
20 (5.49%) |
0 (0%) |
34 (9.34%) |
0 (0%) |
120 (32.97%) |
0 (0%) |
32 (8.79%) |
10 (2.75%) |
142 (39.01%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
6 (1.65%) |
0 (0%) |
slowpoke 2018-04-29 (incomplete) |
364 | 38 (10.44%) |
0 (0%) |
0 (0%) |
38 (10.44%) |
0 (0%) |
125 (34.34%) |
0 (0%) |
34 (9.34%) |
11 (3.02%) |
156 (42.86%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
364 | 89 (24.45%) |
31 (8.52%) |
35 (9.62%) |
23 (6.32%) |
0 (0%) |
88 (24.18%) |
0 (0%) |
31 (8.52%) |
10 (2.75%) |
137 (37.64%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
9 (2.47%) |
0 (0%) |
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
364 | 31 (8.52%) |
0 (0%) |
0 (0%) |
31 (8.52%) |
122 (33.52%) |
0 (0%) |
0 (0%) |
48 (13.19%) |
10 (2.75%) |
149 (40.93%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (1.1%) |
0 (0%) |
0 (0%) |
0 (0%) |
The dodo solver 2018-04-29 (complete) |
364 | 25 (6.87%) |
25 (6.87%) |
0 (0%) |
0 (0%) |
0 (0%) |
158 (43.41%) |
0 (0%) |
32 (8.79%) |
23 (6.32%) |
126 (34.62%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | cosoco 1.12 (complete) |
|
GG's minicp 2018-04-29 (complete) |
|
MiniCPFever 2018-04-29 (complete) |
|
slowpoke 2018-04-29 (incomplete) |
|
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
|
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
|
The dodo solver 2018-04-29 (complete) |
|
---|
This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
cosoco 1.12 (complete) |
364 | 118 (32.42%) |
30 (8.24%) |
46 (12.64%) |
42 (11.54%) |
0 (0%) |
127 (34.89%) |
0 (0%) |
0 (0%) |
0 (0%) |
108 (29.67%) |
1 (0.27%) |
10 (2.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
GG's minicp 2018-04-29 (complete) |
364 | 59 (16.21%) |
19 (5.22%) |
3 (0.82%) |
37 (10.16%) |
0 (0%) |
118 (32.42%) |
0 (0%) |
34 (9.34%) |
11 (3.02%) |
142 (39.01%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
miniBTD 2018.07.27_3 (complete) |
176 | 77 (43.75%) |
44 (25%) |
0 (0%) |
33 (18.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
31 (17.61%) |
0 (0%) |
68 (38.64%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
miniBTD_12 2018.07.27_12 (complete) |
176 | 79 (44.89%) |
43 (24.43%) |
0 (0%) |
36 (20.45%) |
0 (0%) |
0 (0%) |
0 (0%) |
19 (10.8%) |
0 (0%) |
78 (44.32%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
MiniCPFever 2018-04-29 (complete) |
364 | 54 (14.84%) |
20 (5.49%) |
0 (0%) |
34 (9.34%) |
0 (0%) |
120 (32.97%) |
0 (0%) |
32 (8.79%) |
10 (2.75%) |
142 (39.01%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
6 (1.65%) |
0 (0%) |
minimacht 2018.07.27 (complete) |
176 | 69 (39.2%) |
32 (18.18%) |
0 (0%) |
37 (21.02%) |
0 (0%) |
0 (0%) |
0 (0%) |
16 (9.09%) |
0 (0%) |
91 (51.7%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NACRE 1.0.4 (complete) |
176 | 86 (48.86%) |
43 (24.43%) |
0 (0%) |
43 (24.43%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
90 (51.14%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
slowpoke 2018-04-29 (incomplete) |
364 | 38 (10.44%) |
0 (0%) |
0 (0%) |
38 (10.44%) |
0 (0%) |
125 (34.34%) |
0 (0%) |
34 (9.34%) |
11 (3.02%) |
156 (42.86%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
364 | 89 (24.45%) |
31 (8.52%) |
35 (9.62%) |
23 (6.32%) |
0 (0%) |
88 (24.18%) |
0 (0%) |
31 (8.52%) |
10 (2.75%) |
137 (37.64%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
9 (2.47%) |
0 (0%) |
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
364 | 31 (8.52%) |
0 (0%) |
0 (0%) |
31 (8.52%) |
122 (33.52%) |
0 (0%) |
0 (0%) |
48 (13.19%) |
10 (2.75%) |
149 (40.93%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (1.1%) |
0 (0%) |
0 (0%) |
0 (0%) |
The dodo solver 2018-04-29 (complete) |
364 | 25 (6.87%) |
25 (6.87%) |
0 (0%) |
0 (0%) |
0 (0%) |
158 (43.41%) |
0 (0%) |
32 (8.79%) |
23 (6.32%) |
126 (34.62%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | cosoco 1.12 (complete) |
|
GG's minicp 2018-04-29 (complete) |
|
miniBTD 2018.07.27_3 (complete) |
|
miniBTD_12 2018.07.27_12 (complete) |
|
MiniCPFever 2018-04-29 (complete) |
|
minimacht 2018.07.27 (complete) |
|
NACRE 1.0.4 (complete) |
|
slowpoke 2018-04-29 (incomplete) |
|
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete) |
|
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete) |
|
The dodo solver 2018-04-29 (complete) |
|
---|