solver | a "light red" background denotes a solver which was disqualified because it gave an incorrect answer, and which will not appear in the ranking |
solver | a "light yellow" background denotes a solver which was replaced by another version, and which will not appear in the ranking |
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Choco-solver 4.0.7b par (e747e1e) (complete) |
346 | 126 (36.42%) |
0 (0%) |
126 (36.42%) |
0 (0%) |
0 (0%) |
198 (57.23%) |
0 (0%) |
12 (3.47%) |
0 (0%) |
10 (2.89%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
OscaR - Parallel with EPS 2018-07-02 (complete) |
346 | 110 (31.79%) |
0 (0%) |
110 (31.79%) |
0 (0%) |
0 (0%) |
138 (39.88%) |
0 (0%) |
21 (6.07%) |
0 (0%) |
55 (15.9%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (1.16%) |
9 (2.6%) |
9 (2.6%) |
0 (0%) |
OscaR - Parallel with EPS 2018-08-14 (complete) |
346 | 113 (32.66%) |
0 (0%) |
113 (32.66%) |
0 (0%) |
0 (0%) |
145 (41.91%) |
0 (0%) |
21 (6.07%) |
0 (0%) |
64 (18.5%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (0.58%) |
1 (0.29%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | Choco-solver 4.0.7b par (e747e1e) (complete) |
|
OscaR - Parallel with EPS 2018-07-02 (complete) |
|
OscaR - Parallel with EPS 2018-08-14 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Choco-solver 4.0.7b par (e747e1e) (complete) |
236 | 134 (56.78%) |
46 (19.49%) |
0 (0%) |
88 (37.29%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
102 (43.22%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
OscaR - Parallel with EPS 2018-07-02 (complete) |
236 | 88 (37.29%) |
33 (13.98%) |
0 (0%) |
55 (23.31%) |
0 (0%) |
6 (2.54%) |
0 (0%) |
0 (0%) |
0 (0%) |
142 (60.17%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
OscaR - Parallel with EPS 2018-08-14 (complete) |
236 | 89 (37.71%) |
33 (13.98%) |
0 (0%) |
56 (23.73%) |
0 (0%) |
6 (2.54%) |
0 (0%) |
0 (0%) |
0 (0%) |
141 (59.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop both+glucose-syrup (2018-07-07) (complete) |
236 | 140 (59.32%) |
55 (23.31%) |
0 (0%) |
85 (36.02%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
60 (25.42%) |
36 (15.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop both+glucose-syrup (2018-07-31) (complete) |
236 | 139 (58.9%) |
57 (24.15%) |
0 (0%) |
82 (34.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
97 (41.1%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop order+glucose-syrup (2018-07-07) (complete) |
236 | 153 (64.83%) |
57 (24.15%) |
0 (0%) |
96 (40.68%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
54 (22.88%) |
29 (12.29%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop order+glucose-syrup (2018-07-31) (complete) |
236 | 152 (64.41%) |
56 (23.73%) |
0 (0%) |
96 (40.68%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
82 (34.75%) |
2 (0.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | Choco-solver 4.0.7b par (e747e1e) (complete) |
|
OscaR - Parallel with EPS 2018-07-02 (complete) |
|
OscaR - Parallel with EPS 2018-08-14 (complete) |
|
scop both+glucose-syrup (2018-07-07) (complete) |
|
scop both+glucose-syrup (2018-07-31) (complete) |
|
scop order+glucose-syrup (2018-07-07) (complete) |
|
scop order+glucose-syrup (2018-07-31) (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Choco-solver 4.0.7b par (e747e1e) (complete) |
582 | 260 (44.67%) |
46 (7.9%) |
126 (21.65%) |
88 (15.12%) |
0 (0%) |
198 (34.02%) |
0 (0%) |
12 (2.06%) |
0 (0%) |
112 (19.24%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
OscaR - Parallel with EPS 2018-07-02 (complete) |
582 | 198 (34.02%) |
33 (5.67%) |
110 (18.9%) |
55 (9.45%) |
0 (0%) |
144 (24.74%) |
0 (0%) |
21 (3.61%) |
0 (0%) |
197 (33.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (0.69%) |
9 (1.55%) |
9 (1.55%) |
0 (0%) |
OscaR - Parallel with EPS 2018-08-14 (complete) |
582 | 202 (34.71%) |
33 (5.67%) |
113 (19.42%) |
56 (9.62%) |
0 (0%) |
151 (25.95%) |
0 (0%) |
21 (3.61%) |
0 (0%) |
205 (35.22%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (0.34%) |
1 (0.17%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | Choco-solver 4.0.7b par (e747e1e) (complete) |
|
OscaR - Parallel with EPS 2018-07-02 (complete) |
|
OscaR - Parallel with EPS 2018-08-14 (complete) |
|
---|
This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.
Solver Name | #benchs | Solved | UNSAT. | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (conv. script) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG CERT. | WRONG OPT. | WRONG UNSAT. | WRONG ANS. |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Choco-solver 4.0.7b par (e747e1e) (complete) |
582 | 260 (44.67%) |
46 (7.9%) |
126 (21.65%) |
88 (15.12%) |
0 (0%) |
198 (34.02%) |
0 (0%) |
12 (2.06%) |
0 (0%) |
112 (19.24%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
OscaR - Parallel with EPS 2018-07-02 (complete) |
582 | 198 (34.02%) |
33 (5.67%) |
110 (18.9%) |
55 (9.45%) |
0 (0%) |
144 (24.74%) |
0 (0%) |
21 (3.61%) |
0 (0%) |
197 (33.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
4 (0.69%) |
9 (1.55%) |
9 (1.55%) |
0 (0%) |
OscaR - Parallel with EPS 2018-08-14 (complete) |
582 | 202 (34.71%) |
33 (5.67%) |
113 (19.42%) |
56 (9.62%) |
0 (0%) |
151 (25.95%) |
0 (0%) |
21 (3.61%) |
0 (0%) |
205 (35.22%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (0.34%) |
1 (0.17%) |
0 (0%) |
0 (0%) |
scop both+glucose-syrup (2018-07-07) (complete) |
236 | 140 (59.32%) |
55 (23.31%) |
0 (0%) |
85 (36.02%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
60 (25.42%) |
36 (15.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop both+glucose-syrup (2018-07-31) (complete) |
236 | 139 (58.9%) |
57 (24.15%) |
0 (0%) |
82 (34.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
97 (41.1%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop order+glucose-syrup (2018-07-07) (complete) |
236 | 153 (64.83%) |
57 (24.15%) |
0 (0%) |
96 (40.68%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
54 (22.88%) |
29 (12.29%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
scop order+glucose-syrup (2018-07-31) (complete) |
236 | 152 (64.41%) |
56 (23.73%) |
0 (0%) |
96 (40.68%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
82 (34.75%) |
2 (0.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | Choco-solver 4.0.7b par (e747e1e) (complete) |
|
OscaR - Parallel with EPS 2018-07-02 (complete) |
|
OscaR - Parallel with EPS 2018-08-14 (complete) |
|
scop both+glucose-syrup (2018-07-07) (complete) |
|
scop both+glucose-syrup (2018-07-31) (complete) |
|
scop order+glucose-syrup (2018-07-07) (complete) |
|
scop order+glucose-syrup (2018-07-31) (complete) |
|
---|