Solver Name | #benchs | Solved | UNSAT | OPT/SAT | SAT (TO/MO) | UNKNOWN | Bad Answers |
---|---|---|---|---|---|---|---|
borg pb-dec-11.04.03 (complete) |
100 | 73 (73%) |
20 (20%) |
53 (53%) |
0 (0%) |
27 (27%) |
0 (0%) |
bsolo 3.2 (complete) |
100 | 62 (62%) |
15 (15%) |
47 (47%) |
0 (0%) |
38 (38%) |
0 (0%) |
clasp 2.0-R4191 (complete) |
100 | 64 (64%) |
15 (15%) |
49 (49%) |
0 (0%) |
36 (36%) |
0 (0%) |
MinisatID 2.4.8 [DEPRECATED] (complete) |
100 | 57 (57%) |
10 (10%) |
47 (47%) |
0 (0%) |
43 (43%) |
0 (0%) |
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
100 | 57 (57%) |
10 (10%) |
47 (47%) |
0 (0%) |
43 (43%) |
0 (0%) |
MinisatID 2.5.2 (fixed) (complete) |
100 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
100 (100%) |
0 (0%) |
MinisatID 2.5.2-gmp (fixed) (complete) |
100 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
100 (100%) |
0 (0%) |
Sat4j CuttingPlanes 2.3.0 (complete) |
100 | 65 (65%) |
15 (15%) |
50 (50%) |
0 (0%) |
35 (35%) |
0 (0%) |
Sat4j Res//CP 2.3.0 (complete) |
100 | 65 (65%) |
15 (15%) |
50 (50%) |
0 (0%) |
35 (35%) |
0 (0%) |
Sat4j Resolution 2.3.0 (complete) |
100 | 27 (27%) |
15 (15%) |
12 (12%) |
0 (0%) |
73 (73%) |
0 (0%) |
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
100 | 75 (75%) |
21 (21%) |
54 (54%) |
0 (0%) |
25 (25%) |
0 (0%) |
SCIP spx 2 2011-06-10 (fixed) (complete) |
100 | 74 (74%) |
20 (20%) |
54 (54%) |
0 (0%) |
26 (26%) |
0 (0%) |
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
100 | 75 (75%) |
20 (20%) |
55 (55%) |
0 (0%) |
25 (25%) |
0 (0%) |
SCIP spx E_2 2011-06-10 (fixed) (complete) |
100 | 75 (75%) |
20 (20%) |
55 (55%) |
0 (0%) |
25 (25%) |
0 (0%) |
Solver Name | Progress | borg pb-dec-11.04.03 (complete) |
|
bsolo 3.2 (complete) |
|
clasp 2.0-R4191 (complete) |
|
MinisatID 2.4.8 [DEPRECATED] (complete) |
|
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
|
MinisatID 2.5.2 (fixed) (complete) |
|
MinisatID 2.5.2-gmp (fixed) (complete) |
|
Sat4j CuttingPlanes 2.3.0 (complete) |
|
Sat4j Res//CP 2.3.0 (complete) |
|
Sat4j Resolution 2.3.0 (complete) |
|
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx 2 2011-06-10 (fixed) (complete) |
|
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx E_2 2011-06-10 (fixed) (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT | OPT/SAT | SAT (TO/MO) | UNKNOWN | Bad Answers |
---|---|---|---|---|---|---|---|
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
557 | 107 (19.21%) |
57 (10.23%) |
50 (8.98%) |
0 (0%) |
450 (80.79%) |
0 (0%) |
MinisatID 2.5.2-gmp (fixed) (complete) |
557 | 117 (21.01%) |
57 (10.23%) |
60 (10.77%) |
0 (0%) |
440 (78.99%) |
0 (0%) |
Sat4j CuttingPlanes 2.3.0 (complete) |
557 | 175 (31.42%) |
59 (10.59%) |
116 (20.83%) |
175 (31.42%) |
207 (37.16%) |
0 (0%) |
Sat4j Res//CP 2.3.0 (complete) |
557 | 208 (37.34%) |
59 (10.59%) |
149 (26.75%) |
188 (33.75%) |
161 (28.9%) |
0 (0%) |
Sat4j Resolution 2.3.0 (complete) |
557 | 201 (36.09%) |
57 (10.23%) |
144 (25.85%) |
192 (34.47%) |
164 (29.44%) |
0 (0%) |
Solver Name | Progress | MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
|
MinisatID 2.5.2-gmp (fixed) (complete) |
|
Sat4j CuttingPlanes 2.3.0 (complete) |
|
Sat4j Res//CP 2.3.0 (complete) |
|
Sat4j Resolution 2.3.0 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT | OPT/SAT | SAT (TO/MO) | UNKNOWN | Bad Answers |
---|---|---|---|---|---|---|---|
borg pb-opt-11.04.03 (complete) |
731 | 413 (56.5%) |
35 (4.79%) |
378 (51.71%) |
0 (0%) |
283 (38.71%) |
35 (4.79%) |
bsolo 3.2 (complete) |
731 | 372 (50.89%) |
34 (4.65%) |
338 (46.24%) |
10 (1.37%) |
45 (6.16%) |
0 (0%) |
clasp 2.0-R4191 (complete) |
731 | 320 (43.78%) |
34 (4.65%) |
286 (39.12%) |
336 (45.96%) |
75 (10.26%) |
0 (0%) |
MinisatID 2.4.8 [DEPRECATED] (complete) |
731 | 264 (36.11%) |
34 (4.65%) |
230 (31.46%) |
0 (0%) |
467 (63.89%) |
0 (0%) |
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
731 | 238 (32.56%) |
30 (4.1%) |
208 (28.45%) |
0 (0%) |
493 (67.44%) |
0 (0%) |
MinisatID 2.5.2 (fixed) (complete) |
731 | 288 (39.4%) |
32 (4.38%) |
256 (35.02%) |
0 (0%) |
443 (60.6%) |
0 (0%) |
MinisatID 2.5.2-gmp (fixed) (complete) |
731 | 262 (35.84%) |
30 (4.1%) |
232 (31.74%) |
0 (0%) |
469 (64.16%) |
0 (0%) |
pwbo 1.1 (complete) |
731 | 383 (52.39%) |
33 (4.51%) |
350 (47.88%) |
297 (40.63%) |
51 (6.98%) |
0 (0%) |
Sat4j CuttingPlanes 2.3.0 (complete) |
731 | 270 (36.94%) |
30 (4.1%) |
240 (32.83%) |
381 (52.12%) |
80 (10.94%) |
0 (0%) |
Sat4j Res//CP 2.3.0 (complete) |
731 | 329 (45.01%) |
34 (4.65%) |
295 (40.36%) |
357 (48.84%) |
45 (6.16%) |
0 (0%) |
Sat4j Resolution 2.3.0 (complete) |
731 | 316 (43.23%) |
34 (4.65%) |
282 (38.58%) |
367 (50.21%) |
48 (6.57%) |
0 (0%) |
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
731 | 395 (54.04%) |
35 (4.79%) |
360 (49.25%) |
120 (16.42%) |
203 (27.77%) |
0 (0%) |
SCIP spx 2 2011-06-10 (fixed) (complete) |
731 | 424 (58%) |
35 (4.79%) |
389 (53.21%) |
9 (1.23%) |
164 (22.44%) |
0 (0%) |
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
731 | 413 (56.5%) |
35 (4.79%) |
378 (51.71%) |
114 (15.6%) |
196 (26.81%) |
1 (0.14%) |
SCIP spx E_2 2011-06-10 (fixed) (complete) |
731 | 424 (58%) |
35 (4.79%) |
389 (53.21%) |
12 (1.64%) |
163 (22.3%) |
0 (0%) |
wbo 1.6 (complete) |
731 | 271 (37.07%) |
33 (4.51%) |
238 (32.56%) |
0 (0%) |
451 (61.7%) |
0 (0%) |
Solver Name | Progress | borg pb-opt-11.04.03 (complete) |
|
bsolo 3.2 (complete) |
|
clasp 2.0-R4191 (complete) |
|
MinisatID 2.4.8 [DEPRECATED] (complete) |
|
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
|
MinisatID 2.5.2 (fixed) (complete) |
|
MinisatID 2.5.2-gmp (fixed) (complete) |
|
pwbo 1.1 (complete) |
|
Sat4j CuttingPlanes 2.3.0 (complete) |
|
Sat4j Res//CP 2.3.0 (complete) |
|
Sat4j Resolution 2.3.0 (complete) |
|
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx 2 2011-06-10 (fixed) (complete) |
|
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx E_2 2011-06-10 (fixed) (complete) |
|
wbo 1.6 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT | OPT/SAT | SAT (TO/MO) | UNKNOWN | Bad Answers |
---|---|---|---|---|---|---|---|
borg pb-opt-11.04.03 (complete) |
409 | 274 (66.99%) |
0 (0%) |
274 (66.99%) |
0 (0%) |
128 (31.3%) |
7 (1.71%) |
bsolo 3.2 (complete) |
409 | 234 (57.21%) |
0 (0%) |
234 (57.21%) |
6 (1.47%) |
6 (1.47%) |
0 (0%) |
clasp 2.0-R4191 [DEPRECATED] (complete) |
409 | 280 (68.46%) |
0 (0%) |
280 (68.46%) |
125 (30.56%) |
0 (0%) |
4 (0.98%) |
clasp 2.0-R4191-patched (fixed) (complete) |
409 | 280 (68.46%) |
0 (0%) |
280 (68.46%) |
129 (31.54%) |
0 (0%) |
0 (0%) |
MinisatID 2.4.8 [DEPRECATED] (complete) |
409 | 181 (44.25%) |
0 (0%) |
181 (44.25%) |
0 (0%) |
228 (55.75%) |
0 (0%) |
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
409 | 158 (38.63%) |
0 (0%) |
158 (38.63%) |
0 (0%) |
251 (61.37%) |
0 (0%) |
MinisatID 2.5.2 (fixed) (complete) |
409 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
409 (100%) |
0 (0%) |
MinisatID 2.5.2-gmp (fixed) (complete) |
409 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
409 (100%) |
0 (0%) |
Sat4j CuttingPlanes 2.3.0 (complete) |
409 | 120 (29.34%) |
0 (0%) |
120 (29.34%) |
240 (58.68%) |
49 (11.98%) |
0 (0%) |
Sat4j Res//CP 2.3.0 (complete) |
409 | 274 (66.99%) |
0 (0%) |
274 (66.99%) |
135 (33.01%) |
0 (0%) |
0 (0%) |
Sat4j Resolution 2.3.0 (complete) |
409 | 277 (67.73%) |
0 (0%) |
277 (67.73%) |
132 (32.27%) |
0 (0%) |
0 (0%) |
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
409 | 294 (71.88%) |
0 (0%) |
294 (71.88%) |
81 (19.8%) |
20 (4.89%) |
0 (0%) |
SCIP spx 2 2011-06-10 (fixed) (complete) |
409 | 294 (71.88%) |
0 (0%) |
294 (71.88%) |
0 (0%) |
12 (2.93%) |
0 (0%) |
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
409 | 294 (71.88%) |
0 (0%) |
294 (71.88%) |
79 (19.32%) |
30 (7.33%) |
0 (0%) |
SCIP spx E_2 2011-06-10 (fixed) (complete) |
409 | 297 (72.62%) |
0 (0%) |
297 (72.62%) |
0 (0%) |
8 (1.96%) |
0 (0%) |
Solver Name | Progress | borg pb-opt-11.04.03 (complete) |
|
bsolo 3.2 (complete) |
|
clasp 2.0-R4191 [DEPRECATED] (complete) |
|
clasp 2.0-R4191-patched (fixed) (complete) |
|
MinisatID 2.4.8 [DEPRECATED] (complete) |
|
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
|
MinisatID 2.5.2 (fixed) (complete) |
|
MinisatID 2.5.2-gmp (fixed) (complete) |
|
Sat4j CuttingPlanes 2.3.0 (complete) |
|
Sat4j Res//CP 2.3.0 (complete) |
|
Sat4j Resolution 2.3.0 (complete) |
|
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx 2 2011-06-10 (fixed) (complete) |
|
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx E_2 2011-06-10 (fixed) (complete) |
|
---|
Solver Name | #benchs | Solved | UNSAT | OPT/SAT | SAT (TO/MO) | UNKNOWN | Bad Answers |
---|---|---|---|---|---|---|---|
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
2278 | 824 (36.17%) |
212 (9.31%) |
612 (26.87%) |
0 (0%) |
1453 (63.78%) |
1 (0.04%) |
MinisatID 2.5.2-gmp (fixed) (complete) |
2278 | 741 (32.53%) |
284 (12.47%) |
457 (20.06%) |
0 (0%) |
1537 (67.47%) |
0 (0%) |
Sat4j CuttingPlanes 2.3.0 (complete) |
2278 | 872 (38.28%) |
232 (10.18%) |
640 (28.09%) |
796 (34.94%) |
610 (26.78%) |
0 (0%) |
Sat4j Res//CP 2.3.0 (complete) |
2278 | 1296 (56.89%) |
345 (15.14%) |
951 (41.75%) |
680 (29.85%) |
302 (13.26%) |
0 (0%) |
Sat4j Resolution 2.3.0 (complete) |
2278 | 1213 (53.25%) |
314 (13.78%) |
899 (39.46%) |
691 (30.33%) |
374 (16.42%) |
0 (0%) |
Solver Name | Progress | MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
|
MinisatID 2.5.2-gmp (fixed) (complete) |
|
Sat4j CuttingPlanes 2.3.0 (complete) |
|
Sat4j Res//CP 2.3.0 (complete) |
|
Sat4j Resolution 2.3.0 (complete) |
|
---|
This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.
Solver Name | #benchs | Solved | UNSAT | OPT/SAT | SAT (TO/MO) | UNKNOWN | Bad Answers |
---|---|---|---|---|---|---|---|
borg pb-dec-11.04.03 (complete) |
581 | 504 (86.75%) |
268 (46.13%) |
236 (40.62%) |
0 (0%) |
77 (13.25%) |
0 (0%) |
borg pb-opt-11.04.03 (complete) |
1140 | 687 (60.26%) |
35 (3.07%) |
652 (57.19%) |
0 (0%) |
411 (36.05%) |
42 (3.68%) |
bsolo 3.2 (complete) |
1721 | 1084 (62.99%) |
286 (16.62%) |
798 (46.37%) |
16 (0.93%) |
154 (8.95%) |
0 (0%) |
clasp 2.0-R4191 (complete) |
1312 | 764 (58.23%) |
261 (19.89%) |
503 (38.34%) |
336 (25.61%) |
212 (16.16%) |
0 (0%) |
clasp 2.0-R4191 [DEPRECATED] (complete) |
409 | 280 (68.46%) |
0 (0%) |
280 (68.46%) |
125 (30.56%) |
0 (0%) |
4 (0.98%) |
clasp 2.0-R4191-patched (fixed) (complete) |
409 | 280 (68.46%) |
0 (0%) |
280 (68.46%) |
129 (31.54%) |
0 (0%) |
0 (0%) |
MinisatID 2.4.8 [DEPRECATED] (complete) |
1721 | 769 (44.68%) |
160 (9.3%) |
609 (35.39%) |
0 (0%) |
951 (55.26%) |
1 (0.06%) |
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
2278 | 824 (36.17%) |
212 (9.31%) |
612 (26.87%) |
0 (0%) |
1453 (63.78%) |
1 (0.04%) |
MinisatID 2.5.2 (fixed) (complete) |
1721 | 656 (38.12%) |
231 (13.42%) |
425 (24.69%) |
0 (0%) |
1065 (61.88%) |
0 (0%) |
MinisatID 2.5.2-gmp (fixed) (complete) |
2278 | 741 (32.53%) |
284 (12.47%) |
457 (20.06%) |
0 (0%) |
1537 (67.47%) |
0 (0%) |
pwbo 1.1 (complete) |
731 | 383 (52.39%) |
33 (4.51%) |
350 (47.88%) |
297 (40.63%) |
51 (6.98%) |
0 (0%) |
Sat4j CuttingPlanes 2.3.0 (complete) |
2278 | 872 (38.28%) |
232 (10.18%) |
640 (28.09%) |
796 (34.94%) |
610 (26.78%) |
0 (0%) |
Sat4j Res//CP 2.3.0 (complete) |
2278 | 1296 (56.89%) |
345 (15.14%) |
951 (41.75%) |
680 (29.85%) |
302 (13.26%) |
0 (0%) |
Sat4j Resolution 2.3.0 (complete) |
2278 | 1213 (53.25%) |
314 (13.78%) |
899 (39.46%) |
691 (30.33%) |
374 (16.42%) |
0 (0%) |
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
1721 | 1147 (66.65%) |
291 (16.91%) |
856 (49.74%) |
201 (11.68%) |
346 (20.1%) |
0 (0%) |
SCIP spx 2 2011-06-10 (fixed) (complete) |
1721 | 1175 (68.27%) |
290 (16.85%) |
885 (51.42%) |
9 (0.52%) |
300 (17.43%) |
0 (0%) |
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
1721 | 1165 (67.69%) |
290 (16.85%) |
875 (50.84%) |
193 (11.21%) |
349 (20.28%) |
1 (0.06%) |
SCIP spx E_2 2011-06-10 (fixed) (complete) |
1721 | 1180 (68.56%) |
290 (16.85%) |
890 (51.71%) |
12 (0.7%) |
293 (17.02%) |
0 (0%) |
wbo 1.6 (complete) |
1212 | 665 (54.87%) |
247 (20.38%) |
418 (34.49%) |
0 (0%) |
538 (44.39%) |
0 (0%) |
Solver Name | Progress | borg pb-dec-11.04.03 (complete) |
|
borg pb-opt-11.04.03 (complete) |
|
bsolo 3.2 (complete) |
|
clasp 2.0-R4191 (complete) |
|
clasp 2.0-R4191 [DEPRECATED] (complete) |
|
clasp 2.0-R4191-patched (fixed) (complete) |
|
MinisatID 2.4.8 [DEPRECATED] (complete) |
|
MinisatID 2.4.8-gmp [DEPRECATED] (complete) |
|
MinisatID 2.5.2 (fixed) (complete) |
|
MinisatID 2.5.2-gmp (fixed) (complete) |
|
pwbo 1.1 (complete) |
|
Sat4j CuttingPlanes 2.3.0 (complete) |
|
Sat4j Res//CP 2.3.0 (complete) |
|
Sat4j Resolution 2.3.0 (complete) |
|
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx 2 2011-06-10 (fixed) (complete) |
|
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete) |
|
SCIP spx E_2 2011-06-10 (fixed) (complete) |
|
wbo 1.6 (complete) |
|
---|
WARNING: 304 jobs are unclassified for solver 1955 in category OPT-SMALLINT-LIN/
WARNING: 13 jobs are unclassified for solver 1949 in category OPT-SMALLINT-LIN/
WARNING: 134 jobs are unclassified for solver 1977 in category OPT-SMALLINT-LIN/
WARNING: 7 jobs are unclassified for solver 1948 in category OPT-SMALLINT-LIN/
WARNING: 132 jobs are unclassified for solver 1979 in category OPT-SMALLINT-LIN/
WARNING: 9 jobs are unclassified for solver 1953 in category OPT-SMALLINT-LIN/
WARNING: 163 jobs are unclassified for solver 1955 in category OPT-SMALLINT-NLC/
WARNING: 14 jobs are unclassified for solver 1949 in category OPT-SMALLINT-NLC/
WARNING: 103 jobs are unclassified for solver 1977 in category OPT-SMALLINT-NLC/
WARNING: 6 jobs are unclassified for solver 1948 in category OPT-SMALLINT-NLC/
WARNING: 104 jobs are unclassified for solver 1979 in category OPT-SMALLINT-NLC/
WARNING: 467 jobs are unclassified for solver 1955 in category /
WARNING: 27 jobs are unclassified for solver 1949 in category /
WARNING: 237 jobs are unclassified for solver 1977 in category /
WARNING: 13 jobs are unclassified for solver 1948 in category /
WARNING: 236 jobs are unclassified for solver 1979 in category /
WARNING: 9 jobs are unclassified for solver 1953 in category /