solver | a "light red" background denotes a solver which was disqualified because it gave an incorrect answer, and which will not appear in the ranking |
solver | a "light yellow" background denotes a solver which was replaced by another version, and which will not appear in the ranking |
Solver Name | #benchs | Solved | UNSATC | UNSAT. | OPTC | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG MSG. | WRONG CERT. | WRONG OPT. | WRONG OPTC | WRONG UNSAT. | WRONG UNSATC |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CASHWMaxSAT-DisjCad-S 2024-07-10 (complete) |
397 | 273 (68.77%) |
0 (0%) |
149 (37.53%) |
0 (0%) |
0 (0%) |
124 (31.23%) |
0 (0%) |
0 (0%) |
0 (0%) |
15 (3.78%) |
0 (0%) |
109 (27.46%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
CASHWMaxSAT-DisjCad-SP 2024-07-10 (complete) |
397 | 268 (67.51%) |
0 (0%) |
149 (37.53%) |
0 (0%) |
0 (0%) |
119 (29.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
14 (3.53%) |
0 (0%) |
114 (28.72%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
CASHWMaxSAT-DisjCom-S 2024-07-10 (complete) |
397 | 267 (67.25%) |
0 (0%) |
154 (38.79%) |
0 (0%) |
0 (0%) |
113 (28.46%) |
0 (0%) |
0 (0%) |
0 (0%) |
14 (3.53%) |
0 (0%) |
115 (28.97%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
CASHWMaxSAT-DisjCom-SP 2024-07-10 (complete) |
397 | 267 (67.25%) |
0 (0%) |
154 (38.79%) |
0 (0%) |
0 (0%) |
113 (28.46%) |
0 (0%) |
0 (0%) |
0 (0%) |
12 (3.02%) |
0 (0%) |
115 (28.97%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
2 (0.5%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
DLS-PBO 2024-07-09 (incomplete) |
397 | 121 (30.48%) |
0 (0%) |
10 (2.52%) |
0 (0%) |
0 (0%) |
111 (27.96%) |
0 (0%) |
0 (0%) |
0 (0%) |
269 (67.76%) |
0 (0%) |
4 (1.01%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact 2024-07-05 (complete) |
397 | 286 (72.04%) |
0 (0%) |
184 (46.35%) |
0 (0%) |
0 (0%) |
102 (25.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (0.5%) |
0 (0%) |
106 (26.7%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_no_SoPlex 2024-07-05 (complete) |
397 | 275 (69.27%) |
0 (0%) |
171 (43.07%) |
0 (0%) |
0 (0%) |
104 (26.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (0.5%) |
0 (0%) |
117 (29.47%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_veripb2 2024-07-05 (complete) |
397 | 291 (73.3%) |
175 (44.08%) |
8 (2.02%) |
0 (0%) |
0 (0%) |
108 (27.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
106 (26.7%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_veripb2_no_SoPlex 2024-07-05 (complete) |
397 | 284 (71.54%) |
165 (41.56%) |
7 (1.76%) |
0 (0%) |
0 (0%) |
112 (28.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
113 (28.46%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-02 (complete) |
397 | 236 (59.45%) |
0 (0%) |
159 (40.05%) |
0 (0%) |
0 (0%) |
77 (19.4%) |
0 (0%) |
0 (0%) |
0 (0%) |
151 (38.04%) |
0 (0%) |
9 (2.27%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-30 (complete) |
397 | 235 (59.19%) |
0 (0%) |
159 (40.05%) |
0 (0%) |
0 (0%) |
76 (19.14%) |
0 (0%) |
0 (0%) |
0 (0%) |
152 (38.29%) |
0 (0%) |
9 (2.27%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-07-30 (complete) |
397 | 268 (67.51%) |
0 (0%) |
167 (42.07%) |
0 (0%) |
0 (0%) |
101 (25.44%) |
0 (0%) |
0 (0%) |
0 (0%) |
6 (1.51%) |
0 (0%) |
120 (30.23%) |
2 (0.5%) |
0 (0%) |
0 (0%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-09 (complete) |
397 | 269 (67.76%) |
0 (0%) |
170 (42.82%) |
0 (0%) |
0 (0%) |
99 (24.94%) |
0 (0%) |
0 (0%) |
0 (0%) |
6 (1.51%) |
0 (0%) |
119 (29.97%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-12 (complete) |
397 | 270 (68.01%) |
0 (0%) |
169 (42.57%) |
0 (0%) |
0 (0%) |
101 (25.44%) |
0 (0%) |
0 (0%) |
0 (0%) |
92 (23.17%) |
0 (0%) |
21 (5.29%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
11 (2.77%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-CASHWMaxSATDisjCadSP+Exact 2024-07-11 (complete) |
397 | 312 (78.59%) |
0 (0%) |
186 (46.85%) |
0 (0%) |
0 (0%) |
126 (31.74%) |
0 (0%) |
0 (0%) |
0 (0%) |
79 (19.9%) |
0 (0%) |
0 (0%) |
3 (0.76%) |
0 (0%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-CASHWMaxSATDisjComS+Exact 2024-07-11 (complete) |
397 | 309 (77.83%) |
0 (0%) |
186 (46.85%) |
0 (0%) |
0 (0%) |
123 (30.98%) |
0 (0%) |
0 (0%) |
0 (0%) |
84 (21.16%) |
0 (0%) |
0 (0%) |
4 (1.01%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-LSMabP+Exact 2024-07-11 (incomplete) |
397 | 117 (29.47%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
117 (29.47%) |
0 (0%) |
0 (0%) |
0 (0%) |
277 (69.77%) |
0 (0%) |
0 (0%) |
1 (0.25%) |
0 (0%) |
2 (0.5%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-LSMabSP+CASHWMaxSATDisjCom 2024-07-11 (incomplete) |
397 | 130 (32.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
130 (32.75%) |
0 (0%) |
0 (0%) |
0 (0%) |
267 (67.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-LSMabSP+Exact 2024-07-11 (incomplete) |
397 | 115 (28.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
115 (28.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
280 (70.53%) |
0 (0%) |
0 (0%) |
2 (0.5%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-Mab 2024-07-10 (incomplete) |
397 | 105 (26.45%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
105 (26.45%) |
0 (0%) |
0 (0%) |
0 (0%) |
279 (70.28%) |
0 (0%) |
10 (2.52%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-MabP 2024-07-10 (incomplete) |
397 | 108 (27.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
108 (27.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
275 (69.27%) |
0 (0%) |
11 (2.77%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-MabS 2024-07-10 (incomplete) |
397 | 110 (27.71%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
110 (27.71%) |
0 (0%) |
0 (0%) |
0 (0%) |
276 (69.52%) |
0 (0%) |
8 (2.02%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-MabSP 2024-07-10 (incomplete) |
397 | 112 (28.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
112 (28.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
270 (68.01%) |
0 (0%) |
12 (3.02%) |
3 (0.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NaPS 1.03 (complete) |
397 | 273 (68.77%) |
0 (0%) |
151 (38.04%) |
0 (0%) |
0 (0%) |
122 (30.73%) |
0 (0%) |
0 (0%) |
0 (0%) |
11 (2.77%) |
0 (0%) |
108 (27.2%) |
0 (0%) |
5 (1.26%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NaPS-GM 1.03 (complete) |
397 | 241 (60.71%) |
0 (0%) |
150 (37.78%) |
0 (0%) |
0 (0%) |
91 (22.92%) |
0 (0%) |
0 (0%) |
0 (0%) |
11 (2.77%) |
0 (0%) |
140 (35.26%) |
1 (0.25%) |
4 (1.01%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NaPS-PB16 1.02b5 (complete) |
397 | 236 (59.45%) |
0 (0%) |
150 (37.78%) |
0 (0%) |
0 (0%) |
86 (21.66%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.25%) |
0 (0%) |
158 (39.8%) |
2 (0.5%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
ParLS-PBO-1.1 2024-07-09 (incomplete) |
397 | 122 (30.73%) |
0 (0%) |
10 (2.52%) |
0 (0%) |
0 (0%) |
112 (28.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
7 (1.76%) |
0 (0%) |
268 (67.51%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
ParLS-PBO-1.1 20 cores, 2024-07-09 (incomplete) |
397 | 134 (33.75%) |
0 (0%) |
10 (2.52%) |
0 (0%) |
0 (0%) |
124 (31.23%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
263 (66.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Picat 2024-06-18 (complete) |
397 | 273 (68.77%) |
0 (0%) |
150 (37.78%) |
0 (0%) |
0 (0%) |
123 (30.98%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
121 (30.48%) |
0 (0%) |
0 (0%) |
1 (0.25%) |
2 (0.5%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat log (1) (complete) |
397 | 98 (24.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
98 (24.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
110 (27.71%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
189 (47.61%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat log (2) (complete) |
397 | 287 (72.29%) |
189 (47.61%) |
0 (0%) |
0 (0%) |
0 (0%) |
98 (24.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
110 (27.71%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat log (3) (complete) |
397 | 288 (72.54%) |
189 (47.61%) |
0 (0%) |
0 (0%) |
0 (0%) |
99 (24.94%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
109 (27.46%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat nolog (1) (complete) |
397 | 97 (24.43%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
97 (24.43%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
184 (46.35%) |
116 (29.22%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat nolog (2) (complete) |
397 | 282 (71.03%) |
0 (0%) |
184 (46.35%) |
0 (0%) |
0 (0%) |
98 (24.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
115 (28.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
397 | 199 (50.13%) |
0 (0%) |
139 (35.01%) |
0 (0%) |
0 (0%) |
60 (15.11%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
198 (49.87%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
397 | 217 (54.66%) |
0 (0%) |
149 (37.53%) |
0 (0%) |
0 (0%) |
68 (17.13%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
180 (45.34%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Resolution 2024 2024-07-07 (complete) |
397 | 244 (61.46%) |
141 (35.52%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
102 (25.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
153 (38.54%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Resolution Default 2024-07-07 (complete) |
397 | 246 (61.96%) |
0 (0%) |
142 (35.77%) |
0 (0%) |
0 (0%) |
104 (26.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
150 (37.78%) |
0 (0%) |
1 (0.25%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
SCIP 2024-07-07 (complete) |
397 | 241 (60.71%) |
0 (0%) |
161 (40.55%) |
0 (0%) |
0 (0%) |
80 (20.15%) |
0 (0%) |
0 (0%) |
0 (0%) |
131 (33%) |
0 (0%) |
25 (6.3%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | CASHWMaxSAT-DisjCad-S 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCad-SP 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCom-S 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCom-SP 2024-07-10 (complete) |
|
DLS-PBO 2024-07-09 (incomplete) |
|
Exact 2024-07-05 (complete) |
|
Exact_no_SoPlex 2024-07-05 (complete) |
|
Exact_veripb2 2024-07-05 (complete) |
|
Exact_veripb2_no_SoPlex 2024-07-05 (complete) |
|
FiberSCIP 2024-07-02 (complete) |
|
FiberSCIP 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-08-09 (complete) |
|
FiberSCIP 20 cores, 2024-08-12 (complete) |
|
Hybrid-CASHWMaxSATDisjCadSP+Exact 2024-07-11 (complete) |
|
Hybrid-CASHWMaxSATDisjComS+Exact 2024-07-11 (complete) |
|
Hybrid-LSMabP+Exact 2024-07-11 (incomplete) |
|
Hybrid-LSMabSP+CASHWMaxSATDisjCom 2024-07-11 (incomplete) |
|
Hybrid-LSMabSP+Exact 2024-07-11 (incomplete) |
|
LS-Mab 2024-07-10 (incomplete) |
|
LS-MabP 2024-07-10 (incomplete) |
|
LS-MabS 2024-07-10 (incomplete) |
|
LS-MabSP 2024-07-10 (incomplete) |
|
NaPS 1.03 (complete) |
|
NaPS-GM 1.03 (complete) |
|
NaPS-PB16 1.02b5 (complete) |
|
ParLS-PBO-1.1 2024-07-09 (incomplete) |
|
ParLS-PBO-1.1 20 cores, 2024-07-09 (incomplete) |
|
Picat 2024-06-18 (complete) |
|
RoundingSat log (1) (complete) |
|
RoundingSat log (2) (complete) |
|
RoundingSat log (3) (complete) |
|
RoundingSat nolog (1) (complete) |
|
RoundingSat nolog (2) (complete) |
|
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
|
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
|
Sat4j Resolution 2024 2024-07-07 (complete) |
|
Sat4j Resolution Default 2024-07-07 (complete) |
|
SCIP 2024-07-07 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSATC | UNSAT. | OPTC | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG MSG. | WRONG CERT. | WRONG OPT. | WRONG OPTC | WRONG UNSAT. | WRONG UNSATC |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Exact 2024-07-05 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_no_SoPlex 2024-07-05 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-02 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-30 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-07-30 (complete) |
10 | 10 (100%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
8 (80%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-09 (complete) |
10 | 10 (100%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
8 (80%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-12 (complete) |
10 | 10 (100%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
8 (80%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Picat 2024-06-18 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
10 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
10 (100%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Resolution Default 2024-07-07 (complete) |
10 | 3 (30%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
SCIP 2024-07-07 (complete) |
10 | 9 (90%) |
0 (0%) |
2 (20%) |
0 (0%) |
0 (0%) |
7 (70%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (10%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | Exact 2024-07-05 (complete) |
|
Exact_no_SoPlex 2024-07-05 (complete) |
|
FiberSCIP 2024-07-02 (complete) |
|
FiberSCIP 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-08-09 (complete) |
|
FiberSCIP 20 cores, 2024-08-12 (complete) |
|
Picat 2024-06-18 (complete) |
|
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
|
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
|
Sat4j Resolution Default 2024-07-07 (complete) |
|
SCIP 2024-07-07 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSATC | UNSAT. | OPTC | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG MSG. | WRONG CERT. | WRONG OPT. | WRONG OPTC | WRONG UNSAT. | WRONG UNSATC |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CASHWMaxSAT-DisjCad-S 2024-07-10 (complete) |
478 | 235 (49.16%) |
0 (0%) |
12 (2.51%) |
0 (0%) |
223 (46.65%) |
0 (0%) |
0 (0%) |
190 (39.75%) |
1 (0.21%) |
3 (0.63%) |
0 (0%) |
47 (9.83%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
CASHWMaxSAT-DisjCad-SP 2024-07-10 (complete) |
478 | 235 (49.16%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
222 (46.44%) |
0 (0%) |
0 (0%) |
192 (40.17%) |
2 (0.42%) |
2 (0.42%) |
0 (0%) |
44 (9.21%) |
3 (0.63%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
CASHWMaxSAT-DisjCom-S 2024-07-10 (complete) |
478 | 232 (48.54%) |
0 (0%) |
12 (2.51%) |
0 (0%) |
220 (46.03%) |
0 (0%) |
0 (0%) |
191 (39.96%) |
1 (0.21%) |
3 (0.63%) |
0 (0%) |
48 (10.04%) |
3 (0.63%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
CASHWMaxSAT-DisjCom-SP 2024-07-10 (complete) |
478 | 231 (48.33%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
218 (45.61%) |
0 (0%) |
0 (0%) |
195 (40.79%) |
2 (0.42%) |
1 (0.21%) |
0 (0%) |
44 (9.21%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
3 (0.63%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
DLS-PBO 2024-07-09 (incomplete) |
478 | 8 (1.67%) |
0 (0%) |
8 (1.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
399 (83.47%) |
13 (2.72%) |
0 (0%) |
33 (6.9%) |
0 (0%) |
1 (0.21%) |
11 (2.3%) |
0 (0%) |
0 (0%) |
0 (0%) |
11 (2.3%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact 2024-07-05 (complete) |
478 | 252 (52.72%) |
0 (0%) |
14 (2.93%) |
0 (0%) |
238 (49.79%) |
0 (0%) |
4 (0.84%) |
183 (38.28%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
38 (7.95%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_no_SoPlex 2024-07-05 (complete) |
478 | 226 (47.28%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
213 (44.56%) |
0 (0%) |
6 (1.26%) |
226 (47.28%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
19 (3.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_veripb2 2024-07-05 (complete) |
478 | 239 (50%) |
13 (2.72%) |
1 (0.21%) |
169 (35.36%) |
56 (11.72%) |
0 (0%) |
6 (1.26%) |
187 (39.12%) |
0 (0%) |
0 (0%) |
0 (0%) |
46 (9.62%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_veripb2_no_SoPlex 2024-07-05 (complete) |
478 | 195 (40.79%) |
12 (2.51%) |
1 (0.21%) |
144 (30.13%) |
38 (7.95%) |
0 (0%) |
5 (1.05%) |
239 (50%) |
0 (0%) |
0 (0%) |
0 (0%) |
39 (8.16%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-02 (complete) |
478 | 253 (52.93%) |
0 (0%) |
15 (3.14%) |
0 (0%) |
238 (49.79%) |
0 (0%) |
164 (34.31%) |
0 (0%) |
0 (0%) |
43 (9%) |
0 (0%) |
9 (1.88%) |
7 (1.46%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-30 (complete) |
478 | 245 (51.26%) |
0 (0%) |
15 (3.14%) |
0 (0%) |
230 (48.12%) |
0 (0%) |
173 (36.19%) |
0 (0%) |
0 (0%) |
43 (9%) |
0 (0%) |
9 (1.88%) |
7 (1.46%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-07-30 (complete) |
478 | 263 (55.02%) |
0 (0%) |
17 (3.56%) |
0 (0%) |
246 (51.46%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
203 (42.47%) |
7 (1.46%) |
0 (0%) |
0 (0%) |
5 (1.05%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-09 (complete) |
478 | 259 (54.18%) |
0 (0%) |
16 (3.35%) |
0 (0%) |
243 (50.84%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
207 (43.31%) |
8 (1.67%) |
0 (0%) |
0 (0%) |
4 (0.84%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-12 (complete) |
478 | 245 (51.26%) |
0 (0%) |
15 (3.14%) |
0 (0%) |
230 (48.12%) |
0 (0%) |
157 (32.85%) |
0 (0%) |
0 (0%) |
16 (3.35%) |
0 (0%) |
38 (7.95%) |
9 (1.88%) |
0 (0%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-CASHWMaxSATDisjCadSP+Exact 2024-07-11 (complete) |
478 | 259 (54.18%) |
0 (0%) |
14 (2.93%) |
0 (0%) |
245 (51.26%) |
0 (0%) |
197 (41.21%) |
0 (0%) |
0 (0%) |
14 (2.93%) |
0 (0%) |
5 (1.05%) |
0 (0%) |
0 (0%) |
3 (0.63%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-CASHWMaxSATDisjComS+Exact 2024-07-11 (complete) |
478 | 254 (53.14%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
241 (50.42%) |
0 (0%) |
199 (41.63%) |
0 (0%) |
0 (0%) |
21 (4.39%) |
0 (0%) |
4 (0.84%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-LSMabP+Exact 2024-07-11 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
447 (93.51%) |
0 (0%) |
0 (0%) |
24 (5.02%) |
0 (0%) |
5 (1.05%) |
0 (0%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-LSMabSP+CASHWMaxSATDisjCom 2024-07-11 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
439 (91.84%) |
0 (0%) |
0 (0%) |
38 (7.95%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Hybrid-LSMabSP+Exact 2024-07-11 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
449 (93.93%) |
0 (0%) |
0 (0%) |
28 (5.86%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
IPBHS-GUROBI 2024-07-09 (complete) |
478 | 257 (53.77%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
244 (51.05%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
221 (46.23%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
IPBHS-GUROBI-SYM 2024-07-09 (complete) |
478 | 258 (53.97%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
245 (51.26%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
219 (45.82%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
IPBHS-SCIP 2024-07-09 (complete) |
478 | 245 (51.26%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
232 (48.54%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
230 (48.12%) |
0 (0%) |
0 (0%) |
0 (0%) |
3 (0.63%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-Mab 2024-07-10 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
398 (83.26%) |
34 (7.11%) |
0 (0%) |
34 (7.11%) |
0 (0%) |
1 (0.21%) |
11 (2.3%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-MabP 2024-07-10 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
433 (90.59%) |
0 (0%) |
0 (0%) |
34 (7.11%) |
0 (0%) |
1 (0.21%) |
10 (2.09%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-MabS 2024-07-10 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
431 (90.17%) |
0 (0%) |
0 (0%) |
34 (7.11%) |
0 (0%) |
2 (0.42%) |
11 (2.3%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
LS-MabSP 2024-07-10 (incomplete) |
478 | 0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
433 (90.59%) |
0 (0%) |
0 (0%) |
33 (6.9%) |
0 (0%) |
2 (0.42%) |
10 (2.09%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
mixed-bag 2024-07-02 (complete) |
478 | 273 (57.11%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
260 (54.39%) |
0 (0%) |
0 (0%) |
87 (18.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
107 (22.38%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
7 (1.46%) |
0 (0%) |
mixed-bag 2024-07-19 (complete) |
478 | 279 (58.37%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
266 (55.65%) |
0 (0%) |
0 (0%) |
87 (18.2%) |
0 (0%) |
0 (0%) |
0 (0%) |
107 (22.38%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
2 (0.42%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NaPS 1.03 (complete) |
478 | 208 (43.51%) |
0 (0%) |
12 (2.51%) |
0 (0%) |
196 (41%) |
0 (0%) |
1 (0.21%) |
205 (42.89%) |
3 (0.63%) |
1 (0.21%) |
0 (0%) |
30 (6.28%) |
5 (1.05%) |
25 (5.23%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NaPS-GM 1.03 (complete) |
478 | 204 (42.68%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
191 (39.96%) |
0 (0%) |
2 (0.42%) |
207 (43.31%) |
5 (1.05%) |
0 (0%) |
0 (0%) |
35 (7.32%) |
6 (1.26%) |
19 (3.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
NaPS-PB16 1.02b5 (complete) |
478 | 203 (42.47%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
190 (39.75%) |
0 (0%) |
17 (3.56%) |
199 (41.63%) |
8 (1.67%) |
2 (0.42%) |
0 (0%) |
40 (8.37%) |
8 (1.67%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
ParLS-PBO-1.1 2024-07-09 (incomplete) |
478 | 8 (1.67%) |
0 (0%) |
8 (1.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
43 (9%) |
387 (80.96%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
37 (7.74%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
ParLS-PBO-1.1 20 cores, 2024-07-09 (incomplete) |
478 | 8 (1.67%) |
0 (0%) |
8 (1.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
3 (0.63%) |
436 (91.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
31 (6.49%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
pb-oll-rs 2024-07-02 (complete) |
478 | 207 (43.31%) |
0 (0%) |
12 (2.51%) |
0 (0%) |
195 (40.79%) |
0 (0%) |
0 (0%) |
218 (45.61%) |
0 (0%) |
0 (0%) |
0 (0%) |
46 (9.62%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
5 (1.05%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
pb-oll-rs 2024-07-19 (complete) |
478 | 208 (43.51%) |
0 (0%) |
12 (2.51%) |
0 (0%) |
196 (41%) |
0 (0%) |
0 (0%) |
217 (45.4%) |
0 (0%) |
0 (0%) |
0 (0%) |
47 (9.83%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
5 (1.05%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Picat 2024-06-18 (complete) |
478 | 201 (42.05%) |
0 (0%) |
14 (2.93%) |
0 (0%) |
187 (39.12%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
275 (57.53%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat log (1) (complete) |
478 | 229 (47.91%) |
0 (0%) |
0 (0%) |
202 (42.26%) |
27 (5.65%) |
0 (0%) |
4 (0.84%) |
166 (34.73%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
41 (8.58%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
21 (4.39%) |
0 (0%) |
0 (0%) |
0 (0%) |
14 (2.93%) |
0 (0%) |
0 (0%) |
RoundingSat log (2) (complete) |
478 | 245 (51.26%) |
13 (2.72%) |
0 (0%) |
204 (42.68%) |
28 (5.86%) |
0 (0%) |
5 (1.05%) |
91 (19.04%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
53 (11.09%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
69 (14.44%) |
0 (0%) |
0 (0%) |
0 (0%) |
11 (2.3%) |
0 (0%) |
0 (0%) |
RoundingSat log (3) (complete) |
478 | 254 (53.14%) |
13 (2.72%) |
0 (0%) |
218 (45.61%) |
23 (4.81%) |
0 (0%) |
5 (1.05%) |
104 (21.76%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
51 (10.67%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
60 (12.55%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
RoundingSat nolog (1) (complete) |
478 | 243 (50.84%) |
0 (0%) |
0 (0%) |
0 (0%) |
243 (50.84%) |
0 (0%) |
4 (0.84%) |
198 (41.42%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
31 (6.49%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
RoundingSat nolog (2) (complete) |
478 | 256 (53.56%) |
0 (0%) |
13 (2.72%) |
0 (0%) |
243 (50.84%) |
0 (0%) |
4 (0.84%) |
163 (34.1%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
53 (11.09%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
478 | 133 (27.82%) |
0 (0%) |
10 (2.09%) |
0 (0%) |
123 (25.73%) |
0 (0%) |
0 (0%) |
258 (53.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
83 (17.36%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
478 | 146 (30.54%) |
0 (0%) |
12 (2.51%) |
0 (0%) |
134 (28.03%) |
0 (0%) |
0 (0%) |
271 (56.69%) |
0 (0%) |
0 (0%) |
0 (0%) |
61 (12.76%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Resolution 2024 2024-07-07 (complete) |
478 | 126 (26.36%) |
6 (1.26%) |
0 (0%) |
118 (24.69%) |
2 (0.42%) |
0 (0%) |
3 (0.63%) |
271 (56.69%) |
0 (0%) |
6 (1.26%) |
0 (0%) |
62 (12.97%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
10 (2.09%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Resolution Default 2024-07-07 (complete) |
478 | 128 (26.78%) |
0 (0%) |
11 (2.3%) |
0 (0%) |
117 (24.48%) |
0 (0%) |
2 (0.42%) |
260 (54.39%) |
0 (0%) |
0 (0%) |
0 (0%) |
58 (12.13%) |
0 (0%) |
15 (3.14%) |
0 (0%) |
0 (0%) |
14 (2.93%) |
0 (0%) |
1 (0.21%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
SCIP 2024-07-07 (complete) |
478 | 263 (55.02%) |
0 (0%) |
15 (3.14%) |
0 (0%) |
248 (51.88%) |
0 (0%) |
153 (32.01%) |
2 (0.42%) |
0 (0%) |
40 (8.37%) |
0 (0%) |
14 (2.93%) |
2 (0.42%) |
0 (0%) |
0 (0%) |
4 (0.84%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | CASHWMaxSAT-DisjCad-S 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCad-SP 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCom-S 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCom-SP 2024-07-10 (complete) |
|
DLS-PBO 2024-07-09 (incomplete) |
|
Exact 2024-07-05 (complete) |
|
Exact_no_SoPlex 2024-07-05 (complete) |
|
Exact_veripb2 2024-07-05 (complete) |
|
Exact_veripb2_no_SoPlex 2024-07-05 (complete) |
|
FiberSCIP 2024-07-02 (complete) |
|
FiberSCIP 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-08-09 (complete) |
|
FiberSCIP 20 cores, 2024-08-12 (complete) |
|
Hybrid-CASHWMaxSATDisjCadSP+Exact 2024-07-11 (complete) |
|
Hybrid-CASHWMaxSATDisjComS+Exact 2024-07-11 (complete) |
|
Hybrid-LSMabP+Exact 2024-07-11 (incomplete) |
|
Hybrid-LSMabSP+CASHWMaxSATDisjCom 2024-07-11 (incomplete) |
|
Hybrid-LSMabSP+Exact 2024-07-11 (incomplete) |
|
IPBHS-GUROBI 2024-07-09 (complete) |
|
IPBHS-GUROBI-SYM 2024-07-09 (complete) |
|
IPBHS-SCIP 2024-07-09 (complete) |
|
LS-Mab 2024-07-10 (incomplete) |
|
LS-MabP 2024-07-10 (incomplete) |
|
LS-MabS 2024-07-10 (incomplete) |
|
LS-MabSP 2024-07-10 (incomplete) |
|
mixed-bag 2024-07-02 (complete) |
|
mixed-bag 2024-07-19 (complete) |
|
NaPS 1.03 (complete) |
|
NaPS-GM 1.03 (complete) |
|
NaPS-PB16 1.02b5 (complete) |
|
ParLS-PBO-1.1 2024-07-09 (incomplete) |
|
ParLS-PBO-1.1 20 cores, 2024-07-09 (incomplete) |
|
pb-oll-rs 2024-07-02 (complete) |
|
pb-oll-rs 2024-07-19 (complete) |
|
Picat 2024-06-18 (complete) |
|
RoundingSat log (1) (complete) |
|
RoundingSat log (2) (complete) |
|
RoundingSat log (3) (complete) |
|
RoundingSat nolog (1) (complete) |
|
RoundingSat nolog (2) (complete) |
|
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
|
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
|
Sat4j Resolution 2024 2024-07-07 (complete) |
|
Sat4j Resolution Default 2024-07-07 (complete) |
|
SCIP 2024-07-07 (complete) |
|
---|
Solver Name | #benchs | Solved | UNSATC | UNSAT. | OPTC | OPT | SAT | SAT (subopt.) | SAT (timeout) | SAT (out of mem.) | UNKNOWN | UNKNOWN (no support) | UNKNOWN (timeout) | UNKNOWN (out of mem.) | UNKNOWN (exit code) | UNKNOWN (problem) | Sig. Caught | NO CERT. | WRONG MSG. | WRONG CERT. | WRONG OPT. | WRONG OPTC | WRONG UNSAT. | WRONG UNSATC |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Exact 2024-07-05 (complete) |
54 | 30 (55.56%) |
0 (0%) |
0 (0%) |
0 (0%) |
30 (55.56%) |
0 (0%) |
0 (0%) |
21 (38.89%) |
0 (0%) |
0 (0%) |
0 (0%) |
3 (5.56%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Exact_no_SoPlex 2024-07-05 (complete) |
54 | 27 (50%) |
0 (0%) |
0 (0%) |
0 (0%) |
27 (50%) |
0 (0%) |
0 (0%) |
27 (50%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-02 (complete) |
54 | 37 (68.52%) |
0 (0%) |
0 (0%) |
0 (0%) |
37 (68.52%) |
0 (0%) |
17 (31.48%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 2024-07-30 (complete) |
54 | 37 (68.52%) |
0 (0%) |
0 (0%) |
0 (0%) |
37 (68.52%) |
0 (0%) |
17 (31.48%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-07-30 (complete) |
54 | 36 (66.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
36 (66.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
17 (31.48%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (1.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-09 (complete) |
54 | 36 (66.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
36 (66.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
17 (31.48%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
1 (1.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
FiberSCIP 20 cores, 2024-08-12 (complete) |
54 | 36 (66.67%) |
0 (0%) |
0 (0%) |
0 (0%) |
36 (66.67%) |
0 (0%) |
18 (33.33%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Picat 2024-06-18 (complete) |
54 | 25 (46.3%) |
0 (0%) |
0 (0%) |
0 (0%) |
25 (46.3%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
29 (53.7%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
54 | 13 (24.07%) |
0 (0%) |
0 (0%) |
0 (0%) |
13 (24.07%) |
0 (0%) |
9 (16.67%) |
28 (51.85%) |
0 (0%) |
2 (3.7%) |
0 (0%) |
2 (3.7%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
54 | 20 (37.04%) |
0 (0%) |
0 (0%) |
0 (0%) |
20 (37.04%) |
0 (0%) |
0 (0%) |
34 (62.96%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Sat4j Resolution Default 2024-07-07 (complete) |
54 | 19 (35.19%) |
0 (0%) |
0 (0%) |
0 (0%) |
19 (35.19%) |
0 (0%) |
0 (0%) |
32 (59.26%) |
0 (0%) |
0 (0%) |
0 (0%) |
2 (3.7%) |
0 (0%) |
1 (1.85%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
SCIP 2024-07-07 (complete) |
54 | 37 (68.52%) |
0 (0%) |
0 (0%) |
0 (0%) |
37 (68.52%) |
0 (0%) |
17 (31.48%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
0 (0%) |
Solver Name | Progress | Exact 2024-07-05 (complete) |
|
Exact_no_SoPlex 2024-07-05 (complete) |
|
FiberSCIP 2024-07-02 (complete) |
|
FiberSCIP 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-08-09 (complete) |
|
FiberSCIP 20 cores, 2024-08-12 (complete) |
|
Picat 2024-06-18 (complete) |
|
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
|
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
|
Sat4j Resolution Default 2024-07-07 (complete) |
|
SCIP 2024-07-07 (complete) |
|
---|
This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.
Solver Name | Progress | CASHWMaxSAT-DisjCad-S 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCad-SP 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCom-S 2024-07-10 (complete) |
|
CASHWMaxSAT-DisjCom-SP 2024-07-10 (complete) |
|
DLS-PBO 2024-07-09 (incomplete) |
|
Exact 2024-07-05 (complete) |
|
Exact_no_SoPlex 2024-07-05 (complete) |
|
Exact_veripb2 2024-07-05 (complete) |
|
Exact_veripb2_no_SoPlex 2024-07-05 (complete) |
|
FiberSCIP 2024-07-02 (complete) |
|
FiberSCIP 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-07-30 (complete) |
|
FiberSCIP 20 cores, 2024-08-09 (complete) |
|
FiberSCIP 20 cores, 2024-08-12 (complete) |
|
Hybrid-CASHWMaxSATDisjCadSP+Exact 2024-07-11 (complete) |
|
Hybrid-CASHWMaxSATDisjComS+Exact 2024-07-11 (complete) |
|
Hybrid-LSMabP+Exact 2024-07-11 (incomplete) |
|
Hybrid-LSMabSP+CASHWMaxSATDisjCom 2024-07-11 (incomplete) |
|
Hybrid-LSMabSP+Exact 2024-07-11 (incomplete) |
|
IPBHS-GUROBI 2024-07-09 (complete) |
|
IPBHS-GUROBI-SYM 2024-07-09 (complete) |
|
IPBHS-SCIP 2024-07-09 (complete) |
|
LS-Mab 2024-07-10 (incomplete) |
|
LS-MabP 2024-07-10 (incomplete) |
|
LS-MabS 2024-07-10 (incomplete) |
|
LS-MabSP 2024-07-10 (incomplete) |
|
mixed-bag 2024-07-02 (complete) |
|
mixed-bag 2024-07-19 (complete) |
|
NaPS 1.03 (complete) |
|
NaPS-GM 1.03 (complete) |
|
NaPS-PB16 1.02b5 (complete) |
|
ParLS-PBO-1.1 2024-07-09 (incomplete) |
|
ParLS-PBO-1.1 20 cores, 2024-07-09 (incomplete) |
|
pb-oll-rs 2024-07-02 (complete) |
|
pb-oll-rs 2024-07-19 (complete) |
|
Picat 2024-06-18 (complete) |
|
RoundingSat log (1) (complete) |
|
RoundingSat log (2) (complete) |
|
RoundingSat log (3) (complete) |
|
RoundingSat nolog (1) (complete) |
|
RoundingSat nolog (2) (complete) |
|
Sat4j Cutting Planes 2024 2024-07-12 (complete) |
|
Sat4j Partial RoundingSAT 2024 2024-07-12 (complete) |
|
Sat4j Resolution 2024 2024-07-07 (complete) |
|
Sat4j Resolution Default 2024-07-07 (complete) |
|
SCIP 2024-07-07 (complete) |
|
---|