PB'11 competition: satisfaction and optimization track: solvers page (tabular format, summary)

Some explanations

Category "no optimisation, small integers, linear constraints" (DEC-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
borg
pb-dec-11.04.03
(complete)		 481431
(89.6%)		 248
(51.56%)		 183
(38.05%)		 0
(0%)		 50
(10.4%)		 0
(0%)
bsolo
3.2
(complete)		 481416
(86.49%)		 237
(49.27%)		 179
(37.21%)		 0
(0%)		 65
(13.51%)		 0
(0%)
clasp
2.0-R4191
(complete)		 481380
(79%)		 212
(44.07%)		 168
(34.93%)		 0
(0%)		 101
(21%)		 0
(0%)
MinisatID
2.4.8 [DEPRECATED]
(complete)		 481267
(55.51%)		 116
(24.12%)		 151
(31.39%)		 0
(0%)		 213
(44.28%)		 1
(0.21%)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 481264
(54.89%)		 115
(23.91%)		 149
(30.98%)		 0
(0%)		 216
(44.91%)		 1
(0.21%)
MinisatID
2.5.2 (fixed)
(complete)		 481368
(76.51%)		 199
(41.37%)		 169
(35.14%)		 0
(0%)		 113
(23.49%)		 0
(0%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 481362
(75.26%)		 197
(40.96%)		 165
(34.3%)		 0
(0%)		 119
(24.74%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 481242
(50.31%)		 128
(26.61%)		 114
(23.7%)		 0
(0%)		 239
(49.69%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 481420
(87.32%)		 237
(49.27%)		 183
(38.05%)		 0
(0%)		 61
(12.68%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 481392
(81.5%)		 208
(43.24%)		 184
(38.25%)		 0
(0%)		 89
(18.5%)		 0
(0%)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 481383
(79.63%)		 235
(48.86%)		 148
(30.77%)		 0
(0%)		 98
(20.37%)		 0
(0%)
SCIP spx 2
2011-06-10 (fixed)
(complete)		 481383
(79.63%)		 235
(48.86%)		 148
(30.77%)		 0
(0%)		 98
(20.37%)		 0
(0%)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 481383
(79.63%)		 235
(48.86%)		 148
(30.77%)		 0
(0%)		 98
(20.37%)		 0
(0%)
SCIP spx E_2
2011-06-10 (fixed)
(complete)		 481384
(79.83%)		 235
(48.86%)		 149
(30.98%)		 0
(0%)		 97
(20.17%)		 0
(0%)
wbo
1.6
(complete)		 481394
(81.91%)		 214
(44.49%)		 180
(37.42%)		 0
(0%)		 87
(18.09%)		 0
(0%)

Solver NameProgress
borg
pb-dec-11.04.03
(complete)
done
481
UNSAT
248		OPT/SAT
183		?
50
bsolo
3.2
(complete)
done
481
UNSAT
237		OPT/SAT
179		?
65
clasp
2.0-R4191
(complete)
done
481
UNSAT
212		OPT/SAT
168		?
101
MinisatID
2.4.8 [DEPRECATED]
(complete)
done
481
UNSAT
116		OPT/SAT
151		?
213		ERR
1
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
481
UNSAT
115		OPT/SAT
149		?
216		ERR
1
MinisatID
2.5.2 (fixed)
(complete)
done
481
UNSAT
199		OPT/SAT
169		?
113
MinisatID
2.5.2-gmp (fixed)
(complete)
done
481
UNSAT
197		OPT/SAT
165		?
119
Sat4j CuttingPlanes
2.3.0
(complete)
done
481
UNSAT
128		OPT/SAT
114		?
239
Sat4j Res//CP
2.3.0
(complete)
done
481
UNSAT
237		OPT/SAT
183		?
61
Sat4j Resolution
2.3.0
(complete)
done
481
UNSAT
208		OPT/SAT
184		?
89
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
481
UNSAT
235		OPT/SAT
148		?
98
SCIP spx 2
2011-06-10 (fixed)
(complete)
done
481
UNSAT
235		OPT/SAT
148		?
98
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
481
UNSAT
235		OPT/SAT
148		?
98
SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
481
UNSAT
235		OPT/SAT
149		?
97
wbo
1.6
(complete)
done
481
UNSAT
214		OPT/SAT
180		?
87

Category "no optimisation, small integers, non linear constraints" (DEC-SMALLINT-NLC)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
borg
pb-dec-11.04.03
(complete)		 10073
(73%)		 20
(20%)		 53
(53%)		 0
(0%)		 27
(27%)		 0
(0%)
bsolo
3.2
(complete)		 10062
(62%)		 15
(15%)		 47
(47%)		 0
(0%)		 38
(38%)		 0
(0%)
clasp
2.0-R4191
(complete)		 10064
(64%)		 15
(15%)		 49
(49%)		 0
(0%)		 36
(36%)		 0
(0%)
MinisatID
2.4.8 [DEPRECATED]
(complete)		 10057
(57%)		 10
(10%)		 47
(47%)		 0
(0%)		 43
(43%)		 0
(0%)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 10057
(57%)		 10
(10%)		 47
(47%)		 0
(0%)		 43
(43%)		 0
(0%)
MinisatID
2.5.2 (fixed)
(complete)		 1000
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 100
(100%)		 0
(0%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 1000
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 100
(100%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 10065
(65%)		 15
(15%)		 50
(50%)		 0
(0%)		 35
(35%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 10065
(65%)		 15
(15%)		 50
(50%)		 0
(0%)		 35
(35%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 10027
(27%)		 15
(15%)		 12
(12%)		 0
(0%)		 73
(73%)		 0
(0%)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 10075
(75%)		 21
(21%)		 54
(54%)		 0
(0%)		 25
(25%)		 0
(0%)
SCIP spx 2
2011-06-10 (fixed)
(complete)		 10074
(74%)		 20
(20%)		 54
(54%)		 0
(0%)		 26
(26%)		 0
(0%)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 10075
(75%)		 20
(20%)		 55
(55%)		 0
(0%)		 25
(25%)		 0
(0%)
SCIP spx E_2
2011-06-10 (fixed)
(complete)		 10075
(75%)		 20
(20%)		 55
(55%)		 0
(0%)		 25
(25%)		 0
(0%)

Solver NameProgress
borg
pb-dec-11.04.03
(complete)
done
100
UNSAT
20		OPT/SAT
53		?
27
bsolo
3.2
(complete)
done
100
UNSAT
15		OPT/SAT
47		?
38
clasp
2.0-R4191
(complete)
done
100
UNSAT
15		OPT/SAT
49		?
36
MinisatID
2.4.8 [DEPRECATED]
(complete)
done
100
UNSAT
10		OPT/SAT
47		?
43
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
100
UNSAT
10		OPT/SAT
47		?
43
MinisatID
2.5.2 (fixed)
(complete)
done
100
?
100
MinisatID
2.5.2-gmp (fixed)
(complete)
done
100
?
100
Sat4j CuttingPlanes
2.3.0
(complete)
done
100
UNSAT
15		OPT/SAT
50		?
35
Sat4j Res//CP
2.3.0
(complete)
done
100
UNSAT
15		OPT/SAT
50		?
35
Sat4j Resolution
2.3.0
(complete)
done
100
UNSAT
15		OPT/SAT
12		?
73
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
100
UNSAT
21		OPT/SAT
54		?
25
SCIP spx 2
2011-06-10 (fixed)
(complete)
done
100
UNSAT
20		OPT/SAT
54		?
26
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
100
UNSAT
20		OPT/SAT
55		?
25
SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
100
UNSAT
20		OPT/SAT
55		?
25

Category "optimisation, big integers, linear constraints" (OPT-BIGINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 557107
(19.21%)		 57
(10.23%)		 50
(8.98%)		 0
(0%)		 450
(80.79%)		 0
(0%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 557117
(21.01%)		 57
(10.23%)		 60
(10.77%)		 0
(0%)		 440
(78.99%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 557175
(31.42%)		 59
(10.59%)		 116
(20.83%)		 175
(31.42%)		 207
(37.16%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 557208
(37.34%)		 59
(10.59%)		 149
(26.75%)		 188
(33.75%)		 161
(28.9%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 557201
(36.09%)		 57
(10.23%)		 144
(25.85%)		 192
(34.47%)		 164
(29.44%)		 0
(0%)

Solver NameProgress
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
557
UNSAT
57		OPT/SAT
50		?
450
MinisatID
2.5.2-gmp (fixed)
(complete)
done
557
UNSAT
57		OPT/SAT
60		?
440
Sat4j CuttingPlanes
2.3.0
(complete)
done
557
UNSAT
59		OPT/SAT
116		SAT (TO/MO)
175		?
207
Sat4j Res//CP
2.3.0
(complete)
done
557
UNSAT
59		OPT/SAT
149		SAT (TO/MO)
188		?
161
Sat4j Resolution
2.3.0
(complete)
done
557
UNSAT
57		OPT/SAT
144		SAT (TO/MO)
192		?
164

Category "optimisation, small integers, linear constraints" (OPT-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
borg
pb-opt-11.04.03
(complete)		 731413
(56.5%)		 35
(4.79%)		 378
(51.71%)		 0
(0%)		 283
(38.71%)		 35
(4.79%)
bsolo
3.2
(complete)		 731372
(50.89%)		 34
(4.65%)		 338
(46.24%)		 10
(1.37%)		 45
(6.16%)		 0
(0%)
clasp
2.0-R4191
(complete)		 731320
(43.78%)		 34
(4.65%)		 286
(39.12%)		 336
(45.96%)		 75
(10.26%)		 0
(0%)
MinisatID
2.4.8 [DEPRECATED]
(complete)		 731264
(36.11%)		 34
(4.65%)		 230
(31.46%)		 0
(0%)		 467
(63.89%)		 0
(0%)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 731238
(32.56%)		 30
(4.1%)		 208
(28.45%)		 0
(0%)		 493
(67.44%)		 0
(0%)
MinisatID
2.5.2 (fixed)
(complete)		 731288
(39.4%)		 32
(4.38%)		 256
(35.02%)		 0
(0%)		 443
(60.6%)		 0
(0%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 731262
(35.84%)		 30
(4.1%)		 232
(31.74%)		 0
(0%)		 469
(64.16%)		 0
(0%)
pwbo
1.1
(complete)		 731383
(52.39%)		 33
(4.51%)		 350
(47.88%)		 297
(40.63%)		 51
(6.98%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 731270
(36.94%)		 30
(4.1%)		 240
(32.83%)		 381
(52.12%)		 80
(10.94%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 731329
(45.01%)		 34
(4.65%)		 295
(40.36%)		 357
(48.84%)		 45
(6.16%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 731316
(43.23%)		 34
(4.65%)		 282
(38.58%)		 367
(50.21%)		 48
(6.57%)		 0
(0%)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 731395
(54.04%)		 35
(4.79%)		 360
(49.25%)		 120
(16.42%)		 203
(27.77%)		 0
(0%)
SCIP spx 2
2011-06-10 (fixed)
(complete)		 731424
(58%)		 35
(4.79%)		 389
(53.21%)		 9
(1.23%)		 164
(22.44%)		 0
(0%)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 731413
(56.5%)		 35
(4.79%)		 378
(51.71%)		 114
(15.6%)		 196
(26.81%)		 1
(0.14%)
SCIP spx E_2
2011-06-10 (fixed)
(complete)		 731424
(58%)		 35
(4.79%)		 389
(53.21%)		 12
(1.64%)		 163
(22.3%)		 0
(0%)
wbo
1.6
(complete)		 731271
(37.07%)		 33
(4.51%)		 238
(32.56%)		 0
(0%)		 451
(61.7%)		 0
(0%)

Solver NameProgress
borg
pb-opt-11.04.03
(complete)
done
731
UNSAT
35		OPT/SAT
378		?
283		ERR
35
bsolo
3.2
(complete)
done
731
UNSAT
34		OPT/SAT
338		SAT (TO/MO)
10		?
45
clasp
2.0-R4191
(complete)
done
731
UNSAT
34		OPT/SAT
286		SAT (TO/MO)
336		?
75
MinisatID
2.4.8 [DEPRECATED]
(complete)
done
731
UNSAT
34		OPT/SAT
230		?
467
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
731
UNSAT
30		OPT/SAT
208		?
493
MinisatID
2.5.2 (fixed)
(complete)
done
731
UNSAT
32		OPT/SAT
256		?
443
MinisatID
2.5.2-gmp (fixed)
(complete)
done
731
UNSAT
30		OPT/SAT
232		?
469
pwbo
1.1
(complete)
done
731
UNSAT
33		OPT/SAT
350		SAT (TO/MO)
297		?
51
Sat4j CuttingPlanes
2.3.0
(complete)
done
731
UNSAT
30		OPT/SAT
240		SAT (TO/MO)
381		?
80
Sat4j Res//CP
2.3.0
(complete)
done
731
UNSAT
34		OPT/SAT
295		SAT (TO/MO)
357		?
45
Sat4j Resolution
2.3.0
(complete)
done
731
UNSAT
34		OPT/SAT
282		SAT (TO/MO)
367		?
48
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
731
UNSAT
35		OPT/SAT
360		SAT (TO/MO)
120		?
203
SCIP spx 2
2011-06-10 (fixed)
(complete)
done
731
UNSAT
35		OPT/SAT
389		SAT (TO/MO)
9		?
164
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
731
UNSAT
35		OPT/SAT
378		SAT (TO/MO)
114		?
196		ERR
1
SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
731
UNSAT
35		OPT/SAT
389		SAT (TO/MO)
12		?
163
wbo
1.6
(complete)
done
731
UNSAT
33		OPT/SAT
238		?
451

Category "optimisation, small integers, non linear constraints" (OPT-SMALLINT-NLC)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
borg
pb-opt-11.04.03
(complete)		 409274
(66.99%)		 0
(0%)		 274
(66.99%)		 0
(0%)		 128
(31.3%)		 7
(1.71%)
bsolo
3.2
(complete)		 409234
(57.21%)		 0
(0%)		 234
(57.21%)		 6
(1.47%)		 6
(1.47%)		 0
(0%)
clasp
2.0-R4191 [DEPRECATED]
(complete)		 409280
(68.46%)		 0
(0%)		 280
(68.46%)		 125
(30.56%)		 0
(0%)		 4
(0.98%)
clasp
2.0-R4191-patched (fixed)
(complete)		 409280
(68.46%)		 0
(0%)		 280
(68.46%)		 129
(31.54%)		 0
(0%)		 0
(0%)
MinisatID
2.4.8 [DEPRECATED]
(complete)		 409181
(44.25%)		 0
(0%)		 181
(44.25%)		 0
(0%)		 228
(55.75%)		 0
(0%)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 409158
(38.63%)		 0
(0%)		 158
(38.63%)		 0
(0%)		 251
(61.37%)		 0
(0%)
MinisatID
2.5.2 (fixed)
(complete)		 4090
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 409
(100%)		 0
(0%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 4090
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 409
(100%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 409120
(29.34%)		 0
(0%)		 120
(29.34%)		 240
(58.68%)		 49
(11.98%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 409274
(66.99%)		 0
(0%)		 274
(66.99%)		 135
(33.01%)		 0
(0%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 409277
(67.73%)		 0
(0%)		 277
(67.73%)		 132
(32.27%)		 0
(0%)		 0
(0%)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 409294
(71.88%)		 0
(0%)		 294
(71.88%)		 81
(19.8%)		 20
(4.89%)		 0
(0%)
SCIP spx 2
2011-06-10 (fixed)
(complete)		 409294
(71.88%)		 0
(0%)		 294
(71.88%)		 0
(0%)		 12
(2.93%)		 0
(0%)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 409294
(71.88%)		 0
(0%)		 294
(71.88%)		 79
(19.32%)		 30
(7.33%)		 0
(0%)
SCIP spx E_2
2011-06-10 (fixed)
(complete)		 409297
(72.62%)		 0
(0%)		 297
(72.62%)		 0
(0%)		 8
(1.96%)		 0
(0%)

Solver NameProgress
borg
pb-opt-11.04.03
(complete)
done
409
OPT/SAT
274		?
128		ERR
7
bsolo
3.2
(complete)
done
409
OPT/SAT
234		SAT (TO/MO)
6		?
6
clasp
2.0-R4191 [DEPRECATED]
(complete)
done
409
OPT/SAT
280		SAT (TO/MO)
125		ERR
4
clasp
2.0-R4191-patched (fixed)
(complete)
done
409
OPT/SAT
280		SAT (TO/MO)
129
MinisatID
2.4.8 [DEPRECATED]
(complete)
done
409
OPT/SAT
181		?
228
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
409
OPT/SAT
158		?
251
MinisatID
2.5.2 (fixed)
(complete)
done
409
?
409
MinisatID
2.5.2-gmp (fixed)
(complete)
done
409
?
409
Sat4j CuttingPlanes
2.3.0
(complete)
done
409
OPT/SAT
120		SAT (TO/MO)
240		?
49
Sat4j Res//CP
2.3.0
(complete)
done
409
OPT/SAT
274		SAT (TO/MO)
135
Sat4j Resolution
2.3.0
(complete)
done
409
OPT/SAT
277		SAT (TO/MO)
132
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
409
OPT/SAT
294		SAT (TO/MO)
81		?
20
SCIP spx 2
2011-06-10 (fixed)
(complete)
done
409
OPT/SAT
294		?
12
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
409
OPT/SAT
294		SAT (TO/MO)
79		?
30
SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
409
OPT/SAT
297		?
8

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 2278824
(36.17%)		 212
(9.31%)		 612
(26.87%)		 0
(0%)		 1453
(63.78%)		 1
(0.04%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 2278741
(32.53%)		 284
(12.47%)		 457
(20.06%)		 0
(0%)		 1537
(67.47%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 2278872
(38.28%)		 232
(10.18%)		 640
(28.09%)		 796
(34.94%)		 610
(26.78%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 22781296
(56.89%)		 345
(15.14%)		 951
(41.75%)		 680
(29.85%)		 302
(13.26%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 22781213
(53.25%)		 314
(13.78%)		 899
(39.46%)		 691
(30.33%)		 374
(16.42%)		 0
(0%)

Solver NameProgress
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
2278
UNSAT
212		OPT/SAT
612		?
1453		ERR
1
MinisatID
2.5.2-gmp (fixed)
(complete)
done
2278
UNSAT
284		OPT/SAT
457		?
1537
Sat4j CuttingPlanes
2.3.0
(complete)
done
2278
UNSAT
232		OPT/SAT
640		SAT (TO/MO)
796		?
610
Sat4j Res//CP
2.3.0
(complete)
done
2278
UNSAT
345		OPT/SAT
951		SAT (TO/MO)
680		?
302
Sat4j Resolution
2.3.0
(complete)
done
2278
UNSAT
314		OPT/SAT
899		SAT (TO/MO)
691		?
374

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
borg
pb-dec-11.04.03
(complete)		 581504
(86.75%)		 268
(46.13%)		 236
(40.62%)		 0
(0%)		 77
(13.25%)		 0
(0%)
borg
pb-opt-11.04.03
(complete)		 1140687
(60.26%)		 35
(3.07%)		 652
(57.19%)		 0
(0%)		 411
(36.05%)		 42
(3.68%)
bsolo
3.2
(complete)		 17211084
(62.99%)		 286
(16.62%)		 798
(46.37%)		 16
(0.93%)		 154
(8.95%)		 0
(0%)
clasp
2.0-R4191
(complete)		 1312764
(58.23%)		 261
(19.89%)		 503
(38.34%)		 336
(25.61%)		 212
(16.16%)		 0
(0%)
clasp
2.0-R4191 [DEPRECATED]
(complete)		 409280
(68.46%)		 0
(0%)		 280
(68.46%)		 125
(30.56%)		 0
(0%)		 4
(0.98%)
clasp
2.0-R4191-patched (fixed)
(complete)		 409280
(68.46%)		 0
(0%)		 280
(68.46%)		 129
(31.54%)		 0
(0%)		 0
(0%)
MinisatID
2.4.8 [DEPRECATED]
(complete)		 1721769
(44.68%)		 160
(9.3%)		 609
(35.39%)		 0
(0%)		 951
(55.26%)		 1
(0.06%)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		 2278824
(36.17%)		 212
(9.31%)		 612
(26.87%)		 0
(0%)		 1453
(63.78%)		 1
(0.04%)
MinisatID
2.5.2 (fixed)
(complete)		 1721656
(38.12%)		 231
(13.42%)		 425
(24.69%)		 0
(0%)		 1065
(61.88%)		 0
(0%)
MinisatID
2.5.2-gmp (fixed)
(complete)		 2278741
(32.53%)		 284
(12.47%)		 457
(20.06%)		 0
(0%)		 1537
(67.47%)		 0
(0%)
pwbo
1.1
(complete)		 731383
(52.39%)		 33
(4.51%)		 350
(47.88%)		 297
(40.63%)		 51
(6.98%)		 0
(0%)
Sat4j CuttingPlanes
2.3.0
(complete)		 2278872
(38.28%)		 232
(10.18%)		 640
(28.09%)		 796
(34.94%)		 610
(26.78%)		 0
(0%)
Sat4j Res//CP
2.3.0
(complete)		 22781296
(56.89%)		 345
(15.14%)		 951
(41.75%)		 680
(29.85%)		 302
(13.26%)		 0
(0%)
Sat4j Resolution
2.3.0
(complete)		 22781213
(53.25%)		 314
(13.78%)		 899
(39.46%)		 691
(30.33%)		 374
(16.42%)		 0
(0%)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 17211147
(66.65%)		 291
(16.91%)		 856
(49.74%)		 201
(11.68%)		 346
(20.1%)		 0
(0%)
SCIP spx 2
2011-06-10 (fixed)
(complete)		 17211175
(68.27%)		 290
(16.85%)		 885
(51.42%)		 9
(0.52%)		 300
(17.43%)		 0
(0%)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		 17211165
(67.69%)		 290
(16.85%)		 875
(50.84%)		 193
(11.21%)		 349
(20.28%)		 1
(0.06%)
SCIP spx E_2
2011-06-10 (fixed)
(complete)		 17211180
(68.56%)		 290
(16.85%)		 890
(51.71%)		 12
(0.7%)		 293
(17.02%)		 0
(0%)
wbo
1.6
(complete)		 1212665
(54.87%)		 247
(20.38%)		 418
(34.49%)		 0
(0%)		 538
(44.39%)		 0
(0%)

Solver NameProgress
borg
pb-dec-11.04.03
(complete)
done
581
UNSAT
268		OPT/SAT
236		?
77
borg
pb-opt-11.04.03
(complete)
done
1140
UNSAT
35		OPT/SAT
652		?
411		ERR
42
bsolo
3.2
(complete)
done
1721
UNSAT
286		OPT/SAT
798		SAT (TO/MO)
16		?
154
clasp
2.0-R4191
(complete)
done
1312
UNSAT
261		OPT/SAT
503		SAT (TO/MO)
336		?
212
clasp
2.0-R4191 [DEPRECATED]
(complete)
done
409
OPT/SAT
280		SAT (TO/MO)
125		ERR
4
clasp
2.0-R4191-patched (fixed)
(complete)
done
409
OPT/SAT
280		SAT (TO/MO)
129
MinisatID
2.4.8 [DEPRECATED]
(complete)
done
1721
UNSAT
160		OPT/SAT
609		?
951		ERR
1
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
done
2278
UNSAT
212		OPT/SAT
612		?
1453		ERR
1
MinisatID
2.5.2 (fixed)
(complete)
done
1721
UNSAT
231		OPT/SAT
425		?
1065
MinisatID
2.5.2-gmp (fixed)
(complete)
done
2278
UNSAT
284		OPT/SAT
457		?
1537
pwbo
1.1
(complete)
done
731
UNSAT
33		OPT/SAT
350		SAT (TO/MO)
297		?
51
Sat4j CuttingPlanes
2.3.0
(complete)
done
2278
UNSAT
232		OPT/SAT
640		SAT (TO/MO)
796		?
610
Sat4j Res//CP
2.3.0
(complete)
done
2278
UNSAT
345		OPT/SAT
951		SAT (TO/MO)
680		?
302
Sat4j Resolution
2.3.0
(complete)
done
2278
UNSAT
314		OPT/SAT
899		SAT (TO/MO)
691		?
374
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
1721
UNSAT
291		OPT/SAT
856		SAT (TO/MO)
201		?
346
SCIP spx 2
2011-06-10 (fixed)
(complete)
done
1721
UNSAT
290		OPT/SAT
885		SAT (TO/MO)
9		?
300
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
done
1721
UNSAT
290		OPT/SAT
875		SAT (TO/MO)
193		?
349		ERR
1
SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
1721
UNSAT
290		OPT/SAT
890		SAT (TO/MO)
12		?
293
wbo
1.6
(complete)
done
1212
UNSAT
247		OPT/SAT
418		?
538

WARNING: 304 jobs are unclassified for solver 1955 in category OPT-SMALLINT-LIN/
WARNING: 13 jobs are unclassified for solver 1949 in category OPT-SMALLINT-LIN/
WARNING: 134 jobs are unclassified for solver 1977 in category OPT-SMALLINT-LIN/
WARNING: 7 jobs are unclassified for solver 1948 in category OPT-SMALLINT-LIN/
WARNING: 132 jobs are unclassified for solver 1979 in category OPT-SMALLINT-LIN/
WARNING: 9 jobs are unclassified for solver 1953 in category OPT-SMALLINT-LIN/
WARNING: 163 jobs are unclassified for solver 1955 in category OPT-SMALLINT-NLC/
WARNING: 14 jobs are unclassified for solver 1949 in category OPT-SMALLINT-NLC/
WARNING: 103 jobs are unclassified for solver 1977 in category OPT-SMALLINT-NLC/
WARNING: 6 jobs are unclassified for solver 1948 in category OPT-SMALLINT-NLC/
WARNING: 104 jobs are unclassified for solver 1979 in category OPT-SMALLINT-NLC/
WARNING: 467 jobs are unclassified for solver 1955 in category /
WARNING: 27 jobs are unclassified for solver 1949 in category /
WARNING: 237 jobs are unclassified for solver 1977 in category /
WARNING: 13 jobs are unclassified for solver 1948 in category /
WARNING: 236 jobs are unclassified for solver 1979 in category /
WARNING: 9 jobs are unclassified for solver 1953 in category /