PB'12 competition: satisfaction and optimization track: solvers page (tabular format, summary)

Some explanations

Category "no optimisation, big integers, linear constraints" (DEC-BIGINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
npSolver
1.0
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
npSolver
1.0 (fixed)
(complete)		 146
(42.86%)		 0
(0%)		 6
(42.86%)		 0
(0%)		 8
(57.14%)		 0
(0%)
npSolver
inc
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
npSolver
inc (fixed)
(complete)		 146
(42.86%)		 0
(0%)		 6
(42.86%)		 0
(0%)		 8
(57.14%)		 0
(0%)
npSolver
inc-topDown
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
npSolver
inc-topDown (fixed)
(complete)		 146
(42.86%)		 0
(0%)		 6
(42.86%)		 0
(0%)		 8
(57.14%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 146
(42.86%)		 0
(0%)		 6
(42.86%)		 0
(0%)		 8
(57.14%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 147
(50%)		 0
(0%)		 7
(50%)		 0
(0%)		 7
(50%)		 0
(0%)
PB07: minisat+
1.14
(complete)		 1412
(85.71%)		 0
(0%)		 12
(85.71%)		 0
(0%)		 2
(14.29%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 144
(28.57%)		 0
(0%)		 4
(28.57%)		 0
(0%)		 10
(71.43%)		 0
(0%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 143
(21.43%)		 0
(0%)		 3
(21.43%)		 0
(0%)		 11
(78.57%)		 0
(0%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 1410
(71.43%)		 0
(0%)		 10
(71.43%)		 0
(0%)		 4
(28.57%)		 0
(0%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 149
(64.29%)		 0
(0%)		 9
(64.29%)		 0
(0%)		 5
(35.71%)		 0
(0%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 148
(57.14%)		 0
(0%)		 8
(57.14%)		 0
(0%)		 6
(42.86%)		 0
(0%)
PB12: minisatp
1.0-2-g022594c
(complete)		 1413
(92.86%)		 1
(7.14%)		 12
(85.71%)		 0
(0%)		 1
(7.14%)		 0
(0%)
pb2sat
2012-05-19
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
PBPASSolver-CARD.DP
2012-05-28
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
PBPASSolver-CARD.SN
2012-05-28
(complete)		 140
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 14
(100%)		 0
(0%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 145
(35.71%)		 0
(0%)		 5
(35.71%)		 0
(0%)		 9
(64.29%)		 0
(0%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 146
(42.86%)		 0
(0%)		 6
(42.86%)		 0
(0%)		 8
(57.14%)		 0
(0%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 146
(42.86%)		 0
(0%)		 6
(42.86%)		 0
(0%)		 8
(57.14%)		 0
(0%)
toysat
2012-05-17
(complete)		 141
(7.14%)		 0
(0%)		 1
(7.14%)		 0
(0%)		 13
(92.86%)		 0
(0%)
toysat
2012-06-01
(complete)		 141
(7.14%)		 0
(0%)		 1
(7.14%)		 0
(0%)		 13
(92.86%)		 0
(0%)

Solver NameProgress
npSolver
1.0
(complete)
done
14
?
14
npSolver
1.0 (fixed)
(complete)
done
14
OPT/SAT
6		?
8
npSolver
inc
(complete)
done
14
?
14
npSolver
inc (fixed)
(complete)
done
14
OPT/SAT
6		?
8
npSolver
inc-topDown
(complete)
done
14
?
14
npSolver
inc-topDown (fixed)
(complete)
done
14
OPT/SAT
6		?
8
npSolver
inc-topdown-quickBound
(complete)
done
14
?
14
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
14
OPT/SAT
6		?
8
PB07: bsolo
3.0.17
(complete)
done
14
OPT/SAT
7		?
7
PB07: minisat+
1.14
(complete)
done
14
OPT/SAT
12		?
2
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
14
OPT/SAT
4		?
10
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
14
OPT/SAT
3		?
11
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
14
OPT/SAT
10		?
4
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
14
OPT/SAT
9		?
5
PB11: Sat4j Res//CP
2.3.0
(complete)
done
14
OPT/SAT
8		?
6
PB12: minisatp
1.0-2-g022594c
(complete)
done
14
UNSAT
1		OPT/SAT
12		?
1
pb2sat
2012-05-19
(complete)
done
14
?
14
pb2satCp2
2012-05-19
(complete)
done
14
?
14
PBPASSolver-CARD.DP
2012-05-28
(complete)
done
14
?
14
PBPASSolver-CARD.SN
2012-05-28
(complete)
done
14
?
14
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
14
OPT/SAT
5		?
9
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
14
OPT/SAT
6		?
8
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
14
OPT/SAT
6		?
8
toysat
2012-05-17
(complete)
done
14
OPT/SAT
1		?
13
toysat
2012-06-01
(complete)
done
14
OPT/SAT
1		?
13

Category "no optimisation, small integers, linear constraints" (DEC-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
bsolo
3.2
(complete)		 355238
(67.04%)		 115
(32.39%)		 123
(34.65%)		 0
(0%)		 117
(32.96%)		 0
(0%)
clasp
2.0.6-R5325 (dec)
(complete)		 355246
(69.3%)		 97
(27.32%)		 149
(41.97%)		 0
(0%)		 109
(30.7%)		 0
(0%)
npSolver
1.0
(complete)		 3550
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 355
(100%)		 0
(0%)
npSolver
1.0 (fixed)
(complete)		 355222
(62.54%)		 97
(27.32%)		 125
(35.21%)		 0
(0%)		 133
(37.46%)		 0
(0%)
npSolver
inc
(complete)		 3550
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 355
(100%)		 0
(0%)
npSolver
inc (fixed)
(complete)		 355181
(50.99%)		 96
(27.04%)		 85
(23.94%)		 0
(0%)		 174
(49.01%)		 0
(0%)
npSolver
inc-topDown
(complete)		 3550
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 355
(100%)		 0
(0%)
npSolver
inc-topDown (fixed)
(complete)		 355181
(50.99%)		 97
(27.32%)		 84
(23.66%)		 0
(0%)		 174
(49.01%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 3550
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 355
(100%)		 0
(0%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 355182
(51.27%)		 97
(27.32%)		 85
(23.94%)		 0
(0%)		 173
(48.73%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 355198
(55.77%)		 85
(23.94%)		 113
(31.83%)		 0
(0%)		 157
(44.23%)		 0
(0%)
PB07: minisat+
1.14
(complete)		 355221
(62.25%)		 91
(25.63%)		 130
(36.62%)		 0
(0%)		 134
(37.75%)		 0
(0%)
PB07: PB-clasp
2007-04-10
(complete)		 355226
(63.66%)		 93
(26.2%)		 133
(37.46%)		 0
(0%)		 129
(36.34%)		 0
(0%)
PB07: Pueblo
1.4
(incomplete)		 355240
(67.61%)		 112
(31.55%)		 128
(36.06%)		 0
(0%)		 115
(32.39%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 355192
(54.08%)		 89
(25.07%)		 103
(29.01%)		 0
(0%)		 163
(45.92%)		 0
(0%)
PB09: bsolo
3.1
(complete)		 355241
(67.89%)		 118
(33.24%)		 123
(34.65%)		 0
(0%)		 113
(31.83%)		 1
(0.28%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 355194
(54.65%)		 88
(24.79%)		 106
(29.86%)		 0
(0%)		 161
(45.35%)		 0
(0%)
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)		 355193
(54.37%)		 112
(31.55%)		 81
(22.82%)		 0
(0%)		 162
(45.63%)		 0
(0%)
PB10: borg-pb
10.05.30
(complete)		 355241
(67.89%)		 112
(31.55%)		 129
(36.34%)		 0
(0%)		 114
(32.11%)		 0
(0%)
PB10: pb_cplex
2010-06-29
(complete)		 355192
(54.08%)		 104
(29.3%)		 88
(24.79%)		 0
(0%)		 163
(45.92%)		 0
(0%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 355243
(68.45%)		 114
(32.11%)		 129
(36.34%)		 0
(0%)		 112
(31.55%)		 0
(0%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 355194
(54.65%)		 113
(31.83%)		 81
(22.82%)		 0
(0%)		 161
(45.35%)		 0
(0%)
PB11: borg
pb-dec-11.04.03
(complete)		 355243
(68.45%)		 119
(33.52%)		 124
(34.93%)		 0
(0%)		 112
(31.55%)		 0
(0%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 355247
(69.58%)		 118
(33.24%)		 129
(36.34%)		 0
(0%)		 108
(30.42%)		 0
(0%)
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)		 355194
(54.65%)		 113
(31.83%)		 81
(22.82%)		 0
(0%)		 161
(45.35%)		 0
(0%)
PB12: minisatp
1.0-2-g022594c
(complete)		 355220
(61.97%)		 91
(25.63%)		 129
(36.34%)		 0
(0%)		 135
(38.03%)		 0
(0%)
pb2sat
2012-05-19
(complete)		 3550
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 355
(100%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 3550
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 355
(100%)		 0
(0%)
PBPASSolver-CARD.DP
2012-05-28
(complete)		 355144
(40.56%)		 65
(18.31%)		 79
(22.25%)		 0
(0%)		 211
(59.44%)		 0
(0%)
PBPASSolver-CARD.SN
2012-05-28
(complete)		 355165
(46.48%)		 67
(18.87%)		 98
(27.61%)		 0
(0%)		 190
(53.52%)		 0
(0%)
pwbo
2.0
(complete)		 355228
(64.23%)		 94
(26.48%)		 134
(37.75%)		 0
(0%)		 127
(35.77%)		 0
(0%)
pwbo
2.02
(complete)		 355226
(63.66%)		 95
(26.76%)		 131
(36.9%)		 0
(0%)		 129
(36.34%)		 0
(0%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 355249
(70.14%)		 119
(33.52%)		 130
(36.62%)		 0
(0%)		 106
(29.86%)		 0
(0%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 355231
(65.07%)		 91
(25.63%)		 140
(39.44%)		 0
(0%)		 124
(34.93%)		 0
(0%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 355205
(57.75%)		 91
(25.63%)		 114
(32.11%)		 0
(0%)		 150
(42.25%)		 0
(0%)
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)		 355197
(55.49%)		 113
(31.83%)		 84
(23.66%)		 0
(0%)		 158
(44.51%)		 0
(0%)
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)		 355196
(55.21%)		 113
(31.83%)		 83
(23.38%)		 0
(0%)		 159
(44.79%)		 0
(0%)
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)		 355203
(57.18%)		 113
(31.83%)		 90
(25.35%)		 0
(0%)		 152
(42.82%)		 0
(0%)
toysat
2012-05-17
(complete)		 355197
(55.49%)		 83
(23.38%)		 114
(32.11%)		 0
(0%)		 157
(44.23%)		 1
(0.28%)
toysat
2012-06-01
(complete)		 355198
(55.77%)		 84
(23.66%)		 114
(32.11%)		 0
(0%)		 157
(44.23%)		 0
(0%)
wbo
1.7
(complete)		 355225
(63.38%)		 94
(26.48%)		 131
(36.9%)		 0
(0%)		 130
(36.62%)		 0
(0%)
wbo
1.72
(complete)		 355226
(63.66%)		 95
(26.76%)		 131
(36.9%)		 0
(0%)		 129
(36.34%)		 0
(0%)

Solver NameProgress
bsolo
3.2
(complete)
done
355
UNSAT
115		OPT/SAT
123		?
117
clasp
2.0.6-R5325 (dec)
(complete)
done
355
UNSAT
97		OPT/SAT
149		?
109
npSolver
1.0
(complete)
done
355
?
355
npSolver
1.0 (fixed)
(complete)
done
355
UNSAT
97		OPT/SAT
125		?
133
npSolver
inc
(complete)
done
355
?
355
npSolver
inc (fixed)
(complete)
done
355
UNSAT
96		OPT/SAT
85		?
174
npSolver
inc-topDown
(complete)
done
355
?
355
npSolver
inc-topDown (fixed)
(complete)
done
355
UNSAT
97		OPT/SAT
84		?
174
npSolver
inc-topdown-quickBound
(complete)
done
355
?
355
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
355
UNSAT
97		OPT/SAT
85		?
173
PB07: bsolo
3.0.17
(complete)
done
355
UNSAT
85		OPT/SAT
113		?
157
PB07: minisat+
1.14
(complete)
done
355
UNSAT
91		OPT/SAT
130		?
134
PB07: PB-clasp
2007-04-10
(complete)
done
355
UNSAT
93		OPT/SAT
133		?
129
PB07: Pueblo
1.4
(incomplete)
done
355
UNSAT
112		OPT/SAT
128		?
115
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
355
UNSAT
89		OPT/SAT
103		?
163
PB09: bsolo
3.1
(complete)
done
355
UNSAT
118		OPT/SAT
123		?
113		ERR
1
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
355
UNSAT
88		OPT/SAT
106		?
161
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
done
355
UNSAT
112		OPT/SAT
81		?
162
PB10: borg-pb
10.05.30
(complete)
done
355
UNSAT
112		OPT/SAT
129		?
114
PB10: pb_cplex
2010-06-29
(complete)
done
355
UNSAT
104		OPT/SAT
88		?
163
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
355
UNSAT
114		OPT/SAT
129		?
112
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
355
UNSAT
113		OPT/SAT
81		?
161
PB11: borg
pb-dec-11.04.03
(complete)
done
355
UNSAT
119		OPT/SAT
124		?
112
PB11: Sat4j Res//CP
2.3.0
(complete)
done
355
UNSAT
118		OPT/SAT
129		?
108
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
355
UNSAT
113		OPT/SAT
81		?
161
PB12: minisatp
1.0-2-g022594c
(complete)
done
355
UNSAT
91		OPT/SAT
129		?
135
pb2sat
2012-05-19
(complete)
done
355
?
355
pb2satCp2
2012-05-19
(complete)
done
355
?
355
PBPASSolver-CARD.DP
2012-05-28
(complete)
done
355
UNSAT
65		OPT/SAT
79		?
211
PBPASSolver-CARD.SN
2012-05-28
(complete)
done
355
UNSAT
67		OPT/SAT
98		?
190
pwbo
2.0
(complete)
done
355
UNSAT
94		OPT/SAT
134		?
127
pwbo
2.02
(complete)
done
355
UNSAT
95		OPT/SAT
131		?
129
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
355
UNSAT
119		OPT/SAT
130		?
106
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
355
UNSAT
91		OPT/SAT
140		?
124
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
355
UNSAT
91		OPT/SAT
114		?
150
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)
done
355
UNSAT
113		OPT/SAT
84		?
158
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)
done
355
UNSAT
113		OPT/SAT
83		?
159
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)
done
355
UNSAT
113		OPT/SAT
90		?
152
toysat
2012-05-17
(complete)
done
355
UNSAT
83		OPT/SAT
114		?
157		ERR
1
toysat
2012-06-01
(complete)
done
355
UNSAT
84		OPT/SAT
114		?
157
wbo
1.7
(complete)
done
355
UNSAT
94		OPT/SAT
131		?
130
wbo
1.72
(complete)
done
355
UNSAT
95		OPT/SAT
131		?
129

Category "no optimisation, small integers, non linear constraints" (DEC-SMALLINT-NLC)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
bsolo
3.2
(complete)		 3018
(60%)		 5
(16.67%)		 13
(43.33%)		 0
(0%)		 12
(40%)		 0
(0%)
clasp
2.0.6-R5325 (dec)
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
npSolver
1.0
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
npSolver
1.0 (fixed)
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
npSolver
inc
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
npSolver
inc (fixed)
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)
npSolver
inc-topDown
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
npSolver
inc-topDown (fixed)
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 307
(23.33%)		 3
(10%)		 4
(13.33%)		 0
(0%)		 23
(76.67%)		 0
(0%)
PB07: minisat+
1.14
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
PB07: PB-clasp
2007-04-10
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
PB07: Pueblo
1.4
(incomplete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
PB09: bsolo
3.1
(complete)		 3018
(60%)		 5
(16.67%)		 13
(43.33%)		 0
(0%)		 12
(40%)		 0
(0%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)		 3022
(73.33%)		 6
(20%)		 16
(53.33%)		 0
(0%)		 8
(26.67%)		 0
(0%)
PB10: borg-pb
10.05.30
(complete)		 308
(26.67%)		 3
(10%)		 5
(16.67%)		 0
(0%)		 5
(16.67%)		 17
(56.67%)
PB10: pb_cplex
2010-06-29
(complete)		 3024
(80%)		 9
(30%)		 15
(50%)		 0
(0%)		 6
(20%)		 0
(0%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 3022
(73.33%)		 6
(20%)		 16
(53.33%)		 0
(0%)		 8
(26.67%)		 0
(0%)
PB11: borg
pb-dec-11.04.03
(complete)		 3021
(70%)		 6
(20%)		 15
(50%)		 0
(0%)		 9
(30%)		 0
(0%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)		 3022
(73.33%)		 6
(20%)		 16
(53.33%)		 0
(0%)		 8
(26.67%)		 0
(0%)
PB12: minisatp
1.0-2-g022594c
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
pb2sat
2012-05-19
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 300
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 30
(100%)		 0
(0%)
pwbo
2.0
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
pwbo
2.02
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 3016
(53.33%)		 5
(16.67%)		 11
(36.67%)		 0
(0%)		 14
(46.67%)		 0
(0%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 3021
(70%)		 5
(16.67%)		 16
(53.33%)		 0
(0%)		 9
(30%)		 0
(0%)
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)		 3022
(73.33%)		 6
(20%)		 16
(53.33%)		 0
(0%)		 8
(26.67%)		 0
(0%)
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)		 3022
(73.33%)		 6
(20%)		 16
(53.33%)		 0
(0%)		 8
(26.67%)		 0
(0%)
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)		 3023
(76.67%)		 7
(23.33%)		 16
(53.33%)		 0
(0%)		 7
(23.33%)		 0
(0%)
toysat
2012-05-17
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
toysat
2012-06-01
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
wbo
1.7
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)
wbo
1.72
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)

Solver NameProgress
bsolo
3.2
(complete)
done
30
UNSAT
5		OPT/SAT
13		?
12
clasp
2.0.6-R5325 (dec)
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
npSolver
1.0
(complete)
done
30
?
30
npSolver
1.0 (fixed)
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
npSolver
inc
(complete)
done
30
?
30
npSolver
inc (fixed)
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11
npSolver
inc-topDown
(complete)
done
30
?
30
npSolver
inc-topDown (fixed)
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11
npSolver
inc-topdown-quickBound
(complete)
done
30
?
30
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11
PB07: bsolo
3.0.17
(complete)
done
30
UNSAT
3		OPT/SAT
4		?
23
PB07: minisat+
1.14
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
PB07: PB-clasp
2007-04-10
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
PB07: Pueblo
1.4
(incomplete)
done
30
UNSAT
5		OPT/SAT
15		?
10
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
30
?
30
PB09: bsolo
3.1
(complete)
done
30
UNSAT
5		OPT/SAT
13		?
12
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
done
30
UNSAT
6		OPT/SAT
16		?
8
PB10: borg-pb
10.05.30
(complete)
done
30
UNSAT
3		OPT/SAT
5		?
5		ERR
17
PB10: pb_cplex
2010-06-29
(complete)
done
30
UNSAT
9		OPT/SAT
15		?
6
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
30
UNSAT
6		OPT/SAT
16		?
8
PB11: borg
pb-dec-11.04.03
(complete)
done
30
UNSAT
6		OPT/SAT
15		?
9
PB11: Sat4j Res//CP
2.3.0
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
30
UNSAT
6		OPT/SAT
16		?
8
PB12: minisatp
1.0-2-g022594c
(complete)
done
30
?
30
pb2sat
2012-05-19
(complete)
done
30
?
30
pb2satCp2
2012-05-19
(complete)
done
30
?
30
pwbo
2.0
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
pwbo
2.02
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
30
UNSAT
5		OPT/SAT
11		?
14
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
30
UNSAT
5		OPT/SAT
16		?
9
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)
done
30
UNSAT
6		OPT/SAT
16		?
8
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)
done
30
UNSAT
6		OPT/SAT
16		?
8
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)
done
30
UNSAT
7		OPT/SAT
16		?
7
toysat
2012-05-17
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
toysat
2012-06-01
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
wbo
1.7
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11
wbo
1.72
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11

Category "optimisation, big integers, linear constraints" (OPT-BIGINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
npSolver
1.0
(complete)		 41633
(7.93%)		 16
(3.85%)		 17
(4.09%)		 0
(0%)		 383
(92.07%)		 0
(0%)
npSolver
1.0 (fixed)
(complete)		 41637
(8.89%)		 19
(4.57%)		 18
(4.33%)		 0
(0%)		 379
(91.11%)		 0
(0%)
npSolver
inc
(complete)		 41625
(6.01%)		 16
(3.85%)		 9
(2.16%)		 0
(0%)		 391
(93.99%)		 0
(0%)
npSolver
inc (fixed)
(complete)		 41530
(7.23%)		 19
(4.58%)		 11
(2.65%)		 0
(0%)		 385
(92.77%)		 0
(0%)
npSolver
inc-topDown
(complete)		 41624
(5.77%)		 16
(3.85%)		 8
(1.92%)		 0
(0%)		 392
(94.23%)		 0
(0%)
npSolver
inc-topDown (fixed)
(complete)		 41634
(8.17%)		 19
(4.57%)		 15
(3.61%)		 0
(0%)		 382
(91.83%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 41618
(4.33%)		 16
(3.85%)		 2
(0.48%)		 0
(0%)		 398
(95.67%)		 0
(0%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 41627
(6.49%)		 19
(4.57%)		 8
(1.92%)		 0
(0%)		 389
(93.51%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 41666
(15.87%)		 23
(5.53%)		 43
(10.34%)		 88
(21.15%)		 125
(30.05%)		 11
(2.64%)
PB07: minisat+
1.14
(complete)		 41636
(8.65%)		 23
(5.53%)		 13
(3.13%)		 83
(19.95%)		 207
(49.76%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 41692
(22.12%)		 23
(5.53%)		 69
(16.59%)		 106
(25.48%)		 165
(39.66%)		 30
(7.21%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 41689
(21.39%)		 23
(5.53%)		 66
(15.87%)		 100
(24.04%)		 184
(44.23%)		 5
(1.2%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 416112
(26.92%)		 23
(5.53%)		 89
(21.39%)		 104
(25%)		 82
(19.71%)		 1
(0.24%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 416186
(44.71%)		 38
(9.13%)		 148
(35.58%)		 21
(5.05%)		 175
(42.07%)		 34
(8.17%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 416108
(25.96%)		 23
(5.53%)		 85
(20.43%)		 112
(26.92%)		 60
(14.42%)		 1
(0.24%)
PB12: minisatp
1.0-2-g022594c
(complete)		 41638
(9.13%)		 23
(5.53%)		 15
(3.61%)		 79
(18.99%)		 177
(42.55%)		 65
(15.63%)
pb2sat
2012-05-19
(complete)		 41631
(7.45%)		 16
(3.85%)		 15
(3.61%)		 0
(0%)		 385
(92.55%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 41629
(6.97%)		 16
(3.85%)		 13
(3.13%)		 0
(0%)		 387
(93.03%)		 0
(0%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 416103
(24.76%)		 23
(5.53%)		 80
(19.23%)		 123
(29.57%)		 61
(14.66%)		 1
(0.24%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 41697
(23.32%)		 23
(5.53%)		 74
(17.79%)		 120
(28.85%)		 83
(19.95%)		 2
(0.48%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 41677
(18.51%)		 23
(5.53%)		 54
(12.98%)		 145
(34.86%)		 114
(27.4%)		 0
(0%)
toysat
2012-05-17
(complete)		 41659
(14.18%)		 19
(4.57%)		 40
(9.62%)		 0
(0%)		 357
(85.82%)		 0
(0%)
toysat
2012-06-01
(complete)		 41662
(14.9%)		 24
(5.77%)		 38
(9.13%)		 0
(0%)		 353
(84.86%)		 1
(0.24%)

Solver NameProgress
npSolver
1.0
(complete)
done
416
UNSAT
16		OPT/SAT
17		?
383
npSolver
1.0 (fixed)
(complete)
done
416
UNSAT
19		OPT/SAT
18		?
379
npSolver
inc
(complete)
done
416
UNSAT
16		OPT/SAT
9		?
391
npSolver
inc (fixed)
(complete)
done
415
UNSAT
19		OPT/SAT
11		?
385
npSolver
inc-topDown
(complete)
done
416
UNSAT
16		OPT/SAT
8		?
392
npSolver
inc-topDown (fixed)
(complete)
done
416
UNSAT
19		OPT/SAT
15		?
382
npSolver
inc-topdown-quickBound
(complete)
done
416
UNSAT
16		OPT/SAT
2		?
398
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
416
UNSAT
19		OPT/SAT
8		?
389
PB07: bsolo
3.0.17
(complete)
done
416
UNSAT
23		OPT/SAT
43		SAT (TO/MO)
88		?
125		ERR
11
PB07: minisat+
1.14
(complete)
done
416
UNSAT
23		OPT/SAT
13		SAT (TO/MO)
83		?
207
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
416
UNSAT
23		OPT/SAT
69		SAT (TO/MO)
106		?
165		ERR
30
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
416
UNSAT
23		OPT/SAT
66		SAT (TO/MO)
100		?
184		ERR
5
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
416
UNSAT
23		OPT/SAT
89		SAT (TO/MO)
104		?
82		ERR
1
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
416
UNSAT
38		OPT/SAT
148		SAT (TO/MO)
21		?
175		ERR
34
PB11: Sat4j Res//CP
2.3.0
(complete)
done
416
UNSAT
23		OPT/SAT
85		SAT (TO/MO)
112		?
60		ERR
1
PB12: minisatp
1.0-2-g022594c
(complete)
done
416
UNSAT
23		OPT/SAT
15		SAT (TO/MO)
79		?
177		ERR
65
pb2sat
2012-05-19
(complete)
done
416
UNSAT
16		OPT/SAT
15		?
385
pb2satCp2
2012-05-19
(complete)
done
416
UNSAT
16		OPT/SAT
13		?
387
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
416
UNSAT
23		OPT/SAT
80		SAT (TO/MO)
123		?
61		ERR
1
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
416
UNSAT
23		OPT/SAT
74		SAT (TO/MO)
120		?
83		ERR
2
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
416
UNSAT
23		OPT/SAT
54		SAT (TO/MO)
145		?
114
toysat
2012-05-17
(complete)
done
416
UNSAT
19		OPT/SAT
40		?
357
toysat
2012-06-01
(complete)
done
416
UNSAT
24		OPT/SAT
38		?
353		ERR
1

Category "optimisation, small integers, linear constraints" (OPT-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
bsolo
3.2
(complete)		 657306
(46.58%)		 21
(3.2%)		 285
(43.38%)		 286
(43.53%)		 65
(9.89%)		 0
(0%)
clasp
2.0.6-R5325 (opt)
(complete)		 657281
(42.77%)		 21
(3.2%)		 260
(39.57%)		 235
(35.77%)		 141
(21.46%)		 0
(0%)
npSolver
1.0
(complete)		 657325
(49.47%)		 18
(2.74%)		 307
(46.73%)		 0
(0%)		 332
(50.53%)		 0
(0%)
npSolver
1.0 (fixed)
(complete)		 657337
(51.29%)		 22
(3.35%)		 315
(47.95%)		 0
(0%)		 320
(48.71%)		 0
(0%)
npSolver
inc
(complete)		 657203
(30.9%)		 18
(2.74%)		 185
(28.16%)		 0
(0%)		 454
(69.1%)		 0
(0%)
npSolver
inc (fixed)
(complete)		 657201
(30.59%)		 22
(3.35%)		 179
(27.25%)		 0
(0%)		 456
(69.41%)		 0
(0%)
npSolver
inc-topDown
(complete)		 657199
(30.29%)		 18
(2.74%)		 181
(27.55%)		 0
(0%)		 458
(69.71%)		 0
(0%)
npSolver
inc-topDown (fixed)
(complete)		 657192
(29.22%)		 22
(3.35%)		 170
(25.88%)		 0
(0%)		 465
(70.78%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 657183
(27.85%)		 18
(2.74%)		 165
(25.11%)		 0
(0%)		 474
(72.15%)		 0
(0%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 657168
(25.57%)		 22
(3.35%)		 146
(22.22%)		 0
(0%)		 489
(74.43%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 657316
(48.1%)		 20
(3.04%)		 296
(45.05%)		 247
(37.6%)		 82
(12.48%)		 7
(1.07%)
PB07: minisat+
1.14
(complete)		 657277
(42.16%)		 22
(3.35%)		 255
(38.81%)		 248
(37.75%)		 117
(17.81%)		 0
(0%)
PB07: PB-clasp
2007-04-10
(complete)		 657206
(31.35%)		 20
(3.04%)		 186
(28.31%)		 227
(34.55%)		 204
(31.05%)		 0
(0%)
PB07: Pueblo
1.4
(incomplete)		 657283
(43.07%)		 21
(3.2%)		 262
(39.88%)		 324
(49.32%)		 50
(7.61%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 657265
(40.33%)		 22
(3.35%)		 243
(36.99%)		 257
(39.12%)		 98
(14.92%)		 37
(5.63%)
PB09: bsolo
3.1
(complete)		 657305
(46.42%)		 21
(3.2%)		 284
(43.23%)		 279
(42.47%)		 70
(10.65%)		 3
(0.46%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 657265
(40.33%)		 21
(3.2%)		 244
(37.14%)		 292
(44.44%)		 99
(15.07%)		 1
(0.15%)
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)		 657321
(48.86%)		 22
(3.35%)		 299
(45.51%)		 135
(20.55%)		 200
(30.44%)		 1
(0.15%)
PB10: pb_cplex
2010-06-29
(complete)		 657376
(57.23%)		 21
(3.2%)		 355
(54.03%)		 26
(3.96%)		 235
(35.77%)		 20
(3.04%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 657275
(41.86%)		 21
(3.2%)		 254
(38.66%)		 320
(48.71%)		 61
(9.28%)		 1
(0.15%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 657312
(47.49%)		 21
(3.2%)		 291
(44.29%)		 161
(24.51%)		 184
(28.01%)		 0
(0%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 657279
(42.47%)		 21
(3.2%)		 258
(39.27%)		 338
(51.45%)		 40
(6.09%)		 0
(0%)
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)		 657344
(52.36%)		 22
(3.35%)		 322
(49.01%)		 136
(20.7%)		 177
(26.94%)		 0
(0%)
PB12: minisatp
1.0-2-g022594c
(complete)		 657279
(42.47%)		 22
(3.35%)		 257
(39.12%)		 230
(35.01%)		 91
(13.85%)		 57
(8.68%)
pb2sat
2012-05-19
(complete)		 657331
(50.38%)		 22
(3.35%)		 309
(47.03%)		 0
(0%)		 326
(49.62%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 657314
(47.79%)		 22
(3.35%)		 292
(44.44%)		 0
(0%)		 343
(52.21%)		 0
(0%)
pwbo
2.0
(complete)		 657335
(50.99%)		 17
(2.59%)		 318
(48.4%)		 234
(35.62%)		 80
(12.18%)		 8
(1.22%)
pwbo
2.02
(complete)		 657342
(52.05%)		 19
(2.89%)		 323
(49.16%)		 241
(36.68%)		 74
(11.26%)		 0
(0%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 657275
(41.86%)		 22
(3.35%)		 253
(38.51%)		 333
(50.68%)		 49
(7.46%)		 0
(0%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 657278
(42.31%)		 21
(3.2%)		 257
(39.12%)		 314
(47.79%)		 65
(9.89%)		 0
(0%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 657231
(35.16%)		 22
(3.35%)		 209
(31.81%)		 368
(56.01%)		 57
(8.68%)		 1
(0.15%)
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)		 657365
(55.56%)		 22
(3.35%)		 343
(52.21%)		 161
(24.51%)		 131
(19.94%)		 0
(0%)
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)		 657366
(55.71%)		 22
(3.35%)		 344
(52.36%)		 161
(24.51%)		 130
(19.79%)		 0
(0%)
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)		 657364
(55.4%)		 22
(3.35%)		 342
(52.05%)		 163
(24.81%)		 130
(19.79%)		 0
(0%)
toysat
2012-05-17
(complete)		 657247
(37.6%)		 21
(3.2%)		 226
(34.4%)		 6
(0.91%)		 404
(61.49%)		 0
(0%)
toysat
2012-06-01
(complete)		 657248
(37.75%)		 21
(3.2%)		 227
(34.55%)		 0
(0%)		 409
(62.25%)		 0
(0%)
wbo
1.7
(complete)		 657260
(39.57%)		 18
(2.74%)		 242
(36.83%)		 0
(0%)		 389
(59.21%)		 8
(1.22%)
wbo
1.72
(complete)		 657265
(40.33%)		 20
(3.04%)		 245
(37.29%)		 0
(0%)		 392
(59.67%)		 0
(0%)

Solver NameProgress
bsolo
3.2
(complete)
done
657
UNSAT
21		OPT/SAT
285		SAT (TO/MO)
286		?
65
clasp
2.0.6-R5325 (opt)
(complete)
done
657
UNSAT
21		OPT/SAT
260		SAT (TO/MO)
235		?
141
npSolver
1.0
(complete)
done
657
UNSAT
18		OPT/SAT
307		?
332
npSolver
1.0 (fixed)
(complete)
done
657
UNSAT
22		OPT/SAT
315		?
320
npSolver
inc
(complete)
done
657
UNSAT
18		OPT/SAT
185		?
454
npSolver
inc (fixed)
(complete)
done
657
UNSAT
22		OPT/SAT
179		?
456
npSolver
inc-topDown
(complete)
done
657
UNSAT
18		OPT/SAT
181		?
458
npSolver
inc-topDown (fixed)
(complete)
done
657
UNSAT
22		OPT/SAT
170		?
465
npSolver
inc-topdown-quickBound
(complete)
done
657
UNSAT
18		OPT/SAT
165		?
474
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
657
UNSAT
22		OPT/SAT
146		?
489
PB07: bsolo
3.0.17
(complete)
done
657
UNSAT
20		OPT/SAT
296		SAT (TO/MO)
247		?
82		ERR
7
PB07: minisat+
1.14
(complete)
done
657
UNSAT
22		OPT/SAT
255		SAT (TO/MO)
248		?
117
PB07: PB-clasp
2007-04-10
(complete)
done
657
UNSAT
20		OPT/SAT
186		SAT (TO/MO)
227		?
204
PB07: Pueblo
1.4
(incomplete)
done
657
UNSAT
21		OPT/SAT
262		SAT (TO/MO)
324		?
50
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
657
UNSAT
22		OPT/SAT
243		SAT (TO/MO)
257		?
98		ERR
37
PB09: bsolo
3.1
(complete)
done
657
UNSAT
21		OPT/SAT
284		SAT (TO/MO)
279		?
70		ERR
3
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
657
UNSAT
21		OPT/SAT
244		SAT (TO/MO)
292		?
99		ERR
1
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
done
657
UNSAT
22		OPT/SAT
299		SAT (TO/MO)
135		?
200		ERR
1
PB10: pb_cplex
2010-06-29
(complete)
done
657
UNSAT
21		OPT/SAT
355		SAT (TO/MO)
26		?
235		ERR
20
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
657
UNSAT
21		OPT/SAT
254		SAT (TO/MO)
320		?
61		ERR
1
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
657
UNSAT
21		OPT/SAT
291		SAT (TO/MO)
161		?
184
PB11: Sat4j Res//CP
2.3.0
(complete)
done
657
UNSAT
21		OPT/SAT
258		SAT (TO/MO)
338		?
40
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
657
UNSAT
22		OPT/SAT
322		SAT (TO/MO)
136		?
177
PB12: minisatp
1.0-2-g022594c
(complete)
done
657
UNSAT
22		OPT/SAT
257		SAT (TO/MO)
230		?
91		ERR
57
pb2sat
2012-05-19
(complete)
done
657
UNSAT
22		OPT/SAT
309		?
326
pb2satCp2
2012-05-19
(complete)
done
657
UNSAT
22		OPT/SAT
292		?
343
pwbo
2.0
(complete)
done
657
UNSAT
17		OPT/SAT
318		SAT (TO/MO)
234		?
80		ERR
8
pwbo
2.02
(complete)
done
657
UNSAT
19		OPT/SAT
323		SAT (TO/MO)
241		?
74
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
657
UNSAT
22		OPT/SAT
253		SAT (TO/MO)
333		?
49
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
657
UNSAT
21		OPT/SAT
257		SAT (TO/MO)
314		?
65
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
657
UNSAT
22		OPT/SAT
209		SAT (TO/MO)
368		?
57		ERR
1
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)
done
657
UNSAT
22		OPT/SAT
343		SAT (TO/MO)
161		?
131
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)
done
657
UNSAT
22		OPT/SAT
344		SAT (TO/MO)
161		?
130
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)
done
657
UNSAT
22		OPT/SAT
342		SAT (TO/MO)
163		?
130
toysat
2012-05-17
(complete)
done
657
UNSAT
21		OPT/SAT
226		SAT (TO/MO)
6		?
404
toysat
2012-06-01
(complete)
done
657
UNSAT
21		OPT/SAT
227		?
409
wbo
1.7
(complete)
done
657
UNSAT
18		OPT/SAT
242		?
389		ERR
8
wbo
1.72
(complete)
done
657
UNSAT
20		OPT/SAT
245		?
392

Category "optimisation, small integers, non linear constraints" (OPT-SMALLINT-NLC)

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
bsolo
3.2
(complete)		 12444
(35.48%)		 0
(0%)		 44
(35.48%)		 80
(64.52%)		 0
(0%)		 0
(0%)
clasp
2.0.6-R5325 (opt)
(complete)		 12460
(48.39%)		 0
(0%)		 60
(48.39%)		 64
(51.61%)		 0
(0%)		 0
(0%)
npSolver
1.0
(complete)		 12442
(33.87%)		 0
(0%)		 42
(33.87%)		 0
(0%)		 77
(62.1%)		 5
(4.03%)
npSolver
1.0 (fixed)
(complete)		 12460
(48.39%)		 0
(0%)		 60
(48.39%)		 0
(0%)		 64
(51.61%)		 0
(0%)
npSolver
inc
(complete)		 12441
(33.06%)		 0
(0%)		 41
(33.06%)		 0
(0%)		 78
(62.9%)		 5
(4.03%)
npSolver
inc (fixed)
(complete)		 12459
(47.58%)		 0
(0%)		 59
(47.58%)		 0
(0%)		 65
(52.42%)		 0
(0%)
npSolver
inc-topDown
(complete)		 12441
(33.06%)		 0
(0%)		 41
(33.06%)		 0
(0%)		 78
(62.9%)		 5
(4.03%)
npSolver
inc-topDown (fixed)
(complete)		 12459
(47.58%)		 0
(0%)		 59
(47.58%)		 0
(0%)		 65
(52.42%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 12431
(25%)		 0
(0%)		 31
(25%)		 0
(0%)		 88
(70.97%)		 5
(4.03%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 12431
(25%)		 0
(0%)		 31
(25%)		 0
(0%)		 93
(75%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 12447
(37.9%)		 0
(0%)		 47
(37.9%)		 73
(58.87%)		 4
(3.23%)		 0
(0%)
PB07: minisat+
1.14
(complete)		 12459
(47.58%)		 0
(0%)		 59
(47.58%)		 59
(47.58%)		 6
(4.84%)		 0
(0%)
PB07: PB-clasp
2007-04-10
(complete)		 12437
(29.84%)		 0
(0%)		 37
(29.84%)		 26
(20.97%)		 61
(49.19%)		 0
(0%)
PB07: Pueblo
1.4
(incomplete)		 12457
(45.97%)		 0
(0%)		 57
(45.97%)		 67
(54.03%)		 0
(0%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 12438
(30.65%)		 0
(0%)		 38
(30.65%)		 26
(20.97%)		 60
(48.39%)		 0
(0%)
PB09: bsolo
3.1
(complete)		 12436
(29.03%)		 0
(0%)		 36
(29.03%)		 67
(54.03%)		 6
(4.84%)		 15
(12.1%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 12447
(37.9%)		 0
(0%)		 47
(37.9%)		 63
(50.81%)		 13
(10.48%)		 1
(0.81%)
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)		 12465
(52.42%)		 0
(0%)		 65
(52.42%)		 52
(41.94%)		 7
(5.65%)		 0
(0%)
PB10: pb_cplex
2010-06-29
(complete)		 12444
(35.48%)		 0
(0%)		 44
(35.48%)		 0
(0%)		 71
(57.26%)		 9
(7.26%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 12455
(44.35%)		 0
(0%)		 55
(44.35%)		 68
(54.84%)		 1
(0.81%)		 0
(0%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 12462
(50%)		 0
(0%)		 62
(50%)		 61
(49.19%)		 1
(0.81%)		 0
(0%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 12456
(45.16%)		 0
(0%)		 56
(45.16%)		 67
(54.03%)		 1
(0.81%)		 0
(0%)
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)		 12467
(54.03%)		 0
(0%)		 67
(54.03%)		 53
(42.74%)		 4
(3.23%)		 0
(0%)
PB12: minisatp
1.0-2-g022594c
(complete)		 1240
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 124
(100%)		 0
(0%)
pb2sat
2012-05-19
(complete)		 12452
(41.94%)		 0
(0%)		 52
(41.94%)		 0
(0%)		 72
(58.06%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 12452
(41.94%)		 0
(0%)		 52
(41.94%)		 0
(0%)		 72
(58.06%)		 0
(0%)
pwbo
2.0
(complete)		 12454
(43.55%)		 0
(0%)		 54
(43.55%)		 69
(55.65%)		 0
(0%)		 1
(0.81%)
pwbo
2.02
(complete)		 12454
(43.55%)		 0
(0%)		 54
(43.55%)		 69
(55.65%)		 0
(0%)		 1
(0.81%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 12455
(44.35%)		 0
(0%)		 55
(44.35%)		 69
(55.65%)		 0
(0%)		 0
(0%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 12455
(44.35%)		 0
(0%)		 55
(44.35%)		 69
(55.65%)		 0
(0%)		 0
(0%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 12423
(18.55%)		 0
(0%)		 23
(18.55%)		 101
(81.45%)		 0
(0%)		 0
(0%)
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)		 12466
(53.23%)		 0
(0%)		 66
(53.23%)		 56
(45.16%)		 2
(1.61%)		 0
(0%)
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)		 12467
(54.03%)		 0
(0%)		 67
(54.03%)		 56
(45.16%)		 1
(0.81%)		 0
(0%)
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)		 12465
(52.42%)		 0
(0%)		 65
(52.42%)		 54
(43.55%)		 5
(4.03%)		 0
(0%)
toysat
2012-05-17
(complete)		 12445
(36.29%)		 0
(0%)		 45
(36.29%)		 0
(0%)		 79
(63.71%)		 0
(0%)
toysat
2012-06-01
(complete)		 12444
(35.48%)		 0
(0%)		 44
(35.48%)		 0
(0%)		 80
(64.52%)		 0
(0%)
wbo
1.7
(complete)		 12445
(36.29%)		 0
(0%)		 45
(36.29%)		 0
(0%)		 79
(63.71%)		 0
(0%)
wbo
1.72
(complete)		 12445
(36.29%)		 0
(0%)		 45
(36.29%)		 0
(0%)		 79
(63.71%)		 0
(0%)

Solver NameProgress
bsolo
3.2
(complete)
done
124
OPT/SAT
44		SAT (TO/MO)
80
clasp
2.0.6-R5325 (opt)
(complete)
done
124
OPT/SAT
60		SAT (TO/MO)
64
npSolver
1.0
(complete)
done
124
OPT/SAT
42		?
77		ERR
5
npSolver
1.0 (fixed)
(complete)
done
124
OPT/SAT
60		?
64
npSolver
inc
(complete)
done
124
OPT/SAT
41		?
78		ERR
5
npSolver
inc (fixed)
(complete)
done
124
OPT/SAT
59		?
65
npSolver
inc-topDown
(complete)
done
124
OPT/SAT
41		?
78		ERR
5
npSolver
inc-topDown (fixed)
(complete)
done
124
OPT/SAT
59		?
65
npSolver
inc-topdown-quickBound
(complete)
done
124
OPT/SAT
31		?
88		ERR
5
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
124
OPT/SAT
31		?
93
PB07: bsolo
3.0.17
(complete)
done
124
OPT/SAT
47		SAT (TO/MO)
73		?
4
PB07: minisat+
1.14
(complete)
done
124
OPT/SAT
59		SAT (TO/MO)
59		?
6
PB07: PB-clasp
2007-04-10
(complete)
done
124
OPT/SAT
37		SAT (TO/MO)
26		?
61
PB07: Pueblo
1.4
(incomplete)
done
124
OPT/SAT
57		SAT (TO/MO)
67
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
124
OPT/SAT
38		SAT (TO/MO)
26		?
60
PB09: bsolo
3.1
(complete)
done
124
OPT/SAT
36		SAT (TO/MO)
67		?
6		ERR
15
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
124
OPT/SAT
47		SAT (TO/MO)
63		?
13		ERR
1
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
done
124
OPT/SAT
65		SAT (TO/MO)
52		?
7
PB10: pb_cplex
2010-06-29
(complete)
done
124
OPT/SAT
44		?
71		ERR
9
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
124
OPT/SAT
55		SAT (TO/MO)
68		?
1
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
124
OPT/SAT
62		SAT (TO/MO)
61		?
1
PB11: Sat4j Res//CP
2.3.0
(complete)
done
124
OPT/SAT
56		SAT (TO/MO)
67		?
1
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
124
OPT/SAT
67		SAT (TO/MO)
53		?
4
PB12: minisatp
1.0-2-g022594c
(complete)
done
124
?
124
pb2sat
2012-05-19
(complete)
done
124
OPT/SAT
52		?
72
pb2satCp2
2012-05-19
(complete)
done
124
OPT/SAT
52		?
72
pwbo
2.0
(complete)
done
124
OPT/SAT
54		SAT (TO/MO)
69		ERR
1
pwbo
2.02
(complete)
done
124
OPT/SAT
54		SAT (TO/MO)
69		ERR
1
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
124
OPT/SAT
55		SAT (TO/MO)
69
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
124
OPT/SAT
55		SAT (TO/MO)
69
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
124
OPT/SAT
23		SAT (TO/MO)
101
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)
done
124
OPT/SAT
66		SAT (TO/MO)
56		?
2
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)
done
124
OPT/SAT
67		SAT (TO/MO)
56		?
1
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)
done
124
OPT/SAT
65		SAT (TO/MO)
54		?
5
toysat
2012-05-17
(complete)
done
124
OPT/SAT
45		?
79
toysat
2012-06-01
(complete)
done
124
OPT/SAT
44		?
80
wbo
1.7
(complete)
done
124
OPT/SAT
45		?
79
wbo
1.72
(complete)
done
124
OPT/SAT
45		?
79

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
npSolver
1.0
(complete)		 1596400
(25.06%)		 34
(2.13%)		 366
(22.93%)		 0
(0%)		 1191
(74.62%)		 5
(0.31%)
npSolver
1.0 (fixed)
(complete)		 1596682
(42.73%)		 143
(8.96%)		 539
(33.77%)		 0
(0%)		 914
(57.27%)		 0
(0%)
npSolver
inc
(complete)		 1596269
(16.85%)		 34
(2.13%)		 235
(14.72%)		 0
(0%)		 1322
(82.83%)		 5
(0.31%)
npSolver
inc (fixed)
(complete)		 1595496
(31.1%)		 142
(8.9%)		 354
(22.19%)		 0
(0%)		 1099
(68.9%)		 0
(0%)
npSolver
inc-topDown
(complete)		 1596264
(16.54%)		 34
(2.13%)		 230
(14.41%)		 0
(0%)		 1327
(83.15%)		 5
(0.31%)
npSolver
inc-topDown (fixed)
(complete)		 1596491
(30.76%)		 143
(8.96%)		 348
(21.8%)		 0
(0%)		 1105
(69.24%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 1596232
(14.54%)		 34
(2.13%)		 198
(12.41%)		 0
(0%)		 1359
(85.15%)		 5
(0.31%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 1596433
(27.13%)		 143
(8.96%)		 290
(18.17%)		 0
(0%)		 1163
(72.87%)		 0
(0%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 1596715
(44.8%)		 163
(10.21%)		 552
(34.59%)		 492
(30.83%)		 270
(16.92%)		 2
(0.13%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 1596785
(49.19%)		 178
(11.15%)		 607
(38.03%)		 243
(15.23%)		 534
(33.46%)		 34
(2.13%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 1596718
(44.99%)		 167
(10.46%)		 551
(34.52%)		 517
(32.39%)		 225
(14.1%)		 1
(0.06%)
PB12: minisatp
1.0-2-g022594c
(complete)		 1596550
(34.46%)		 137
(8.58%)		 413
(25.88%)		 309
(19.36%)		 558
(34.96%)		 122
(7.64%)
pb2sat
2012-05-19
(complete)		 1596414
(25.94%)		 38
(2.38%)		 376
(23.56%)		 0
(0%)		 1182
(74.06%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 1596395
(24.75%)		 38
(2.38%)		 357
(22.37%)		 0
(0%)		 1201
(75.25%)		 0
(0%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 1596707
(44.3%)		 169
(10.59%)		 538
(33.71%)		 525
(32.89%)		 235
(14.72%)		 1
(0.06%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 1596683
(42.79%)		 140
(8.77%)		 543
(34.02%)		 503
(31.52%)		 294
(18.42%)		 2
(0.13%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 1596563
(35.28%)		 141
(8.83%)		 422
(26.44%)		 614
(38.47%)		 338
(21.18%)		 1
(0.06%)
toysat
2012-05-17
(complete)		 1596569
(35.65%)		 128
(8.02%)		 441
(27.63%)		 6
(0.38%)		 1020
(63.91%)		 1
(0.06%)
toysat
2012-06-01
(complete)		 1596573
(35.9%)		 134
(8.4%)		 439
(27.51%)		 0
(0%)		 1022
(64.04%)		 1
(0.06%)

Solver NameProgress
npSolver
1.0
(complete)
done
1596
UNSAT
34		OPT/SAT
366		?
1191		ERR
5
npSolver
1.0 (fixed)
(complete)
done
1596
UNSAT
143		OPT/SAT
539		?
914
npSolver
inc
(complete)
done
1596
UNSAT
34		OPT/SAT
235		?
1322		ERR
5
npSolver
inc (fixed)
(complete)
done
1595
UNSAT
142		OPT/SAT
354		?
1099
npSolver
inc-topDown
(complete)
done
1596
UNSAT
34		OPT/SAT
230		?
1327		ERR
5
npSolver
inc-topDown (fixed)
(complete)
done
1596
UNSAT
143		OPT/SAT
348		?
1105
npSolver
inc-topdown-quickBound
(complete)
done
1596
UNSAT
34		OPT/SAT
198		?
1359		ERR
5
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
1596
UNSAT
143		OPT/SAT
290		?
1163
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
1596
UNSAT
163		OPT/SAT
552		SAT (TO/MO)
492		?
270		ERR
2
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
1596
UNSAT
178		OPT/SAT
607		SAT (TO/MO)
243		?
534		ERR
34
PB11: Sat4j Res//CP
2.3.0
(complete)
done
1596
UNSAT
167		OPT/SAT
551		SAT (TO/MO)
517		?
225		ERR
1
PB12: minisatp
1.0-2-g022594c
(complete)
done
1596
UNSAT
137		OPT/SAT
413		SAT (TO/MO)
309		?
558		ERR
122
pb2sat
2012-05-19
(complete)
done
1596
UNSAT
38		OPT/SAT
376		?
1182
pb2satCp2
2012-05-19
(complete)
done
1596
UNSAT
38		OPT/SAT
357		?
1201
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
1596
UNSAT
169		OPT/SAT
538		SAT (TO/MO)
525		?
235		ERR
1
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
1596
UNSAT
140		OPT/SAT
543		SAT (TO/MO)
503		?
294		ERR
2
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
1596
UNSAT
141		OPT/SAT
422		SAT (TO/MO)
614		?
338		ERR
1
toysat
2012-05-17
(complete)
done
1596
UNSAT
128		OPT/SAT
441		SAT (TO/MO)
6		?
1020		ERR
1
toysat
2012-06-01
(complete)
done
1596
UNSAT
134		OPT/SAT
439		?
1022		ERR
1

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATOPT/SATSAT (TO/MO)UNKNOWNBad Answers
bsolo
3.2
(complete)		 1166606
(51.97%)		 141
(12.09%)		 465
(39.88%)		 366
(31.39%)		 194
(16.64%)		 0
(0%)
clasp
2.0.6-R5325 (dec)
(complete)		 385266
(69.09%)		 102
(26.49%)		 164
(42.6%)		 0
(0%)		 119
(30.91%)		 0
(0%)
clasp
2.0.6-R5325 (opt)
(complete)		 781341
(43.66%)		 21
(2.69%)		 320
(40.97%)		 299
(38.28%)		 141
(18.05%)		 0
(0%)
npSolver
1.0
(complete)		 1596400
(25.06%)		 34
(2.13%)		 366
(22.93%)		 0
(0%)		 1191
(74.62%)		 5
(0.31%)
npSolver
1.0 (fixed)
(complete)		 1596682
(42.73%)		 143
(8.96%)		 539
(33.77%)		 0
(0%)		 914
(57.27%)		 0
(0%)
npSolver
inc
(complete)		 1596269
(16.85%)		 34
(2.13%)		 235
(14.72%)		 0
(0%)		 1322
(82.83%)		 5
(0.31%)
npSolver
inc (fixed)
(complete)		 1595496
(31.1%)		 142
(8.9%)		 354
(22.19%)		 0
(0%)		 1099
(68.9%)		 0
(0%)
npSolver
inc-topDown
(complete)		 1596264
(16.54%)		 34
(2.13%)		 230
(14.41%)		 0
(0%)		 1327
(83.15%)		 5
(0.31%)
npSolver
inc-topDown (fixed)
(complete)		 1596491
(30.76%)		 143
(8.96%)		 348
(21.8%)		 0
(0%)		 1105
(69.24%)		 0
(0%)
npSolver
inc-topdown-quickBound
(complete)		 1596232
(14.54%)		 34
(2.13%)		 198
(12.41%)		 0
(0%)		 1359
(85.15%)		 5
(0.31%)
npSolver
inc-topdown-quickBound (fixed)
(complete)		 1596433
(27.13%)		 143
(8.96%)		 290
(18.17%)		 0
(0%)		 1163
(72.87%)		 0
(0%)
PB07: bsolo
3.0.17
(complete)		 1596641
(40.16%)		 131
(8.21%)		 510
(31.95%)		 408
(25.56%)		 398
(24.94%)		 18
(1.13%)
PB07: minisat+
1.14
(complete)		 1596625
(39.16%)		 141
(8.83%)		 484
(30.33%)		 390
(24.44%)		 476
(29.82%)		 0
(0%)
PB07: PB-clasp
2007-04-10
(complete)		 1166489
(41.94%)		 118
(10.12%)		 371
(31.82%)		 253
(21.7%)		 404
(34.65%)		 0
(0%)
PB07: Pueblo
1.4
(incomplete)		 1166600
(51.46%)		 138
(11.84%)		 462
(39.62%)		 391
(33.53%)		 175
(15.01%)		 0
(0%)
PB07: SAT4JPseudoResolution
2007-03-23
(complete)		 1596591
(37.03%)		 134
(8.4%)		 457
(28.63%)		 389
(24.37%)		 526
(32.96%)		 67
(4.2%)
PB09: bsolo
3.1
(complete)		 1166600
(51.46%)		 144
(12.35%)		 456
(39.11%)		 346
(29.67%)		 201
(17.24%)		 19
(1.63%)
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)		 1596618
(38.72%)		 137
(8.58%)		 481
(30.14%)		 455
(28.51%)		 478
(29.95%)		 7
(0.44%)
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)		 1166601
(51.54%)		 140
(12.01%)		 461
(39.54%)		 187
(16.04%)		 377
(32.33%)		 1
(0.09%)
PB10: borg-pb
10.05.30
(complete)		 385249
(64.68%)		 115
(29.87%)		 134
(34.81%)		 0
(0%)		 119
(30.91%)		 17
(4.42%)
PB10: pb_cplex
2010-06-29
(complete)		 1166636
(54.55%)		 134
(11.49%)		 502
(43.05%)		 26
(2.23%)		 475
(40.74%)		 29
(2.49%)
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 1596715
(44.8%)		 163
(10.21%)		 552
(34.59%)		 492
(30.83%)		 270
(16.92%)		 2
(0.13%)
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 1596785
(49.19%)		 178
(11.15%)		 607
(38.03%)		 243
(15.23%)		 534
(33.46%)		 34
(2.13%)
PB11: borg
pb-dec-11.04.03
(complete)		 385264
(68.57%)		 125
(32.47%)		 139
(36.1%)		 0
(0%)		 121
(31.43%)		 0
(0%)
PB11: Sat4j Res//CP
2.3.0
(complete)		 1596718
(44.99%)		 167
(10.46%)		 551
(34.52%)		 517
(32.39%)		 225
(14.1%)		 1
(0.06%)
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)		 1166627
(53.77%)		 141
(12.09%)		 486
(41.68%)		 189
(16.21%)		 350
(30.02%)		 0
(0%)
PB12: minisatp
1.0-2-g022594c
(complete)		 1596550
(34.46%)		 137
(8.58%)		 413
(25.88%)		 309
(19.36%)		 558
(34.96%)		 122
(7.64%)
pb2sat
2012-05-19
(complete)		 1596414
(25.94%)		 38
(2.38%)		 376
(23.56%)		 0
(0%)		 1182
(74.06%)		 0
(0%)
pb2satCp2
2012-05-19
(complete)		 1596395
(24.75%)		 38
(2.38%)		 357
(22.37%)		 0
(0%)		 1201
(75.25%)		 0
(0%)
PBPASSolver-CARD.DP
2012-05-28
(complete)		 369144
(39.02%)		 65
(17.62%)		 79
(21.41%)		 0
(0%)		 225
(60.98%)		 0
(0%)
PBPASSolver-CARD.SN
2012-05-28
(complete)		 369165
(44.72%)		 67
(18.16%)		 98
(26.56%)		 0
(0%)		 204
(55.28%)		 0
(0%)
pwbo
2.0
(complete)		 3020
(66.67%)		 5
(16.67%)		 15
(50%)		 0
(0%)		 10
(33.33%)		 0
(0%)
pwbo
2.0
(complete)		 355228
(64.23%)		 94
(26.48%)		 134
(37.75%)		 0
(0%)		 127
(35.77%)		 0
(0%)
pwbo
2.0
(complete)		 657335
(50.99%)		 17
(2.59%)		 318
(48.4%)		 234
(35.62%)		 80
(12.18%)		 8
(1.22%)
pwbo
2.0
(complete)		 12454
(43.55%)		 0
(0%)		 54
(43.55%)		 69
(55.65%)		 0
(0%)		 1
(0.81%)
pwbo
2.02
(complete)		 657342
(52.05%)		 19
(2.89%)		 323
(49.16%)		 241
(36.68%)		 74
(11.26%)		 0
(0%)
pwbo
2.02
(complete)		 3019
(63.33%)		 5
(16.67%)		 14
(46.67%)		 0
(0%)		 11
(36.67%)		 0
(0%)
pwbo
2.02
(complete)		 355226
(63.66%)		 95
(26.76%)		 131
(36.9%)		 0
(0%)		 129
(36.34%)		 0
(0%)
pwbo
2.02
(complete)		 12454
(43.55%)		 0
(0%)		 54
(43.55%)		 69
(55.65%)		 0
(0%)		 1
(0.81%)
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)		 1596707
(44.3%)		 169
(10.59%)		 538
(33.71%)		 525
(32.89%)		 235
(14.72%)		 1
(0.06%)
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)		 1596683
(42.79%)		 140
(8.77%)		 543
(34.02%)		 503
(31.52%)		 294
(18.42%)		 2
(0.13%)
SAT4J PB specific settings
2.3.2 snapshot
(complete)		 1596563
(35.28%)		 141
(8.83%)		 422
(26.44%)		 614
(38.47%)		 338
(21.18%)		 1
(0.06%)
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)		 1166650
(55.75%)		 141
(12.09%)		 509
(43.65%)		 217
(18.61%)		 299
(25.64%)		 0
(0%)
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)		 1166651
(55.83%)		 141
(12.09%)		 510
(43.74%)		 217
(18.61%)		 298
(25.56%)		 0
(0%)
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)		 1166655
(56.17%)		 142
(12.18%)		 513
(44%)		 217
(18.61%)		 294
(25.21%)		 0
(0%)
toysat
2012-05-17
(complete)		 1596569
(35.65%)		 128
(8.02%)		 441
(27.63%)		 6
(0.38%)		 1020
(63.91%)		 1
(0.06%)
toysat
2012-06-01
(complete)		 1596573
(35.9%)		 134
(8.4%)		 439
(27.51%)		 0
(0%)		 1022
(64.04%)		 1
(0.06%)
wbo
1.7
(complete)		 15464
(41.56%)		 5
(3.25%)		 59
(38.31%)		 0
(0%)		 90
(58.44%)		 0
(0%)
wbo
1.7
(complete)		 1012485
(47.92%)		 112
(11.07%)		 373
(36.86%)		 0
(0%)		 519
(51.28%)		 8
(0.79%)
wbo
1.72
(complete)		 1012491
(48.52%)		 115
(11.36%)		 376
(37.15%)		 0
(0%)		 521
(51.48%)		 0
(0%)
wbo
1.72
(complete)		 15464
(41.56%)		 5
(3.25%)		 59
(38.31%)		 0
(0%)		 90
(58.44%)		 0
(0%)

Solver NameProgress
bsolo
3.2
(complete)
done
1166
UNSAT
141		OPT/SAT
465		SAT (TO/MO)
366		?
194
clasp
2.0.6-R5325 (dec)
(complete)
done
385
UNSAT
102		OPT/SAT
164		?
119
clasp
2.0.6-R5325 (opt)
(complete)
done
781
UNSAT
21		OPT/SAT
320		SAT (TO/MO)
299		?
141
npSolver
1.0
(complete)
done
1596
UNSAT
34		OPT/SAT
366		?
1191		ERR
5
npSolver
1.0 (fixed)
(complete)
done
1596
UNSAT
143		OPT/SAT
539		?
914
npSolver
inc
(complete)
done
1596
UNSAT
34		OPT/SAT
235		?
1322		ERR
5
npSolver
inc (fixed)
(complete)
done
1595
UNSAT
142		OPT/SAT
354		?
1099
npSolver
inc-topDown
(complete)
done
1596
UNSAT
34		OPT/SAT
230		?
1327		ERR
5
npSolver
inc-topDown (fixed)
(complete)
done
1596
UNSAT
143		OPT/SAT
348		?
1105
npSolver
inc-topdown-quickBound
(complete)
done
1596
UNSAT
34		OPT/SAT
198		?
1359		ERR
5
npSolver
inc-topdown-quickBound (fixed)
(complete)
done
1596
UNSAT
143		OPT/SAT
290		?
1163
PB07: bsolo
3.0.17
(complete)
done
1596
UNSAT
131		OPT/SAT
510		SAT (TO/MO)
408		?
398		ERR
18
PB07: minisat+
1.14
(complete)
done
1596
UNSAT
141		OPT/SAT
484		SAT (TO/MO)
390		?
476
PB07: PB-clasp
2007-04-10
(complete)
done
1166
UNSAT
118		OPT/SAT
371		SAT (TO/MO)
253		?
404
PB07: Pueblo
1.4
(incomplete)
done
1166
UNSAT
138		OPT/SAT
462		SAT (TO/MO)
391		?
175
PB07: SAT4JPseudoResolution
2007-03-23
(complete)
done
1596
UNSAT
134		OPT/SAT
457		SAT (TO/MO)
389		?
526		ERR
67
PB09: bsolo
3.1
(complete)
done
1166
UNSAT
144		OPT/SAT
456		SAT (TO/MO)
346		?
201		ERR
19
PB09: SAT4J Pseudo Resolution
2.1.1
(complete)
done
1596
UNSAT
137		OPT/SAT
481		SAT (TO/MO)
455		?
478		ERR
7
PB09: SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
done
1166
UNSAT
140		OPT/SAT
461		SAT (TO/MO)
187		?
377		ERR
1
PB10: borg-pb
10.05.30
(complete)
done
385
UNSAT
115		OPT/SAT
134		?
119		ERR
17
PB10: pb_cplex
2010-06-29
(complete)
done
1166
UNSAT
134		OPT/SAT
502		SAT (TO/MO)
26		?
475		ERR
29
PB10: SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
1596
UNSAT
163		OPT/SAT
552		SAT (TO/MO)
492		?
270		ERR
2
PB10: SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
1596
UNSAT
178		OPT/SAT
607		SAT (TO/MO)
243		?
534		ERR
34
PB11: borg
pb-dec-11.04.03
(complete)
done
385
UNSAT
125		OPT/SAT
139		?
121
PB11: Sat4j Res//CP
2.3.0
(complete)
done
1596
UNSAT
167		OPT/SAT
551		SAT (TO/MO)
517		?
225		ERR
1
PB11: SCIP spx E_2
2011-06-10 (fixed)
(complete)
done
1166
UNSAT
141		OPT/SAT
486		SAT (TO/MO)
189		?
350
PB12: minisatp
1.0-2-g022594c
(complete)
done
1596
UNSAT
137		OPT/SAT
413		SAT (TO/MO)
309		?
558		ERR
122
pb2sat
2012-05-19
(complete)
done
1596
UNSAT
38		OPT/SAT
376		?
1182
pb2satCp2
2012-05-19
(complete)
done
1596
UNSAT
38		OPT/SAT
357		?
1201
PBPASSolver-CARD.DP
2012-05-28
(complete)
done
369
UNSAT
65		OPT/SAT
79		?
225
PBPASSolver-CARD.SN
2012-05-28
(complete)
done
369
UNSAT
67		OPT/SAT
98		?
204
pwbo
2.0
(complete)
done
30
UNSAT
5		OPT/SAT
15		?
10
pwbo
2.0
(complete)
done
355
UNSAT
94		OPT/SAT
134		?
127
pwbo
2.0
(complete)
done
657
UNSAT
17		OPT/SAT
318		SAT (TO/MO)
234		?
80		ERR
8
pwbo
2.0
(complete)
done
124
OPT/SAT
54		SAT (TO/MO)
69		ERR
1
pwbo
2.02
(complete)
done
657
UNSAT
19		OPT/SAT
323		SAT (TO/MO)
241		?
74
pwbo
2.02
(complete)
done
30
UNSAT
5		OPT/SAT
14		?
11
pwbo
2.02
(complete)
done
355
UNSAT
95		OPT/SAT
131		?
129
pwbo
2.02
(complete)
done
124
OPT/SAT
54		SAT (TO/MO)
69		ERR
1
SAT 4j PB RES // CP
2.3.2 Snapshot
(complete)
done
1596
UNSAT
169		OPT/SAT
538		SAT (TO/MO)
525		?
235		ERR
1
Sat 4j PB Resolution
2.3.2 Snapshot
(complete)
done
1596
UNSAT
140		OPT/SAT
543		SAT (TO/MO)
503		?
294		ERR
2
SAT4J PB specific settings
2.3.2 snapshot
(complete)
done
1596
UNSAT
141		OPT/SAT
422		SAT (TO/MO)
614		?
338		ERR
1
SCIP spx
SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed
(complete)
done
1166
UNSAT
141		OPT/SAT
509		SAT (TO/MO)
217		?
299
SCIP spx E
SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed
(complete)
done
1166
UNSAT
141		OPT/SAT
510		SAT (TO/MO)
217		?
298
SCIP spx standard
SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
(complete)
done
1166
UNSAT
142		OPT/SAT
513		SAT (TO/MO)
217		?
294
toysat
2012-05-17
(complete)
done
1596
UNSAT
128		OPT/SAT
441		SAT (TO/MO)
6		?
1020		ERR
1
toysat
2012-06-01
(complete)
done
1596
UNSAT
134		OPT/SAT
439		?
1022		ERR
1
wbo
1.7
(complete)
done
154
UNSAT
5		OPT/SAT
59		?
90
wbo
1.7
(complete)
done
1012
UNSAT
112		OPT/SAT
373		?
519		ERR
8
wbo
1.72
(complete)
done
1012
UNSAT
115		OPT/SAT
376		?
521
wbo
1.72
(complete)
done
154
UNSAT
5		OPT/SAT
59		?
90

WARNING: 126 jobs are unclassified for solver 2318 in category OPT-BIGINT-LIN/
WARNING: 90 jobs are unclassified for solver 2306 in category OPT-BIGINT-LIN/
WARNING: 23 jobs are unclassified for solver 2319 in category OPT-BIGINT-LIN/
WARNING: 38 jobs are unclassified for solver 2322 in category OPT-BIGINT-LIN/
WARNING: 117 jobs are unclassified for solver 2324 in category OPT-BIGINT-LIN/
WARNING: 135 jobs are unclassified for solver 2328 in category OPT-BIGINT-LIN/
WARNING: 57 jobs are unclassified for solver 2307 in category OPT-BIGINT-LIN/
WARNING: 128 jobs are unclassified for solver 2268 in category OPT-BIGINT-LIN/
WARNING: 114 jobs are unclassified for solver 2269 in category OPT-BIGINT-LIN/
WARNING: 80 jobs are unclassified for solver 2289 in category OPT-BIGINT-LIN/
WARNING: 5 jobs are unclassified for solver 2318 in category OPT-SMALLINT-LIN/
WARNING: 15 jobs are unclassified for solver 2306 in category OPT-SMALLINT-LIN/
WARNING: 20 jobs are unclassified for solver 2317 in category OPT-SMALLINT-LIN/
WARNING: 117 jobs are unclassified for solver 2324 in category /
WARNING: 135 jobs are unclassified for solver 2328 in category /
WARNING: 57 jobs are unclassified for solver 2307 in category /
WARNING: 128 jobs are unclassified for solver 2268 in category /
WARNING: 114 jobs are unclassified for solver 2269 in category /
WARNING: 80 jobs are unclassified for solver 2289 in category /
WARNING: 131 jobs are unclassified for solver 2318 in category /
WARNING: 105 jobs are unclassified for solver 2306 in category /
WARNING: 20 jobs are unclassified for solver 2317 in category /
WARNING: 23 jobs are unclassified for solver 2319 in category /
WARNING: 38 jobs are unclassified for solver 2322 in category /
WARNING: 117 jobs are unclassified for solver 2324 in category /
WARNING: 135 jobs are unclassified for solver 2328 in category /
WARNING: 57 jobs are unclassified for solver 2307 in category /
WARNING: 128 jobs are unclassified for solver 2268 in category /
WARNING: 114 jobs are unclassified for solver 2269 in category /
WARNING: 80 jobs are unclassified for solver 2289 in category /