PB'10 competition: WBO track: solvers page (tabular format, summary)

Some explanations

Category "both soft and hard constraints, big integers, linear constraints" (PARTIAL-BIGINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
PB/CT
0.1
(complete)		 2633
(1.14%)		 1
(0.38%)		 2
(0.76%)		 0
(0%)		 23
(8.75%)		 83
(31.56%)		 0
(0%)		 154
(58.56%)
PB/CT
0.1 fixed
(complete)		 26378
(29.66%)		 0
(0%)		 78
(29.66%)		 0
(0%)		 156
(59.32%)		 29
(11.03%)		 0
(0%)		 0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		 26323
(8.75%)		 18
(6.84%)		 5
(1.9%)		 0
(0%)		 24
(9.13%)		 9
(3.42%)		 0
(0%)		 207
(78.71%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 26317
(6.46%)		 10
(3.8%)		 7
(2.66%)		 0
(0%)		 17
(6.46%)		 18
(6.84%)		 0
(0%)		 211
(80.23%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		 26325
(9.51%)		 14
(5.32%)		 11
(4.18%)		 0
(0%)		 13
(4.94%)		 11
(4.18%)		 0
(0%)		 214
(81.37%)
SAT4J PB Resolution
2.2.1
(complete)		 263113
(42.97%)		 0
(0%)		 113
(42.97%)		 0
(0%)		 126
(47.91%)		 24
(9.13%)		 0
(0%)		 0
(0%)

Solver NameProgress
PB/CT
0.1
(complete)
done
263
UNSAT
1		MOPT
2		MSAT
23		?
83		ERR
154
PB/CT
0.1 fixed
(complete)
done
263
MOPT
78		MSAT
156		?
29
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
263
UNSAT
18		MOPT
5		MSAT
24		?
9		ERR
207
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
263
UNSAT
10		MOPT
7		MSAT
17		?
18		ERR
211
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
263
UNSAT
14		MOPT
11		MSAT
13		?
11		ERR
214
SAT4J PB Resolution
2.2.1
(complete)
done
263
MOPT
113		MSAT
126		?
24

Category "both soft and hard constraints, small integers, linear constraints" (PARTIAL-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
pb_cplex
2010-07-01
(complete)		 536428
(79.85%)		 0
(0%)		 428
(79.85%)		 0
(0%)		 2
(0.37%)		 106
(19.78%)		 0
(0%)		 0
(0%)
PB/CT
0.1
(complete)		 536209
(38.99%)		 1
(0.19%)		 208
(38.81%)		 0
(0%)		 72
(13.43%)		 150
(27.99%)		 0
(0%)		 105
(19.59%)
PB/CT
0.1 fixed
(complete)		 536375
(69.96%)		 1
(0.19%)		 374
(69.78%)		 0
(0%)		 161
(30.04%)		 0
(0%)		 0
(0%)		 0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		 53660
(11.19%)		 0
(0%)		 60
(11.19%)		 0
(0%)		 250
(46.64%)		 21
(3.92%)		 0
(0%)		 205
(38.25%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 53694
(17.54%)		 1
(0.19%)		 93
(17.35%)		 0
(0%)		 66
(12.31%)		 0
(0%)		 0
(0%)		 376
(70.15%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		 53679
(14.74%)		 1
(0.19%)		 78
(14.55%)		 0
(0%)		 92
(17.16%)		 2
(0.37%)		 0
(0%)		 363
(67.72%)
SAT4J PB Resolution
2.2.1
(complete)		 536446
(83.21%)		 1
(0.19%)		 445
(83.02%)		 0
(0%)		 89
(16.6%)		 1
(0.19%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 536297
(55.41%)		 1
(0.19%)		 296
(55.22%)		 0
(0%)		 191
(35.63%)		 48
(8.96%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 536282
(52.61%)		 1
(0.19%)		 281
(52.43%)		 0
(0%)		 203
(37.87%)		 51
(9.51%)		 0
(0%)		 0
(0%)
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)		 536146
(27.24%)		 1
(0.19%)		 145
(27.05%)		 0
(0%)		 190
(35.45%)		 200
(37.31%)		 0
(0%)		 0
(0%)
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 536269
(50.19%)		 1
(0.19%)		 268
(50%)		 0
(0%)		 210
(39.18%)		 57
(10.63%)		 0
(0%)		 0
(0%)
wbo
1.4a
(complete)		 536373
(69.59%)		 1
(0.19%)		 372
(69.4%)		 0
(0%)		 25
(4.66%)		 138
(25.75%)		 0
(0%)		 0
(0%)
wbo
1.4b
(complete)		 5361
(0.19%)		 1
(0.19%)		 0
(0%)		 0
(0%)		 24
(4.48%)		 76
(14.18%)		 0
(0%)		 435
(81.16%)

Solver NameProgress
pb_cplex
2010-07-01
(complete)
done
536
MOPT
428		MSAT
2		?
106
PB/CT
0.1
(complete)
done
536
UNSAT
1		MOPT
208		MSAT
72		?
150		ERR
105
PB/CT
0.1 fixed
(complete)
done
536
UNSAT
1		MOPT
374		MSAT
161
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
536
MOPT
60		MSAT
250		?
21		ERR
205
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
536
UNSAT
1		MOPT
93		MSAT
66		ERR
376
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
536
UNSAT
1		MOPT
78		MSAT
92		?
2		ERR
363
SAT4J PB Resolution
2.2.1
(complete)
done
536
UNSAT
1		MOPT
445		MSAT
89		?
1
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
536
UNSAT
1		MOPT
296		MSAT
191		?
48
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
536
UNSAT
1		MOPT
281		MSAT
203		?
51
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
done
536
UNSAT
1		MOPT
145		MSAT
190		?
200
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
536
UNSAT
1		MOPT
268		MSAT
210		?
57
wbo
1.4a
(complete)
done
536
UNSAT
1		MOPT
372		MSAT
25		?
138
wbo
1.4b
(complete)
done
536
UNSAT
1		MSAT
24		?
76		ERR
435

Category "only soft constraints, big integers, linear constraints" (SOFT-BIGINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
PB/CT
0.1
(complete)		 460
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 46
(100%)		 0
(0%)		 0
(0%)
PB/CT
0.1 fixed
(complete)		 460
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 45
(97.83%)		 1
(2.17%)		 0
(0%)		 0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		 461
(2.17%)		 0
(0%)		 1
(2.17%)		 0
(0%)		 25
(54.35%)		 0
(0%)		 0
(0%)		 20
(43.48%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 464
(8.7%)		 0
(0%)		 4
(8.7%)		 0
(0%)		 17
(36.96%)		 0
(0%)		 0
(0%)		 25
(54.35%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		 467
(15.22%)		 0
(0%)		 7
(15.22%)		 0
(0%)		 11
(23.91%)		 0
(0%)		 0
(0%)		 28
(60.87%)
SAT4J PB Resolution
2.2.1
(complete)		 4633
(71.74%)		 0
(0%)		 33
(71.74%)		 0
(0%)		 13
(28.26%)		 0
(0%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 460
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 5
(10.87%)		 41
(89.13%)		 0
(0%)		 0
(0%)

Solver NameProgress
PB/CT
0.1
(complete)
done
46
?
46
PB/CT
0.1 fixed
(complete)
done
46
MSAT
45		?
1
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
46
MOPT
1		MSAT
25		ERR
20
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
46
MOPT
4		MSAT
17		ERR
25
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
46
MOPT
7		MSAT
11		ERR
28
SAT4J PB Resolution
2.2.1
(complete)
done
46
MOPT
33		MSAT
13
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
46
MSAT
5		?
41

Category "only soft constraints, small integers, linear constraints" (SOFT-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
pb_cplex
2010-07-01
(complete)		 201160
(79.6%)		 0
(0%)		 160
(79.6%)		 0
(0%)		 0
(0%)		 41
(20.4%)		 0
(0%)		 0
(0%)
PB/CT
0.1
(complete)		 20159
(29.35%)		 0
(0%)		 59
(29.35%)		 0
(0%)		 45
(22.39%)		 54
(26.87%)		 0
(0%)		 43
(21.39%)
PB/CT
0.1 fixed
(complete)		 201138
(68.66%)		 0
(0%)		 138
(68.66%)		 0
(0%)		 63
(31.34%)		 0
(0%)		 0
(0%)		 0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		 20119
(9.45%)		 0
(0%)		 19
(9.45%)		 0
(0%)		 132
(65.67%)		 0
(0%)		 0
(0%)		 50
(24.88%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 20133
(16.42%)		 0
(0%)		 33
(16.42%)		 0
(0%)		 42
(20.9%)		 0
(0%)		 0
(0%)		 126
(62.69%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		 20123
(11.44%)		 0
(0%)		 23
(11.44%)		 0
(0%)		 44
(21.89%)		 0
(0%)		 0
(0%)		 134
(66.67%)
SAT4J PB Resolution
2.2.1
(complete)		 201160
(79.6%)		 0
(0%)		 160
(79.6%)		 0
(0%)		 41
(20.4%)		 0
(0%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 201113
(56.22%)		 0
(0%)		 113
(56.22%)		 0
(0%)		 76
(37.81%)		 12
(5.97%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 201113
(56.22%)		 0
(0%)		 113
(56.22%)		 0
(0%)		 88
(43.78%)		 0
(0%)		 0
(0%)		 0
(0%)
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)		 20122
(10.95%)		 0
(0%)		 22
(10.95%)		 0
(0%)		 120
(59.7%)		 59
(29.35%)		 0
(0%)		 0
(0%)
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 201113
(56.22%)		 0
(0%)		 113
(56.22%)		 0
(0%)		 76
(37.81%)		 12
(5.97%)		 0
(0%)		 0
(0%)
wbo
1.4a
(complete)		 201161
(80.1%)		 0
(0%)		 161
(80.1%)		 0
(0%)		 0
(0%)		 40
(19.9%)		 0
(0%)		 0
(0%)
wbo
1.4b
(complete)		 2010
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 36
(17.91%)		 0
(0%)		 165
(82.09%)

Solver NameProgress
pb_cplex
2010-07-01
(complete)
done
201
MOPT
160		?
41
PB/CT
0.1
(complete)
done
201
MOPT
59		MSAT
45		?
54		ERR
43
PB/CT
0.1 fixed
(complete)
done
201
MOPT
138		MSAT
63
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
201
MOPT
19		MSAT
132		ERR
50
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
201
MOPT
33		MSAT
42		ERR
126
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
201
MOPT
23		MSAT
44		ERR
134
SAT4J PB Resolution
2.2.1
(complete)
done
201
MOPT
160		MSAT
41
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
201
MOPT
113		MSAT
76		?
12
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
201
MOPT
113		MSAT
88
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
done
201
MOPT
22		MSAT
120		?
59
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
201
MOPT
113		MSAT
76		?
12
wbo
1.4a
(complete)
done
201
MOPT
161		?
40
wbo
1.4b
(complete)
done
201
?
36		ERR
165

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
PB/CT
0.1
(complete)		 1046271
(25.91%)		 2
(0.19%)		 269
(25.72%)		 0
(0%)		 140
(13.38%)		 333
(31.84%)		 0
(0%)		 302
(28.87%)
PB/CT
0.1 fixed
(complete)		 1046591
(56.5%)		 1
(0.1%)		 590
(56.41%)		 0
(0%)		 425
(40.63%)		 30
(2.87%)		 0
(0%)		 0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		 1046103
(9.85%)		 18
(1.72%)		 85
(8.13%)		 0
(0%)		 431
(41.2%)		 30
(2.87%)		 0
(0%)		 482
(46.08%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 1046148
(14.15%)		 11
(1.05%)		 137
(13.1%)		 0
(0%)		 142
(13.58%)		 18
(1.72%)		 0
(0%)		 738
(70.55%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		 1046134
(12.81%)		 15
(1.43%)		 119
(11.38%)		 0
(0%)		 160
(15.3%)		 13
(1.24%)		 0
(0%)		 739
(70.65%)
SAT4J PB Resolution
2.2.1
(complete)		 1046752
(71.89%)		 1
(0.1%)		 751
(71.8%)		 0
(0%)		 269
(25.72%)		 25
(2.39%)		 0
(0%)		 0
(0%)

Solver NameProgress
PB/CT
0.1
(complete)
done
1046
UNSAT
2		MOPT
269		MSAT
140		?
333		ERR
302
PB/CT
0.1 fixed
(complete)
done
1046
UNSAT
1		MOPT
590		MSAT
425		?
30
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
18		MOPT
85		MSAT
431		?
30		ERR
482
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
1046
UNSAT
11		MOPT
137		MSAT
142		?
18		ERR
738
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
15		MOPT
119		MSAT
160		?
13		ERR
739
SAT4J PB Resolution
2.2.1
(complete)
done
1046
UNSAT
1		MOPT
751		MSAT
269		?
25

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
pb_cplex
2010-07-01
(complete)		 737588
(79.78%)		 0
(0%)		 588
(79.78%)		 0
(0%)		 2
(0.27%)		 147
(19.95%)		 0
(0%)		 0
(0%)
PB/CT
0.1
(complete)		 1046271
(25.91%)		 2
(0.19%)		 269
(25.72%)		 0
(0%)		 140
(13.38%)		 333
(31.84%)		 0
(0%)		 302
(28.87%)
PB/CT
0.1 fixed
(complete)		 1046591
(56.5%)		 1
(0.1%)		 590
(56.41%)		 0
(0%)		 425
(40.63%)		 30
(2.87%)		 0
(0%)		 0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		 1046103
(9.85%)		 18
(1.72%)		 85
(8.13%)		 0
(0%)		 431
(41.2%)		 30
(2.87%)		 0
(0%)		 482
(46.08%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		 1046148
(14.15%)		 11
(1.05%)		 137
(13.1%)		 0
(0%)		 142
(13.58%)		 18
(1.72%)		 0
(0%)		 738
(70.55%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		 1046134
(12.81%)		 15
(1.43%)		 119
(11.38%)		 0
(0%)		 160
(15.3%)		 13
(1.24%)		 0
(0%)		 739
(70.65%)
SAT4J PB Resolution
2.2.1
(complete)		 1046752
(71.89%)		 1
(0.1%)		 751
(71.8%)		 0
(0%)		 269
(25.72%)		 25
(2.39%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 737410
(55.63%)		 1
(0.14%)		 409
(55.5%)		 0
(0%)		 267
(36.23%)		 60
(8.14%)		 0
(0%)		 0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)		 783395
(50.45%)		 1
(0.13%)		 394
(50.32%)		 0
(0%)		 296
(37.8%)		 92
(11.75%)		 0
(0%)		 0
(0%)
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)		 737168
(22.8%)		 1
(0.14%)		 167
(22.66%)		 0
(0%)		 310
(42.06%)		 259
(35.14%)		 0
(0%)		 0
(0%)
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		 737382
(51.83%)		 1
(0.14%)		 381
(51.7%)		 0
(0%)		 286
(38.81%)		 69
(9.36%)		 0
(0%)		 0
(0%)
wbo
1.4a
(complete)		 737534
(72.46%)		 1
(0.14%)		 533
(72.32%)		 0
(0%)		 25
(3.39%)		 178
(24.15%)		 0
(0%)		 0
(0%)
wbo
1.4b
(complete)		 7371
(0.14%)		 1
(0.14%)		 0
(0%)		 0
(0%)		 24
(3.26%)		 112
(15.2%)		 0
(0%)		 600
(81.41%)

Solver NameProgress
pb_cplex
2010-07-01
(complete)
done
737
MOPT
588		MSAT
2		?
147
PB/CT
0.1
(complete)
done
1046
UNSAT
2		MOPT
269		MSAT
140		?
333		ERR
302
PB/CT
0.1 fixed
(complete)
done
1046
UNSAT
1		MOPT
590		MSAT
425		?
30
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
18		MOPT
85		MSAT
431		?
30		ERR
482
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
1046
UNSAT
11		MOPT
137		MSAT
142		?
18		ERR
738
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
15		MOPT
119		MSAT
160		?
13		ERR
739
SAT4J PB Resolution
2.2.1
(complete)
done
1046
UNSAT
1		MOPT
751		MSAT
269		?
25
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
737
UNSAT
1		MOPT
409		MSAT
267		?
60
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
783
UNSAT
1		MOPT
394		MSAT
296		?
92
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
done
737
UNSAT
1		MOPT
167		MSAT
310		?
259
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
737
UNSAT
1		MOPT
381		MSAT
286		?
69
wbo
1.4a
(complete)
done
737
UNSAT
1		MOPT
533		MSAT
25		?
178
wbo
1.4b
(complete)
done
737
UNSAT
1		MSAT
24		?
112		ERR
600