PB'10 competition: WBO track: solvers page (tabular format, summary)

Some explanations

Category "both soft and hard constraints, big integers, linear constraints" (PARTIAL-BIGINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
PB/CT
0.1
(complete)
2633
(1.14%)
1
(0.38%)
2
(0.76%)
0
(0%)
23
(8.75%)
83
(31.56%)
0
(0%)
154
(58.56%)
PB/CT
0.1 fixed
(complete)
26378
(29.66%)
0
(0%)
78
(29.66%)
0
(0%)
156
(59.32%)
29
(11.03%)
0
(0%)
0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
26323
(8.75%)
18
(6.84%)
5
(1.9%)
0
(0%)
24
(9.13%)
9
(3.42%)
0
(0%)
207
(78.71%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
26317
(6.46%)
10
(3.8%)
7
(2.66%)
0
(0%)
17
(6.46%)
18
(6.84%)
0
(0%)
211
(80.23%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
26325
(9.51%)
14
(5.32%)
11
(4.18%)
0
(0%)
13
(4.94%)
11
(4.18%)
0
(0%)
214
(81.37%)
SAT4J PB Resolution
2.2.1
(complete)
263113
(42.97%)
0
(0%)
113
(42.97%)
0
(0%)
126
(47.91%)
24
(9.13%)
0
(0%)
0
(0%)

Solver NameProgress
PB/CT
0.1
(complete)
done
263
UNSAT
1
MOPT
2
MSAT
23
?
83
ERR
154
PB/CT
0.1 fixed
(complete)
done
263
MOPT
78
MSAT
156
?
29
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
263
UNSAT
18
MOPT
5
MSAT
24
?
9
ERR
207
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
263
UNSAT
10
MOPT
7
MSAT
17
?
18
ERR
211
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
263
UNSAT
14
MOPT
11
MSAT
13
?
11
ERR
214
SAT4J PB Resolution
2.2.1
(complete)
done
263
MOPT
113
MSAT
126
?
24

Category "both soft and hard constraints, small integers, linear constraints" (PARTIAL-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
pb_cplex
2010-07-01
(complete)
536428
(79.85%)
0
(0%)
428
(79.85%)
0
(0%)
2
(0.37%)
106
(19.78%)
0
(0%)
0
(0%)
PB/CT
0.1
(complete)
536209
(38.99%)
1
(0.19%)
208
(38.81%)
0
(0%)
72
(13.43%)
150
(27.99%)
0
(0%)
105
(19.59%)
PB/CT
0.1 fixed
(complete)
536375
(69.96%)
1
(0.19%)
374
(69.78%)
0
(0%)
161
(30.04%)
0
(0%)
0
(0%)
0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
53660
(11.19%)
0
(0%)
60
(11.19%)
0
(0%)
250
(46.64%)
21
(3.92%)
0
(0%)
205
(38.25%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
53694
(17.54%)
1
(0.19%)
93
(17.35%)
0
(0%)
66
(12.31%)
0
(0%)
0
(0%)
376
(70.15%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
53679
(14.74%)
1
(0.19%)
78
(14.55%)
0
(0%)
92
(17.16%)
2
(0.37%)
0
(0%)
363
(67.72%)
SAT4J PB Resolution
2.2.1
(complete)
536446
(83.21%)
1
(0.19%)
445
(83.02%)
0
(0%)
89
(16.6%)
1
(0.19%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
536297
(55.41%)
1
(0.19%)
296
(55.22%)
0
(0%)
191
(35.63%)
48
(8.96%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
536282
(52.61%)
1
(0.19%)
281
(52.43%)
0
(0%)
203
(37.87%)
51
(9.51%)
0
(0%)
0
(0%)
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
536146
(27.24%)
1
(0.19%)
145
(27.05%)
0
(0%)
190
(35.45%)
200
(37.31%)
0
(0%)
0
(0%)
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
536269
(50.19%)
1
(0.19%)
268
(50%)
0
(0%)
210
(39.18%)
57
(10.63%)
0
(0%)
0
(0%)
wbo
1.4a
(complete)
536373
(69.59%)
1
(0.19%)
372
(69.4%)
0
(0%)
25
(4.66%)
138
(25.75%)
0
(0%)
0
(0%)
wbo
1.4b
(complete)
5361
(0.19%)
1
(0.19%)
0
(0%)
0
(0%)
24
(4.48%)
76
(14.18%)
0
(0%)
435
(81.16%)

Solver NameProgress
pb_cplex
2010-07-01
(complete)
done
536
MOPT
428
MSAT
2
?
106
PB/CT
0.1
(complete)
done
536
UNSAT
1
MOPT
208
MSAT
72
?
150
ERR
105
PB/CT
0.1 fixed
(complete)
done
536
UNSAT
1
MOPT
374
MSAT
161
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
536
MOPT
60
MSAT
250
?
21
ERR
205
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
536
UNSAT
1
MOPT
93
MSAT
66
ERR
376
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
536
UNSAT
1
MOPT
78
MSAT
92
?
2
ERR
363
SAT4J PB Resolution
2.2.1
(complete)
done
536
UNSAT
1
MOPT
445
MSAT
89
?
1
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
536
UNSAT
1
MOPT
296
MSAT
191
?
48
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
536
UNSAT
1
MOPT
281
MSAT
203
?
51
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
done
536
UNSAT
1
MOPT
145
MSAT
190
?
200
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
536
UNSAT
1
MOPT
268
MSAT
210
?
57
wbo
1.4a
(complete)
done
536
UNSAT
1
MOPT
372
MSAT
25
?
138
wbo
1.4b
(complete)
done
536
UNSAT
1
MSAT
24
?
76
ERR
435

Category "only soft constraints, big integers, linear constraints" (SOFT-BIGINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
PB/CT
0.1
(complete)
460
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
46
(100%)
0
(0%)
0
(0%)
PB/CT
0.1 fixed
(complete)
460
(0%)
0
(0%)
0
(0%)
0
(0%)
45
(97.83%)
1
(2.17%)
0
(0%)
0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
461
(2.17%)
0
(0%)
1
(2.17%)
0
(0%)
25
(54.35%)
0
(0%)
0
(0%)
20
(43.48%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
464
(8.7%)
0
(0%)
4
(8.7%)
0
(0%)
17
(36.96%)
0
(0%)
0
(0%)
25
(54.35%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
467
(15.22%)
0
(0%)
7
(15.22%)
0
(0%)
11
(23.91%)
0
(0%)
0
(0%)
28
(60.87%)
SAT4J PB Resolution
2.2.1
(complete)
4633
(71.74%)
0
(0%)
33
(71.74%)
0
(0%)
13
(28.26%)
0
(0%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
460
(0%)
0
(0%)
0
(0%)
0
(0%)
5
(10.87%)
41
(89.13%)
0
(0%)
0
(0%)

Solver NameProgress
PB/CT
0.1
(complete)
done
46
?
46
PB/CT
0.1 fixed
(complete)
done
46
MSAT
45
?
1
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
46
MOPT
1
MSAT
25
ERR
20
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
46
MOPT
4
MSAT
17
ERR
25
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
46
MOPT
7
MSAT
11
ERR
28
SAT4J PB Resolution
2.2.1
(complete)
done
46
MOPT
33
MSAT
13
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
46
MSAT
5
?
41

Category "only soft constraints, small integers, linear constraints" (SOFT-SMALLINT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
pb_cplex
2010-07-01
(complete)
201160
(79.6%)
0
(0%)
160
(79.6%)
0
(0%)
0
(0%)
41
(20.4%)
0
(0%)
0
(0%)
PB/CT
0.1
(complete)
20159
(29.35%)
0
(0%)
59
(29.35%)
0
(0%)
45
(22.39%)
54
(26.87%)
0
(0%)
43
(21.39%)
PB/CT
0.1 fixed
(complete)
201138
(68.66%)
0
(0%)
138
(68.66%)
0
(0%)
63
(31.34%)
0
(0%)
0
(0%)
0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
20119
(9.45%)
0
(0%)
19
(9.45%)
0
(0%)
132
(65.67%)
0
(0%)
0
(0%)
50
(24.88%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
20133
(16.42%)
0
(0%)
33
(16.42%)
0
(0%)
42
(20.9%)
0
(0%)
0
(0%)
126
(62.69%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
20123
(11.44%)
0
(0%)
23
(11.44%)
0
(0%)
44
(21.89%)
0
(0%)
0
(0%)
134
(66.67%)
SAT4J PB Resolution
2.2.1
(complete)
201160
(79.6%)
0
(0%)
160
(79.6%)
0
(0%)
41
(20.4%)
0
(0%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
201113
(56.22%)
0
(0%)
113
(56.22%)
0
(0%)
76
(37.81%)
12
(5.97%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
201113
(56.22%)
0
(0%)
113
(56.22%)
0
(0%)
88
(43.78%)
0
(0%)
0
(0%)
0
(0%)
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
20122
(10.95%)
0
(0%)
22
(10.95%)
0
(0%)
120
(59.7%)
59
(29.35%)
0
(0%)
0
(0%)
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
201113
(56.22%)
0
(0%)
113
(56.22%)
0
(0%)
76
(37.81%)
12
(5.97%)
0
(0%)
0
(0%)
wbo
1.4a
(complete)
201161
(80.1%)
0
(0%)
161
(80.1%)
0
(0%)
0
(0%)
40
(19.9%)
0
(0%)
0
(0%)
wbo
1.4b
(complete)
2010
(0%)
0
(0%)
0
(0%)
0
(0%)
0
(0%)
36
(17.91%)
0
(0%)
165
(82.09%)

Solver NameProgress
pb_cplex
2010-07-01
(complete)
done
201
MOPT
160
?
41
PB/CT
0.1
(complete)
done
201
MOPT
59
MSAT
45
?
54
ERR
43
PB/CT
0.1 fixed
(complete)
done
201
MOPT
138
MSAT
63
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
201
MOPT
19
MSAT
132
ERR
50
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
201
MOPT
33
MSAT
42
ERR
126
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
201
MOPT
23
MSAT
44
ERR
134
SAT4J PB Resolution
2.2.1
(complete)
done
201
MOPT
160
MSAT
41
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
201
MOPT
113
MSAT
76
?
12
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
201
MOPT
113
MSAT
88
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
done
201
MOPT
22
MSAT
120
?
59
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
201
MOPT
113
MSAT
76
?
12
wbo
1.4a
(complete)
done
201
MOPT
161
?
40
wbo
1.4b
(complete)
done
201
?
36
ERR
165

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
PB/CT
0.1
(complete)
1046271
(25.91%)
2
(0.19%)
269
(25.72%)
0
(0%)
140
(13.38%)
333
(31.84%)
0
(0%)
302
(28.87%)
PB/CT
0.1 fixed
(complete)
1046591
(56.5%)
1
(0.1%)
590
(56.41%)
0
(0%)
425
(40.63%)
30
(2.87%)
0
(0%)
0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
1046103
(9.85%)
18
(1.72%)
85
(8.13%)
0
(0%)
431
(41.2%)
30
(2.87%)
0
(0%)
482
(46.08%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
1046148
(14.15%)
11
(1.05%)
137
(13.1%)
0
(0%)
142
(13.58%)
18
(1.72%)
0
(0%)
738
(70.55%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
1046134
(12.81%)
15
(1.43%)
119
(11.38%)
0
(0%)
160
(15.3%)
13
(1.24%)
0
(0%)
739
(70.65%)
SAT4J PB Resolution
2.2.1
(complete)
1046752
(71.89%)
1
(0.1%)
751
(71.8%)
0
(0%)
269
(25.72%)
25
(2.39%)
0
(0%)
0
(0%)

Solver NameProgress
PB/CT
0.1
(complete)
done
1046
UNSAT
2
MOPT
269
MSAT
140
?
333
ERR
302
PB/CT
0.1 fixed
(complete)
done
1046
UNSAT
1
MOPT
590
MSAT
425
?
30
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
18
MOPT
85
MSAT
431
?
30
ERR
482
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
1046
UNSAT
11
MOPT
137
MSAT
142
?
18
ERR
738
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
15
MOPT
119
MSAT
160
?
13
ERR
739
SAT4J PB Resolution
2.2.1
(complete)
done
1046
UNSAT
1
MOPT
751
MSAT
269
?
25

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
pb_cplex
2010-07-01
(complete)
737588
(79.78%)
0
(0%)
588
(79.78%)
0
(0%)
2
(0.27%)
147
(19.95%)
0
(0%)
0
(0%)
PB/CT
0.1
(complete)
1046271
(25.91%)
2
(0.19%)
269
(25.72%)
0
(0%)
140
(13.38%)
333
(31.84%)
0
(0%)
302
(28.87%)
PB/CT
0.1 fixed
(complete)
1046591
(56.5%)
1
(0.1%)
590
(56.41%)
0
(0%)
425
(40.63%)
30
(2.87%)
0
(0%)
0
(0%)
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
1046103
(9.85%)
18
(1.72%)
85
(8.13%)
0
(0%)
431
(41.2%)
30
(2.87%)
0
(0%)
482
(46.08%)
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
1046148
(14.15%)
11
(1.05%)
137
(13.1%)
0
(0%)
142
(13.58%)
18
(1.72%)
0
(0%)
738
(70.55%)
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
1046134
(12.81%)
15
(1.43%)
119
(11.38%)
0
(0%)
160
(15.3%)
13
(1.24%)
0
(0%)
739
(70.65%)
SAT4J PB Resolution
2.2.1
(complete)
1046752
(71.89%)
1
(0.1%)
751
(71.8%)
0
(0%)
269
(25.72%)
25
(2.39%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
737410
(55.63%)
1
(0.14%)
409
(55.5%)
0
(0%)
267
(36.23%)
60
(8.14%)
0
(0%)
0
(0%)
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
783395
(50.45%)
1
(0.13%)
394
(50.32%)
0
(0%)
296
(37.8%)
92
(11.75%)
0
(0%)
0
(0%)
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
737168
(22.8%)
1
(0.14%)
167
(22.66%)
0
(0%)
310
(42.06%)
259
(35.14%)
0
(0%)
0
(0%)
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
737382
(51.83%)
1
(0.14%)
381
(51.7%)
0
(0%)
286
(38.81%)
69
(9.36%)
0
(0%)
0
(0%)
wbo
1.4a
(complete)
737534
(72.46%)
1
(0.14%)
533
(72.32%)
0
(0%)
25
(3.39%)
178
(24.15%)
0
(0%)
0
(0%)
wbo
1.4b
(complete)
7371
(0.14%)
1
(0.14%)
0
(0%)
0
(0%)
24
(3.26%)
112
(15.2%)
0
(0%)
600
(81.41%)

Solver NameProgress
pb_cplex
2010-07-01
(complete)
done
737
MOPT
588
MSAT
2
?
147
PB/CT
0.1
(complete)
done
1046
UNSAT
2
MOPT
269
MSAT
140
?
333
ERR
302
PB/CT
0.1 fixed
(complete)
done
1046
UNSAT
1
MOPT
590
MSAT
425
?
30
SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
18
MOPT
85
MSAT
431
?
30
ERR
482
SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)
done
1046
UNSAT
11
MOPT
137
MSAT
142
?
18
ERR
738
SAT4J PB Resolution
2.2.0 2010-05-26
(complete)
done
1046
UNSAT
15
MOPT
119
MSAT
160
?
13
ERR
739
SAT4J PB Resolution
2.2.1
(complete)
done
1046
UNSAT
1
MOPT
751
MSAT
269
?
25
SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
737
UNSAT
1
MOPT
409
MSAT
267
?
60
SCIPclp
SCIP 1.2.1.3 with Clp 1.11.1 (Release Version) as LP solver
(complete)
done
783
UNSAT
1
MOPT
394
MSAT
296
?
92
SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)
done
737
UNSAT
1
MOPT
167
MSAT
310
?
259
SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)
done
737
UNSAT
1
MOPT
381
MSAT
286
?
69
wbo
1.4a
(complete)
done
737
UNSAT
1
MOPT
533
MSAT
25
?
178
wbo
1.4b
(complete)
done
737
UNSAT
1
MSAT
24
?
112
ERR
600