PB'06 evaluation: results by benchmark

Results by benchmark for category optimisation, medium integers (OPT-MEDINT)

This page displays the results of the different solvers for each benchmark for category optimisation, medium integers (OPT-MEDINT)

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
absconPseudo 1
bsolo 2006/05
glpPB 0.2
minisat+ 1.14
PBS 4.1L
Pueblo 1.3
Pueblo 1.4
SAT4JPSEUDO 2006.2
SAT4JPSEUDO 2006.2 Heuristics

Bench nameBest results
on this
instance		absconPseudo
1		bsolo
2006/05		glpPB
0.2		minisat+
1.14		PBS
4.1L		Pueblo
1.3		Pueblo
1.4		SAT4JPSEUDO
2006.2		SAT4JPSEUDO
2006.2 Heuristics
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/ftp.netlib.org/lp/data/
normalized-mps-v2-20-10-fit1d.opb		 SAT
f=-251
TT=1800.89
T1=168.33
 ? (TO)

TT=1800.11

 ?

TT=1800.59

 ? (TO)

TT=1800.65

 ? (TO)

TT=1800.77

 ? (TO)

TT=1800.61

 SAT
f=-29
TT=1783.82
T1=1784.99
 SAT
f=-29
TT=1782.91
T1=1783.46
 SAT (TO)
f=-251
TT=1800.89
T1=168.33
 ? (TO)

TT=1800.74
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-aflow30a.opb		 SAT
f=2754
TT=1782.75
T1=179.72
 ? (TO)

TT=1800.85

 ?

TT=1800.03

 ? (TO)

TT=1800.52

 ? (TO)

TT=1800.58

 ? (TO)

TT=1800.62

 SAT
f=2856
TT=1782.76
T1=1314.9
 SAT
f=2754
TT=1782.75
T1=179.72
 ? (TO)

TT=1800.85

 ? (TO)

TT=1800.68
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-aflow40b.opb		 SAT
f=5507
TT=1782.8
T1=1058.58
 ? (TO)

TT=1800.59

 ?

TT=1800.03

 ? (TO)

TT=1800.47

 ? (TO)

TT=1800.72

 ? (TO)

TT=1800.51

 SAT
f=5507
TT=1782.81
T1=1058.58
 SAT
f=5507
TT=1782.8
T1=1066.77
 ? (TO)

TT=1800.6

 ? (TO)

TT=1801.11
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-air04.opb



? (TO)

TT=1800.9

 ?

TT=1800.03

 ? (TO)

TT=1800.43

 ? (TO)

TT=1800.57

 ? (TO)

TT=1800.64

 ? (TO)

TT=1800.63

 ? (TO)

TT=1800.61

 ? (TO)

TT=1800.23

 ? (TO)

TT=1800.23
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-air05.opb		 SAT
f=37422
TT=1800.03
T1=1110.49
 ? (TO)

TT=1800.84

 SAT
f=37422
TT=1800.03
T1=1110.49
 ? (TO)

TT=1800.61

 SAT (TO)
f=41670
TT=1800.83
T1=467.58
 ? (TO)

TT=1800.67

 SAT
f=39907
TT=1782.74
T1=1294.73
 SAT
f=39486
TT=1782.75
T1=493.47
 SAT (TO)
f=43735
TT=1801.12
T1=766.92
 SAT (TO)
f=44222
TT=1800.74
T1=1574.96
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-nw04.opb		 OPT
f=8431
TT=351.901
T1=351.99
 ? (TO)

TT=1800.85

 ? (TO)

TT=1800.69

 OPT
f=8431
TT=351.901
T1=351.99
 ? (TO)

TT=1800.6

 ? (TO)

TT=1800.52

 SAT
f=30407
TT=1783.6
T1=40.71
 SAT
f=30407
TT=1783.25
T1=37.84
 ? (TO)

TT=1800.86

 ? (TO)

TT=1801.1
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-t1717.opb		 SAT
f=201342
TT=1782.77
T1=16.21
 ? (TO)

TT=1800.36

 SAT (TO)
f=359991
TT=1800.52
T1=1311.85
 ? (TO)

TT=1800.59

 SAT (TO)
f=372547
TT=1800.6
T1=462.96
 ? (TO)

TT=1800.5

 SAT
f=201342
TT=1782.77
T1=16.38
 SAT
f=201342
TT=1783.33
T1=16.21
 ? (TO)

TT=1802.83

 SAT (TO)
f=404993
TT=1805.08
T1=166.38
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib2003/
normalized-mps-v2-20-10-vpm2.opb



? (TO)

TT=1801.01

 ?

TT=1800.15

 ? (TO)

TT=1800.61

 ? (TO)

TT=1800.66

 ? (TO)

TT=1800.62

 ? (TO)

TT=1800.51

 ? (TO)

TT=1800.51

 ? (TO)

TT=1801.14

 ? (TO)

TT=1800.37
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib3/
normalized-mps-v2-20-10-gt2.opb		 SAT
f=21166
TT=1782.73
T1=627.36
 ? (TO)

TT=1800.29

 SAT
f=44712
TT=1800.05
T1=1437.11
 ? (TO)

TT=1800.75

 SAT (TO)
f=64860
TT=1800.59
T1=1607.03
 ? (TO)

TT=1800.52

 SAT
f=21166
TT=1782.73
T1=627.36
 SAT
f=34976
TT=1782.73
T1=1387.29
 SAT (TO)
f=47803
TT=1800.95
T1=1339.27
 SAT (TO)
f=24647
TT=1800.66
T1=1695.79
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-air01.opb		 OPT
f=3398
TT=0.043
T1=0.04
 OPT
f=3398
TT=22.044
T1=2.14
 OPT
f=3398
TT=0.077
T1=0.41
 OPT
f=3398
TT=0.043
T1=0.04
 SAT (TO)
f=3440
TT=1800.51
T1=1066.95
 OPT
f=3398
TT=19.221
T1=13.58
 OPT
f=3398
TT=54.634
T1=37.98
 OPT
f=3398
TT=10.646
T1=8.47
 OPT
f=3398
TT=538.235
T1=357.09
 OPT
f=3398
TT=34.559
T1=3.01
Bench nameBest results
on this
instance		absconPseudo
1		bsolo
2006/05		glpPB
0.2		minisat+
1.14		PBS
4.1L		Pueblo
1.3		Pueblo
1.4		SAT4JPSEUDO
2006.2		SAT4JPSEUDO
2006.2 Heuristics
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-air02.opb		 OPT
f=3905
TT=0.822
T1=0.82
 SAT (TO)
f=4492
TT=1801.11
T1=377.22
 OPT
f=3905
TT=84.487
T1=84.01
 OPT
f=3905
TT=0.822
T1=0.82
 SAT (TO)
f=34793
TT=1800.71
T1=702.32
 ? (TO)

TT=1800.52

 SAT
f=5830
TT=1782.77
T1=949.92
 SAT
f=4792
TT=1782.75
T1=1728.85
 SAT (TO)
f=4489
TT=1800.4
T1=751.7
 SAT (TO)
f=4492
TT=1800.98
T1=265.36
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-air03.opb		 OPT
f=170080
TT=1.719
T1=1.72
 ? (TO)

TT=1800.08

 OPT
f=170080
TT=323.724
T1=323.81
 OPT
f=170080
TT=1.719
T1=1.72
 SAT (TO)
f=300125
TT=1800.73
T1=1662.47
 ? (TO)

TT=1800.57

 SAT
f=225693
TT=1783.13
T1=1361.48
 SAT
f=186908
TT=1782.78
T1=1719.56
 SAT (TO)
f=246113
TT=1800.66
T1=206.98
 SAT (TO)
f=200394
TT=1801
T1=901.88
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-air06.opb		 OPT
f=49649
TT=43.281
T1=43.29
 ? (TO)

TT=1800.92

 ?

TT=1800.03

 OPT
f=49649
TT=43.281
T1=43.29
 SAT (TO)
f=66761
TT=1800.77
T1=1417.34
 ? (TO)

TT=1800.57

 ? (TO)

TT=1800.67

 SAT
f=64955
TT=1782.75
T1=1443.63
 ? (TO)

TT=1800.77

 ? (TO)

TT=1800.43
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-enigma.opb		 OPT
f=0
TT=0.174
T1=0.17
 OPT
f=0
TT=1.479
T1=1.06
 OPT
f=0
TT=40.185
T1=40.18
 OPT
f=0
TT=11.221
T1=11.22
 OPT
f=0
TT=47.793
T1=47.8
 OPT
f=0
TT=0.174
T1=0.17
 OPT
f=0
TT=23.746
T1=23.75
 OPT
f=0
TT=6.29
T1=6.29
 ? (TO)

TT=1801.08

 OPT
f=0
TT=73.199
T1=71.37
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-p0282.opb		 OPT
f=258411
TT=1.17
T1=1.17
 OPT
f=258411
TT=9.843
T1=9.16
 SAT
f=321925
TT=1800.08
T1=0.5
 OPT
f=258411
TT=1.17
T1=1.17
 OPT
f=258411
TT=1101.48
T1=1101.84
 OPT
f=258411
TT=20.976
T1=20.98
 SAT
f=280704
TT=1782.74
T1=435.6
 SAT
f=258411
TT=1782.74
T1=1541.17
 SAT (TO)
f=282007
TT=1800.45
T1=916.99
 SAT (TO)
f=267459
TT=1801.16
T1=1339.02
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-p0291.opb		 OPT
f=10447498
TT=0.057
T1=0.05
 OPT
f=10447498
TT=1.628
T1=1
 OPT
f=10447498
TT=23.99
T1=21.16
 OPT
f=10447498
TT=0.159
T1=0.16
 SAT (TO)
f=29395365
TT=1800.65
T1=1512.19
 OPT
f=10447498
TT=0.577
T1=0.57
 OPT
f=10447498
TT=0.057
T1=0.05
 OPT
f=10447498
TT=0.061
T1=0.06
 OPT
f=10447498
TT=14.816
T1=13.07
 OPT
f=10447498
TT=8.518
T1=6.71
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-p6000.opb		 SAT
f=-950703
TT=1800.94
T1=677.5
 ? (TO)

TT=1800.36

 SAT
f=-432546
TT=1800.04
T1=969.36
 ? (TO)

TT=1800.45

 ? (TO)

TT=1800.6

 ? (TO)

TT=1800.61

 SAT
f=-103424
TT=1782.78
T1=1783.37
 SAT
f=-103424
TT=1782.79
T1=1783.42
 ? (TO)

TT=1802.57

 SAT (TO)
f=-950703
TT=1800.94
T1=677.5
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-pipex.opb		 OPT
f=788263
TT=0.186
T1=0.18
 OPT
f=788263
TT=5.903
T1=3.57
 OPT
f=788263
TT=9.41
T1=9.29
 OPT
f=788263
TT=0.186
T1=0.18
 OPT
f=788263
TT=206.105
T1=152.3
 OPT
f=788263
TT=165.968
T1=85
 OPT
f=788263
TT=9.407
T1=8.43
 OPT
f=788263
TT=15.733
T1=13.77
 OPT
f=788263
TT=341.904
T1=170.69
 OPT
f=788263
TT=176.543
T1=94.44
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/MIPLIB/miplib/
normalized-mps-v2-20-10-vpm1.opb		 SAT
f=38
TT=1782.73
T1=1470.47
 ? (TO)

TT=1801.04

 Signal

TT=1065.07

 ? (TO)

TT=1800.46

 ? (TO)

TT=1800.73

 ? (TO)

TT=1800.71

 SAT
f=38
TT=1782.73
T1=1470.47
 SAT
f=53
TT=1782.73
T1=819.02
 ? (TO)

TT=1800.39

 ? (TO)

TT=1800.99
normalized-PB06/OPT-MEDINT/
mps-v2-20-10/plato.asu.edu/pub/milp/
normalized-mps-v2-20-10-neos9.opb



?

TT=30.285

 ?

TT=1800.24

 ? (TO)

TT=1800.7

 Signal

TT=540.954

 ? (TO)

TT=1800.63

 Signal

TT=1726.81

 Signal

TT=1707.73

 ? (TO)

TT=1800.89

 ? (TO)

TT=1800.95
Bench nameBest results
on this
instance		absconPseudo
1		bsolo
2006/05		glpPB
0.2		minisat+
1.14		PBS
4.1L		Pueblo
1.3		Pueblo
1.4		SAT4JPSEUDO
2006.2		SAT4JPSEUDO
2006.2 Heuristics
normalized-PB06/OPT-MEDINT/mps-v2-20-10/
plato.asu.edu/pub/milp/
normalized-mps-v2-20-10-seymour1.opb		 SAT
f=456321
TT=1800.6
T1=4.53
 SAT (TO)
f=456321
TT=1800.6
T1=4.53
 SAT
f=498929
TT=1800.19
T1=356.52
 ? (TO)

TT=1800.59

 SAT (TO)
f=1008256
TT=1800.61
T1=265.94
 ? (TO)

TT=1800.53

 SAT
f=521216
TT=1782.76
T1=1.64
 SAT
f=521216
TT=1782.77
T1=1.67
 SAT (TO)
f=495616
TT=1801.32
T1=139.38
 SAT (TO)
f=471080
TT=1800.61
T1=28.15
normalized-PB06/OPT-MEDINT/reduced/mps-v2-20-10/MIPLIB/
miplib2003/normalized-reduced-mps-v2-20-10-ds.opb



?

TT=115.726

 ? (TO)

TT=1800.67

 ? (TO)

TT=1800.83

 Signal

TT=411.702

 ? (TO)

TT=1800.72

 ? (TO)

TT=1800.7

 ? (TO)

TT=1800.93

 ?

TT=376.33

 ?

TT=363.936
normalized-PB06/OPT-MEDINT/reduced/mps-v2-20-10/
plato.asu.edu/pub/lptestset/fome/
normalized-reduced-mps-v2-20-10-fome12.opb



?

TT=39.77

 ? (TO)

TT=1800.34

 ? (TO)

TT=1800.72

 Signal

TT=1578.86

 ? (TO)

TT=1800.59

 ? (TO)

TT=1800.81

 ? (TO)

TT=1800.73

 ? (TO)

TT=1800.97

 ? (TO)

TT=1800.77
normalized-PB06/OPT-MEDINT/reduced/mps-v2-20-10/
plato.asu.edu/pub/lptestset/fome/
normalized-reduced-mps-v2-20-10-fome13.opb



?

TT=70.809

 ? (TO)

TT=1800.6

 ? (TO)

TT=1800.91

 Signal

TT=1037.72

 ? (TO)

TT=1800.67

 ? (TO)

TT=1800.88

 ? (TO)

TT=1800.86

 ? (TO)

TT=1800.37

 ? (TO)

TT=1800.36
normalized-PB06/OPT-MEDINT/web/
www.ps.uni-sb.de/~walser/benchmarks/
course-ass/normalized-ws97-3.opb		 SAT
f=3052
TT=1800.73
T1=1081.07
 ? (TO)

TT=1800.87

 SAT
f=69117
TT=1800.14
T1=1768.85
 ? (TO)

TT=1800.63

 ? (TO)

TT=1800.7

 ? (TO)

TT=1800.75

 SAT
f=207320
TT=1782.75
T1=1783.25
 SAT
f=284446
TT=1782.75
T1=1783.27
 SAT (TO)
f=22070
TT=1800.71
T1=234.73
 SAT (TO)
f=3052
TT=1800.73
T1=1081.07

Some statistics...

absconPseudo
1		bsolo
2006/05		glpPB
0.2		minisat+
1.14		PBS
4.1L		Pueblo
1.3		Pueblo
1.4		SAT4JPSEUDO
2006.2		SAT4JPSEUDO
2006.2 Heuristics
Number of times the solver is able to give the best known answer679358746
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)679358846
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)117014212
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)117014212