PB'09 competition: results by benchmark

Results by benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Handmade, subsubcategory Graph Problems

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Handmade, subsubcategory Graph Problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
BoolVar 2009-04-26 (complete)
bsolo 3.1 (complete)
bsolo 3.1 cl (complete)
bsolo 3.1 pb (complete)
pbclasp 2009-04-24 (complete)
SAT4J Pseudo CP 2.1.1 (complete)
SAT4J Pseudo Resolution 2.1.1 (complete)
SCIPclp SCIP 1.1.0.7 with CLP 1.8.2 (complete)
SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)
wbo 1.0 (complete)

Bench nameBest results
on this
instance
BoolVar
2009-04-26
(complete)
bsolo
3.1
(complete)
bsolo
3.1 cl
(complete)
bsolo
3.1 pb
(complete)
pbclasp
2009-04-24
(complete)
SAT4J Pseudo CP
2.1.1
(complete)
SAT4J Pseudo Resolution
2.1.1
(complete)
SCIPclp
SCIP 1.1.0.7 with CLP 1.8.2
(complete)
SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
wbo
1.0
(complete)
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_1.opb.PB06.opb
SAT
TT=1.445
T1=1.44
? (MO)
TT=180.669

SAT
TT=58.048
T1=58.06
SAT
TT=57.972
T1=57.99
SAT
TT=57.909
T1=57.91
SAT
TT=4.989
T1=3.9
SAT
TT=13.214
T1=9.42
SAT
TT=8.831
T1=6.61
? (TO)
TT=1800.21

?
TT=1799.3

SAT
TT=1.445
T1=1.44
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_2.opb.PB06.opb
SAT
TT=1.435
T1=1.43
? (MO)
TT=180.738

SAT
TT=58.115
T1=58.12
SAT
TT=58.043
T1=58.04
SAT
TT=57.995
T1=58
SAT
TT=4.535
T1=3.44
SAT
TT=10.965
T1=8.04
SAT
TT=8.576
T1=6.52
?
TT=1799.56

?
TT=1799.34

SAT
TT=1.435
T1=1.43
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_3.opb.PB06.opb
SAT
TT=1.446
T1=1.44
? (MO)
TT=183.751

SAT
TT=57.945
T1=57.96
SAT
TT=58.05
T1=58.06
SAT
TT=57.917
T1=57.94
SAT
TT=4.918
T1=3.81
SAT
TT=13.072
T1=9.14
SAT
TT=9.189
T1=6.82
? (TO)
TT=1800.2

? (TO)
TT=1802.16

SAT
TT=1.446
T1=1.44
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_8.opb.PB06.opb
SAT
TT=1.547
T1=1.54
? (MO)
TT=185.282

SAT
TT=55.624
T1=55.63
SAT
TT=57.406
T1=57.41
SAT
TT=57.389
T1=57.4
SAT
TT=4.94
T1=3.84
SAT
TT=12.161
T1=8.45
SAT
TT=8.322
T1=6.47
? (TO)
TT=1800.07

?
TT=1799.19

SAT
TT=1.547
T1=1.54
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_9.opb.PB06.opb
SAT
TT=1.538
T1=1.53
? (MO)
TT=187.102

SAT
TT=57.178
T1=57.18
SAT
TT=57.425
T1=57.44
SAT
TT=57.459
T1=57.48
SAT
TT=4.886
T1=3.78
SAT
TT=12.458
T1=8.65
SAT
TT=8.137
T1=6.32
?
TT=1799.43

? (TO)
TT=1802.18

SAT
TT=1.538
T1=1.53
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_0.opb.PB06.opb
SAT
TT=4.518
T1=4.5
? (exit code)
TT=422.596

SAT
TT=184.594
T1=184.63
SAT
TT=184.594
T1=184.61
SAT
TT=184.588
T1=184.63
SAT
TT=15.334
T1=11.41
SAT
TT=19.916
T1=15.81
SAT
TT=16.833
T1=13.88
? (MO)
TT=47.884

? (MO)
TT=36.081

SAT
TT=4.518
T1=4.5
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_2.opb.PB06.opb
SAT
TT=4.551
T1=4.54
? (exit code)
TT=455.895

SAT
TT=184.943
T1=184.95
SAT
TT=184.934
T1=184.96
SAT
TT=184.935
T1=184.94
SAT
TT=15.3
T1=11.46
SAT
TT=20.68
T1=16.44
SAT
TT=17.63
T1=14.65
? (MO)
TT=47.639

? (MO)
TT=37.994

SAT
TT=4.551
T1=4.54
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_5.opb.PB06.opb
SAT
TT=4.871
T1=4.86
? (exit code)
TT=480.999

SAT
TT=184.877
T1=184.89
SAT
TT=184.932
T1=184.98
SAT
TT=184.937
T1=184.9
SAT
TT=15.311
T1=11.47
SAT
TT=20.185
T1=15.88
SAT
TT=17.001
T1=13.96
? (MO)
TT=47.323

? (MO)
TT=38.302

SAT
TT=4.871
T1=4.86
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_6.opb.PB06.opb
SAT
TT=4.843
T1=4.83
? (exit code)
TT=485.917

SAT
TT=184.938
T1=184.98
SAT
TT=184.933
T1=184.9
SAT
TT=184.951
T1=184.99
SAT
TT=15.749
T1=11.86
SAT
TT=19.998
T1=15.95
SAT
TT=17.181
T1=14.22
? (MO)
TT=48.249

? (MO)
TT=38.002

SAT
TT=4.843
T1=4.83
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_9.opb.PB06.opb
SAT
TT=4.547
T1=4.53
? (exit code)
TT=455.076

SAT
TT=184.601
T1=184.64
SAT
TT=184.565
T1=184.61
SAT
TT=184.959
T1=184.99
SAT
TT=15.326
T1=11.48
SAT
TT=19.889
T1=15.75
SAT
TT=17.695
T1=14.62
? (MO)
TT=48.219

? (MO)
TT=38.047

SAT
TT=4.547
T1=4.53
Bench nameBest resultsBoolVar
2009-04-26
(complete)
bsolo
3.1
(complete)
bsolo
3.1 cl
(complete)
bsolo
3.1 pb
(complete)
pbclasp
2009-04-24
(complete)
SAT4J Pseudo CP
2.1.1
(complete)
SAT4J Pseudo Resolution
2.1.1
(complete)
SCIPclp
SCIP 1.1.0.7 with CLP 1.8.2
(complete)
SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
wbo
1.0
(complete)
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_1.opb.PB06.opb
SAT
TT=11.962
T1=11.94
? (exit code)
TT=442.218

SAT
TT=194.573
T1=194.53
SAT
TT=194.603
T1=194.57
SAT
TT=194.575
T1=194.54
SAT
TT=42.462
T1=31.35
SAT
TT=39.412
T1=33.63
SAT
TT=36.854
T1=31.87
? (MO)
TT=40.294

? (MO)
TT=39.842

SAT
TT=11.962
T1=11.94
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_3.opb.PB06.opb
SAT
TT=11.933
T1=11.91
? (exit code)
TT=539.591

SAT
TT=194.475
T1=194.44
SAT
TT=194.56
T1=194.53
SAT
TT=194.527
T1=194.49
SAT
TT=41.738
T1=30.39
SAT
TT=40.742
T1=35.02
SAT
TT=36.446
T1=31.64
? (MO)
TT=42.734

? (MO)
TT=42.793

SAT
TT=11.933
T1=11.91
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_5.opb.PB06.opb
SAT
TT=11.922
T1=11.9
? (exit code)
TT=480.98

SAT
TT=193.604
T1=193.6
SAT
TT=194.617
T1=194.6
SAT
TT=193.688
T1=193.69
SAT
TT=41.939
T1=30.79
SAT
TT=43.672
T1=37.46
SAT
TT=38.903
T1=34.2
? (MO)
TT=42.546

? (MO)
TT=42.918

SAT
TT=11.922
T1=11.9
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_6.opb.PB06.opb
SAT
TT=11.94
T1=11.91
? (exit code)
TT=421.855

SAT
TT=193.608
T1=193.6
SAT
TT=194.613
T1=194.6
SAT
TT=194.544
T1=194.52
SAT
TT=42.144
T1=30.69
SAT
TT=41.975
T1=36.25
SAT
TT=36.681
T1=31.83
? (MO)
TT=42.884

? (MO)
TT=41.006

SAT
TT=11.94
T1=11.91
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_9.opb.PB06.opb
SAT
TT=12.768
T1=12.74
? (exit code)
TT=451.186

SAT
TT=194.601
T1=194.5
SAT
TT=194.609
T1=194.57
SAT
TT=194.603
T1=194.59
SAT
TT=42.253
T1=30.89
SAT
TT=37.749
T1=32.71
SAT
TT=38.517
T1=33.83
? (MO)
TT=43.057

? (MO)
TT=42.806

SAT
TT=12.768
T1=12.74



StatisticsBoolVar
2009-04-26
(complete)
bsolo
3.1
(complete)
bsolo
3.1 cl
(complete)
bsolo
3.1 pb
(complete)
pbclasp
2009-04-24
(complete)
SAT4J Pseudo CP
2.1.1
(complete)
SAT4J Pseudo Resolution
2.1.1
(complete)
SCIPclp
SCIP 1.1.0.7 with CLP 1.8.2
(complete)
SCIPspx
SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
(complete)
wbo
1.0
(complete)
Number of times the solver is able to give the best known answer01515151515150015
Number of times the solver is the best solver from a complete solver point of view
(i.e. best known answer and best TT time)
00000000015