PB'11 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Handmade, subsubcategory Graph Problems

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Handmade, subsubcategory Graph Problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg pb-dec-11.04.03 (complete)
bsolo 3.2 (complete)
clasp 2.0-R4191 (complete)
MinisatID 2.4.8 [DEPRECATED] (complete)
MinisatID 2.4.8-gmp [DEPRECATED] (complete)
MinisatID 2.5.2 (fixed) (complete)
MinisatID 2.5.2-gmp (fixed) (complete)
Sat4j CuttingPlanes 2.3.0 (complete)
Sat4j Res//CP 2.3.0 (complete)
Sat4j Resolution 2.3.0 (complete)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx 2 2011-06-10 (fixed) (complete)
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx E_2 2011-06-10 (fixed) (complete)
wbo 1.6 (complete)

Bench nameBest results
on this
instance
borg
pb-dec-11.04.03
(complete)
bsolo
3.2
(complete)
clasp
2.0-R4191
(complete)
MinisatID
2.4.8 [DEPRECATED]
(complete)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
MinisatID
2.5.2 (fixed)
(complete)
MinisatID
2.5.2-gmp (fixed)
(complete)
Sat4j CuttingPlanes
2.3.0
(complete)
Sat4j Res//CP
2.3.0
(complete)
Sat4j Resolution
2.3.0
(complete)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
SCIP spx 2
2011-06-10 (fixed)
(complete)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
SCIP spx E_2
2011-06-10 (fixed)
(complete)
wbo
1.6
(complete)
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_1.opb.PB06.opb
SAT
TT=0.834
T1=0.83
SAT
TT=9.84
T1=10.03
SAT
TT=22.179
T1=22.17
SAT
TT=3.88
T1=3.87
SAT
TT=4.665
T1=4.66
SAT
TT=13.776
T1=13.73
SAT
TT=5.209
T1=5.18
SAT
TT=13.963
T1=13.92
SAT
TT=5.095
T1=3.38
SAT
TT=4.629
T1=3.07
SAT
TT=3.096
T1=2.28
? (TO)
TT=1800.78

?
TT=1797.73

? (TO)
TT=1800.78

?
TT=1797.74

SAT
TT=0.834
T1=0.83
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_2.opb.PB06.opb
SAT
TT=0.835
T1=0.83
SAT
TT=9.877
T1=9.96
SAT
TT=22.205
T1=22.19
SAT
TT=3.878
T1=3.87
SAT
TT=4.52
T1=4.49
SAT
TT=13.127
T1=13.08
SAT
TT=4.5
T1=4.48
SAT
TT=13.328
T1=13.28
SAT
TT=4.595
T1=2.95
SAT
TT=4.892
T1=3.07
SAT
TT=3.25
T1=2.34
SAT
TT=1734.45
T1=1733.85
SAT
TT=1701.03
T1=1700.4
? (TO)
TT=1802.09

SAT
TT=1720.32
T1=1719.69
SAT
TT=0.835
T1=0.83
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_3.opb.PB06.opb
SAT
TT=0.835
T1=0.83
SAT
TT=9.486
T1=9.58
SAT
TT=24.203
T1=24.2
SAT
TT=3.889
T1=3.88
SAT
TT=5.856
T1=5.83
SAT
TT=18.162
T1=18.11
SAT
TT=5.121
T1=5.1
SAT
TT=15.825
T1=15.78
SAT
TT=5.122
T1=3.3
SAT
TT=4.488
T1=3.11
SAT
TT=3.189
T1=2.33
SAT
TT=1712.97
T1=1712.31
SAT
TT=1676.74
T1=1676.05
SAT
TT=1717.85
T1=1717.17
SAT
TT=1694.37
T1=1693.71
SAT
TT=0.835
T1=0.83
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_8.opb.PB06.opb
SAT
TT=0.833
T1=0.83
SAT
TT=9.43
T1=9.55
SAT
TT=23.411
T1=23.41
SAT
TT=3.874
T1=3.87
SAT
TT=4.474
T1=4.45
SAT
TT=13.046
T1=13
SAT
TT=4.501
T1=4.47
SAT
TT=12.997
T1=12.95
SAT
TT=4.805
T1=3
SAT
TT=3.905
T1=3.08
SAT
TT=3.103
T1=2.29
SAT
TT=1544.48
T1=1543.89
SAT
TT=1516.37
T1=1515.74
SAT
TT=1537.21
T1=1536.55
SAT
TT=1462.61
T1=1461.97
SAT
TT=0.833
T1=0.83
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v30_e350_d15_mw10_9.opb.PB06.opb
SAT
TT=0.836
T1=0.83
SAT
TT=10.457
T1=10.49
SAT
TT=21.797
T1=21.79
SAT
TT=3.883
T1=3.88
SAT
TT=4.927
T1=4.9
SAT
TT=15.226
T1=15.18
SAT
TT=5.444
T1=5.42
SAT
TT=18.005
T1=17.96
SAT
TT=4.971
T1=3.19
SAT
TT=4.285
T1=3.08
SAT
TT=3.082
T1=2.28
SAT
TT=1473.98
T1=1473.33
SAT
TT=1581.36
T1=1580.66
SAT
TT=1481.48
T1=1481.02
SAT
TT=1477.8
T1=1477.16
SAT
TT=0.836
T1=0.83
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_0.opb.PB06.opb
SAT
TT=2.595
T1=2.6
SAT
TT=25.416
T1=24.24
SAT
TT=149.937
T1=149.92
SAT
TT=24.039
T1=24.03
SAT
TT=16.223
T1=16.15
SAT
TT=48.21
T1=48.09
SAT
TT=14.44
T1=14.38
SAT
TT=43.213
T1=43.09
SAT
TT=8.19
T1=6.24
SAT
TT=8.818
T1=6.88
SAT
TT=7.158
T1=6.2
? (TO)
TT=1802.16

? (TO)
TT=1800.14

? (TO)
TT=1802.14

?
TT=1799.89

SAT
TT=2.595
T1=2.6
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_2.opb.PB06.opb
SAT
TT=2.6
T1=2.59
SAT
TT=26.889
T1=24.96
SAT
TT=150.81
T1=150.82
SAT
TT=24.184
T1=24.17
SAT
TT=13.155
T1=13.09
SAT
TT=35.754
T1=35.64
SAT
TT=13.009
T1=12.94
SAT
TT=38.662
T1=38.54
SAT
TT=8.709
T1=6.47
SAT
TT=8.843
T1=6.97
SAT
TT=6.588
T1=5.67
? (TO)
TT=1802.14

? (TO)
TT=1800.14

? (TO)
TT=1802.15

? (TO)
TT=1802.12

SAT
TT=2.6
T1=2.59
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_5.opb.PB06.opb
SAT
TT=2.597
T1=2.58
SAT
TT=26.809
T1=23.73
SAT
TT=148.262
T1=148.27
SAT
TT=24.154
T1=24.14
SAT
TT=15.27
T1=15.2
SAT
TT=44.537
T1=44.42
SAT
TT=17.274
T1=17.21
SAT
TT=54.374
T1=54.25
SAT
TT=8.495
T1=6.44
SAT
TT=8.451
T1=6.99
SAT
TT=6.597
T1=5.64
? (TO)
TT=1802.13

? (TO)
TT=1800.17

? (TO)
TT=1802.16

?
TT=1799.24

SAT
TT=2.597
T1=2.58
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_6.opb.PB06.opb
SAT
TT=2.599
T1=2.59
SAT
TT=24.55
T1=23.01
SAT
TT=150.131
T1=150.11
SAT
TT=24.092
T1=24.08
SAT
TT=15.35
T1=15.29
SAT
TT=39.88
T1=39.8
SAT
TT=13.95
T1=13.89
SAT
TT=39.791
T1=39.67
SAT
TT=7.84
T1=6.13
SAT
TT=8.966
T1=6.91
SAT
TT=7.189
T1=6.16
? (TO)
TT=1802.17

? (TO)
TT=1800.17

? (TO)
TT=1802.17

?
TT=1799.6

SAT
TT=2.599
T1=2.59
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v40_e600_d20_mw10_9.opb.PB06.opb
SAT
TT=2.601
T1=2.59
SAT
TT=26.683
T1=25.24
SAT
TT=150.311
T1=150.3
SAT
TT=24.184
T1=24.17
SAT
TT=13.527
T1=13.46
SAT
TT=37.696
T1=37.58
SAT
TT=13.485
T1=13.42
SAT
TT=37.424
T1=37.31
SAT
TT=7.655
T1=6
SAT
TT=8.904
T1=6.93
SAT
TT=6.602
T1=5.67
? (TO)
TT=1802.16

?
TT=1799.87

? (TO)
TT=1802.16

? (TO)
TT=1800.13

SAT
TT=2.601
T1=2.59
Bench nameBest resultsborg
pb-dec-11.04.03
(complete)
bsolo
3.2
(complete)
clasp
2.0-R4191
(complete)
MinisatID
2.4.8 [DEPRECATED]
(complete)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
MinisatID
2.5.2 (fixed)
(complete)
MinisatID
2.5.2-gmp (fixed)
(complete)
Sat4j CuttingPlanes
2.3.0
(complete)
Sat4j Res//CP
2.3.0
(complete)
Sat4j Resolution
2.3.0
(complete)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
SCIP spx 2
2011-06-10 (fixed)
(complete)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
SCIP spx E_2
2011-06-10 (fixed)
(complete)
wbo
1.6
(complete)
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_1.opb.PB06.opb
SAT
TT=6.798
T1=6.78
SAT
TT=303.488
T1=296.73
SAT
TT=186.155
T1=186.12
SAT
TT=106.777
T1=106.77
SAT
TT=36.069
T1=35.94
SAT
TT=99.088
T1=98.83
SAT
TT=42.027
T1=41.86
SAT
TT=107.364
T1=107.06
SAT
TT=15.15
T1=12.45
SAT
TT=18.261
T1=14.18
SAT
TT=13.305
T1=11.79
? (TO)
TT=1802.26

? (TO)
TT=1800.32

? (TO)
TT=1802.3

? (TO)
TT=1802.27

SAT
TT=6.798
T1=6.78
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_3.opb.PB06.opb
SAT
TT=6.779
T1=6.78
SAT
TT=303.472
T1=303.59
SAT
TT=186.633
T1=186.6
SAT
TT=106.999
T1=106.98
SAT
TT=40.842
T1=40.67
SAT
TT=104.484
T1=104.24
SAT
TT=36.888
T1=36.76
SAT
TT=100.414
T1=100.16
SAT
TT=15.182
T1=12.48
SAT
TT=18.864
T1=14.61
SAT
TT=14.375
T1=12.8
? (TO)
TT=1802.28

? (TO)
TT=1800.27

? (TO)
TT=1802.28

? (TO)
TT=1802.24

SAT
TT=6.779
T1=6.78
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_5.opb.PB06.opb
SAT
TT=6.787
T1=6.76
SAT
TT=302.751
T1=302.3
SAT
TT=186.225
T1=186.19
SAT
TT=106.402
T1=106.38
SAT
TT=36.887
T1=36.74
SAT
TT=101.219
T1=100.96
SAT
TT=40.205
T1=40.06
SAT
TT=106.269
T1=105.94
SAT
TT=17.104
T1=14.23
SAT
TT=18.717
T1=14.39
SAT
TT=13.91
T1=12.53
? (TO)
TT=1802.32

? (TO)
TT=1800.27

? (TO)
TT=1802.26

? (TO)
TT=1800.24

SAT
TT=6.787
T1=6.76
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_6.opb.PB06.opb
SAT
TT=7.326
T1=7.32
SAT
TT=277.685
T1=277.38
SAT
TT=186.184
T1=186.28
SAT
TT=114.404
T1=114.4
SAT
TT=36.367
T1=36.2
SAT
TT=99.391
T1=99.11
SAT
TT=36.453
T1=36.31
SAT
TT=110.3
T1=109.94
SAT
TT=15.358
T1=12.69
SAT
TT=18.318
T1=14.37
SAT
TT=13.327
T1=11.88
? (TO)
TT=1802.31

? (TO)
TT=1800.28

? (TO)
TT=1802.3

? (TO)
TT=1802.26

SAT
TT=7.326
T1=7.32
normalized-PB06/SATUNSAT-SMALLINT/
submitted-PB06/liu/dbst/
normalized-dbst_v50_e1000_d25_mw10_9.opb.PB06.opb
SAT
TT=6.791
T1=6.77
SAT
TT=302.857
T1=303.15
SAT
TT=186.142
T1=186.13
SAT
TT=107.762
T1=107.74
SAT
TT=35.133
T1=34.98
SAT
TT=95.511
T1=95.23
SAT
TT=34.942
T1=34.81
SAT
TT=94.039
T1=93.79
SAT
TT=14.315
T1=11.91
SAT
TT=18.605
T1=14.41
SAT
TT=13.798
T1=12.39
? (TO)
TT=1802.31

? (TO)
TT=1800.24

? (TO)
TT=1802.29

? (TO)
TT=1802.26

SAT
TT=6.791
T1=6.77



Statisticsborg
pb-dec-11.04.03
(complete)
bsolo
3.2
(complete)
clasp
2.0-R4191
(complete)
MinisatID
2.4.8 [DEPRECATED]
(complete)
MinisatID
2.4.8-gmp [DEPRECATED]
(complete)
MinisatID
2.5.2 (fixed)
(complete)
MinisatID
2.5.2-gmp (fixed)
(complete)
Sat4j CuttingPlanes
2.3.0
(complete)
Sat4j Res//CP
2.3.0
(complete)
Sat4j Resolution
2.3.0
(complete)
SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
SCIP spx 2
2011-06-10 (fixed)
(complete)
SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)
SCIP spx E_2
2011-06-10 (fixed)
(complete)
wbo
1.6
(complete)
Number of times the solver is able to give the best known answer151515001515151515443415
Number of times the solver is the best solver from a complete solver point of view
(i.e. best known answer and best TT time)
0000000000000015