PB'16 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Routing

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Routing

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)
cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)
minisatp 2012-10-02 git-d91742b (complete)
NaPS 1.02 (complete)
Open-WBO PB16 (complete)
Open-WBO-LSU PB16 (complete)
Sat4j PB 2.3.6 Res+CP PB16 (complete)
Sat4j PB 2.3.6 Resolution PB16 (complete)
toysat 2016-05-02 (complete)

Bench nameBest results
on this
instance		cdcl-cuttingplanes OPT binary search
2016-05-01
(complete)		cdcl-cuttingplanes OPT linear search
2016-05-01
(complete)		minisatp
2012-10-02 git-d91742b
(complete)		NaPS
1.02
(complete)		Open-WBO
PB16
(complete)		Open-WBO-LSU
PB16
(complete)		Sat4j PB 2.3.6 Res+CP
PB16
(complete)		Sat4j PB 2.3.6 Resolution
PB16
(complete)		toysat
2016-05-02
(complete)
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s3-3-3-1pb.opb 		OPT
f=36
TT=0.011
T1=0.01
 OPT
f=36
TT=0.031
T1=0.03
 OPT
f=36
TT=0.016
T1=0.01
 OPT
f=36
TT=0.062
T1=0.06
 OPT
f=36
TT=0.121
T1=0.12
 OPT
f=36
TT=0.011
T1=0.01
 OPT
f=36
TT=0.02
T1=0.02
 OPT
f=36
TT=1.115
T1=0.7
 OPT
f=36
TT=0.503
T1=0.24
 OPT
f=36
TT=0.134
T1=0.05
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s3-3-3-2pb.opb 		OPT
f=36
TT=0.008
T1=0
 OPT
f=36
TT=0.036
T1=0.03
 OPT
f=36
TT=0.018
T1=0.01
 OPT
f=36
TT=0.099
T1=0.09
 OPT
f=36
TT=0.139
T1=0.14
 OPT
f=36
TT=0.008
T1=0
 OPT
f=36
TT=0.014
T1=0.01
 OPT
f=36
TT=0.874
T1=0.72
 OPT
f=36
TT=0.603
T1=0.3
 OPT
f=36
TT=0.136
T1=0.07
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s3-3-3-3pb.opb 		OPT
f=36
TT=0.008
T1=0
 OPT
f=36
TT=0.036
T1=0.03
 OPT
f=36
TT=0.018
T1=0.01
 OPT
f=36
TT=0.1
T1=0.1
 OPT
f=36
TT=0.124
T1=0.13
 OPT
f=36
TT=0.008
T1=0
 OPT
f=36
TT=0.023
T1=0.02
 OPT
f=36
TT=1.153
T1=0.71
 OPT
f=36
TT=0.586
T1=0.29
 OPT
f=36
TT=0.108
T1=0.04
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s3-3-3-4pb.opb 		OPT
f=36
TT=0.01
T1=0
 OPT
f=36
TT=0.044
T1=0.04
 OPT
f=36
TT=0.022
T1=0.02
 OPT
f=36
TT=0.082
T1=0.08
 OPT
f=36
TT=0.142
T1=0.14
 OPT
f=36
TT=0.01
T1=0
 OPT
f=36
TT=0.018
T1=0.01
 OPT
f=36
TT=1.194
T1=0.72
 OPT
f=36
TT=0.615
T1=0.24
 OPT
f=36
TT=0.102
T1=0.04
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s3-3-3-5pb.opb 		OPT
f=34
TT=0.008
T1=0
 OPT
f=34
TT=0.041
T1=0.04
 OPT
f=34
TT=0.032
T1=0.03
 OPT
f=34
TT=0.083
T1=0.08
 OPT
f=34
TT=0.158
T1=0.15
 OPT
f=34
TT=0.008
T1=0
 OPT
f=34
TT=0.015
T1=0.01
 OPT
f=34
TT=1.723
T1=1.23
 OPT
f=34
TT=0.673
T1=0.19
 OPT
f=34
TT=0.151
T1=0.14
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-10pb.opb 		OPT
f=70
TT=0.104
T1=0.1
 OPT
f=70
TT=0.375
T1=0.37
 OPT
f=70
TT=0.104
T1=0.1
 OPT
f=70
TT=1.041
T1=1.04
 OPT
f=70
TT=1.007
T1=1
 OPT
f=70
TT=0.21
T1=0.21
 OPT
f=70
TT=0.295
T1=0.29
 OPT
f=70
TT=4.872
T1=1.33
 OPT
f=70
TT=1.584
T1=0.66
 OPT
f=70
TT=1.62
T1=1.56
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-1pb.opb 		OPT
f=62
TT=0.038
T1=0.03
 OPT
f=62
TT=15.602
T1=15.6
 OPT
f=62
TT=7.934
T1=7.93
 OPT
f=62
TT=3.467
T1=3.46
 OPT
f=62
TT=2.051
T1=2.04
 OPT
f=62
TT=0.038
T1=0.03
 OPT
f=62
TT=0.266
T1=0.26
 OPT
f=62
TT=188.007
T1=5.84
 OPT
f=62
TT=536.192
T1=7.92
 OPT
f=62
TT=7.436
T1=7.38
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-2pb.opb 		OPT
f=64
TT=0.397
T1=0.39
 OPT
f=64
TT=6.932
T1=6.93
 OPT
f=64
TT=1.935
T1=1.93
 OPT
f=64
TT=11.348
T1=11.34
 OPT
f=64
TT=3.075
T1=3.07
 OPT
f=64
TT=0.397
T1=0.39
 OPT
f=64
TT=1.428
T1=1.42
 OPT
f=64
TT=227.618
T1=6.83
 OPT
f=64
TT=82.73
T1=9.66
 OPT
f=64
TT=1.677
T1=1.62
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-3pb.opb 		OPT
f=62
TT=0.299
T1=0.29
 OPT
f=62
TT=13.44
T1=13.44
 OPT
f=62
TT=8.496
T1=8.5
 OPT
f=62
TT=4.467
T1=4.46
 OPT
f=62
TT=2.576
T1=2.57
 OPT
f=62
TT=0.299
T1=0.29
 OPT
f=62
TT=0.593
T1=0.59
 OPT
f=62
TT=155.529
T1=31.31
 OPT
f=62
TT=84.78
T1=51.64
 OPT
f=62
TT=21.491
T1=21.39
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-4pb.opb 		OPT
f=60
TT=0.184
T1=0.18
 OPT
f=60
TT=12.047
T1=12.04
 OPT
f=60
TT=15.762
T1=15.76
 OPT
f=60
TT=9.712
T1=9.7
 OPT
f=60
TT=2.403
T1=2.43
 OPT
f=60
TT=0.184
T1=0.18
 OPT
f=60
TT=0.342
T1=0.34
 OPT
f=60
TT=951.36
T1=22.82
 OPT
f=60
TT=405.95
T1=101.34
 OPT
f=60
TT=8.299
T1=8.24
Bench nameBest resultscdcl-cuttingplanes OPT binary search
2016-05-01
(complete)		cdcl-cuttingplanes OPT linear search
2016-05-01
(complete)		minisatp
2012-10-02 git-d91742b
(complete)		NaPS
1.02
(complete)		Open-WBO
PB16
(complete)		Open-WBO-LSU
PB16
(complete)		Sat4j PB 2.3.6 Res+CP
PB16
(complete)		Sat4j PB 2.3.6 Resolution
PB16
(complete)		toysat
2016-05-02
(complete)
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-5pb.opb 		OPT
f=60
TT=0.376
T1=0.37
 OPT
f=60
TT=10.853
T1=10.85
 OPT
f=60
TT=19.979
T1=19.98
 OPT
f=60
TT=6.431
T1=6.43
 OPT
f=60
TT=0.922
T1=0.91
 OPT
f=60
TT=0.376
T1=0.37
 OPT
f=60
TT=0.775
T1=0.77
 OPT
f=60
TT=258.277
T1=29.87
 OPT
f=60
TT=116.56
T1=46.04
 OPT
f=60
TT=11.049
T1=10.99
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-6pb.opb 		OPT
f=66
TT=0.229
T1=0.22
 OPT
f=66
TT=4.668
T1=4.66
 OPT
f=66
TT=1.747
T1=1.74
 OPT
f=66
TT=4.547
T1=4.54
 OPT
f=66
TT=2.534
T1=2.53
 OPT
f=66
TT=0.229
T1=0.22
 OPT
f=66
TT=0.699
T1=0.82
 OPT
f=66
TT=47.936
T1=2.8
 OPT
f=66
TT=11.573
T1=1.66
 OPT
f=66
TT=61.889
T1=1.04
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-7pb.opb 		OPT
f=64
TT=0.405
T1=0.4
 OPT
f=64
TT=1.779
T1=1.77
 OPT
f=64
TT=1.871
T1=1.87
 OPT
f=64
TT=3.663
T1=3.66
 OPT
f=64
TT=0.924
T1=0.92
 OPT
f=64
TT=0.405
T1=0.4
 OPT
f=64
TT=0.845
T1=0.84
 OPT
f=64
TT=21.091
T1=3.81
 OPT
f=64
TT=7.793
T1=2.31
 OPT
f=64
TT=26.579
T1=26.52
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-8pb.opb 		OPT
f=36
TT=0.021
T1=0.02
 OPT
f=36
TT=6.463
T1=6.46
 OPT
f=36
TT=1.937
T1=1.93
 OPT
f=36
TT=4.712
T1=4.71
 OPT
f=36
TT=0.341
T1=0.34
 OPT
f=36
TT=0.021
T1=0.02
 OPT
f=36
TT=0.134
T1=0.13
 OPT
f=36
TT=50.916
T1=10.31
 OPT
f=36
TT=107.82
T1=2.34
 OPT
f=36
TT=0.234
T1=0.22
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/routing/
normalized-s4-4-3-9pb.opb 		OPT
f=68
TT=0.193
T1=0.19
 OPT
f=68
TT=2.122
T1=2.12
 OPT
f=68
TT=1.524
T1=1.52
 OPT
f=68
TT=14.856
T1=14.85
 OPT
f=68
TT=1.913
T1=1.9
 OPT
f=68
TT=0.193
T1=0.19
 OPT
f=68
TT=0.401
T1=0.4
 OPT
f=68
TT=10.371
T1=1.32
 OPT
f=68
TT=4.592
T1=2.1
 OPT
f=68
TT=130.572
T1=1.21



Statisticscdcl-cuttingplanes OPT binary search
2016-05-01
(complete)		cdcl-cuttingplanes OPT linear search
2016-05-01
(complete)		minisatp
2012-10-02 git-d91742b
(complete)		NaPS
1.02
(complete)		Open-WBO
PB16
(complete)		Open-WBO-LSU
PB16
(complete)		Sat4j PB 2.3.6 Res+CP
PB16
(complete)		Sat4j PB 2.3.6 Resolution
PB16
(complete)		toysat
2016-05-02
(complete)
Number of times the solver is able to give the best known answer151515151515151515
Number of times the solver is the best solver from a complete solver point of view
(i.e. best known answer and best TT time)		0100140000