PB'10 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Queens Problems

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Queens Problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
bsolo 3.2 Card (complete)
bsolo 3.2 Cl (complete)
pb_cplex 2010-06-29 (complete)
PB/CT 0.1 (complete)
PB/CT 0.1 fixed (complete)
PBPASSolver 2010-06-13 (complete)
SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)
SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)
SAT4J PB Resolution 2.2.0 2010-05-26 (complete)
SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)
SCIPnone SCIP 1.2.1.2 without any LP solver (complete)
SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
wbo 1.4b (complete)
wbo 1.4b (fixed) (complete)

Bench nameBest results
on this
instance		bsolo
3.2 Card
(complete)		bsolo
3.2 Cl
(complete)		pb_cplex
2010-06-29
(complete)		PB/CT
0.1
(complete)		PB/CT
0.1 fixed
(complete)		PBPASSolver
2010-06-13
(complete)		SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)		SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)		SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		wbo
1.4b
(complete)		wbo
1.4b (fixed)
(complete)
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_0.opb.PB06.opb 		SAT
f=1297
TT=1800.67
T1=144.7
 SAT
f=5661
TT=1798.1
T1=1546.71
 SAT
f=5483
TT=1798.13
T1=233.16
 ? (TO)

TT=1800.03

 SAT (MO)
f=5592
TT=471.022
T1=371.69
 SAT (MO)
f=5592
TT=507.333
T1=401.85
 ? (MO)

TT=13.26

 ? (TO)

TT=1800.47

 SAT (TO)
f=1297
TT=1800.67
T1=144.7
 SAT (TO)
f=1303
TT=1800.48
T1=8.93
 ? (TO)

TT=1802.24

 SAT (TO)
f=8907
TT=1802.16
T1=1384.18
 ? (TO)

TT=1802.17

 ?

TT=1795.37

 ? (MO)

TT=1299.44

 ? (MO)

TT=1300.97
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_10.opb.PB06.opb 		SAT
f=668
TT=1800.54
T1=8.87
 SAT
f=5755
TT=1798.12
T1=1721.58
 SAT
f=5589
TT=1798.14
T1=567.26
 ? (TO)

TT=1800.06

 SAT (MO)
f=5422
TT=549.049
T1=345.24
 SAT (MO)
f=5385
TT=1144.91
T1=937.81
 ? (MO)

TT=13.2

 ? (TO)

TT=1800.42

 SAT (TO)
f=668
TT=1800.64
T1=12.79
 SAT (TO)
f=668
TT=1800.54
T1=8.87
 ? (TO)

TT=1802.16

 SAT (TO)
f=8680
TT=1802.12
T1=1737.03
 ? (TO)

TT=1802.16

 ?

TT=1795.36

 ? (MO)

TT=1461.98

 ? (MO)

TT=1467.48
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_11.opb.PB06.opb 		SAT
f=833
TT=1800.5
T1=8.9
 SAT
f=5402
TT=1798.1
T1=767.69
 SAT
f=5461
TT=1798.11
T1=42.91
 ? (TO)

TT=1800.12

 SAT (MO)
f=5286
TT=1066.49
T1=843.71
 SAT (MO)
f=5618
TT=512.335
T1=418.2
 ? (MO)

TT=13.16

 ? (TO)

TT=1800.39

 SAT (TO)
f=833
TT=1800.57
T1=11.66
 SAT (TO)
f=833
TT=1800.5
T1=8.9
 ? (TO)

TT=1802.17

 SAT (TO)
f=9201
TT=1802.16
T1=660.06
 ? (TO)

TT=1802.18

 ?

TT=1795.36

 ? (MO)

TT=1415.02

 ? (MO)

TT=1418.59
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_12.opb.PB06.opb 		SAT
f=448
TT=1796.51
T1=589.03
 SAT
f=5673
TT=1798.27
T1=183.78
 SAT
f=5489
TT=1798.14
T1=1088.18
 ? (TO)

TT=1800.03

 SAT (MO)
f=5482
TT=993.645
T1=818.03
 SAT (TO)
f=5243
TT=1800.2
T1=1177.79
 ? (MO)

TT=13.76

 ? (TO)

TT=1800.35

 SAT (TO)
f=488
TT=1800.78
T1=11.94
 SAT (TO)
f=488
TT=1800.5
T1=8.88
 ? (TO)

TT=1802.18

 SAT (TO)
f=8479
TT=1802.08
T1=1540.67
 ? (TO)

TT=1802.21

 SAT
f=448
TT=1796.51
T1=589.03
 ? (MO)

TT=1712.32

 ? (MO)

TT=1720.53
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_13.opb.PB06.opb 		SAT
f=486
TT=1795.97
T1=804.45
 SAT
f=5918
TT=1798.19
T1=926.35
 SAT
f=5527
TT=1798.12
T1=1530.54
 ? (TO)

TT=1800.09

 SAT (MO)
f=5693
TT=468.627
T1=352.14
 SAT (MO)
f=5693
TT=420.565
T1=313.31
 ? (MO)

TT=13.3

 ? (TO)

TT=1800.52

 SAT (TO)
f=840
TT=1800.61
T1=13.11
 SAT (TO)
f=840
TT=1800.45
T1=9.12
 ? (TO)

TT=1802.13

 SAT (TO)
f=8716
TT=1802.18
T1=1725
 ? (TO)

TT=1802.23

 SAT
f=486
TT=1795.97
T1=804.45
 ? (MO)

TT=1479.36

 ? (MO)

TT=1413.96
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_14.opb.PB06.opb 		SAT
f=453
TT=1802.16
T1=664.83
 SAT
f=5891
TT=1798.16
T1=1127.61
 SAT
f=5640
TT=1798.22
T1=1448.32
 ? (TO)

TT=1800.15

 SAT (MO)
f=5534
TT=1161.03
T1=985.44
 SAT (MO)
f=5583
TT=1201.74
T1=999.05
 ? (MO)

TT=13.2

 ? (TO)

TT=1800.37

 SAT (TO)
f=585
TT=1800.83
T1=13.09
 SAT (TO)
f=585
TT=1800.53
T1=9.35
 ? (TO)

TT=1802.19

 SAT (TO)
f=9742
TT=1802.15
T1=1344.55
 SAT (TO)
f=453
TT=1802.16
T1=664.83
 SAT
f=488
TT=1796.5
T1=1119.83
 ? (MO)

TT=1393.03

 ? (MO)

TT=1404.93
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_1.opb.PB06.opb 		SAT
f=515
TT=1802.15
T1=1398.62
 SAT
f=5583
TT=1798.24
T1=798.46
 SAT
f=5450
TT=1798.22
T1=643.45
 ? (TO)

TT=1800.03

 SAT (MO)
f=5293
TT=530.436
T1=402.43
 SAT (MO)
f=5550
TT=540.65
T1=393.12
 ? (MO)

TT=13.22

 ? (TO)

TT=1800.33

 SAT (TO)
f=790
TT=1800.66
T1=11.62
 SAT (TO)
f=790
TT=1800.5
T1=8.78
 ? (TO)

TT=1802.19

 SAT (TO)
f=8666
TT=1802.15
T1=1590.69
 SAT (TO)
f=515
TT=1802.15
T1=1398.62
 ?

TT=1795.17

 ? (MO)

TT=1681.07

 ? (MO)

TT=1603.8
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_2.opb.PB06.opb 		SAT
f=699
TT=1800.5
T1=9.47
 SAT
f=5740
TT=1798.11
T1=595.67
 SAT
f=5630
TT=1798.15
T1=1492.86
 ? (TO)

TT=1800.17

 SAT (MO)
f=5591
TT=484.614
T1=313.82
 SAT (TO)
f=5403
TT=1800.22
T1=1514.94
 ? (MO)

TT=13.29

 ? (TO)

TT=1800.88

 SAT (TO)
f=699
TT=1800.73
T1=13.13
 SAT (TO)
f=699
TT=1800.5
T1=9.47
 ? (TO)

TT=1802.15

 SAT (TO)
f=9969
TT=1802.1
T1=598.32
 ? (TO)

TT=1802.21

 ?

TT=1795.06

 ? (MO)

TT=1558.62

 ? (MO)

TT=1563.05
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_3.opb.PB06.opb 		SAT
f=390
TT=1796.04
T1=764.69
 SAT
f=5942
TT=1798.19
T1=1606.29
 SAT
f=5622
TT=1798.12
T1=886.1
 ? (TO)

TT=1800.12

 SAT (MO)
f=5741
TT=460.565
T1=357.41
 SAT (MO)
f=5672
TT=496.422
T1=467.06
 ? (MO)

TT=12.85

 ? (TO)

TT=1800.45

 SAT (TO)
f=1237
TT=1800.7
T1=12
 SAT (TO)
f=1237
TT=1800.54
T1=8.79
 ? (TO)

TT=1802.18

 SAT (TO)
f=8954
TT=1802.12
T1=1660.61
 ? (TO)

TT=1802.2

 SAT
f=390
TT=1796.04
T1=764.69
 ? (MO)

TT=1481.25

 ? (MO)

TT=1486.88
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_4.opb.PB06.opb 		SAT
f=483
TT=1796.63
T1=1207.59
 SAT
f=5625
TT=1798.18
T1=1307.56
 SAT
f=5248
TT=1798.14
T1=165.99
 ? (TO)

TT=1800.01

 SAT (MO)
f=5440
TT=548.454
T1=334.33
 SAT (MO)
f=5440
TT=501.932
T1=304.1
 ? (MO)

TT=13.29

 ? (TO)

TT=1800.39

 SAT (TO)
f=780
TT=1800.74
T1=12.7
 SAT (TO)
f=780
TT=1800.57
T1=8.84
 ? (TO)

TT=1802.2

 SAT (TO)
f=9591
TT=1802.1
T1=1653.42
 ? (TO)

TT=1802.23

 SAT
f=483
TT=1796.63
T1=1207.59
 ? (MO)

TT=1232.85

 ? (MO)

TT=1236.06
Bench nameBest resultsbsolo
3.2 Card
(complete)		bsolo
3.2 Cl
(complete)		pb_cplex
2010-06-29
(complete)		PB/CT
0.1
(complete)		PB/CT
0.1 fixed
(complete)		PBPASSolver
2010-06-13
(complete)		SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)		SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)		SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		wbo
1.4b
(complete)		wbo
1.4b (fixed)
(complete)
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_5.opb.PB06.opb 		SAT
f=557
TT=1802.16
T1=1618.83
 SAT
f=5715
TT=1798.18
T1=1689.65
 SAT
f=5530
TT=1798.12
T1=1336.57
 ? (TO)

TT=1800.04

 SAT (MO)
f=5488
TT=1145.5
T1=956.92
 SAT (MO)
f=5482
TT=1020.44
T1=740.28
 ? (MO)

TT=12.91

 ? (TO)

TT=1800.45

 SAT (TO)
f=891
TT=1800.71
T1=11.98
 SAT (TO)
f=891
TT=1800.51
T1=8.6
 ? (TO)

TT=1802.16

 SAT (TO)
f=9770
TT=1802.1
T1=1062.31
 SAT (TO)
f=557
TT=1802.16
T1=1618.83
 ?

TT=1794.91

 ? (MO)

TT=1721.92

 ? (MO)

TT=1727.39
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_6.opb.PB06.opb 		SAT
f=371
TT=1796.29
T1=1626.26
 SAT
f=5518
TT=1798.09
T1=825.32
 SAT
f=5674
TT=1798.13
T1=1511.28
 ? (TO)

TT=1800.16

 SAT (MO)
f=5359
TT=1068.01
T1=912.67
 SAT (MO)
f=5566
TT=506.025
T1=365.54
 ? (MO)

TT=13.3

 ? (TO)

TT=1800.41

 SAT (TO)
f=802
TT=1800.75
T1=10.98
 SAT (TO)
f=802
TT=1800.53
T1=8.71
 ? (TO)

TT=1802.14

 SAT (TO)
f=9235
TT=1802.12
T1=578.94
 ? (TO)

TT=1802.2

 SAT
f=371
TT=1796.29
T1=1626.26
 ? (MO)

TT=1315.56

 ? (MO)

TT=1317.32
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_7.opb.PB06.opb 		SAT
f=396
TT=1796.11
T1=764.81
 SAT
f=5521
TT=1798.17
T1=661.35
 SAT
f=5483
TT=1798.11
T1=1335.11
 ? (TO)

TT=1800.05

 SAT (MO)
f=5487
TT=1068.29
T1=765.11
 SAT (MO)
f=5447
TT=548.587
T1=323.3
 ? (MO)

TT=13.2

 ? (TO)

TT=1800.37

 SAT (TO)
f=944
TT=1800.64
T1=11.74
 SAT (TO)
f=944
TT=1800.51
T1=8.87
 ? (TO)

TT=1802.19

 SAT (TO)
f=8785
TT=1802.09
T1=1159.32
 SAT (TO)
f=486
TT=1802.15
T1=677.87
 SAT
f=396
TT=1796.11
T1=764.81
 ? (MO)

TT=1416.3

 ? (MO)

TT=1428.52
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_8.opb.PB06.opb 		SAT
f=545
TT=1802.2
T1=1198.31
 SAT
f=5693
TT=1798.11
T1=1581.59
 SAT
f=5597
TT=1798.11
T1=964.11
 ? (TO)

TT=1800.18

 SAT (MO)
f=5374
TT=504.453
T1=236.03
 SAT (MO)
f=5374
TT=543.766
T1=250.77
 ? (MO)

TT=12.79

 ? (TO)

TT=1800.39

 SAT (TO)
f=855
TT=1800.54
T1=11.92
 SAT (TO)
f=855
TT=1800.51
T1=9.01
 ? (TO)

TT=1802.2

 SAT (TO)
f=9183
TT=1802.14
T1=981.75
 SAT (TO)
f=545
TT=1802.2
T1=1198.31
 ?

TT=1795.13

 ? (TO)

TT=1800.2

 ? (TO)

TT=1800.17
normalized-PB06/OPT-SMALLINT/
submitted-PB06/liu/wnq/
normalized-wnq_n100_mw99_9.opb.PB06.opb 		SAT
f=419
TT=1796.43
T1=1088.63
 SAT
f=5739
TT=1798.11
T1=983.71
 SAT
f=5711
TT=1798.11
T1=686.75
 ? (TO)

TT=1800.2

 SAT (MO)
f=5484
TT=1122.68
T1=834.68
 SAT (MO)
f=5208
TT=1744.73
T1=1073.8
 ? (MO)

TT=13.19

 ? (TO)

TT=1800.42

 SAT (TO)
f=717
TT=1800.56
T1=11.74
 SAT (TO)
f=717
TT=1800.53
T1=9.05
 ? (TO)

TT=1802.14

 SAT (TO)
f=9403
TT=1802.13
T1=1228.59
 ? (TO)

TT=1802.21

 SAT
f=419
TT=1796.43
T1=1088.63
 ? (MO)

TT=1548.14

 ? (MO)

TT=1557.9



Statisticsbsolo
3.2 Card
(complete)		bsolo
3.2 Cl
(complete)		pb_cplex
2010-06-29
(complete)		PB/CT
0.1
(complete)		PB/CT
0.1 fixed
(complete)		PBPASSolver
2010-06-13
(complete)		SAT4J PB CuttingPlanes
2.2.0 2010-05-26
(complete)		SAT4J PB RES // CP
2.2.0 2010-05-31
(complete)		SAT4J PB Resolution
2.2.0 2010-05-26
(complete)		SCIPclp
SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver
(complete)		SCIPnone
SCIP 1.2.1.2 without any LP solver
(complete)		SCIPspx
SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		SCIPspx
SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver
(complete)		wbo
1.4b
(complete)		wbo
1.4b (fixed)
(complete)
Number of times the solver is able to give the best known answer000000043004700
Number of times the solver is the best solver from a complete solver point of view
(i.e. best known answer and best TT time)		000000043004700