PB'11 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Synthesis PTL/CMOS Circuits

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Synthesis PTL/CMOS Circuits

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg pb-opt-11.04.03 (complete)
bsolo 3.2 (complete)
clasp 2.0-R4191 (complete)
MinisatID 2.4.8 [DEPRECATED] (complete)
MinisatID 2.4.8-gmp [DEPRECATED] (complete)
MinisatID 2.5.2 (fixed) (complete)
MinisatID 2.5.2-gmp (fixed) (complete)
pwbo 1.1 (complete)
Sat4j CuttingPlanes 2.3.0 (complete)
Sat4j Res//CP 2.3.0 (complete)
Sat4j Resolution 2.3.0 (complete)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx 2 2011-06-10 (fixed) (complete)
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx E_2 2011-06-10 (fixed) (complete)
wbo 1.6 (complete)

Bench nameBest results
on this
instance		borg
pb-opt-11.04.03
(complete)		bsolo
3.2
(complete)		clasp
2.0-R4191
(complete)		MinisatID
2.4.8 [DEPRECATED]
(complete)		MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		MinisatID
2.5.2 (fixed)
(complete)		MinisatID
2.5.2-gmp (fixed)
(complete)		pwbo
1.1
(complete)		Sat4j CuttingPlanes
2.3.0
(complete)		Sat4j Res//CP
2.3.0
(complete)		Sat4j Resolution
2.3.0
(complete)		SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		SCIP spx 2
2011-06-10 (fixed)
(complete)		SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		SCIP spx E_2
2011-06-10 (fixed)
(complete)		wbo
1.6
(complete)
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-9symml.opb 		OPT
f=4517
TT=2.445
T1=2.43
 OPT
f=4517
TT=2.489
T1=2.69
 SAT
f=4742
TT=1798
T1=1730.75
 SAT (TO)
f=6525
TT=1800.05
T1=185.62
 ? (TO)
f=6453
TT=1800.07
T1=0.02
 ? (TO)
f=6453
TT=1800.05
T1=0.06
 ? (TO)
f=6453
TT=1800.08
T1=1800
 ? (TO)
f=6453
TT=1800.05
T1=1800
 SAT (TO)
f=5225
TT=1800.07
T1=325.25
 SAT (TO)
f=5576
TT=1800.56
T1=214.62
 SAT (TO)
f=5352
TT=1800.36
T1=989.95
 SAT (TO)
f=5351
TT=1800.11
T1=1668.62
 OPT
f=4517
TT=2.475
T1=2.46
 OPT
f=4517
TT=2.445
T1=2.43
 OPT
f=4517
TT=2.479
T1=2.46
 OPT
f=4517
TT=2.462
T1=2.45
 ? (TO)

TT=1800.12
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-C432.opb 		OPT
f=4822
TT=1.358
T1=1.34
 OPT
f=4822
TT=1.879
T1=1.93
 SAT
f=5429
TT=1798.13
T1=1542.11
 SAT (TO)
f=7170
TT=1800.03
T1=0
 ? (TO)
f=7342
TT=1800.06
T1=990.81
 ? (TO)
f=7672
TT=1800.05
T1=0.13
 ? (TO)
f=7638
TT=1800.07
T1=1800
 ? (TO)
f=7656
TT=1800.06
T1=1800
 SAT (TO)
f=6171
TT=1800.07
T1=1.97
 SAT (TO)
f=5811
TT=1800.25
T1=498.42
 SAT (TO)
f=6038
TT=1800.76
T1=228.93
 SAT (TO)
f=6023
TT=1800.08
T1=1214.73
 OPT
f=4822
TT=1.393
T1=1.37
 OPT
f=4822
TT=1.358
T1=1.34
 OPT
f=4822
TT=1.387
T1=1.37
 OPT
f=4822
TT=1.363
T1=1.34
 ? (TO)

TT=1800.14
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-c8.opb 		OPT
f=1194
TT=0.1
T1=0.09
 OPT
f=1194
TT=0.624
T1=0.87
 OPT
f=1194
TT=0.39
T1=0.36
 SAT (TO)
f=1439
TT=1800.06
T1=57.31
 ? (TO)
f=1497
TT=1800.06
T1=910.75
 ? (TO)
f=1587
TT=1800.09
T1=1764.93
 ? (TO)
f=1447
TT=1800.06
T1=1800
 ? (TO)
f=1488
TT=1800.05
T1=1800
 OPT
f=1194
TT=3.737
T1=1.87
 SAT (TO)
f=1495
TT=1800.85
T1=107.85
 SAT (TO)
f=1322
TT=1800.35
T1=499.1
 SAT (TO)
f=1351
TT=1800.16
T1=1550.45
 OPT
f=1194
TT=0.1
T1=0.09
 OPT
f=1194
TT=0.105
T1=0.09
 OPT
f=1194
TT=0.101
T1=0.09
 OPT
f=1194
TT=0.1
T1=0.09
 OPT
f=1194
TT=186.092
T1=186.1
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-cc.opb 		OPT
f=1567
TT=0.039
T1=0.01
 OPT
f=1567
TT=0.462
T1=0.55
 OPT
f=1567
TT=0.197
T1=0.18
 SAT (TO)
f=1713
TT=1800.08
T1=0.03
 ? (TO)
f=1655
TT=1800.05
T1=427.71
 ? (TO)
f=1731
TT=1800.06
T1=68.57
 ? (TO)
f=1662
TT=1800.06
T1=1800
 ? (TO)
f=1725
TT=1800.06
T1=1800
 SAT (TO)
f=1620
TT=1800.09
T1=761.97
 SAT (TO)
f=1567
TT=1800.33
T1=766.99
 SAT (TO)
f=1567
TT=1800.21
T1=52.87
 SAT (TO)
f=1604
TT=1800.16
T1=816.83
 OPT
f=1567
TT=0.04
T1=0.01
 OPT
f=1567
TT=0.039
T1=0.01
 OPT
f=1567
TT=0.041
T1=0.01
 OPT
f=1567
TT=0.041
T1=0.01
 ? (TO)

TT=1800.13
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-cm42a.opb 		OPT
f=694
TT=0.008
T1=0
 OPT
f=694
TT=0.529
T1=0.8
 OPT
f=694
TT=0.008
T1=0
 OPT
f=694
TT=704.315
T1=60.34
 ? (TO)
f=703
TT=1800.03
T1=1416.37
 ? (TO)
f=721
TT=1800.02
T1=1752.8
 ? (TO)
f=705
TT=1800.05
T1=1800
 ? (TO)
f=714
TT=1800.06
T1=1800
 SAT (TO)
f=696
TT=1800.09
T1=340.92
 SAT (TO)
f=784
TT=1800.26
T1=1240.39
 SAT (TO)
f=694
TT=1800.24
T1=192.3
 OPT
f=694
TT=758.677
T1=83.75
 OPT
f=694
TT=0.036
T1=0.03
 OPT
f=694
TT=0.035
T1=0.03
 OPT
f=694
TT=0.038
T1=0.03
 OPT
f=694
TT=0.038
T1=0.03
 ? (TO)

TT=1800.09
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-cmb.opb 		OPT
f=1053
TT=0.054
T1=0.05
 OPT
f=1053
TT=0.559
T1=0.85
 OPT
f=1053
TT=0.378
T1=0.34
 SAT (TO)
f=1389
TT=1800.06
T1=1524
 ? (TO)
f=1442
TT=1800.03
T1=1243.83
 ? (TO)
f=1474
TT=1800.04
T1=833.08
 ? (TO)
f=1435
TT=1800.05
T1=1800
 ? (TO)
f=1498
TT=1800.06
T1=1800
 OPT
f=1053
TT=0.115
T1=0.06
 SAT (TO)
f=1373
TT=1800.2
T1=13.08
 SAT (TO)
f=1161
TT=1800.3
T1=515.17
 SAT (TO)
f=1161
TT=1800.07
T1=223.95
 OPT
f=1053
TT=0.075
T1=0.07
 OPT
f=1053
TT=0.073
T1=0.06
 OPT
f=1053
TT=0.077
T1=0.07
 OPT
f=1053
TT=0.075
T1=0.07
 OPT
f=1053
TT=0.054
T1=0.05
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/
synthesis-ptl-cmos-circuits/normalized-mux.opb 		OPT
f=872
TT=0.011
T1=0.01
 OPT
f=872
TT=0.534
T1=0.73
 OPT
f=872
TT=0.011
T1=0.01
 SAT (TO)
f=1069
TT=1800.08
T1=607.71
 ? (TO)
f=1248
TT=1800.08
T1=1655.26
 ? (TO)
f=1342
TT=1800.05
T1=290.88
 ? (TO)
f=1277
TT=1800.06
T1=1800
 ? (TO)
f=1357
TT=1800.07
T1=1800
 OPT
f=872
TT=0.055
T1=0.03
 SAT (TO)
f=1086
TT=1800.63
T1=9.54
 SAT (TO)
f=1093
TT=1800.35
T1=170.95
 SAT (TO)
f=1247
TT=1800.18
T1=2.44
 OPT
f=872
TT=0.034
T1=0.02
 OPT
f=872
TT=0.032
T1=0.02
 OPT
f=872
TT=0.034
T1=0.02
 OPT
f=872
TT=0.032
T1=0.02
 OPT
f=872
TT=0.027
T1=0.02
normalized-PB06/OPT-SMALLINT/
submitted-PB05/manquinho/synthesis-ptl-cmos-circuits/
normalized-my_adder.opb 		OPT
f=4561
TT=0.46
T1=0.34
 OPT
f=4561
TT=1.048
T1=1.31
 OPT
f=4561
TT=13.505
T1=1.27
 SAT (TO)
f=5548
TT=1800.07
T1=0
 ? (TO)
f=6443
TT=1800.08
T1=160.89
 ? (TO)
f=6443
TT=1800.07
T1=1057.87
 ? (TO)
f=6446
TT=1800.05
T1=1800
 ? (TO)
f=6446
TT=1800.04
T1=1800
 SAT (TO)
f=5566
TT=1800.08
T1=757.62
 SAT (TO)
f=5089
TT=1800.72
T1=49.43
 SAT (TO)
f=5176
TT=1800.18
T1=1053.6
 SAT (TO)
f=5465
TT=1800.08
T1=201.24
 OPT
f=4561
TT=0.484
T1=0.37
 OPT
f=4561
TT=0.518
T1=0.39
 OPT
f=4561
TT=0.467
T1=0.35
 OPT
f=4561
TT=0.46
T1=0.34
 ? (TO)

TT=1800.07



Statisticsborg
pb-opt-11.04.03
(complete)		bsolo
3.2
(complete)		clasp
2.0-R4191
(complete)		MinisatID
2.4.8 [DEPRECATED]
(complete)		MinisatID
2.4.8-gmp [DEPRECATED]
(complete)		MinisatID
2.5.2 (fixed)
(complete)		MinisatID
2.5.2-gmp (fixed)
(complete)		pwbo
1.1
(complete)		Sat4j CuttingPlanes
2.3.0
(complete)		Sat4j Res//CP
2.3.0
(complete)		Sat4j Resolution
2.3.0
(complete)		SCIP spx
SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		SCIP spx 2
2011-06-10 (fixed)
(complete)		SCIP spx E
SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED]
(complete)		SCIP spx E_2
2011-06-10 (fixed)
(complete)		wbo
1.6
(complete)
Number of times the solver is able to give the best known answer0610000300188083
Number of times the solver is the best solver from a complete solver point of view
(i.e. best known answer and best TT time)		0200000000013021