PB'12 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Radar Surveillance

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Radar Surveillance

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
bsolo 3.2 (complete)
clasp 2.0.6-R5325 (opt) (complete)
npSolver 1.0 (complete)
npSolver 1.0 (fixed) (complete)
npSolver inc (complete)
npSolver inc (fixed) (complete)
npSolver inc-topDown (complete)
npSolver inc-topDown (fixed) (complete)
npSolver inc-topdown-quickBound (complete)
npSolver inc-topdown-quickBound (fixed) (complete)
PB07: bsolo 3.0.17 (complete)
PB07: minisat+ 1.14 (complete)
PB07: PB-clasp 2007-04-10 (complete)
PB07: Pueblo 1.4 (incomplete)
PB07: SAT4JPseudoResolution 2007-03-23 (complete)
PB09: bsolo 3.1 (complete)
PB09: SAT4J Pseudo Resolution 2.1.1 (complete)
PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)
PB10: pb_cplex 2010-06-29 (complete)
PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)
PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
PB11: Sat4j Res//CP 2.3.0 (complete)
PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)
PB12: minisatp 1.0-2-g022594c (complete)
pb2sat 2012-05-19 (complete)
pb2satCp2 2012-05-19 (complete)
pwbo 2.0 (complete)
pwbo 2.02 (complete)
SAT 4j PB RES // CP 2.3.2 Snapshot (complete)
Sat 4j PB Resolution 2.3.2 Snapshot (complete)
SAT4J PB specific settings 2.3.2 snapshot (complete)
SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)
SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)
SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)
toysat 2012-05-17 (complete)
toysat 2012-06-01 (complete)
wbo 1.7 (complete)
wbo 1.72 (complete)

Bench name	Best results on this instance	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-10:10:4.5:0.5:100.opb	OPT f=3 TT=0.028 T1=0.02	OPT f=3 TT=0.274 T1=0.26	SAT (TO) f=6 TT=1800.02 T1=8.55	? (TO) TT=1800.04	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.14	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.14	OPT f=3 TT=0.02 T1=0.02	SAT (TO) f=7 TT=1800.08 T1=1234.43	? (TO) TT=1800.1	SAT f=7 TT=1783 T1=1783.27	SAT (TO) f=12 TT=1800.07 T1=0.21	OPT f=3 TT=0.292 T1=0.28	? (exit code) TT=875.453	OPT f=3 TT=0.36 T1=0.35	OPT f=3 TT=0.028 T1=0.02	SAT (TO) f=7 TT=1801.08 T1=284.92	OPT f=3 TT=0.691 T1=0.68	SAT (TO) f=7 TT=1800.15 T1=147.05	OPT f=3 TT=0.347 T1=0.34	SAT (TO) f=7 TT=1800.07 T1=1800.4	? (TO) TT=1800.04	? (TO) TT=1800.06	SAT (TO) f=5 TT=1800.51 T1=900.31	SAT (TO) f=5 TT=1800.17 T1=900.31	SAT (TO) f=9 TT=1800.1 T1=2.23	SAT (TO) f=12 TT=1800.89 T1=0.21	SAT (TO) f=21 TT=1800.03 T1=276.87	OPT f=3 TT=0.387 T1=0.37	OPT f=3 TT=0.394 T1=0.37	OPT f=3 TT=0.324 T1=0.31	? (TO) TT=1800.02	? (TO) TT=1800.1	? TT=1799.89	? TT=1799.79
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-10:10:4.5:0.95:100.opb	OPT f=0 TT=0.04 T1=0.03	OPT f=0 TT=0.668 T1=0.64	SAT (TO) f=3 TT=1800.02 T1=1086.1	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) TT=1800.09	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.12	? (TO) TT=1800.08	OPT f=0 TT=0.107 T1=0.1	SAT (TO) f=6 TT=1800.05 T1=274.47	? (TO) TT=1800.29	SAT f=5 TT=1783 T1=1783.3	? (exit code) TT=984.157	OPT f=0 TT=0.706 T1=0.68	? (exit code) TT=970.48	OPT f=0 TT=0.547 T1=0.53	OPT f=0 TT=0.04 T1=0.03	SAT (TO) f=3 TT=1800.52 T1=2.21	OPT f=0 TT=0.062 T1=0.05	SAT (TO) f=3 TT=1800.08 T1=1.23	OPT f=0 TT=0.069 T1=0.06	SAT (TO) f=6 TT=1800.04 T1=1800.4	? (TO) TT=1800.13	? (TO) TT=1800.13	SAT (TO) f=1 TT=1800.17 T1=900.31	SAT (TO) f=1 TT=1800.07 T1=900.31	SAT (TO) f=3 TT=1800.2 T1=1.22	SAT (TO) f=5 TT=1800.69 T1=0.2	SAT (TO) f=21 TT=1800.03 T1=0.25	OPT f=0 TT=0.176 T1=0.16	OPT f=0 TT=0.166 T1=0.15	OPT f=0 TT=0.115 T1=0.11	? (TO) TT=1800.01	? (TO) TT=1800.02	? TT=1799.46	? TT=1799.81
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-10:10:4.5:0.95:98.opb	OPT f=1 TT=0.033 T1=0.03	OPT f=1 TT=0.146 T1=0.14	SAT (TO) f=7 TT=1800.02 T1=1122.84	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.12	OPT f=1 TT=0.045 T1=0.04	SAT (TO) f=9 TT=1800.07 T1=120.4	? (TO) TT=1800.12	SAT f=11 TT=1783.01 T1=1783.29	? (exit code) TT=1015.99	OPT f=1 TT=0.156 T1=0.15	? (exit code) TT=949.3	OPT f=1 TT=0.488 T1=0.48	OPT f=1 TT=0.033 T1=0.03	SAT (TO) f=6 TT=1800.82 T1=3.24	OPT f=1 TT=0.533 T1=0.52	SAT (TO) f=6 TT=1800.06 T1=1.74	OPT f=1 TT=0.492 T1=0.47	SAT (TO) f=6 TT=1800.07 T1=1800.4	? (TO) TT=1800.09	? (TO) TT=1800.06	SAT (TO) f=7 TT=1800.16 T1=900.31	SAT (TO) f=7 TT=1800.16 T1=900.31	SAT (TO) f=6 TT=1800.07 T1=1.72	SAT (TO) f=8 TT=1800.09 T1=0.2	SAT (TO) f=31 TT=1800.04 T1=1069.8	OPT f=1 TT=0.406 T1=0.39	OPT f=1 TT=0.391 T1=0.37	OPT f=1 TT=0.315 T1=0.3	? (TO) TT=1800.02	? (TO) TT=1800.02	? TT=1799.56	? TT=1799.48
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-10:20:4.5:0.5:100.opb	OPT f=0 TT=0.161 T1=0.16	OPT f=0 TT=7.411 T1=7.38	SAT (TO) f=14 TT=1800.02 T1=1396.27	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) TT=1800.11	OPT f=0 TT=0.3 T1=0.26	SAT (TO) f=22 TT=1800.09 T1=1165.85	? (TO) TT=1613.78	SAT f=19 TT=1783 T1=1783.38	? (exit code) TT=407.91	OPT f=0 TT=7.424 T1=7.39	? (exit code) TT=1007.2	OPT f=0 TT=2.881 T1=2.86	OPT f=0 TT=0.161 T1=0.16	SAT (TO) f=9 TT=1800.78 T1=8.27	OPT f=0 TT=5.674 T1=5.66	SAT (TO) f=9 TT=1800.08 T1=4.29	OPT f=0 TT=6.476 T1=6.46	SAT (TO) f=21 TT=1800.06 T1=1800.5	? (TO) TT=1800.07	? (TO) TT=1800.14	SAT (TO) f=11 TT=1800.55 T1=900.31	SAT (TO) f=13 TT=1800.06 T1=900.21	SAT (TO) f=9 TT=1800.02 T1=4.29	SAT (TO) f=19 TT=1800.07 T1=0.25	SAT (TO) f=55 TT=1800.65 T1=0.27	OPT f=0 TT=1.172 T1=1.15	OPT f=0 TT=1.117 T1=1.09	OPT f=0 TT=0.941 T1=0.92	? (TO) TT=1800.01	? (TO) TT=1800.02	? TT=1799.5	? TT=1799.83
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-10:20:4.5:0.95:100.opb	OPT f=1 TT=0.183 T1=0.18	OPT f=1 TT=13.864 T1=13.8	SAT (TO) f=12 TT=1800.02 T1=0	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.1	? (TO) TT=1800.14	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.11	OPT f=1 TT=0.4 T1=0.32	SAT (TO) f=78 TT=1800.07 T1=1246.56	? (TO) TT=1800.1	SAT f=16 TT=1783 T1=1783.28	? (exit code) TT=370.24	OPT f=1 TT=14.695 T1=14.63	? (exit code) TT=921.55	OPT f=1 TT=1.175 T1=1.16	OPT f=1 TT=0.183 T1=0.18	SAT (TO) f=10 TT=1800.15 T1=5.27	OPT f=1 TT=0.85 T1=0.83	SAT (TO) f=10 TT=1800.04 T1=2.81	OPT f=1 TT=1.371 T1=1.35	SAT (TO) f=85 TT=1800.1 T1=1800.4	? (TO) TT=1800.06	? (TO) TT=1800.04	SAT (TO) f=16 TT=1800.07 T1=900.31	SAT (TO) f=13 TT=1800.04 T1=900.31	SAT (TO) f=10 TT=1800.65 T1=2.79	SAT (TO) f=17 TT=1800.03 T1=0.26	SAT (TO) f=50 TT=1800.1 T1=0.33	OPT f=1 TT=1.67 T1=1.65	OPT f=1 TT=1.722 T1=1.7	OPT f=1 TT=1.34 T1=1.32	? (TO) TT=1800.1	? (TO) TT=1800.09	? TT=1799.87	? TT=1799.5
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-10:20:4.5:0.95:98.opb	OPT f=2 TT=0.182 T1=0.18	OPT f=2 TT=5.73 T1=5.73	SAT (TO) f=19 TT=1800.1 T1=0	? (TO) TT=1800.11	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.12	OPT f=2 TT=0.384 T1=0.34	SAT (TO) f=88 TT=1800.07 T1=1642.29	? (TO) TT=1800.3	SAT f=25 TT=1783.01 T1=1783.28	? (exit code) TT=338.932	OPT f=2 TT=5.825 T1=5.82	? (exit code) TT=748.17	OPT f=2 TT=2.066 T1=2.06	OPT f=2 TT=0.182 T1=0.18	SAT (TO) f=16 TT=1800.2 T1=6.25	OPT f=2 TT=17.192 T1=17.18	SAT (TO) f=16 TT=1800.13 T1=4.81	OPT f=2 TT=1.495 T1=1.47	SAT (TO) f=94 TT=1800.06 T1=1800.4	? (TO) TT=1800.07	? (TO) TT=1800.1	SAT (TO) f=19 TT=1800.38 T1=900.31	SAT (TO) f=20 TT=1800.05 T1=900.31	SAT (TO) f=16 TT=1800.07 T1=3.27	SAT (TO) f=25 TT=1800.05 T1=0.25	SAT (TO) f=73 TT=1800.07 T1=0.27	OPT f=2 TT=1.544 T1=1.52	OPT f=2 TT=1.548 T1=1.53	OPT f=2 TT=1.405 T1=1.38	? (TO) TT=1800.08	? (TO) TT=1800.1	? TT=1799.5	? TT=1799.53
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-30:30:4.5:0.5:100.opb	OPT f=5 TT=3.303 T1=3.28	SAT f=83 TT=1798.09 T1=1787.9	SAT (TO) f=61 TT=1800.08 T1=0.01	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.07	OPT f=5 TT=20.487 T1=19.32	SAT (TO) f=607 TT=1800.06 T1=1114.57	? (TO) TT=1800.05	SAT f=109 TT=1783 T1=1783.3	No Cert. TT=465.57	SAT f=101 TT=1798.12 T1=1782.96	SAT (TO) f=51 TT=1800.84 T1=0.5	OPT f=5 TT=379.803 T1=379.83	OPT f=5 TT=3.303 T1=3.28	SAT (TO) f=45 TT=1800.59 T1=41.57	OPT f=5 TT=217.202 T1=217.21	SAT (TO) f=45 TT=1800.17 T1=33.54	OPT f=5 TT=222.845 T1=222.83	SAT (TO) f=679 TT=1800.06 T1=1800.5	? (TO) TT=1800.07	? (TO) TT=1800.05	SAT (TO) f=194 TT=1800.53 T1=900.31	SAT (TO) f=189 TT=1800.09 T1=900.31	SAT (TO) f=45 TT=1800.09 T1=14.71	SAT (TO) f=51 TT=1800.09 T1=0.44	SAT (TO) f=246 TT=1800.09 T1=0.48	OPT f=5 TT=89.067 T1=89.03	OPT f=5 TT=85.898 T1=85.87	OPT f=5 TT=204.469 T1=204.45	? (TO) TT=1800.08	? (TO) TT=1800.07	? TT=1799.83	? TT=1799.46
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-30:30:4.5:0.95:100.opb	OPT f=2 TT=4.242 T1=4.22	SAT f=251 TT=1798.01 T1=302.58	SAT (TO) f=60 TT=1800.08 T1=0.02	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.07	SAT (TO) f=251 TT=1800.06 T1=1779.01	SAT (TO) f=714 TT=1800.04 T1=1419.78	? (TO) TT=1800.07	SAT f=107 TT=1783.01 T1=1783.32	No Cert. TT=318.24	SAT f=253 TT=1798.03 T1=0.49	? (exit code) TT=940.38	OPT f=2 TT=690.109 T1=690.17	OPT f=2 TT=4.242 T1=4.22	SAT (TO) f=54 TT=1800.72 T1=121.66	OPT f=2 TT=238.791 T1=238.79	SAT (TO) f=54 TT=1800.03 T1=120.59	OPT f=2 TT=454.428 T1=454.45	SAT (TO) f=720 TT=1800.1 T1=1800.6	? (TO) TT=1800.04	? (TO) TT=1800.14	SAT (TO) f=226 TT=1800.07 T1=900.31	SAT (TO) f=232 TT=1800.05 T1=900.31	SAT (TO) f=54 TT=1800.18 T1=29.56	SAT (TO) f=69 TT=1800.03 T1=0.45	SAT (TO) f=295 TT=1800.04 T1=0.49	OPT f=2 TT=372.418 T1=372.42	OPT f=2 TT=365.326 T1=365.31	OPT f=2 TT=655.529 T1=655.58	? (TO) TT=1800.08	? (TO) TT=1800.07	? TT=1799.9	? TT=1799.89
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-30:30:4.5:0.95:98.opb	OPT f=4 TT=3.544 T1=3.52	SAT f=283 TT=1798.02 T1=1570.94	SAT (TO) f=92 TT=1800.11 T1=0.01	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.04	SAT (TO) f=272 TT=1800.13 T1=11.31	SAT (TO) f=731 TT=1800.07 T1=821.55	? (TO) TT=1800.25	SAT f=168 TT=1783.01 T1=1783.32	No Cert. TT=345.31	SAT f=275 TT=1798.01 T1=637.58	? (exit code) TT=890.14	OPT f=4 TT=271.565 T1=271.57	OPT f=4 TT=3.544 T1=3.52	SAT (TO) f=80 TT=1800.85 T1=65.49	OPT f=4 TT=137.072 T1=137.06	SAT (TO) f=80 TT=1800.07 T1=64.15	OPT f=4 TT=14.865 T1=14.8	SAT (TO) f=761 TT=1800.05 T1=1800.5	? (TO) TT=1800.06	? (TO) TT=1800.09	SAT (TO) f=268 TT=1800.48 T1=900.31	SAT (TO) f=276 TT=1800.48 T1=900.31	SAT (TO) f=80 TT=1800.05 T1=22.01	SAT (TO) f=88 TT=1800.02 T1=0.44	SAT (TO) f=337 TT=1800.02 T1=0.61	OPT f=4 TT=167.416 T1=167.37	OPT f=4 TT=161.755 T1=161.74	OPT f=4 TT=140.924 T1=140.9	? (TO) TT=1800.04	? (TO) TT=1800.02	? TT=1799.84	? TT=1799.39
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-30:70:4.5:0.5:100.opb	OPT f=9 TT=20.584 T1=19.32	SAT f=614 TT=1798.01 T1=931.54	SAT (TO) f=130 TT=1800.08 T1=0.03	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.05	Signal TT=602.933	SAT (TO) f=1978 TT=1800.02 T1=1308.29	? (TO) TT=1765.72	SAT f=283 TT=1783.01 T1=1783.35	No Cert. TT=327.266	SAT f=612 TT=1798.27 T1=232.87	SAT (TO) f=146 TT=1800.8 T1=0.78	SAT f=16 TT=1793.19 T1=711.53	OPT f=9 TT=20.584 T1=19.32	SAT (TO) f=130 TT=1800.12 T1=788.57	SAT f=15 TT=1789.82 T1=1732.77	SAT (TO) f=130 TT=1800.61 T1=677.11	SAT f=13 TT=1796.8 T1=1377.46	SAT (TO) f=1952 TT=1800.05 T1=1800.5	? (TO) TT=1800.1	? (TO) TT=1800.03	SAT (TO) f=566 TT=1800.14 T1=900.31	SAT (TO) f=565 TT=1800.11 T1=900.31	SAT (TO) f=123 TT=1800.03 T1=601.15	SAT (TO) f=146 TT=1800.03 T1=0.6	SAT (TO) f=639 TT=1800.67 T1=0.69	SAT f=10 TT=1796.84 T1=1728.13	SAT f=10 TT=1796.81 T1=1663.33	SAT f=10 TT=1796.85 T1=1669.86	? (TO) TT=1800.08	? (TO) TT=1800.12	? TT=1799.61	? TT=1799.63
Bench name	Best results	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-30:70:4.5:0.95:100.opb	OPT f=3 TT=18.669 T1=18.53	SAT f=641 TT=1798.02 T1=1421.59	SAT (TO) f=140 TT=1800.07 T1=0.02	? (TO) TT=1800.01	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800	Signal TT=999.199	SAT (TO) f=2016 TT=1800.03 T1=1113.47	? (TO) TT=1800.11	SAT f=276 TT=1783.01 T1=1783.34	No Cert. TT=302.55	SAT f=641 TT=1798.21 T1=1670.98	SAT (TO) f=135 TT=1800.06 T1=0.76	OPT f=3 TT=1465.25 T1=1465.41	OPT f=3 TT=18.669 T1=18.53	SAT (TO) f=127 TT=1800.2 T1=85.81	SAT f=9 TT=1789.83 T1=1424	SAT (TO) f=127 TT=1800.15 T1=79.58	SAT f=7 TT=1796.82 T1=288.17	SAT (TO) f=1694 TT=1800.03 T1=1800.5	? (TO) TT=1800.04	? (TO) TT=1800.03	SAT (TO) f=575 TT=1800.58 T1=900.31	SAT (TO) f=593 TT=1800.14 T1=900.31	SAT (TO) f=127 TT=1800.19 T1=55.59	SAT (TO) f=135 TT=1800.02 T1=0.61	SAT (TO) f=688 TT=1800.01 T1=0.61	OPT f=3 TT=1611.71 T1=1611.8	OPT f=3 TT=1627.71 T1=1627.79	OPT f=3 TT=1560.37 T1=1560.48	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.03	? TT=1799.44
normalized-PB06/OPT-SMALLINT/web/ www.ps.uni-sb.de/~walser/benchmarks/ radar/normalized-30:70:4.5:0.95:98.opb	OPT f=12 TT=20.71 T1=20.57	SAT f=729 TT=1798.02 T1=5.37	SAT (TO) f=249 TT=1800.09 T1=0.03	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.13	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.01	? (TO) TT=1800.09	? (TO) TT=1800.02	Signal TT=660.294	SAT (TO) f=2084 TT=1800.03 T1=589.84	? (TO) TT=1796.24	SAT f=390 TT=1783.01 T1=1783.33	? (exit code) TT=266.7	Signal TT=1575.95	SAT (TO) f=203 TT=1800.05 T1=0.78	OPT f=12 TT=1242.19 T1=1242.29	OPT f=12 TT=20.71 T1=20.57	SAT (TO) f=203 TT=1800.57 T1=1.65	SAT f=15 TT=1789.84 T1=1051.91	SAT (TO) f=203 TT=1800.02 T1=1.14	SAT f=15 TT=1796.91 T1=1000.33	SAT (TO) f=2084 TT=1800.06 T1=1800.61	? (TO) TT=1800.11	? (TO) TT=1800.1	SAT (TO) f=737 TT=1800.52 T1=900.31	SAT (TO) f=718 TT=1800.09 T1=900.31	SAT (TO) f=203 TT=1800.66 T1=1.18	SAT (TO) f=203 TT=1800.09 T1=0.63	SAT (TO) f=765 TT=1800.08 T1=0.69	SAT f=17 TT=1796.82 T1=537.71	SAT f=17 TT=1796.86 T1=536.83	SAT f=17 TT=1796.85 T1=520.33	? (TO) TT=1800.04	? (TO) TT=1800.06	? TT=1799.51	? TT=1799.53

Statistics		bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
Number of times the solver is able to give the best known answer		6	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	12	0	9	0	9	0	0	0	0	0	0	0	0	10	10	10	0	0	0	0
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	12	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0