PB'12 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory MPS to OPB

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory MPS to OPB

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
bsolo 3.2 (complete)
clasp 2.0.6-R5325 (opt) (complete)
npSolver 1.0 (complete)
npSolver 1.0 (fixed) (complete)
npSolver inc (complete)
npSolver inc (fixed) (complete)
npSolver inc-topDown (complete)
npSolver inc-topDown (fixed) (complete)
npSolver inc-topdown-quickBound (complete)
npSolver inc-topdown-quickBound (fixed) (complete)
PB07: bsolo 3.0.17 (complete)
PB07: minisat+ 1.14 (complete)
PB07: PB-clasp 2007-04-10 (complete)
PB07: Pueblo 1.4 (incomplete)
PB07: SAT4JPseudoResolution 2007-03-23 (complete)
PB09: bsolo 3.1 (complete)
PB09: SAT4J Pseudo Resolution 2.1.1 (complete)
PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)
PB10: pb_cplex 2010-06-29 (complete)
PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)
PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
PB11: Sat4j Res//CP 2.3.0 (complete)
PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)
PB12: minisatp 1.0-2-g022594c (complete)
pb2sat 2012-05-19 (complete)
pb2satCp2 2012-05-19 (complete)
pwbo 2.0 (complete)
pwbo 2.02 (complete)
SAT 4j PB RES // CP 2.3.2 Snapshot (complete)
Sat 4j PB Resolution 2.3.2 Snapshot (complete)
SAT4J PB specific settings 2.3.2 snapshot (complete)
SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)
SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)
SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)
toysat 2012-05-17 (complete)
toysat 2012-06-01 (complete)
wbo 1.7 (complete)
wbo 1.72 (complete)

Bench name	Best results on this instance	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-fast0507.opb	OPT f=174 TT=1166.64 T1=663.2	SAT f=215 TT=1798.02 T1=1675.57	? (TO) TT=1800	? (TO) TT=1800.13	? (TO) TT=1800.14	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.12	? (TO) TT=1801.88	? (TO) TT=1800.11	SAT (TO) f=176 TT=1800.1 T1=1187.55	Signal TT=3.631	SAT (TO) f=340 TT=1802.09 T1=195.47	SAT f=318 TT=1783.02 T1=73.76	SAT (TO) f=259 TT=1803.9 T1=7.53	SAT f=215 TT=1798.03 T1=1715.43	SAT (TO) f=259 TT=1800.1 T1=5.12	SAT f=175 TT=1794.25 T1=616.98	OPT f=174 TT=1166.64 T1=663.2	SAT (TO) f=253 TT=1800.84 T1=158.41	SAT f=176 TT=1790.12 T1=15.27	SAT (TO) f=253 TT=1800.74 T1=199.75	SAT f=174 TT=1797.27 T1=1139.8	No Cert. TT=1800.03	? (TO) TT=1800.15	? (TO) TT=1800.11	SAT (TO) f=307 TT=1800.49 T1=900.41	SAT (TO) f=307 TT=1800.59 T1=900.51	SAT (TO) f=259 TT=1800.14 T1=97.4	SAT (TO) f=259 TT=1801.27 T1=1.35	SAT (TO) f=363 TT=1800.1 T1=1.44	SAT f=176 TT=1797.21 T1=388.49	SAT f=176 TT=1797.22 T1=382.85	SAT f=176 TT=1797.09 T1=375.28	? (TO) TT=1800.01	? (TO) TT=1800.11	? TT=1799.89	? TT=1799.69
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-manna81.opb	OPT f=-13164 TT=75.406 T1=26.81	SAT f=-11464 TT=1798.04 T1=615.11	SAT (TO) f=-11688 TT=1800.07 T1=42.13	? (TO) TT=1800.14	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.07	Signal TT=366.985	SAT (TO) f=-8204 TT=1800.14 T1=0.61	? (TO) TT=1800.09	SAT f=-358 TT=1783.24 T1=1783.49	No Cert. TT=247.886	Signal TT=857.661	? (exit code) TT=192.8	OPT f=-13164 TT=189.716 T1=189.67	SAT f=-13164 TT=42.268 T1=26.81	SAT (TO) f=-11570 TT=1800.8 T1=31.97	OPT f=-13164 TT=75.406 T1=75.35	SAT (TO) f=-11530 TT=1800.05 T1=31.54	OPT f=-13164 TT=101.093 T1=101.04	SAT (TO) f=-8204 TT=1800.09 T1=1800.6	? (TO) TT=1800.09	? (TO) TT=1800.07	SAT (TO) f=-11525 TT=1800.39 T1=900.31	SAT (TO) f=-11525 TT=1800.48 T1=900.41	SAT (TO) f=-11521 TT=1800.13 T1=71.87	SAT (TO) f=-10909 TT=1800.09 T1=0.9	SAT (TO) f=-11493 TT=1800.02 T1=38.81	OPT f=-13164 TT=173.379 T1=173.32	OPT f=-13164 TT=172.777 T1=172.71	OPT f=-13164 TT=175.34 T1=175.28	? (TO) TT=1800.01	? (TO) TT=1800.05	? TT=1799.92	? TT=1799.99
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-mzzv11.opb	OPT f=-10859 TT=76.86 T1=68.69	SAT f=-8919 TT=1798.02 T1=1787.85	OPT f=-10859 TT=76.86 T1=76.27	? (TO) TT=1800.07	OPT f=-10859 TT=680.895 T1=602.67	? (TO) TT=1800.12	? (problem) TT=2.75	? (TO) TT=1800.08	? (problem) TT=2.75	? (TO) TT=1800.36	? (problem) TT=2.75	SAT (TO) f=-8089 TT=1800.07 T1=1651.68	SAT (TO) f=-8820 TT=1800.08 T1=1713.41	? (TO) TT=1800.07	SAT f=-5329 TT=1783.01 T1=1783.28	OPT f=-10859 TT=1725.16 T1=1753.86	SAT f=-8700 TT=1798.01 T1=1708.56	OPT f=-10859 TT=1370.57 T1=1234.32	SAT f=-10449 TT=1793.26 T1=1748.63	OPT f=-10859 TT=116.025 T1=68.69	SAT (TO) f=-10765 TT=1800.11 T1=819.18	? TT=1790.26	SAT (TO) f=-10405 TT=1800.06 T1=399.17	OPT f=-10859 TT=836.177 T1=648.64	SAT (TO) f=-6329 TT=1800.09 T1=1800.5	? (TO) TT=1800.05	? (TO) TT=1800.01	SAT (TO) f=-7944 TT=1800.01 T1=900.31	SAT (TO) f=-8136 TT=1800.17 T1=900.41	SAT (TO) f=-10674 TT=1800.71 T1=82.67	OPT f=-10859 TT=98.23 T1=93.89	SAT (TO) f=-9050 TT=1800.03 T1=1695.76	OPT f=-10859 TT=981.397 T1=927.61	OPT f=-10859 TT=992.414 T1=936.78	OPT f=-10859 TT=687.9 T1=585.43	? (TO) TT=1800.11	? (TO) TT=1800.01	? TT=1799.98	? TT=1799.96
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-mzzv42z.opb	OPT f=-10270 TT=26.399 T1=18.18	SAT f=-9476 TT=1798.1 T1=1655.87	OPT f=-10270 TT=26.399 T1=18.18	? (TO) TT=1800.06	OPT f=-10270 TT=239.96 T1=196.64	? (TO) TT=1800.02	? (problem) TT=3.18	? (TO) TT=1800.07	? (problem) TT=3.17	? (TO) TT=1800.09	? (problem) TT=3	SAT (TO) f=-9340 TT=1800.04 T1=1787.38	SAT (TO) f=-9830 TT=1800.07 T1=1765.47	? (TO) TT=1800.05	SAT f=-9215 TT=1783.01 T1=1783.29	OPT f=-10270 TT=305.47 T1=301.68	SAT f=-9476 TT=1798.03 T1=1761.97	OPT f=-10270 TT=208.901 T1=195.24	OPT f=-10270 TT=1079.33 T1=1079.37	OPT f=-10270 TT=39.987 T1=27.16	OPT f=-10270 TT=355.549 T1=199.36	SAT f=-10130 TT=1789.86 T1=1754.17	OPT f=-10270 TT=137.695 T1=65	OPT f=-10270 TT=265.669 T1=265.57	SAT (TO) f=-9610 TT=1800.02 T1=1800.4	OPT f=-10270 TT=1381.59 T1=1211.6	? (TO) TT=1800.09	SAT (TO) f=-9410 TT=1800.18 T1=900.41	SAT (TO) f=-9467 TT=1800.08 T1=900.41	SAT (TO) f=-10255 TT=1800.03 T1=110.01	OPT f=-10270 TT=104.88 T1=83.87	SAT (TO) f=-9570 TT=1800.03 T1=1754.73	SAT f=-10250 TT=1796.88 T1=1675.37	SAT f=-10250 TT=1796.85 T1=1658.92	SAT f=-10250 TT=1796.85 T1=1651.25	? (TO) TT=1800.09	? (TO) TT=1800.02	? TT=1799.53	? TT=1799.54
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-net12.opb	OPT f=214 TT=6.829 T1=2.28	OPT f=214 TT=7.467 T1=2.28	OPT f=214 TT=6.829 T1=3.46	? (problem) TT=789.363	? (TO) TT=1800.04	? (problem) TT=774.191	? (TO) TT=1800	? (problem) TT=775.063	? (TO) TT=1800.04	? (problem) TT=760.826	? (TO) TT=1800.03	OPT f=214 TT=7.061 T1=2.3	? (TO) TT=1800.04	SAT (TO) f=214 TT=1802.09 T1=1788.15	OPT f=214 TT=22.573 T1=16.79	OPT f=214 TT=15.512 T1=11.65	OPT f=214 TT=9.357 T1=2.08	OPT f=214 TT=17.56 T1=12.79	? TT=1797.39	? (TO) TT=1800.2	OPT f=214 TT=17.682 T1=4.23	? TT=1790.03	OPT f=214 TT=10.473 T1=5	? TT=1797.09	? (TO) TT=1800.11	? (problem) TT=633.335	? (problem) TT=636.477	OPT f=214 TT=7.626 T1=4.1	OPT f=214 TT=10.484 T1=5.53	OPT f=214 TT=25.26 T1=11.84	OPT f=214 TT=9.622 T1=5.47	OPT f=214 TT=9.984 T1=6.27	? TT=1797.09	? TT=1797.13	? TT=1797.29	OPT f=214 TT=46.409 T1=29.16	OPT f=214 TT=42.362 T1=26.93	? TT=1799.53	? TT=1799.52
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-p2756.opb	OPT f=3124 TT=1.663 T1=1.2	SAT f=3694 TT=1798.09 T1=438	SAT (TO) f=3731 TT=1800.02 T1=1792.29	? (TO) TT=1800.01	? (problem) TT=512.51	? (TO) TT=1800.01	? (problem) TT=566.84	? (TO) TT=1800.04	? (problem) TT=434.48	? (TO) TT=1800.08	? (problem) TT=6.92	SAT (TO) f=5407 TT=1800.1 T1=310.41	SAT (TO) f=64559 TT=1800.02 T1=295.02	SAT (TO) f=4138 TT=1802.06 T1=1640.5	SAT f=17727 TT=1783.01 T1=1.71	SAT (TO) f=3842 TT=1800.46 T1=176.69	SAT f=3925 TT=1798.04 T1=1245.8	SAT (TO) f=4102 TT=1800.85 T1=577.62	OPT f=3124 TT=2 T1=1.98	OPT f=3124 TT=1.663 T1=1.2	SAT (TO) f=3853 TT=1800.82 T1=804.69	OPT f=3124 TT=2.24 T1=2.22	SAT (TO) f=3703 TT=1800.21 T1=876	OPT f=3124 TT=1.881 T1=1.86	SAT (TO) f=78115 TT=1800.08 T1=1800.61	? (TO) TT=1800.01	? (TO) TT=1800.09	SAT (TO) f=3635 TT=1800.16 T1=900.31	SAT (TO) f=3618 TT=1800.13 T1=900.31	SAT (TO) f=3831 TT=1800.44 T1=1061.88	SAT (TO) f=3968 TT=1800.09 T1=1729.01	SAT (TO) f=5745 TT=1800.05 T1=1707.22	OPT f=3124 TT=2.158 T1=2.14	OPT f=3124 TT=2.393 T1=2.37	OPT f=3124 TT=2.321 T1=2.3	? (TO) TT=1800.02	? (TO) TT=1800.03	? TT=1799.59	? TT=1799.52
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-protfold.opb	SAT f=-28 TT=1800.03 T1=1760.37	SAT f=-23 TT=1798.02 T1=1206.14	SAT (TO) f=-28 TT=1800.03 T1=1760.37	? (TO) TT=1800.13	? (TO) TT=1800.07	? (TO) TT=1800.11	? (TO) TT=1800.11	? (TO) TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.06	SAT (TO) f=-23 TT=1800.1 T1=517.06	SAT (TO) f=-24 TT=1800.11 T1=126.97	? (TO) TT=1800.03	SAT f=-21 TT=1783.01 T1=1783.29	SAT (TO) f=-24 TT=1800.02 T1=1148.5	SAT f=-22 TT=1798.07 T1=1.76	SAT (TO) f=-25 TT=1801.08 T1=745.31	? TT=1797.09	? (TO) TT=1800.22	SAT (TO) f=-25 TT=1800.66 T1=350.2	SAT f=-19 TT=1789.75 T1=1783.9	SAT (TO) f=-25 TT=1800.19 T1=744.55	? TT=1796.81	SAT (TO) f=-24 TT=1800.08 T1=1800.5	? (TO) TT=1800.05	? (TO) TT=1800.11	SAT (TO) f=-25 TT=1800.54 T1=900.41	SAT (TO) f=-24 TT=1800.09 T1=900.31	SAT (TO) f=-24 TT=1801.73 T1=916.83	SAT (TO) f=-24 TT=1800.01 T1=465.6	SAT (TO) f=-26 TT=1800.62 T1=1130.36	SAT f=-18 TT=1796.74 T1=155.06	SAT f=-18 TT=1796.76 T1=155.21	? TT=1796.76	? (TO) TT=1800.11	? (TO) TT=1800.07	? TT=1799.54	? TT=1799.48
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-seymour.opb	SAT f=425 TT=1789.74 T1=215.71	SAT f=461 TT=1798.02 T1=28.78	SAT (TO) f=483 TT=1800.11 T1=564.15	? (TO) TT=1800.13	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) TT=1800.14	? (TO) TT=1800.14	? (TO) TT=1800.09	? (TO) TT=1800.07	SAT (TO) f=434 TT=1800.02 T1=1533.03	SAT (TO) f=466 TT=1800.05 T1=0.17	SAT (TO) f=481 TT=1802.07 T1=0.48	SAT f=492 TT=1783.01 T1=64.73	? (exit code) TT=703.53	SAT f=461 TT=1798.06 T1=28.83	SAT (TO) f=490 TT=1800 T1=0.39	SAT f=425 TT=1797.99 T1=710.98	? (TO) TT=1800.02	SAT (TO) f=481 TT=1800.89 T1=820.34	SAT f=425 TT=1789.74 T1=461	SAT (TO) f=481 TT=1800.18 T1=962.22	SAT f=425 TT=1796.74 T1=1537.51	SAT (TO) f=466 TT=1800.06 T1=1800.4	? (TO) TT=1800.1	? (TO) TT=1800.09	SAT (TO) f=464 TT=1800.38 T1=900.31	SAT (TO) f=465 TT=1800.02 T1=900.31	SAT (TO) f=481 TT=1800.55 T1=317.79	SAT (TO) f=490 TT=1800.06 T1=0.36	SAT (TO) f=484 TT=1800.1 T1=0.37	SAT f=425 TT=1796.76 T1=218.48	SAT f=425 TT=1796.76 T1=215.71	SAT f=426 TT=1796.75 T1=471.81	? (TO) TT=1800.1	? (TO) TT=1800.09	? TT=1799.49	? TT=1799.47
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib3/ normalized-mps-v2-20-10-mitre.opb	OPT f=7677 TT=5.933 T1=5.88	SAT f=8863 TT=1798.03 T1=13.56	SAT (TO) f=8933 TT=1800.09 T1=1708.25	? (problem) TT=103.642	? (TO) TT=1800.13	? (problem) TT=89.9	? (TO) TT=1800.04	? (problem) TT=90.27	? (TO) TT=1800.08	? (problem) TT=89.1	? (TO) TT=1800.07	Signal TT=1347.16	SAT (TO) f=9253 TT=1800.08 T1=1186.1	? (TO) TT=1694.13	SAT f=8008 TT=1783.01 T1=1783.32	No Cert. TT=831.797	SAT f=8967 TT=1798.08 T1=13.47	? (exit code) TT=451.056	OPT f=7677 TT=13.681 T1=13.63	OPT f=7677 TT=10.558 T1=9.61	SAT (TO) f=7680 TT=1800.11 T1=15.1	OPT f=7677 TT=16.121 T1=16.07	SAT (TO) f=7709 TT=1800.2 T1=12.24	OPT f=7677 TT=9.778 T1=9.7	SAT (TO) f=9101 TT=1800.03 T1=1800.5	? (TO) TT=1800.09	? (TO) TT=1800.05	SAT (TO) f=9280 TT=1800.8 T1=900.41	SAT (TO) f=9250 TT=1800.33 T1=900.41	SAT (TO) f=7767 TT=1800.11 T1=3.7	SAT (TO) f=7743 TT=1800.08 T1=2.39	SAT (TO) f=9648 TT=1800.74 T1=1712.6	OPT f=7677 TT=24.458 T1=24.4	OPT f=7677 TT=25.78 T1=25.72	OPT f=7677 TT=5.933 T1=5.88	? (TO) TT=1800.06	? (TO) TT=1800.07	? TT=1799.8	? (TO) TT=1800.43
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-bm23.opb	OPT f=34 TT=0.024 T1=0.02	OPT f=34 TT=0.298 T1=0.21	OPT f=34 TT=0.024 T1=0.02	OPT f=34 TT=4.741 T1=3.7	OPT f=34 TT=0.822 T1=0.82	OPT f=34 TT=3.391 T1=3.21	OPT f=34 TT=0.753 T1=0.59	OPT f=34 TT=1.683 T1=1.36	OPT f=34 TT=0.719 T1=0.68	OPT f=34 TT=1.802 T1=1.47	OPT f=34 TT=0.821 T1=0.78	OPT f=34 TT=0.624 T1=0.62	OPT f=34 TT=3.115 T1=2.65	OPT f=34 TT=0.091 T1=0.05	OPT f=34 TT=0.047 T1=0.03	OPT f=34 TT=0.644 T1=0.38	OPT f=34 TT=0.219 T1=0.13	OPT f=34 TT=0.834 T1=0.32	OPT f=34 TT=0.145 T1=0.09	OPT f=34 TT=0.026 T1=0.03	OPT f=34 TT=1.593 T1=3.13	OPT f=34 TT=0.226 T1=0.16	OPT f=34 TT=1.284 T1=1.65	OPT f=34 TT=0.164 T1=0.11	OPT f=34 TT=2.746 T1=2.74	OPT f=34 TT=1.739 T1=2.36	OPT f=34 TT=1.302 T1=0.79	OPT f=34 TT=0.842 T1=0.42	OPT f=34 TT=0.511 T1=0.25	OPT f=34 TT=1.643 T1=2.15	OPT f=34 TT=0.666 T1=0.28	OPT f=34 TT=0.844 T1=0.45	OPT f=34 TT=0.185 T1=0.13	OPT f=34 TT=0.213 T1=0.16	OPT f=34 TT=0.147 T1=0.1	OPT f=34 TT=0.699 T1=0.42	OPT f=34 TT=0.72 T1=0.44	? TT=1799.58	? TT=1799.76
Bench name	Best results	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-cracpb1.opb	OPT f=22199 TT=0.203 T1=0.25	SAT f=56943 TT=1798 T1=372.86	SAT (TO) f=26372 TT=1800.03 T1=717.04	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.08	? (TO) TT=1800.01	? (TO) TT=1800.06	OPT f=22199 TT=0.52 T1=0.5	SAT (TO) f=57945 TT=1800.05 T1=1186	SAT (TO) f=43373 TT=1800.09 T1=896.1	SAT f=118356 TT=1783 T1=1069.42	SAT (TO) f=50958 TT=1800.59 T1=4.54	SAT f=47122 TT=1798.01 T1=870.62	? (exit code) TT=934.72	OPT f=22199 TT=0.172 T1=0.16	OPT f=22199 TT=0.203 T1=0.84	SAT (TO) f=45222 TT=1800.52 T1=395.49	OPT f=22199 TT=0.264 T1=0.25	SAT (TO) f=40666 TT=1800.88 T1=94.01	OPT f=22199 TT=0.347 T1=0.34	SAT (TO) f=44375 TT=1800.02 T1=1800.4	? (TO) TT=1800.06	? (TO) TT=1800.08	SAT (TO) f=82343 TT=1800.09 T1=900.31	SAT (TO) f=103143 TT=1800.12 T1=900.31	SAT (TO) f=40027 TT=1800.9 T1=11.91	SAT (TO) f=69260 TT=1800.95 T1=13.62	SAT (TO) f=35997 TT=1800.04 T1=780.66	OPT f=22199 TT=0.597 T1=0.58	OPT f=22199 TT=0.706 T1=0.69	OPT f=22199 TT=0.342 T1=0.33	? (TO) TT=1800.02	? (TO) TT=1800.02	? TT=1799.85	? TT=1799.84
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-diamond.opb	UNSAT TT=0	UNSAT TT=0.001	UNSAT TT=0.003	UNSAT TT=0.019	UNSAT TT=0.03	UNSAT TT=0.01	UNSAT TT=0.018	UNSAT TT=0.008	UNSAT TT=0.017	UNSAT TT=0.01	UNSAT TT=0.015	UNSAT TT=0.001	UNSAT TT=0	? (TO) TT=1800.05	UNSAT TT=0.001	UNSAT TT=0.115	UNSAT TT=0.001	UNSAT TT=0.123	UNSAT TT=0.009	UNSAT TT=0.003	UNSAT TT=0.155	UNSAT TT=0.011	UNSAT TT=0.156	UNSAT TT=0.013	UNSAT TT=0.003	UNSAT TT=0.005	UNSAT TT=0.004	UNSAT TT=0.002	UNSAT TT=0.002	UNSAT TT=0.154	UNSAT TT=0.145	UNSAT TT=0.165	UNSAT TT=0.009	UNSAT TT=0.01	UNSAT TT=0.01	UNSAT TT=0.002	UNSAT TT=0.002	UNSAT TT=0.002	UNSAT TT=0.001
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-l152lav.opb	OPT f=4722 TT=3.049 T1=1.93	SAT f=4891 TT=1798.02 T1=1736.09	? (TO) TT=1800.02	? (TO) TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.1	? (TO) TT=1800	? (TO) TT=1800.07	? (TO) TT=1800.01	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1802.1	? (TO) TT=1800.02	? (exit code) TT=743.98	SAT f=4745 TT=1798 T1=1537.93	? (TO) TT=1800.47	OPT f=4722 TT=5.005 T1=4.71	OPT f=4722 TT=3.049 T1=1.93	? (TO) TT=1801.32	OPT f=4722 TT=7.266 T1=7.19	? (TO) TT=1800.68	OPT f=4722 TT=4.77 T1=4.28	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.03	? (TO) TT=1800.19	? (TO) TT=1800.1	? (TO) TT=1800.2	? (TO) TT=1800.02	? (TO) TT=1800.34	OPT f=4722 TT=6.051 T1=5.77	OPT f=4722 TT=5.915 T1=5.65	OPT f=4722 TT=3.365 T1=2.89	? (TO) TT=1800.02	? (TO) TT=1800.02	? TT=1799.49	? TT=1799.33
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-lp4l.opb	OPT f=2967 TT=0.209 T1=0.18	SAT f=3096 TT=1798.02 T1=363.34	SAT (TO) f=3788 TT=1800.11 T1=325.15	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.06	OPT f=2967 TT=3.071 T1=0.58	SAT (TO) f=3659 TT=1800.11 T1=1156.8	SAT (TO) f=3696 TT=1902.5 T1=366.35	SAT f=4224 TT=1783.01 T1=1603.75	SAT f=3964 TT=491.42 T1=84.19	SAT f=3097 TT=1798.02 T1=294.2	SAT (TO) f=4062 TT=1800.09 T1=272.44	OPT f=2967 TT=0.236 T1=0.2	OPT f=2967 TT=0.316 T1=0.31	SAT (TO) f=3647 TT=1800.63 T1=387.6	OPT f=2967 TT=0.322 T1=0.28	SAT (TO) f=3979 TT=1800.17 T1=233.38	OPT f=2967 TT=0.332 T1=0.3	SAT (TO) f=3722 TT=1800.07 T1=1800.5	? (TO) TT=1800.13	? (TO) TT=1800.11	SAT (TO) f=4000 TT=1800.05 T1=900.61	SAT (TO) f=4017 TT=1800.1 T1=900.31	SAT (TO) f=3859 TT=1800.53 T1=806.65	SAT (TO) f=4018 TT=1800.05 T1=0.4	SAT (TO) f=3700 TT=1800.05 T1=1170.43	OPT f=2967 TT=0.625 T1=0.59	OPT f=2967 TT=0.629 T1=0.59	OPT f=2967 TT=0.209 T1=0.18	? (TO) TT=1800.02	? (TO) TT=1800.03	? TT=1799.57	? TT=1799.54
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-lseu.opb	OPT f=1120 TT=0.171 T1=0.08	OPT f=1120 TT=217.281 T1=189.96	OPT f=1120 TT=7.404 T1=5.8	OPT f=1120 TT=636.511 T1=398.94	OPT f=1120 TT=593.847 T1=420.56	OPT f=1120 TT=473.639 T1=311.13	OPT f=1120 TT=358.642 T1=347.35	OPT f=1120 TT=465.22 T1=328.98	OPT f=1120 TT=739.306 T1=724.74	? (TO) TT=1800.29	? (TO) TT=1800.39	OPT f=1120 TT=18.455 T1=5.51	OPT f=1120 TT=63.342 T1=54.79	OPT f=1120 TT=133.456 T1=71.64	OPT f=1120 TT=270.587 T1=229.31	OPT f=1120 TT=130.77 T1=36.64	OPT f=1120 TT=214.632 T1=163.23	OPT f=1120 TT=289.67 T1=120.56	OPT f=1120 TT=0.264 T1=0.23	OPT f=1120 TT=0.171 T1=0.08	OPT f=1120 TT=408.702 T1=16.13	OPT f=1120 TT=0.396 T1=0.32	OPT f=1120 TT=522.317 T1=58.57	OPT f=1120 TT=0.307 T1=0.25	OPT f=1120 TT=50.195 T1=50.2	OPT f=1120 TT=807.457 T1=157.47	OPT f=1120 TT=181.61 T1=64.16	OPT f=1120 TT=203.34 T1=101.71	OPT f=1120 TT=141.133 T1=70.57	OPT f=1120 TT=41.719 T1=13.17	OPT f=1120 TT=17.502 T1=4.57	OPT f=1120 TT=28.97 T1=16.09	OPT f=1120 TT=0.391 T1=0.3	OPT f=1120 TT=0.399 T1=0.3	OPT f=1120 TT=0.37 T1=0.26	OPT f=1120 TT=187.278 T1=171.37	OPT f=1120 TT=184.038 T1=168.39	? TT=1799.66	? TT=1799.97
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-mod008.opb	OPT f=307 TT=0.306 T1=0.03	OPT f=307 TT=1378.3 T1=1214.56	SAT (TO) f=307 TT=1800.03 T1=4.35	? (TO) TT=1800.07	? (TO) TT=1800.12	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.31	? (TO) TT=1800.4	OPT f=307 TT=9.424 T1=0.01	SAT (TO) f=333 TT=1800.04 T1=1471.64	SAT (TO) f=310 TT=1800.14 T1=248.43	SAT f=602 TT=1783.01 T1=0.02	SAT (TO) f=313 TT=1800.04 T1=1186.59	OPT f=307 TT=410.366 T1=191.17	SAT (TO) f=307 TT=1800.57 T1=359.14	OPT f=307 TT=0.415 T1=0.25	OPT f=307 TT=0.306 T1=0.18	OPT f=307 TT=370.47 T1=8.19	OPT f=307 TT=0.843 T1=0.06	OPT f=307 TT=836.866 T1=14.73	OPT f=307 TT=0.736 T1=0.05	SAT (TO) f=356 TT=1800.01 T1=1800.4	? (TO) TT=1800.11	? (TO) TT=1800.12	SAT (TO) f=313 TT=1800.03 T1=900.21	SAT (TO) f=309 TT=1800.04 T1=900.21	OPT f=307 TT=1293.12 T1=2.76	SAT (TO) f=307 TT=1800.88 T1=1281.21	SAT (TO) f=327 TT=1800.04 T1=1728.58	OPT f=307 TT=0.712 T1=0.14	OPT f=307 TT=0.893 T1=0.33	OPT f=307 TT=0.59 T1=0.03	? (TO) TT=1800.03	? (TO) TT=1800.03	? TT=1800	? TT=1799.65
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-mod010.opb	OPT f=6548 TT=0.592 T1=0.48	SAT (TO) f=6576 TT=1800.12 T1=1780.61	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.08	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1802.07	? (TO) TT=1800.02	? (exit code) TT=746.34	? TT=1798.02	? (TO) TT=1800.86	OPT f=6548 TT=1.469 T1=1.38	OPT f=6548 TT=1.621 T1=1.61	SAT (TO) f=7901 TT=1800.62 T1=757.07	OPT f=6548 TT=1.602 T1=1.59	SAT (TO) f=7901 TT=1800.05 T1=795.67	OPT f=6548 TT=1.162 T1=1.14	? (TO) TT=1800.08	? (TO) TT=1800.04	? (TO) TT=1800.01	? (TO) TT=1800.42	? (TO) TT=1800.08	SAT (TO) f=7984 TT=1800.08 T1=897.88	? (TO) TT=1800.07	? (TO) TT=1800.08	OPT f=6548 TT=2.455 T1=2.44	OPT f=6548 TT=2.681 T1=2.66	OPT f=6548 TT=0.592 T1=0.48	? (TO) TT=1800.11	? (TO) TT=1800.1	? TT=1799.77	? TT=1799.55
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0040.opb	OPT f=62027 TT=0.002 T1=0	OPT f=62027 TT=0.026 T1=0.02	OPT f=62027 TT=9.862 T1=0	OPT f=62027 TT=1.904 T1=1.58	OPT f=62027 TT=1.067 T1=1.07	OPT f=62027 TT=1.786 T1=1.67	OPT f=62027 TT=0.996 T1=1	OPT f=62027 TT=1.243 T1=1.13	OPT f=62027 TT=0.672 T1=0.56	OPT f=62027 TT=1.281 T1=1.16	OPT f=62027 TT=0.688 T1=0.57	OPT f=62027 TT=0.033 T1=0.03	OPT f=62027 TT=35.513 T1=33.9	OPT f=62027 TT=6.037 T1=2.65	OPT f=62027 TT=4.848 T1=4.81	OPT f=62027 TT=16.681 T1=0.14	OPT f=62027 TT=0.025 T1=0.02	OPT f=62027 TT=20.84 T1=0.15	OPT f=62027 TT=0.017 T1=0.01	OPT f=62027 TT=0.005 T1=0	OPT f=62027 TT=0.795 T1=3.13	OPT f=62027 TT=0.039 T1=0.03	OPT f=62027 TT=0.875 T1=1.66	OPT f=62027 TT=0.035 T1=0.03	OPT f=62027 TT=5.194 T1=5.19	OPT f=62027 TT=1.814 T1=1.52	OPT f=62027 TT=1.301 T1=1.17	OPT f=62027 TT=0.002 T1=0	OPT f=62027 TT=0.002 T1=0	OPT f=62027 TT=0.843 T1=1.63	OPT f=62027 TT=7.076 T1=0.15	OPT f=62027 TT=9.113 T1=4.86	OPT f=62027 TT=0.06 T1=0.05	OPT f=62027 TT=0.089 T1=0.08	OPT f=62027 TT=0.02 T1=0.01	OPT f=62027 TT=19.36 T1=0	OPT f=62027 TT=19.984 T1=0	OPT f=62027 TT=0.002 T1=0	OPT f=62027 TT=0.002 T1=0
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0201.opb	OPT f=1523 TT=0.201 T1=0.13	OPT f=1523 TT=18.19 T1=16.36	SAT (TO) f=1523 TT=1800.03 T1=71.15	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.11	OPT f=1523 TT=4.096 T1=1.41	OPT f=1523 TT=1564.12 T1=1250.46	OPT f=1523 TT=1470.08 T1=259.36	SAT f=1561 TT=1783.01 T1=1648.32	SAT (TO) f=1523 TT=1800.06 T1=122.65	OPT f=1523 TT=20.457 T1=18.05	SAT (TO) f=1523 TT=1800.78 T1=270.42	OPT f=1523 TT=1.683 T1=0.84	OPT f=1523 TT=0.201 T1=0.13	SAT (TO) f=1523 TT=1801.15 T1=6.2	OPT f=1523 TT=1.578 T1=1.24	SAT (TO) f=1523 TT=1800.03 T1=2.22	OPT f=1523 TT=1.428 T1=0.84	OPT f=1523 TT=764.23 T1=764.37	? (TO) TT=1800.1	? (TO) TT=1800.06	SAT (TO) f=1523 TT=1800.53 T1=900.31	SAT (TO) f=1533 TT=1800.07 T1=900.21	OPT f=1523 TT=1424.17 T1=56.93	OPT f=1523 TT=593.28 T1=109.22	SAT (TO) f=1523 TT=1800.55 T1=266.72	OPT f=1523 TT=0.987 T1=0.82	OPT f=1523 TT=1.128 T1=0.94	OPT f=1523 TT=1.712 T1=1.33	? (TO) TT=1800.02	? (TO) TT=1800.02	? TT=1799.6	? TT=1799.56
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-sentoy.opb	OPT f=-7772 TT=0.048 T1=0.02	OPT f=-7772 TT=36.978 T1=36.96	OPT f=-7772 TT=0.672 T1=0.48	? (problem) TT=33.002	? (TO) TT=1800.12	? (problem) TT=28.48	? (TO) TT=1800.08	? (problem) TT=28.59	? (TO) TT=1800.09	? (problem) TT=29.22	? (TO) TT=1800.1	OPT f=-7772 TT=1.023 T1=0.68	SAT (TO) f=-6609 TT=1800.1 T1=1741.63	? (TO) TT=1800.12	SAT f=-3883 TT=1783.01 T1=1783.28	OPT f=-7772 TT=6.331 T1=3.38	OPT f=-7772 TT=28.246 T1=28.22	OPT f=-7772 TT=6.274 T1=3.47	OPT f=-7772 TT=0.35 T1=0.34	OPT f=-7772 TT=0.048 T1=0.05	OPT f=-7772 TT=2.914 T1=4.18	OPT f=-7772 TT=0.48 T1=0.17	OPT f=-7772 TT=3.167 T1=2.22	OPT f=-7772 TT=0.324 T1=0.04	SAT (TO) f=-6193 TT=1800.09 T1=1800.4	? (TO) TT=1800.03	? (TO) TT=1800.07	OPT f=-7772 TT=7.642 T1=3.81	OPT f=-7772 TT=6.762 T1=3.38	OPT f=-7772 TT=3.066 T1=2.7	OPT f=-7772 TT=2.195 T1=0.55	OPT f=-7772 TT=4.704 T1=2.88	OPT f=-7772 TT=0.427 T1=0.42	OPT f=-7772 TT=0.44 T1=0.43	OPT f=-7772 TT=0.316 T1=0.02	OPT f=-7772 TT=38.902 T1=12.59	OPT f=-7772 TT=38.16 T1=12.36	? TT=1799.56	? TT=1799.97
Bench name	Best results	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein15.opb	OPT f=9 TT=0.001 T1=0	OPT f=9 TT=0.021 T1=0	OPT f=9 TT=0.003 T1=0	OPT f=9 TT=0.057 T1=0.02	OPT f=9 TT=0.068 T1=0.03	OPT f=9 TT=0.022 T1=0.01	OPT f=9 TT=0.03 T1=0.02	OPT f=9 TT=0.015 T1=0.01	OPT f=9 TT=0.026 T1=0.02	OPT f=9 TT=0.019 T1=0.01	OPT f=9 TT=0.024 T1=0.02	OPT f=9 TT=0.023 T1=0	OPT f=9 TT=0.005 T1=0	OPT f=9 TT=0.029 T1=0.01	OPT f=9 TT=0.005 T1=0	OPT f=9 TT=0.223 T1=0.12	OPT f=9 TT=0.021 T1=0	OPT f=9 TT=0.263 T1=0.11	OPT f=9 TT=0.056 T1=0.01	OPT f=9 TT=0.018 T1=0.02	OPT f=9 TT=0.483 T1=1.11	OPT f=9 TT=0.085 T1=0.03	OPT f=9 TT=0.445 T1=0.64	OPT f=9 TT=0.067 T1=0.02	OPT f=9 TT=0.003 T1=0	OPT f=9 TT=0.022 T1=0	OPT f=9 TT=0.016 T1=0	OPT f=9 TT=0.002 T1=0	OPT f=9 TT=0.001 T1=0	OPT f=9 TT=0.415 T1=0.63	OPT f=9 TT=0.234 T1=0.12	OPT f=9 TT=0.234 T1=0.12	OPT f=9 TT=0.1 T1=0.03	OPT f=9 TT=0.132 T1=0.05	OPT f=9 TT=0.079 T1=0.01	OPT f=9 TT=0.012 T1=0	OPT f=9 TT=0.012 T1=0	OPT f=9 TT=0.442 T1=0.43	OPT f=9 TT=0.442 T1=0.43
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein27.opb	OPT f=18 TT=0.151 T1=0	OPT f=18 TT=2.015 T1=0	OPT f=18 TT=0.186 T1=0	OPT f=18 TT=0.216 T1=0.21	OPT f=18 TT=0.228 T1=0.22	OPT f=18 TT=0.175 T1=0.17	OPT f=18 TT=0.182 T1=0.18	OPT f=18 TT=0.152 T1=0.01	OPT f=18 TT=0.171 T1=0.02	OPT f=18 TT=0.158 T1=0.01	OPT f=18 TT=0.162 T1=0.02	OPT f=18 TT=1.724 T1=0	OPT f=18 TT=0.22 T1=0	OPT f=18 TT=0.199 T1=0.03	OPT f=18 TT=0.48 T1=0	OPT f=18 TT=2.044 T1=0.14	OPT f=18 TT=2.004 T1=0	OPT f=18 TT=2.154 T1=0.12	OPT f=18 TT=0.918 T1=0.09	OPT f=18 TT=0.358 T1=0.2	OPT f=18 TT=5.112 T1=1.12	OPT f=18 TT=1.556 T1=0.04	OPT f=18 TT=5.134 T1=0.65	OPT f=18 TT=0.966 T1=0.07	OPT f=18 TT=0.151 T1=0.15	OPT f=18 TT=0.17 T1=0.17	OPT f=18 TT=0.157 T1=0.01	OPT f=18 TT=0.942 T1=0.46	OPT f=18 TT=0.941 T1=0.46	OPT f=18 TT=4.453 T1=0.63	OPT f=18 TT=2.124 T1=0.13	OPT f=18 TT=2.393 T1=0.13	OPT f=18 TT=1.593 T1=0.21	OPT f=18 TT=1.359 T1=0.18	OPT f=18 TT=1.174 T1=0.18	OPT f=18 TT=0.954 T1=0	OPT f=18 TT=0.964 T1=0	? TT=1799.52	? TT=1799.83
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein45.opb	OPT f=30 TT=8.3 T1=0.01	OPT f=30 TT=159.573 T1=2.05	OPT f=30 TT=52.783 T1=0.05	OPT f=30 TT=11.808 T1=3.21	OPT f=30 TT=9.452 T1=9.45	OPT f=30 TT=11.679 T1=3.16	OPT f=30 TT=9.416 T1=9.42	OPT f=30 TT=8.561 T1=0.05	OPT f=30 TT=9.138 T1=0.2	OPT f=30 TT=9.042 T1=0.06	OPT f=30 TT=8.788 T1=0.06	OPT f=30 TT=78.921 T1=0.5	OPT f=30 TT=11.386 T1=0.21	OPT f=30 TT=36.657 T1=0.06	OPT f=30 TT=54.656 T1=0.01	SAT (TO) f=30 TT=1800.03 T1=0.22	OPT f=30 TT=159.515 T1=2.05	SAT (TO) f=30 TT=1800.62 T1=0.28	OPT f=30 TT=19.744 T1=1.69	OPT f=30 TT=12.532 T1=4.45	OPT f=30 TT=171.628 T1=4.16	OPT f=30 TT=20.375 T1=3.67	OPT f=30 TT=189.068 T1=2.18	OPT f=30 TT=16.424 T1=1.01	OPT f=30 TT=8.3 T1=8.29	OPT f=30 TT=11.327 T1=2.61	OPT f=30 TT=8.659 T1=0.08	OPT f=30 TT=203.024 T1=101.54	OPT f=30 TT=195.202 T1=97.61	OPT f=30 TT=868.463 T1=7.67	OPT f=30 TT=499.72 T1=2.11	OPT f=30 TT=313.176 T1=0.27	OPT f=30 TT=16.816 T1=1.27	OPT f=30 TT=15.585 T1=1.2	OPT f=30 TT=18.746 T1=2.49	OPT f=30 TT=252.75 T1=5.35	OPT f=30 TT=254.156 T1=5.38	? TT=1799.42	? TT=1799.76
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein9.opb	OPT f=5 TT=0.001 T1=0	OPT f=5 TT=0.004 T1=0	OPT f=5 TT=0.002 T1=0	OPT f=5 TT=0.041 T1=0.02	OPT f=5 TT=0.052 T1=0.03	OPT f=5 TT=0.018 T1=0.01	OPT f=5 TT=0.027 T1=0.02	OPT f=5 TT=0.013 T1=0.01	OPT f=5 TT=0.021 T1=0.02	OPT f=5 TT=0.013 T1=0.01	OPT f=5 TT=0.02 T1=0.01	OPT f=5 TT=0.003 T1=0	OPT f=5 TT=0.002 T1=0	OPT f=5 TT=0.027 T1=0.01	OPT f=5 TT=0.002 T1=0	OPT f=5 TT=0.144 T1=0.12	OPT f=5 TT=0.004 T1=0	OPT f=5 TT=0.142 T1=0.1	OPT f=5 TT=0.019 T1=0	OPT f=5 TT=0.009 T1=0.01	OPT f=5 TT=0.153 T1=1.11	OPT f=5 TT=0.036 T1=0.01	OPT f=5 TT=0.165 T1=0.63	OPT f=5 TT=0.046 T1=0.01	OPT f=5 TT=0.005 T1=0	OPT f=5 TT=0.015 T1=0	OPT f=5 TT=0.011 T1=0	OPT f=5 TT=0.002 T1=0	OPT f=5 TT=0.002 T1=0	OPT f=5 TT=0.163 T1=0.62	OPT f=5 TT=0.155 T1=0.11	OPT f=5 TT=0.175 T1=0.12	OPT f=5 TT=0.028 T1=0.02	OPT f=5 TT=0.036 T1=0.02	OPT f=5 TT=0.018 T1=0	OPT f=5 TT=0.002 T1=0	OPT f=5 TT=0.003 T1=0	OPT f=5 TT=0.001 T1=0	OPT f=5 TT=0.001 T1=0
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos10.opb	OPT f=-1135 TT=2.06 T1=1.13	OPT f=-1135 TT=8.119 T1=8.06	OPT f=-1135 TT=2.774 T1=2.13	OPT f=-1135 TT=202.222 T1=131.41	OPT f=-1135 TT=767.297 T1=767.84	? (problem) TT=12.3	? (problem) TT=29.34	? (problem) TT=10.75	? (problem) TT=29.08	? (problem) TT=10.69	? (problem) TT=29.18	OPT f=-1135 TT=9.1 T1=9.08	OPT f=-1135 TT=113.078 T1=107.63	? (TO) TT=1800.06	OPT f=-1135 TT=11.288 T1=11.28	OPT f=-1135 TT=5.543 T1=3.73	OPT f=-1135 TT=9.014 T1=8.95	OPT f=-1135 TT=6.703 T1=4.56	OPT f=-1135 TT=49.68 T1=49.57	OPT f=-1135 TT=18.829 T1=18.15	OPT f=-1135 TT=15.782 T1=29.12	OPT f=-1135 TT=25.392 T1=25.16	OPT f=-1135 TT=11.166 T1=13.09	OPT f=-1135 TT=29.866 T1=29.7	OPT f=-1135 TT=50.389 T1=50.4	OPT f=-1135 TT=185.555 T1=160.21	OPT f=-1135 TT=558.285 T1=549.11	OPT f=-1135 TT=2.486 T1=1.35	OPT f=-1135 TT=2.06 T1=1.13	OPT f=-1135 TT=35.758 T1=39.25	OPT f=-1135 TT=3.382 T1=1.9	OPT f=-1135 TT=4.776 T1=3.35	OPT f=-1135 TT=27.27 T1=27.1	OPT f=-1135 TT=27.038 T1=26.86	OPT f=-1135 TT=32.307 T1=32.17	OPT f=-1135 TT=22.486 T1=17.13	OPT f=-1135 TT=21.698 T1=16.45	? TT=1799.55	? TT=1799.55
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos16.opb	OPT f=114 TT=1076.07 T1=5.32	SAT f=118 TT=1798 T1=8.12	SAT (TO) f=114 TT=1800.03 T1=5.32	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) TT=1800.1	? (TO) TT=1800.13	? (TO) TT=1800.13	? (TO) TT=1800.12	? (TO) TT=1800.12	SAT (TO) f=119 TT=1800.11 T1=466.31	SAT (TO) f=115 TT=1800.02 T1=422.86	SAT (TO) f=114 TT=1802.05 T1=151.22	OPT f=114 TT=1076.07 T1=112.2	SAT (TO) f=114 TT=1800.59 T1=217.79	Wrong Opt. f=121 TT=81.402 T1=10.29	SAT (TO) f=114 TT=1800.07 T1=41.72	SAT (TO) f=115 TT=1800.23 T1=840.42	? (TO) TT=1800.32	SAT (TO) f=115 TT=1800.81 T1=861.84	? TT=1789.8	SAT (TO) f=116 TT=1800.15 T1=315.63	? TT=1797.23	SAT (TO) f=114 TT=1800.11 T1=1800.4	? (TO) TT=1800.06	? (TO) TT=1800.09	SAT (TO) f=116 TT=1800.5 T1=900.31	SAT (TO) f=116 TT=1800.15 T1=900.41	SAT (TO) f=114 TT=1800.71 T1=627.15	SAT (TO) f=114 TT=1800.58 T1=70.03	SAT (TO) f=114 TT=1800.01 T1=7.98	SAT f=114 TT=1797.17 T1=1676.29	SAT f=115 TT=1796.97 T1=1493.61	SAT f=114 TT=1797.01 T1=1442.01	? (TO) TT=1800.03	? (TO) TT=1800.03	? TT=1799.6	? TT=1800
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos19.opb	SAT f=-3 TT=1783.2 T1=189.69	SAT f=-3 TT=1798.17 T1=189.69	? (TO) TT=1800.07	? (MO) TT=535.72	? (problem) TT=164.986	? (MO) TT=533.72	? (problem) TT=158.752	? (MO) TT=536.81	? (problem) TT=159.968	? (MO) TT=547.01	? (problem) TT=160.82	Signal TT=141.798	Signal TT=182.188	? (TO) TT=1711.37	SAT f=-3 TT=1783.2 T1=1783.39	? (TO) TT=1800.09	SAT (TO) f=-3 TT=1800.27 T1=189.23	? (TO) TT=1800.03	? (TO) TT=1802.15	No Cert. TT=1800.1	? (TO) TT=1801.08	SAT f=0 TT=1794.03 T1=9.86	? (TO) TT=1800.13	? (TO) TT=1802.15	? (TO) TT=1800.09	? (problem) TT=504.252	? (problem) TT=506.771	SAT (TO) f=-1 TT=1801.33 T1=903.61	SAT (TO) f=-1 TT=1800.48 T1=903.81	? (TO) TT=1800.08	? (TO) TT=1800.07	SAT (TO) f=0 TT=1800.97 T1=9.22	SAT (TO) f=0 TT=1801.86 T1=7.75	SAT (TO) f=0 TT=1801.95 T1=9	SAT (TO) f=0 TT=1802.01 T1=9.56	? (exit code) TT=15.279	? (TO) TT=1800.11	? TT=1799.57	? TT=1799.86
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos1.opb	OPT f=19 TT=1.197 T1=0.67	SAT f=23 TT=1798 T1=473.26	SAT (TO) f=28 TT=1800.03 T1=913.5	OPT f=19 TT=143.066 T1=54.15	OPT f=19 TT=26.744 T1=20.27	OPT f=19 TT=140.497 T1=51.96	? (problem) TT=0.771	OPT f=19 TT=139.768 T1=105.6	? (problem) TT=0.78	OPT f=19 TT=135.032 T1=100.77	? (problem) TT=0.83	SAT (TO) f=26 TT=1800.08 T1=9.99	OPT f=19 TT=10.591 T1=7.41	SAT (TO) f=27 TT=1802.11 T1=5.96	SAT f=24 TT=1783 T1=1771.46	SAT (TO) f=24 TT=1800.08 T1=0.72	SAT f=25 TT=1798 T1=730.86	SAT (TO) f=24 TT=1800.25 T1=0.45	OPT f=19 TT=4.794 T1=4.76	OPT f=19 TT=2.517 T1=1.04	SAT (TO) f=21 TT=1801.39 T1=119.62	OPT f=19 TT=1.197 T1=1.17	SAT (TO) f=21 TT=1800.1 T1=603.73	OPT f=19 TT=2.789 T1=0.67	OPT f=19 TT=6.096 T1=6.09	OPT f=19 TT=147.451 T1=28.36	OPT f=19 TT=191.491 T1=138.32	OPT f=19 TT=65.019 T1=32.52	OPT f=19 TT=534.33 T1=267.2	SAT (TO) f=21 TT=1800.41 T1=2.39	SAT (TO) f=21 TT=1800.05 T1=1290.24	SAT (TO) f=28 TT=1800.02 T1=1461.6	OPT f=19 TT=6.229 T1=6.2	OPT f=19 TT=5.854 T1=5.83	OPT f=19 TT=5.344 T1=5.31	? (TO) TT=1800.08	? (TO) TT=1800.11	? TT=1799.91	? TT=1799.6
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos5.opb	SAT f=15360 TT=1352.86 T1=10.65	SAT f=17408 TT=1798.01 T1=0.17	SAT (TO) f=16634 TT=1800.03 T1=1771.91	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.92	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.12	SAT (TO) f=15681 TT=1800.01 T1=1098.66	SAT (TO) f=15872 TT=1800.1 T1=63.66	SAT (TO) f=16384 TT=1802.05 T1=102.49	SAT f=17408 TT=1783.01 T1=246.19	SAT (TO) f=17599 TT=1800.09 T1=782.65	SAT f=17280 TT=1798 T1=1020.85	SAT (TO) f=17393 TT=1800.09 T1=273.63	SAT (TO) f=15360 TT=1800.79 T1=30.22	SAT f=15360 TT=1352.86 T1=481.15	SAT (TO) f=16384 TT=1800.16 T1=840.95	SAT f=15360 TT=1790.22 T1=79.73	SAT (TO) f=16384 TT=1800.72 T1=133.01	SAT f=15360 TT=1797.89 T1=24.32	SAT (TO) f=15396 TT=1800.07 T1=1800.4	? (TO) TT=1800.1	? (TO) TT=1800.07	SAT (TO) f=16384 TT=1800.4 T1=900.21	SAT (TO) f=16384 TT=1800.39 T1=900.21	SAT (TO) f=16384 TT=1800.13 T1=436.53	SAT (TO) f=18543 TT=1800.02 T1=1647.81	SAT (TO) f=17396 TT=1800.51 T1=862.9	SAT f=15360 TT=1797.56 T1=11.63	SAT f=15360 TT=1797.62 T1=10.65	SAT f=15360 TT=1797.79 T1=14.64	? (TO) TT=1800.03	? (TO) TT=1800.03	? TT=1799.94	? TT=1799.98
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos8.opb	OPT f=-3719 TT=1.004 T1=0.9	OPT f=-3719 TT=8.308 T1=8.29	OPT f=-3719 TT=1.004 T1=0.9	OPT f=-3719 TT=127.848 T1=117.78	OPT f=-3719 TT=1131.13 T1=1031.17	? (problem) TT=14.29	? (problem) TT=21.59	? (problem) TT=12.46	? (problem) TT=19.2	? (problem) TT=13.01	? (problem) TT=20.07	OPT f=-3719 TT=8.386 T1=8.38	OPT f=-3719 TT=139.854 T1=139.2	? (TO) TT=1800.07	OPT f=-3719 TT=3.373 T1=3.37	OPT f=-3719 TT=4.724 T1=3.04	OPT f=-3719 TT=8.689 T1=8.68	OPT f=-3719 TT=4.613 T1=2.88	OPT f=-3719 TT=43.014 T1=42.91	OPT f=-3719 TT=35.785 T1=35.18	OPT f=-3719 TT=9.671 T1=10.25	OPT f=-3719 TT=13.709 T1=13.6	OPT f=-3719 TT=6.314 T1=6.65	OPT f=-3719 TT=13.47 T1=13.34	OPT f=-3719 TT=29.494 T1=29.45	OPT f=-3719 TT=160.53 T1=149.04	OPT f=-3719 TT=610.527 T1=606.63	OPT f=-3719 TT=1.546 T1=0.91	OPT f=-3719 TT=1.569 T1=0.92	OPT f=-3719 TT=7.113 T1=10.15	OPT f=-3719 TT=2.955 T1=1.7	OPT f=-3719 TT=5.752 T1=4.36	OPT f=-3719 TT=19.173 T1=19.04	OPT f=-3719 TT=18.008 T1=17.88	OPT f=-3719 TT=19.113 T1=18.99	OPT f=-3719 TT=14.56 T1=12.5	OPT f=-3719 TT=14.616 T1=12.6	? TT=1799.73	? TT=1799.44

Statistics		bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
Number of times the solver is able to give the best known answer		15	15	11	13	9	8	9	8	8	7	0	12	8	13	13	0	13	0	24	14	23	14	25	12	12	11	0	13	14	15	0	24	24	23	12	12	0	4
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	6	0	0	0	0	0	0	0	0	0	1	0	2	0	0	0	0	9	0	3	0	0	2	0	0	0	3	0	0	0	0	0	3	0	0	0	2