PB'12 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Queens Problems

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Queens Problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
bsolo 3.2 (complete)
clasp 2.0.6-R5325 (opt) (complete)
npSolver 1.0 (complete)
npSolver 1.0 (fixed) (complete)
npSolver inc (complete)
npSolver inc (fixed) (complete)
npSolver inc-topDown (complete)
npSolver inc-topDown (fixed) (complete)
npSolver inc-topdown-quickBound (complete)
npSolver inc-topdown-quickBound (fixed) (complete)
PB07: bsolo 3.0.17 (complete)
PB07: minisat+ 1.14 (complete)
PB07: PB-clasp 2007-04-10 (complete)
PB07: Pueblo 1.4 (incomplete)
PB07: SAT4JPseudoResolution 2007-03-23 (complete)
PB09: bsolo 3.1 (complete)
PB09: SAT4J Pseudo Resolution 2.1.1 (complete)
PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)
PB10: pb_cplex 2010-06-29 (complete)
PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)
PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
PB11: Sat4j Res//CP 2.3.0 (complete)
PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)
PB12: minisatp 1.0-2-g022594c (complete)
pb2sat 2012-05-19 (complete)
pb2satCp2 2012-05-19 (complete)
pwbo 2.0 (complete)
pwbo 2.02 (complete)
SAT 4j PB RES // CP 2.3.2 Snapshot (complete)
Sat 4j PB Resolution 2.3.2 Snapshot (complete)
SAT4J PB specific settings 2.3.2 snapshot (complete)
SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)
SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)
SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)
toysat 2012-05-17 (complete)
toysat 2012-06-01 (complete)
wbo 1.7 (complete)
wbo 1.72 (complete)

Bench name	Best results on this instance	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_0.opb.PB06.opb	SAT f=765 TT=1800.19 T1=3.94	SAT f=5634 TT=1798.02 T1=1598.77	SAT (TO) f=9934 TT=1800.09 T1=2.77	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.03	SAT (TO) f=5435 TT=1800.19 T1=1667.23	? (TO) TT=1798.13	? (TO) TT=1802	SAT f=5407 TT=1783.08 T1=881.06	? (exit code) TT=1495.27	SAT f=5600 TT=1798.08 T1=843.16	? (TO) TT=1800.04	? TT=1797.58	? (TO) TT=1800.09	SAT (TO) f=1303 TT=1800.99 T1=4.36	? TT=1792.92	SAT (TO) f=886 TT=1800.71 T1=3.87	? TT=1799.64	SAT (TO) f=4635 TT=1800.01 T1=1800.9	? (TO) TT=1800.12	? (TO) TT=1800.07	SAT (TO) f=5335 TT=1800.66 T1=901.01	SAT (TO) f=5335 TT=1800.3 T1=901.11	SAT (TO) f=765 TT=1800.19 T1=3.94	SAT (TO) f=788 TT=1800.05 T1=3.31	? (TO) TT=1800.56	? (TO) TT=1800.03	? TT=1799.35	? TT=1799.89	? (TO) TT=1800.07	? (TO) TT=1800.08	? TT=1800.04	? TT=1799.66
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_10.opb.PB06.opb	SAT f=668 TT=1800.2 T1=4.33	SAT f=5518 TT=1798.07 T1=1326.97	SAT (TO) f=9218 TT=1800.06 T1=3.74	? (TO) TT=1800.14	? (TO) TT=1800.14	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.1	? (TO) TT=1800.1	SAT (TO) f=5524 TT=1800.03 T1=1546.5	SAT (TO) f=5456 TT=1800.19 T1=1284	? (TO) TT=1802.04	Signal TT=1638.53	? TT=1080.18	SAT f=5684 TT=1798.03 T1=1793.03	? (TO) TT=1800.08	? TT=1798.58	? (TO) TT=1800.36	SAT (TO) f=668 TT=1800.2 T1=4.33	? TT=1791.72	SAT (TO) f=840 TT=1800.48 T1=3.99	? TT=1799.61	SAT (TO) f=5160 TT=1800.1 T1=1801.1	? (TO) TT=1800.04	? (TO) TT=1800.09	SAT (TO) f=5238 TT=1800.25 T1=901.01	SAT (TO) f=5238 TT=1800.61 T1=901.01	? (TO) TT=1802.63	SAT (TO) f=1234 TT=1800.01 T1=3.03	? (TO) TT=1800.05	? TT=1800	? (TO) TT=1802.04	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.08	? TT=1799.73	? TT=1799.56
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_11.opb.PB06.opb	SAT f=761 TT=1800.23 T1=3.89	SAT f=5402 TT=1798.01 T1=358.63	SAT (TO) f=9745 TT=1800.08 T1=3.7	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800	? (TO) TT=1800.07	SAT (TO) f=5657 TT=1800.2 T1=1773.03	Signal TT=1544.17	? (TO) TT=1801.93	Signal TT=1735.34	? TT=1291.45	SAT f=5626 TT=1798.07 T1=1323.45	? (TO) TT=1800.59	? TT=1797.8	? (TO) TT=1800.34	SAT (TO) f=833 TT=1800.02 T1=4.28	? TT=1791.79	SAT (TO) f=761 TT=1800.23 T1=3.89	? TT=1799.03	SAT (TO) f=6106 TT=1800.1 T1=1801.11	? (TO) TT=1800.12	? (TO) TT=1800	SAT (TO) f=5371 TT=1800.69 T1=901.11	SAT (TO) f=5371 TT=1800.18 T1=901.21	SAT (TO) f=887 TT=1800.05 T1=4.13	SAT (TO) f=818 TT=1800.61 T1=3.22	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.07	? TT=1799.53	? TT=1799.34
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_12.opb.PB06.opb	SAT f=448 TT=1793.17 T1=209.36	SAT f=5667 TT=1798.01 T1=1795.95	SAT (TO) f=9520 TT=1800.04 T1=2.98	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.1	SAT (TO) f=5602 TT=1800.12 T1=658.98	SAT (TO) f=5702 TT=1800.12 T1=1024.64	? (TO) TT=1802.01	? (TO) TT=1782.59	? (exit code) TT=1128.75	SAT f=5746 TT=1798.11 T1=175.27	? (TO) TT=1801.43	? TT=1798.45	? (TO) TT=1800.05	SAT (TO) f=488 TT=1800.8 T1=4.35	SAT f=448 TT=1793.17 T1=209.36	SAT (TO) f=488 TT=1800.02 T1=3.94	? TT=1799.09	SAT (TO) f=6032 TT=1800.02 T1=1800.3	? (TO) TT=1800.07	? (TO) TT=1800.15	SAT (TO) f=5299 TT=1800.12 T1=901.21	SAT (TO) f=5299 TT=1800.16 T1=901.01	? (TO) TT=1802.2	SAT (TO) f=852 TT=1800.07 T1=3.27	? (TO) TT=1800.05	? TT=1799.19	? TT=1799.93	? TT=1799.39	? (TO) TT=1800.05	? (TO) TT=1800.08	? TT=1799.69	? TT=1800.06
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_13.opb.PB06.opb	SAT f=486 TT=1792.84 T1=385.79	SAT f=5734 TT=1798.01 T1=1660.64	SAT (TO) f=9431 TT=1800.09 T1=2.77	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.07	SAT (TO) f=5375 TT=1800.32 T1=365.45	SAT (TO) f=5270 TT=1800.14 T1=522.8	? (TO) TT=1800.11	SAT f=5175 TT=1783.07 T1=697.24	? (exit code) TT=1103.09	SAT f=5473 TT=1798.27 T1=121.46	? (TO) TT=1800.01	SAT f=543 TT=1799.01 T1=1798.1	? (TO) TT=1800.2	SAT (TO) f=840 TT=1801.3 T1=5.41	SAT f=486 TT=1792.84 T1=385.79	SAT (TO) f=1188 TT=1800.69 T1=5.12	? TT=1799.01	SAT (TO) f=5688 TT=1800.01 T1=1800.3	? (TO) TT=1800.05	? (TO) TT=1800.08	SAT (TO) f=5498 TT=1800.83 T1=901.01	SAT (TO) f=5498 TT=1800.23 T1=901.11	SAT (TO) f=1183 TT=1800.8 T1=3.92	SAT (TO) f=1104 TT=1800.84 T1=2.91	? (TO) TT=1800.07	? TT=1799.29	? TT=1799.18	? (TO) TT=1800.04	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.08	? TT=1799.8
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_14.opb.PB06.opb	SAT f=488 TT=1791.96 T1=435.5	SAT f=5800 TT=1798.05 T1=1005.36	SAT (TO) f=9614 TT=1800.08 T1=2.8	? (TO) TT=1800.12	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800	? (TO) TT=1800.05	? (TO) TT=1800.06	SAT (TO) f=5567 TT=1800.18 T1=1777.06	Signal TT=1513.91	? (TO) TT=1802.09	Signal TT=1493.9	? TT=1494.91	SAT f=5700 TT=1798.08 T1=1735.95	? (TO) TT=1800.02	SAT f=453 TT=1799.41 T1=1798.22	? (TO) TT=1800.04	SAT (TO) f=585 TT=1801.29 T1=5.7	SAT f=488 TT=1791.96 T1=435.5	SAT (TO) f=768 TT=1800.1 T1=4.62	? TT=1799.56	SAT (TO) f=5504 TT=1800.01 T1=1801.01	? (TO) TT=1800.14	? (TO) TT=1800.07	SAT (TO) f=5404 TT=1800.29 T1=901.21	SAT (TO) f=5404 TT=1800.22 T1=901.21	SAT (TO) f=615 TT=1800.72 T1=4.22	SAT (TO) f=745 TT=1800.08 T1=3.08	? (TO) TT=1800.01	? TT=1799.82	? TT=1799.31	? TT=1799.3	? (TO) TT=1800.08	? (TO) TT=1800.09	? TT=1799.63	? TT=1799.91
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_1.opb.PB06.opb	SAT f=712 TT=1800.19 T1=3.91	SAT f=5583 TT=1798.02 T1=364.34	SAT (TO) f=9507 TT=1800.09 T1=3.7	? (TO) TT=1800.1	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.01	SAT (TO) f=5041 TT=1800.31 T1=1784.01	Signal TT=1254.56	? (TO) TT=1801.98	Signal TT=1609.36	? TT=966.317	SAT f=5771 TT=1798.11 T1=805.72	? (TO) TT=1801	SAT f=515 TT=1798.6 T1=1797.68	? (TO) TT=1800.35	SAT (TO) f=790 TT=1800.01 T1=4.59	? TT=1792.5	SAT (TO) f=712 TT=1800.19 T1=3.91	? TT=1799	SAT (TO) f=5470 TT=1800.06 T1=1800.9	? (TO) TT=1800.08	? (TO) TT=1800	SAT (TO) f=5282 TT=1800.22 T1=901.11	SAT (TO) f=5282 TT=1800.27 T1=901.11	SAT (TO) f=789 TT=1800.12 T1=4.62	SAT (TO) f=779 TT=1800 T1=3.06	? (TO) TT=1800.04	? TT=1799.93	? TT=1799.65	? TT=1799.19	? (TO) TT=1800.03	? (TO) TT=1800.07	? TT=1799.34	? (TO) TT=1800.06
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_2.opb.PB06.opb	SAT f=687 TT=1800.04 T1=4.77	SAT f=5740 TT=1798.35 T1=290.84	SAT (TO) f=10035 TT=1800.09 T1=2.78	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.1	SAT (TO) f=5328 TT=1800.16 T1=92.05	SAT (TO) f=5803 TT=1800.2 T1=925.87	? (TO) TT=1801.99	SAT f=5448 TT=1783.06 T1=1454.49	? TT=1069.18	SAT f=5587 TT=1798.13 T1=1779.09	? (TO) TT=1800.07	SAT f=767 TT=1799.3 T1=1797.99	? (TO) TT=1800.4	SAT (TO) f=699 TT=1801.18 T1=5.66	? TT=1793	SAT (TO) f=840 TT=1800.15 T1=4.53	? TT=1799.58	SAT (TO) f=4849 TT=1800.01 T1=1800.3	? (TO) TT=1800.01	? (TO) TT=1800.09	SAT (TO) f=5168 TT=1800.21 T1=900.91	SAT (TO) f=5168 TT=1800.25 T1=901.21	SAT (TO) f=687 TT=1800.04 T1=4.77	SAT (TO) f=878 TT=1800.71 T1=3.34	? (TO) TT=1763.55	? TT=1799.18	? (TO) TT=1802.05	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.05	? TT=1799.66	? TT=1799.58
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_3.opb.PB06.opb	SAT f=390 TT=1792.91 T1=274.64	SAT f=5830 TT=1798.18 T1=1094.11	SAT (TO) f=9725 TT=1800.09 T1=3.7	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) TT=1800.03	SAT (TO) f=5643 TT=1800.26 T1=1749.68	Signal TT=1376.63	? (TO) TT=1801.94	SAT f=5303 TT=1783.1 T1=1353.91	? TT=1057.44	SAT f=5851 TT=1798.05 T1=1497.95	? (TO) TT=1800.05	? (TO) TT=1802.04	? (TO) TT=1800.12	SAT (TO) f=1237 TT=1800.08 T1=4.7	SAT f=390 TT=1792.91 T1=274.64	SAT (TO) f=1077 TT=1800.1 T1=6.2	? TT=1798.96	SAT (TO) f=5903 TT=1800.01 T1=1800.3	? (TO) TT=1800.08	? (TO) TT=1800.09	SAT (TO) f=5436 TT=1800.68 T1=900.91	SAT (TO) f=5436 TT=1800.24 T1=901.01	SAT (TO) f=1012 TT=1800.1 T1=4.17	SAT (TO) f=774 TT=1800.01 T1=3.17	? (TO) TT=1800.74	? TT=1799.23	? TT=1799.35	? TT=1799.27	? (TO) TT=1800.08	? (TO) TT=1800.08	? TT=1799.64	? TT=1800.01
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_4.opb.PB06.opb	SAT f=483 TT=1792.1 T1=438.02	SAT f=5623 TT=1798.08 T1=881.15	SAT (TO) f=9461 TT=1800.06 T1=2.79	? (TO) TT=1800.01	? (TO) TT=1800.01	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.13	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.02	SAT (TO) f=5609 TT=1800.19 T1=61.49	Signal TT=1580.56	? (TO) TT=1801.94	? (TO) TT=1756.68	? TT=1302.79	SAT f=5680 TT=1798.03 T1=1351.11	? (TO) TT=1800.24	? TT=1798.75	? (TO) TT=1800.01	SAT (TO) f=780 TT=1800.14 T1=4.51	SAT f=483 TT=1792.1 T1=438.02	SAT (TO) f=836 TT=1800.78 T1=3.75	? TT=1799.91	SAT (TO) f=5616 TT=1800 T1=1800.9	? (TO) TT=1800.07	? (TO) TT=1800.03	SAT (TO) f=5359 TT=1800.67 T1=901.01	SAT (TO) f=5359 TT=1800.15 T1=901.11	SAT (TO) f=975 TT=1800.04 T1=4.68	SAT (TO) f=796 TT=1800.76 T1=2.99	? (TO) TT=1800.2	? TT=1799.17	? TT=1799.94	? TT=1799.83	? (TO) TT=1800.07	? (TO) TT=1800.08	? TT=1799.66	? TT=1799.68
Bench name	Best results	bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_5.opb.PB06.opb	SAT f=659 TT=1800.79 T1=4.09	SAT f=5464 TT=1798.02 T1=1307.08	SAT (TO) f=9976 TT=1800.06 T1=2.93	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.03	SAT (TO) f=5259 TT=1800.2 T1=1704.87	Signal TT=1320.27	? (TO) TT=1801.97	Signal TT=1585	? (exit code) TT=821.14	SAT f=5588 TT=1798.13 T1=897.97	? (TO) TT=1800.07	SAT f=557 TT=1798.4 T1=1797.16	? (TO) TT=1800.18	SAT (TO) f=891 TT=1800.12 T1=4.38	? TT=1792.51	SAT (TO) f=891 TT=1800.09 T1=3.94	? (TO) TT=1802.04	SAT (TO) f=5758 TT=1800.09 T1=1801.01	? (TO) TT=1800.1	? (TO) TT=1800.09	SAT (TO) f=5376 TT=1800.83 T1=901.11	SAT (TO) f=5382 TT=1800.13 T1=901.01	SAT (TO) f=659 TT=1800.79 T1=4.09	SAT (TO) f=825 TT=1800.06 T1=3.23	? (TO) TT=1800.5	? TT=1799.19	? TT=1799.89	? TT=1799.28	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.12	? TT=1799.95
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_6.opb.PB06.opb	SAT f=371 TT=1793.18 T1=624.67	SAT f=5484 TT=1798.03 T1=1427.17	SAT (TO) f=9743 TT=1800.01 T1=2.96	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.01	SAT (TO) f=5164 TT=1800.09 T1=1380.29	Signal TT=1170.45	? (TO) TT=1801.9	Signal TT=1684.98	? TT=1052.94	SAT f=5465 TT=1798.06 T1=1644.12	? (TO) TT=1800.04	SAT f=423 TT=1798.74 T1=1797.72	? (TO) TT=1800.13	SAT (TO) f=802 TT=1800.95 T1=4.53	SAT f=371 TT=1793.18 T1=624.67	SAT (TO) f=977 TT=1800.86 T1=5.09	? TT=1799.27	SAT (TO) f=5303 TT=1800.02 T1=1800.3	? (TO) TT=1800	? (TO) TT=1800.1	SAT (TO) f=5332 TT=1800.65 T1=900.91	SAT (TO) f=5332 TT=1800.25 T1=901.11	SAT (TO) f=956 TT=1800.1 T1=4.23	SAT (TO) f=784 TT=1800.03 T1=3.26	? (TO) TT=1800.03	? TT=1799.26	? TT=1799.94	? TT=1799.31	? (TO) TT=1800.09	? (TO) TT=1800.1	? TT=1800.03	? TT=1799.5
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_7.opb.PB06.opb	SAT f=396 TT=1793.24 T1=356.15	SAT f=5521 TT=1798.04 T1=311.36	SAT (TO) f=9424 TT=1800.09 T1=3.75	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.09	SAT (TO) f=5621 TT=1800.16 T1=1164.64	Signal TT=1576.67	? (TO) TT=1801.95	Signal TT=1545.65	? TT=991.323	SAT f=5552 TT=1798.13 T1=1322.73	? (TO) TT=1800.87	SAT f=486 TT=1798.47 T1=232.77	? (TO) TT=1800.26	SAT (TO) f=944 TT=1801.27 T1=4.35	SAT f=396 TT=1793.24 T1=356.15	SAT (TO) f=944 TT=1800.17 T1=3.73	? (TO) TT=1802.04	SAT (TO) f=5490 TT=1800.05 T1=1801.1	? (TO) TT=1800.06	? (TO) TT=1800.06	SAT (TO) f=5182 TT=1800.99 T1=901.11	SAT (TO) f=5182 TT=1800.19 T1=901.11	SAT (TO) f=907 TT=1800.02 T1=4.16	SAT (TO) f=907 TT=1800 T1=3.08	? (TO) TT=1801.4	? (TO) TT=1800.04	? TT=1799.13	? TT=1799.34	? (TO) TT=1800.1	? (TO) TT=1800.09	? TT=1799.49	? TT=1800.03
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_8.opb.PB06.opb	SAT f=736 TT=1800.83 T1=3.21	SAT f=5679 TT=1798.05 T1=1445.97	SAT (TO) f=9426 TT=1800.1 T1=3.78	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.07	SAT (TO) f=5599 TT=1800.13 T1=46.59	Signal TT=1294.07	? (TO) TT=1801.94	SAT f=5156 TT=1783.07 T1=1667.61	? (exit code) TT=1457.05	SAT f=5586 TT=1798.08 T1=138.13	? (TO) TT=1800.12	SAT f=545 TT=1799.34 T1=1798	? (TO) TT=1800.08	SAT (TO) f=855 TT=1800.13 T1=4.41	? TT=1791.8	SAT (TO) f=744 TT=1800.02 T1=4.91	? TT=1799.1	SAT (TO) f=6307 TT=1800.06 T1=1801	? (TO) TT=1800.04	? (TO) TT=1800.03	SAT (TO) f=5254 TT=1800.08 T1=900.91	SAT (TO) f=5254 TT=1800.12 T1=901.11	SAT (TO) f=844 TT=1800.76 T1=5.53	SAT (TO) f=736 TT=1800.83 T1=3.21	? (TO) TT=1800	? (TO) TT=1800.05	? (TO) TT=1802.04	? TT=1799.34	? (TO) TT=1800.08	? (TO) TT=1800.08	? TT=1799.6	? TT=1799.61
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_9.opb.PB06.opb	SAT f=419 TT=1792.04 T1=443.73	SAT f=5730 TT=1798.04 T1=1458.95	SAT (TO) f=9697 TT=1800.08 T1=4	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.02	SAT (TO) f=5633 TT=1800.29 T1=730.05	Signal TT=1511.22	? (TO) TT=1800.74	Signal TT=1620.81	? TT=977.628	SAT f=5773 TT=1798.11 T1=1666.92	? (TO) TT=1800.88	SAT f=426 TT=1798.66 T1=435.13	? (TO) TT=1800.01	SAT (TO) f=717 TT=1800.04 T1=4.38	SAT f=419 TT=1792.04 T1=443.73	SAT (TO) f=736 TT=1800.09 T1=3.76	? TT=1799.11	SAT (TO) f=5545 TT=1800 T1=1800.3	? (TO) TT=1800.03	? (TO) TT=1800.02	SAT (TO) f=5342 TT=1800.12 T1=901.01	SAT (TO) f=5342 TT=1800.33 T1=901.11	SAT (TO) f=775 TT=1800.7 T1=4.65	SAT (TO) f=981 TT=1800.84 T1=3.21	? (TO) TT=1800.02	? TT=1799.2	? (TO) TT=1800.04	? TT=1799.84	? (TO) TT=1800.07	? (TO) TT=1800.08	? TT=1799.73	? TT=1799.72

Statistics		bsolo 3.2 (complete)	clasp 2.0.6-R5325 (opt) (complete)	npSolver 1.0 (complete)	npSolver 1.0 (fixed) (complete)	npSolver inc (complete)	npSolver inc (fixed) (complete)	npSolver inc-topDown (complete)	npSolver inc-topDown (fixed) (complete)	npSolver inc-topdown-quickBound (complete)	npSolver inc-topdown-quickBound (fixed) (complete)	PB07: bsolo 3.0.17 (complete)	PB07: minisat+ 1.14 (complete)	PB07: PB-clasp 2007-04-10 (complete)	PB07: Pueblo 1.4 (incomplete)	PB07: SAT4JPseudoResolution 2007-03-23 (complete)	PB09: bsolo 3.1 (complete)	PB09: SAT4J Pseudo Resolution 2.1.1 (complete)	PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009) (complete)	PB10: pb_cplex 2010-06-29 (complete)	PB10: SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	PB10: SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	PB11: Sat4j Res//CP 2.3.0 (complete)	PB11: SCIP spx E_2 2011-06-10 (fixed) (complete)	PB12: minisatp 1.0-2-g022594c (complete)	pb2sat 2012-05-19 (complete)	pb2satCp2 2012-05-19 (complete)	pwbo 2.0 (complete)	pwbo 2.02 (complete)	SAT 4j PB RES // CP 2.3.2 Snapshot (complete)	Sat 4j PB Resolution 2.3.2 Snapshot (complete)	SAT4J PB specific settings 2.3.2 snapshot (complete)	SCIP spx SCIP 2.1.1.4. with SoPlex 1.6.0.3 fixed (complete)	SCIP spx E SCIP 2.1.1.4. Exp with SoPlex 1.6.0.3 fixed (complete)	SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed (complete)	toysat 2012-05-17 (complete)	toysat 2012-06-01 (complete)	wbo 1.7 (complete)	wbo 1.72 (complete)
Number of times the solver is able to give the best known answer		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	8	2	0	0	0	0	0	0	3	1	0	0	0	0	0	0	0	0
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	8	2	0	0	0	0	0	0	3	1	0	0	0	0	0	0	0	0