PB'11 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Random, subsubcategory Market Split Problem

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Random, subsubcategory Market Split Problem

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg pb-opt-11.04.03 (complete)
bsolo 3.2 (complete)
clasp 2.0-R4191 (complete)
MinisatID 2.4.8 [DEPRECATED] (complete)
MinisatID 2.4.8-gmp [DEPRECATED] (complete)
MinisatID 2.5.2 (fixed) (complete)
MinisatID 2.5.2-gmp (fixed) (complete)
pwbo 1.1 (complete)
Sat4j CuttingPlanes 2.3.0 (complete)
Sat4j Res//CP 2.3.0 (complete)
Sat4j Resolution 2.3.0 (complete)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx 2 2011-06-10 (fixed) (complete)
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx E_2 2011-06-10 (fixed) (complete)
wbo 1.6 (complete)

Bench name	Best results on this instance	borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_4_30_1.opb	UNSAT TT=30.597	UNSAT TT=8.2	UNSAT TT=206.204	UNSAT TT=30.597	UNSAT TT=787.285	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.05	UNSAT TT=134.73	? (TO) TT=1800.35	UNSAT TT=158.917	UNSAT TT=77.202	UNSAT TT=223.04	UNSAT TT=226.447	UNSAT TT=223.155	UNSAT TT=227.361	UNSAT TT=183.047
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_4_30_2.opb	UNSAT TT=15.266	UNSAT TT=56.507	UNSAT TT=78.634	UNSAT TT=15.266	UNSAT TT=287.519	? (TO) TT=1800.05	UNSAT TT=876.071	? (TO) TT=1800.03	UNSAT TT=72.249	? (TO) TT=1800.25	UNSAT TT=98.158	UNSAT TT=45.778	UNSAT TT=101.709	UNSAT TT=103.859	UNSAT TT=102.086	UNSAT TT=103.282	UNSAT TT=98.524
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_4_30_3.opb	UNSAT TT=48.602	UNSAT TT=14.896	UNSAT TT=454.156	UNSAT TT=48.602	UNSAT TT=1356.05	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.07	UNSAT TT=293.638	? (TO) TT=1800.34	UNSAT TT=270.572	UNSAT TT=124.55	UNSAT TT=240.564	UNSAT TT=222.181	UNSAT TT=213.244	UNSAT TT=219.594	UNSAT TT=314.451
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_4_30_4.opb	UNSAT TT=17.34	UNSAT TT=5.222	UNSAT TT=124.804	UNSAT TT=17.34	UNSAT TT=599.498	? (TO) TT=1800.06	UNSAT TT=1406.93	? (TO) TT=1800.06	UNSAT TT=115.743	? (TO) TT=1800.32	UNSAT TT=106.066	UNSAT TT=52.592	UNSAT TT=70.019	UNSAT TT=71.372	UNSAT TT=70.045	UNSAT TT=70.764	UNSAT TT=110.673
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_5_40_1.opb		? (MO) TT=493.75	? TT=1798	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.24	? (TO) TT=1800.33	? (TO) TT=1800.14	? (TO) TT=1800.08	? TT=1798.68	? (TO) TT=1800.08	? TT=1798.65	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_5_40_2.opb		? (MO) TT=495.34	? TT=1798	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.29	? (TO) TT=1800.24	? (TO) TT=1800.12	? (TO) TT=1800.09	? TT=1798.74	? (TO) TT=1800.08	? TT=1798.71	? (TO) TT=1800.13
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_5_40_3.opb		? (MO) TT=491.36	? TT=1798	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.25	? (TO) TT=1800.22	? (TO) TT=1800.06	? (TO) TT=1800.05	? TT=1798.78	? (TO) TT=1800.07	? TT=1798.68	? (TO) TT=1800.12
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_5_40_4.opb		? (MO) TT=493.52	? TT=1798	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.19	? (TO) TT=1800.17	? (TO) TT=1800.08	? (TO) TT=1800.11	? TT=1799.22	? (TO) TT=1800.1	? TT=1799.17	? (TO) TT=1800.12
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_6_50_1.opb		? (MO) TT=495.8	? TT=1798.05	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.2	? (TO) TT=1800.24	? (TO) TT=1800.14	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.17
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_6_50_2.opb		? (MO) TT=493.43	? TT=1798	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.22	? (TO) TT=1800.26	? (TO) TT=1800.15	? (TO) TT=1800.13	? (TO) TT=1800.12	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.12
Bench name	Best results	borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_6_50_3.opb		? (MO) TT=501.46	? TT=1798	? (TO) TT=1800.08	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.21	? (TO) TT=1800.15	? (TO) TT=1800.16	? (TO) TT=1800.17	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.11	? (TO) TT=1800.12
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_6_50_4.opb		? (MO) TT=493.89	? TT=1798	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.25	? (TO) TT=1800.19	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) TT=1800.1	? (TO) TT=1800.13	? (TO) TT=1800.1	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_7_60_1.opb		? (MO) TT=503.19	? TT=1798	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.25	? (TO) TT=1800.29	? (TO) TT=1800.22	? (TO) TT=1800.14	? (TO) TT=1800.16	? (TO) TT=1800.15	? (TO) TT=1800.2	? (TO) TT=1800.12	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_7_60_2.opb		? (MO) TT=479.54	? TT=1798	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.19	? (TO) TT=1800.21	? (TO) TT=1800.13	? (TO) TT=1800.15	? (TO) TT=1800.12	? (TO) TT=1800.16	? (TO) TT=1800.17	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_7_60_3.opb		? (MO) TT=499.36	? TT=1798	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.23	? (TO) TT=1800.22	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) TT=1800.09	? (TO) TT=1800.16	? (TO) TT=1800.09	? (TO) TT=1800.15
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_7_60_4.opb		? (MO) TT=496.28	? TT=1798	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.19	? (TO) TT=1800.3	? (TO) TT=1800.11	? (TO) TT=1800.16	? (TO) TT=1800.14	? (TO) TT=1800.18	? (TO) TT=1800.09	? (TO) TT=1800.19
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_8_70_1.opb		? (MO) TT=498.03	? TT=1798	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.5	? (TO) TT=1800.35	? (TO) TT=1800.07	? (TO) TT=1800.15	? (TO) TT=1800.16	? (TO) TT=1800.11	? (TO) TT=1800.14	? (TO) TT=1800.13
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_8_70_2.opb		? (MO) TT=485.08	? TT=1798	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.19	? (TO) TT=1800.77	? (TO) TT=1800.07	? (TO) TT=1800.16	? (TO) TT=1800.15	? (TO) TT=1800.15	? (TO) TT=1800.15	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_8_70_3.opb		? (MO) TT=481.89	? TT=1798	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.21	? (TO) TT=1800.77	? (TO) TT=1800.14	? (TO) TT=1800.14	? (TO) TT=1800.13	? (TO) TT=1800.17	? (TO) TT=1800.13	? (TO) TT=1800.17
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-market-split_8_70_4.opb		? (MO) TT=484.99	? TT=1798	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.48	? (TO) TT=1800.15	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.16	? (TO) TT=1800.14	? (TO) TT=1800.13	? (TO) TT=1800.17
Bench name	Best results	borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_4_30_1.opb	OPT f=2 TT=84.767 T1=20.67	? (MO) TT=344.84	OPT f=2 TT=275.687 T1=32.27	OPT f=2 TT=257.757 T1=232.86	OPT f=2 TT=1041.03 T1=139.25	? (TO) f=2 TT=1800.02 T1=1063.39	? (TO) f=2 TT=1800.06 T1=1800	? (TO) f=2 TT=1800.06 T1=1800	OPT f=2 TT=683.869 T1=252.99	SAT (TO) f=11 TT=1800.26 T1=1321.07	OPT f=2 TT=186.762 T1=29.84	OPT f=2 TT=84.767 T1=20.67	OPT f=2 TT=267.001 T1=66.33	OPT f=2 TT=272.667 T1=67.67	OPT f=2 TT=269.129 T1=66.84	OPT f=2 TT=270.822 T1=67.26	OPT f=2 TT=637.679 T1=637.68
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_4_30_2.opb	OPT f=1 TT=43.694 T1=6.32	? (MO) TT=359.47	OPT f=1 TT=112.027 T1=6.32	OPT f=1 TT=43.694 T1=43.69	OPT f=1 TT=389.481 T1=8.46	? (TO) f=1 TT=1800.04 T1=74.17	OPT f=1 TT=1059.93 T1=1059.9	? (TO) f=1 TT=1800.07 T1=1800	OPT f=1 TT=217.488 T1=108.73	SAT (TO) f=202 TT=1800.34 T1=1024.4	OPT f=1 TT=108.869 T1=18.76	OPT f=1 TT=49.121 T1=8.56	OPT f=1 TT=333.924 T1=17.38	OPT f=1 TT=355.756 T1=17.68	OPT f=1 TT=331.334 T1=17.47	OPT f=1 TT=374.841 T1=18.64	OPT f=1 TT=112.132 T1=112.12
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_4_30_3.opb	OPT f=2 TT=134.677 T1=7.42	? (MO) TT=363.65	OPT f=2 TT=711.197 T1=52.78	OPT f=2 TT=375.397 T1=332.95	? (TO) f=2 TT=1800.07 T1=54.51	? (TO) f=2 TT=1800.06 T1=453.31	? (TO) f=2 TT=1800.06 T1=1800	? (TO) f=2 TT=1800.06 T1=1800	OPT f=2 TT=1447.61 T1=678.07	SAT (TO) f=204 TT=1800.24 T1=1015.13	OPT f=2 TT=464.772 T1=145.44	OPT f=2 TT=182.897 T1=95.01	OPT f=2 TT=137.521 T1=7.42	OPT f=2 TT=135.778 T1=7.6	OPT f=2 TT=138.653 T1=7.48	OPT f=2 TT=134.677 T1=7.55	OPT f=2 TT=1308.71 T1=1308.64
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_4_30_4.opb	OPT f=1 TT=38.305 T1=30.25	? (MO) TT=339.25	OPT f=1 TT=225.964 T1=30.25	OPT f=1 TT=38.305 T1=38.3	OPT f=1 TT=744.549 T1=91.83	? (TO) f=1 TT=1800.1 T1=769.08	OPT f=1 TT=1312.64 T1=1312.61	? (TO) f=1 TT=1800.06 T1=1800	OPT f=1 TT=321.545 T1=160.77	SAT (TO) f=29 TT=1800.26 T1=1516.86	OPT f=1 TT=164.91 T1=75.74	OPT f=1 TT=69.168 T1=49.74	OPT f=1 TT=378.171 T1=243.96	OPT f=1 TT=492.917 T1=52.8	OPT f=1 TT=354.282 T1=229.98	OPT f=1 TT=489.485 T1=52.74	OPT f=1 TT=184.71 T1=184.81
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_5_40_1.opb	OPT f=0 TT=1320.51 T1=1318.31	? (MO) TT=250.05	SAT f=2 TT=1798 T1=1520.11	SAT (TO) f=994 TT=1800.07 T1=1357.02	? (TO) f=5 TT=1800.05 T1=908.28	? (TO) f=9 TT=1800.07 T1=15.9	? (TO) f=2 TT=1800.06 T1=1800	? (TO) f=5 TT=1800.07 T1=1800	SAT (TO) f=1 TT=1800.08 T1=900.01	SAT (TO) f=324 TT=1800.27 T1=1663.59	SAT (TO) f=2 TT=1800.15 T1=833.86	OPT f=0 TT=1320.51 T1=1318.31	SAT (TO) f=1 TT=1800.07 T1=158.38	SAT f=1 TT=1797.2 T1=91.31	SAT (TO) f=1 TT=1800.13 T1=159.24	SAT f=1 TT=1797.17 T1=91.43	? (TO) TT=1800.1
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_5_40_2.opb	SAT f=3 TT=1797.64 T1=285.95	? (MO) TT=457.74	SAT f=4 TT=1798 T1=498.68	SAT (TO) f=965 TT=1800.1 T1=1368.09	? (TO) f=3 TT=1800.08 T1=1505.87	? (TO) f=5 TT=1800.06 T1=473.25	? (TO) f=2 TT=1800.06 T1=1800	? (TO) f=8 TT=1800.05 T1=1800	SAT (TO) f=4 TT=1800.1 T1=900.01	SAT (TO) f=352 TT=1800.22 T1=764.6	SAT (TO) f=5 TT=1800.27 T1=104.06	SAT (TO) f=3 TT=1800.12 T1=1169.23	SAT (TO) f=3 TT=1800.12 T1=285.95	SAT f=3 TT=1797.66 T1=305.67	SAT (TO) f=3 TT=1800.08 T1=291.6	SAT f=3 TT=1797.64 T1=305.81	? (TO) TT=1800.11
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_5_40_3.opb	SAT f=2 TT=1797.14 T1=946.63	? (MO) TT=242.46	SAT f=3 TT=1798 T1=753.65	SAT (TO) f=1069 TT=1800.07 T1=1357.58	? (TO) f=2 TT=1800.04 T1=735.15	? (TO) f=9 TT=1800.06 T1=1702.1	? (TO) f=2 TT=1800.09 T1=1800	? (TO) f=4 TT=1800.11 T1=1800.11	SAT (TO) f=3 TT=1800.08 T1=900.01	SAT (TO) f=243 TT=1800.3 T1=1322.71	SAT (TO) f=3 TT=1800.6 T1=102.28	SAT (TO) f=3 TT=1800.1 T1=82.11	SAT (TO) f=2 TT=1800.07 T1=946.63	SAT f=2 TT=1797.14 T1=1061.99	SAT (TO) f=2 TT=1800.11 T1=1069.64	SAT f=2 TT=1797.18 T1=965.75	? (TO) TT=1800.11
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_5_40_4.opb	SAT f=2 TT=1797.1 T1=1081.15	? (MO) TT=240.99	SAT f=3 TT=1798 T1=593.52	SAT (TO) f=1007 TT=1800.07 T1=1368.25	? (TO) f=5 TT=1800.05 T1=329.53	? (TO) f=11 TT=1800.03 T1=1094.14	? (TO) f=4 TT=1800.06 T1=1800	? (TO) f=6 TT=1800.09 T1=1800	SAT (TO) f=3 TT=1800.09 T1=900.01	SAT (TO) f=548 TT=1800.34 T1=1498.46	SAT (TO) f=3 TT=1800.36 T1=141.17	SAT (TO) f=3 TT=1800.1 T1=111.22	SAT (TO) f=2 TT=1800.07 T1=1432.46	SAT f=2 TT=1797.24 T1=1081.15	SAT (TO) f=2 TT=1800.05 T1=1448.52	SAT f=2 TT=1797.1 T1=1170.56	? (TO) TT=1800.11
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_6_50_1.opb	SAT f=3 TT=1797.53 T1=219.37	? (MO) TT=343.83	SAT f=9 TT=1798 T1=996.37	SAT (TO) f=1188 TT=1800.09 T1=1463.53	? (TO) f=12 TT=1800.1 T1=668.23	? (TO) f=17 TT=1800.03 T1=1174.22	? (TO) f=9 TT=1800.06 T1=1800	? (TO) f=12 TT=1800.05 T1=1800	SAT (TO) f=10 TT=1800.13 T1=900.01	SAT (TO) f=502 TT=1800.3 T1=660.77	SAT (TO) f=8 TT=1800.18 T1=473.52	SAT (TO) f=7 TT=1800.1 T1=1250.2	SAT (TO) f=6 TT=1800.08 T1=487.18	SAT f=3 TT=1797.53 T1=219.37	SAT (TO) f=3 TT=1800.06 T1=1530.09	SAT f=5 TT=1797.87 T1=86.83	? (TO) TT=1800.12
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_6_50_2.opb	SAT f=2 TT=1800.08 T1=1087.1	? (MO) TT=336.01	SAT f=10 TT=1798 T1=148.5	SAT (TO) f=1182 TT=1800.07 T1=1452.22	? (TO) f=17 TT=1800.07 T1=241.46	? (TO) f=17 TT=1800.06 T1=1011.05	? (TO) f=7 TT=1800.04 T1=1800	? (TO) f=17 TT=1800.06 T1=1800	SAT (TO) f=12 TT=1800.08 T1=900.01	SAT (TO) f=184 TT=1800.21 T1=1480.57	SAT (TO) f=5 TT=1800.14 T1=632.02	SAT (TO) f=4 TT=1800.11 T1=1049.41	SAT (TO) f=2 TT=1800.08 T1=1087.1	SAT f=5 TT=1798.04 T1=1269.65	SAT (TO) f=2 TT=1800.08 T1=1092.96	SAT f=5 TT=1798.3 T1=21.3	? (TO) TT=1800.11
Bench name	Best results	borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_6_50_3.opb	SAT f=5 TT=1798.37 T1=54.75	? (MO) TT=350.73	SAT f=10 TT=1798 T1=1504.11	SAT (TO) f=1231 TT=1800.05 T1=1456.77	? (TO) f=11 TT=1800.04 T1=612.3	? (TO) f=17 TT=1800.07 T1=1058.36	? (TO) f=11 TT=1800.06 T1=1800	? (TO) f=16 TT=1800.05 T1=1800	SAT (TO) f=6 TT=1800.09 T1=900.01	SAT (TO) f=627 TT=1800.31 T1=1630.22	SAT (TO) f=6 TT=1800.18 T1=865.01	SAT (TO) f=6 TT=1800.18 T1=678.55	SAT (TO) f=6 TT=1800.09 T1=539.71	SAT f=5 TT=1798.43 T1=54.75	SAT (TO) f=6 TT=1800.1 T1=1308.83	SAT f=5 TT=1798.37 T1=55.52	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_6_50_4.opb	SAT f=6 TT=1798.93 T1=137.01	? (MO) TT=353.28	SAT f=7 TT=1798.05 T1=92.71	SAT (TO) f=1129 TT=1800.06 T1=1593.09	? (TO) f=10 TT=1800.08 T1=343.19	? (TO) f=15 TT=1800.07 T1=1133.96	? (TO) f=8 TT=1800.06 T1=1800	? (TO) f=17 TT=1800.07 T1=1800	SAT (TO) f=10 TT=1800.11 T1=900.01	SAT (TO) f=487 TT=1800.28 T1=1587.95	SAT (TO) f=10 TT=1800.27 T1=157.98	SAT (TO) f=7 TT=1800.11 T1=1394.62	SAT (TO) f=6 TT=1800.1 T1=137.01	SAT f=6 TT=1799.44 T1=139.7	SAT (TO) f=6 TT=1800.09 T1=150.9	SAT f=6 TT=1798.93 T1=156.72	? (TO) TT=1800.14
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_7_60_1.opb	SAT f=4 TT=1800.07 T1=593.88	? (MO) TT=301.3	SAT f=19 TT=1798 T1=1504.58	SAT (TO) f=1444 TT=1800.11 T1=1533.81	? (TO) f=39 TT=1800.06 T1=1414.79	? (TO) f=48 TT=1800.05 T1=149.91	? (TO) f=22 TT=1800.05 T1=1800	? (TO) f=27 TT=1800.06 T1=1800	SAT (TO) f=24 TT=1800.09 T1=900.01	SAT (TO) f=941 TT=1800.25 T1=1354.12	SAT (TO) f=19 TT=1800.29 T1=282.75	SAT (TO) f=17 TT=1800.13 T1=856.2	SAT (TO) f=4 TT=1800.07 T1=593.88	SAT f=9 TT=1800 T1=1291.3	SAT (TO) f=4 TT=1800.08 T1=595.78	SAT f=9 TT=1799.95 T1=1276.59	? (TO) TT=1800.12
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_7_60_2.opb	SAT f=8 TT=1800.11 T1=1165.48	? (MO) TT=288.61	SAT f=80 TT=1798 T1=774.81	SAT (TO) f=1274 TT=1800.06 T1=1529.23	? (TO) f=38 TT=1800.07 T1=162.99	? (TO) f=38 TT=1800.08 T1=1140.32	? (TO) f=25 TT=1800.06 T1=1800	? (TO) f=31 TT=1800.05 T1=1800	SAT (TO) f=19 TT=1800.1 T1=900.01	SAT (TO) f=846 TT=1800.23 T1=785.4	SAT (TO) f=17 TT=1800.22 T1=419.44	SAT (TO) f=13 TT=1800.11 T1=1389.86	SAT (TO) f=8 TT=1800.11 T1=1165.48	SAT (TO) f=12 TT=1800.1 T1=385.77	SAT (TO) f=8 TT=1800.1 T1=1169.71	SAT (TO) f=12 TT=1800.08 T1=388.38	? (TO) TT=1800.1
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_7_60_3.opb	SAT f=8 TT=1800.08 T1=561.8	? (MO) TT=303.53	SAT f=72 TT=1798 T1=23.39	SAT (TO) f=1092 TT=1800.08 T1=1540.81	? (TO) f=36 TT=1800.06 T1=1182.25	? (TO) f=44 TT=1800.05 T1=958.47	? (TO) f=28 TT=1800.07 T1=1800	? (TO) f=31 TT=1800.06 T1=1800	SAT (TO) f=25 TT=1800.09 T1=900.01	SAT (TO) f=796 TT=1800.24 T1=1483.5	SAT (TO) f=20 TT=1800.41 T1=420.85	SAT (TO) f=17 TT=1800.11 T1=1651.31	SAT (TO) f=8 TT=1800.08 T1=561.8	SAT (TO) f=11 TT=1800.1 T1=1442.94	SAT (TO) f=8 TT=1800.08 T1=562.13	SAT (TO) f=11 TT=1800.07 T1=1444.83	? (TO) TT=1800.1
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_7_60_4.opb	SAT f=9 TT=1799.73 T1=431.76	? (MO) TT=290.56	SAT f=34 TT=1798 T1=1283.81	SAT (TO) f=1350 TT=1800.06 T1=1535.3	? (TO) f=35 TT=1800.06 T1=1111.35	? (TO) f=38 TT=1800.05 T1=808.78	? (TO) f=18 TT=1800.04 T1=1800	? (TO) f=23 TT=1800.04 T1=1800	SAT (TO) f=21 TT=1800.09 T1=900.01	SAT (TO) f=961 TT=1800.25 T1=1625.42	SAT (TO) f=16 TT=1800.29 T1=770.02	SAT (TO) f=17 TT=1800.15 T1=567.63	SAT (TO) f=10 TT=1800.16 T1=1127.76	SAT (TO) f=11 TT=1800.09 T1=965.66	SAT (TO) f=10 TT=1800.1 T1=1138.98	SAT f=9 TT=1799.73 T1=431.76	? (TO) TT=1800.1
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_8_70_1.opb	SAT f=11 TT=1800.14 T1=620.71	? (MO) TT=298.47	SAT f=99 TT=1798.24 T1=340.92	SAT (TO) f=1257 TT=1800.07 T1=1682	? (TO) f=56 TT=1800.06 T1=1495.38	? (TO) f=99 TT=1800.08 T1=527.12	? (TO) f=25 TT=1800.04 T1=1800	? (TO) f=51 TT=1800.08 T1=1800	SAT (TO) f=31 TT=1800.09 T1=900.01	SAT (TO) f=2049 TT=1800.22 T1=900.25	SAT (TO) f=35 TT=1800.22 T1=798.54	SAT (TO) f=35 TT=1800.08 T1=1663.77	SAT (TO) f=11 TT=1800.14 T1=620.71	SAT (TO) f=13 TT=1800.12 T1=1279.89	SAT (TO) f=11 TT=1800.13 T1=634.6	SAT (TO) f=16 TT=1800.1 T1=1119.77	? (TO) TT=1800.13
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_8_70_2.opb	SAT f=13 TT=1800.16 T1=724.86	? (MO) TT=287.74	SAT f=190 TT=1798 T1=21.76	SAT (TO) f=1554 TT=1800.09 T1=1622.44	? (TO) f=65 TT=1800.08 T1=139.24	? (TO) f=65 TT=1800.05 T1=1410.75	? (TO) f=49 TT=1800.05 T1=1800	? (TO) f=55 TT=1800.06 T1=1800	SAT (TO) f=39 TT=1800.1 T1=900.01	SAT (TO) f=1346 TT=1800.22 T1=1486.19	SAT (TO) f=45 TT=1800.16 T1=794.27	SAT (TO) f=30 TT=1800.12 T1=1702.68	SAT (TO) f=13 TT=1800.16 T1=724.86	SAT (TO) f=18 TT=1800.11 T1=1509.66	SAT (TO) f=13 TT=1800.11 T1=661.59	SAT (TO) f=18 TT=1800.08 T1=1518.72	? (TO) TT=1800.09
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_8_70_3.opb	SAT f=15 TT=1800.08 T1=1643.86	? (MO) TT=297.51	SAT f=160 TT=1798 T1=642.73	SAT (TO) f=1214 TT=1800.11 T1=1626.44	? (TO) f=58 TT=1800.07 T1=1408.17	? (TO) f=82 TT=1800.06 T1=394.39	? (TO) f=43 TT=1800.07 T1=1800	? (TO) f=52 TT=1800.09 T1=1800	SAT (TO) f=37 TT=1800.08 T1=900.01	SAT (TO) f=1473 TT=1800.22 T1=348.65	SAT (TO) f=108 TT=1800.29 T1=936.68	SAT (TO) f=51 TT=1800.13 T1=1404.37	SAT (TO) f=16 TT=1800.09 T1=752.45	SAT (TO) f=16 TT=1800.15 T1=745.14	SAT (TO) f=16 TT=1800.13 T1=688.94	SAT (TO) f=15 TT=1800.08 T1=1643.86	? (TO) TT=1800.13
normalized-PB06/OPT-SMALLINT/ submitted-PB06/manquiho/market-split/ normalized-opt-market-split_8_70_4.opb	SAT f=10 TT=1800.13 T1=1190.94	? (MO) TT=288.8	SAT f=93 TT=1798 T1=1214.18	SAT (TO) f=1463 TT=1800.06 T1=1646.6	? (TO) f=51 TT=1800.08 T1=1567.21	? (TO) f=64 TT=1800.06 T1=649.77	? (TO) f=38 TT=1800.07 T1=1800	? (TO) f=49 TT=1800.05 T1=1800	SAT (TO) f=47 TT=1800.12 T1=900.01	SAT (TO) f=1543 TT=1800.2 T1=395.2	SAT (TO) f=50 TT=1800.3 T1=402.25	SAT (TO) f=40 TT=1800.12 T1=688.8	SAT (TO) f=10 TT=1800.13 T1=1190.94	SAT (TO) f=14 TT=1800.16 T1=448.99	SAT (TO) f=10 TT=1800.13 T1=1196.11	SAT (TO) f=17 TT=1800.12 T1=1054.56	? (TO) TT=1800.13

Statistics		borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
Number of times the solver is able to give the best known answer		0	8	8	7	0	4	0	8	0	8	10	19	14	0	15	8
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	0	6	0	0	0	0	0	0	0	2	7	2	0	7	0