CSP 2008 Competition: results by benchmark

Results by benchmark for category n-ary constraints in extension (N-ARY-EXT), subcategory quasi random

This page displays the results of the different solvers for each benchmark for category n-ary constraints in extension (N-ARY-EXT), subcategory quasi random

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...	Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 112v4 AC
Abscon 112v4 ESAC
bpsolver 2008-06-27
casper zao
casper zito
choco2_dwdeg 2008-06-26
choco2_impwdeg 2008-06-26
Concrete + CPS4J 2008-05-30
Concrete + CSP4J - Tabu Engine 2008-05-30
Concrete + CSP4J - WMC Engine 2008-05-30
cpHydra k_10
cpHydra k_40
galac E
galac M
mddc-solv 2008-06-09
MDG-noprobe 2008-06-27
MDG-probe 2008-06-27
Minion/Tailor 2008-07-04
Mistral-option 1.314
Mistral-prime 1.313
SAT4J CSP 2008-06-13
spider 2008-06-14
Sugar v1.13+minisat
Sugar v1.13+picosat

Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddLarge/ normalized-bdd-21-2713-15-79-10_ext.xml	UNSAT TT=20.161	UNSAT TT=75.449 CHECKS=3377 NODES=1579	? (MO) TT=1.737	? (problem) TT=540.272	? (TO) TT=1800.14	? (TO) TT=1800.18	UNSAT TT=502.74 ASSIGNMENTS=1017	UNSAT TT=932.526 ASSIGNMENTS=1413	UNSAT TT=104.103 CHECKS=96745200 NODES=2174	? (TO) TT=1800.09	? (TO) TT=1800.01	UNSAT TT=43.467 ASSIGNMENTS=1080 CHECKS=2744120	UNSAT TT=43.568 ASSIGNMENTS=1080 CHECKS=2744120	? (MO) TT=1764.19	? (MO) TT=1775.55	UNSAT TT=20.161 CHECKS=1905990	UNSAT TT=29.044 ASSIGNMENTS=953 CHECKS=2516650	UNSAT TT=138.93 ASSIGNMENTS=5088 CHECKS=11761200	? (MO) TT=9.6	UNSAT TT=31.676 ASSIGNMENTS=990 CHECKS=2587020	UNSAT TT=34.385 ASSIGNMENTS=1080 CHECKS=2744120	? TT=92.272	UNSAT TT=1.681 ASSIGNMENTS=2103 CHECKS=195944	? TT=82.751	? TT=77.592
csp/bddLarge/ normalized-bdd-21-2713-15-79-11_ext.xml	UNSAT TT=17.451	UNSAT TT=73.451 CHECKS=4068 NODES=1788	UNSAT TT=80.089 CHECKS=3913 NODES=1688	? (problem) TT=562.562	? (TO) TT=1800.22	? (TO) TT=1800.22	UNSAT TT=911.868 ASSIGNMENTS=1049	UNSAT TT=948.987 ASSIGNMENTS=1358	UNSAT TT=97.693 CHECKS=95507400 NODES=2241	? (TO) TT=1800.03	? (TO) TT=1800.08	UNSAT TT=49.223 ASSIGNMENTS=1051 CHECKS=3479530	UNSAT TT=46.629 ASSIGNMENTS=1051 CHECKS=3479530	? (MO) TT=1737.24	? (TO) TT=1800	UNSAT TT=17.451 CHECKS=2083980	UNSAT TT=38.9 ASSIGNMENTS=1045 CHECKS=3517360	UNSAT TT=130.019 ASSIGNMENTS=5016 CHECKS=11739500	? (MO) TT=7.4	UNSAT TT=39.778 ASSIGNMENTS=1092 CHECKS=3604380	UNSAT TT=37.212 ASSIGNMENTS=1051 CHECKS=3479530	? TT=98.06	UNSAT TT=1.684 ASSIGNMENTS=2174 CHECKS=182016	? TT=278.531	? TT=175.473
csp/bddLarge/ normalized-bdd-21-2713-15-79-12_ext.xml	UNSAT TT=23.58	UNSAT TT=76.401 CHECKS=4420 NODES=1946	UNSAT TT=76.343 CHECKS=3818 NODES=1663	? (problem) TT=530.24	? (TO) TT=1800.19	? (TO) TT=1800.18	UNSAT TT=597.324 ASSIGNMENTS=1096	UNSAT TT=918.954 ASSIGNMENTS=1376	UNSAT TT=99.969 CHECKS=101402000 NODES=2194	? (TO) TT=1800.05	? (TO) TT=1800.09	UNSAT TT=47.555 ASSIGNMENTS=1020 CHECKS=3227590	UNSAT TT=44.215 ASSIGNMENTS=1020 CHECKS=3227590	? (MO) TT=1792.49	? (MO) TT=1749.75	UNSAT TT=23.58 CHECKS=2283190	UNSAT TT=41.345 ASSIGNMENTS=1099 CHECKS=3385670	UNSAT TT=133.813 ASSIGNMENTS=5056 CHECKS=11798900	? (MO) TT=7.58	UNSAT TT=40.519 ASSIGNMENTS=1044 CHECKS=3212260	UNSAT TT=38.918 ASSIGNMENTS=1020 CHECKS=3227590	? TT=87.828	UNSAT TT=2.379 ASSIGNMENTS=2380 CHECKS=388061	? TT=173.51	? TT=171.019
csp/bddLarge/ normalized-bdd-21-2713-15-79-13_ext.xml	SAT TT=1.829	SAT TT=10.945 CHECKS=2800 NODES=84	SAT TT=13.537 CHECKS=2784 NODES=63	? (problem) TT=528.163	SAT TT=1429.09 CHECKS=3023260	SAT TT=154.394 CHECKS=223124	SAT TT=378.668 ASSIGNMENTS=1008	SAT TT=112.528 ASSIGNMENTS=264	SAT TT=129.181 CHECKS=127266000 NODES=3166	SAT TT=173.695 CHECKS=252842000 NODES=4656	SAT TT=4.292 CHECKS=865242 NODES=11	SAT TT=33.92 ASSIGNMENTS=996 CHECKS=2326930	SAT TT=36.259 ASSIGNMENTS=996 CHECKS=2326930	? (TO) TT=1800.09	? (MO) TT=1740.44	SAT TT=4.989 CHECKS=560173	SAT TT=25.485 ASSIGNMENTS=1011 CHECKS=2348490	SAT TT=126.027 ASSIGNMENTS=4913 CHECKS=11824300	? (MO) TT=7.67	SAT TT=1.829 ASSIGNMENTS=23 CHECKS=64014	SAT TT=25.527 ASSIGNMENTS=996 CHECKS=2326930	? TT=88.127	SAT TT=1.812 ASSIGNMENTS=2137 CHECKS=156346	? TT=118.121	? TT=108.251
csp/bddLarge/ normalized-bdd-21-2713-15-79-14_ext.xml	UNSAT TT=20.182	UNSAT TT=76.442 CHECKS=4337 NODES=1797	UNSAT TT=78.339 CHECKS=4293 NODES=1797	? (problem) TT=544.606	? (TO) TT=1800.18	? (TO) TT=1800.22	UNSAT TT=553.099 ASSIGNMENTS=1063	UNSAT TT=1088.96 ASSIGNMENTS=1378	UNSAT TT=96.332 CHECKS=94786000 NODES=2179	? (TO) TT=1800.02	? (TO) TT=1800.09	UNSAT TT=40.446 ASSIGNMENTS=1064 CHECKS=2658980	UNSAT TT=40.31 ASSIGNMENTS=1064 CHECKS=2658980	? (MO) TT=1730.2	? (TO) TT=1800.07	UNSAT TT=20.182 CHECKS=2077040	UNSAT TT=29.608 ASSIGNMENTS=1045 CHECKS=2543630	UNSAT TT=118.182 ASSIGNMENTS=4753 CHECKS=11415200	? (MO) TT=7.41	UNSAT TT=28.473 ASSIGNMENTS=1036 CHECKS=2370010	UNSAT TT=29.492 ASSIGNMENTS=1064 CHECKS=2658980	? TT=93.322	UNSAT TT=1.807 ASSIGNMENTS=2180 CHECKS=231236	? TT=156.024	? TT=149.998
csp/bddLarge/ normalized-bdd-21-2713-15-79-15_ext.xml	UNSAT TT=18.489	UNSAT TT=75.053 CHECKS=3364 NODES=1552	UNSAT TT=89.695 CHECKS=3657 NODES=1544	? (problem) TT=526.969	? (TO) TT=1800.16	? (TO) TT=1800.16	UNSAT TT=418.5 ASSIGNMENTS=949	UNSAT TT=1090.46 ASSIGNMENTS=1408	UNSAT TT=95.81 CHECKS=90291000 NODES=2275	? (TO) TT=1800.07	? (TO) TT=1800.03	UNSAT TT=32.96 ASSIGNMENTS=972 CHECKS=2279050	UNSAT TT=34.558 ASSIGNMENTS=972 CHECKS=2279050	? (MO) TT=1795.53	? (MO) TT=1699.29	UNSAT TT=18.489 CHECKS=1939770	UNSAT TT=27.632 ASSIGNMENTS=950 CHECKS=2297690	UNSAT TT=129.538 ASSIGNMENTS=4968 CHECKS=11733700	? (MO) TT=7.59	UNSAT TT=28.547 ASSIGNMENTS=988 CHECKS=2377590	UNSAT TT=26.737 ASSIGNMENTS=972 CHECKS=2279050	? TT=95.974	UNSAT TT=1.578 ASSIGNMENTS=2075 CHECKS=144580	? TT=151.406	? TT=135.077
csp/bddLarge/ normalized-bdd-21-2713-15-79-16_ext.xml	UNSAT TT=20.834	UNSAT TT=75.513 CHECKS=2848 NODES=1647	UNSAT TT=87.346 CHECKS=2839 NODES=1666	? (problem) TT=551.773	? (TO) TT=1800.22	? (TO) TT=1800.14	UNSAT TT=517.709 ASSIGNMENTS=976	UNSAT TT=987.725 ASSIGNMENTS=1403	UNSAT TT=103.609 CHECKS=95101700 NODES=2251	? (TO) TT=1800.03	? (TO) TT=1800.07	UNSAT TT=39.029 ASSIGNMENTS=993 CHECKS=2558580	UNSAT TT=40.195 ASSIGNMENTS=993 CHECKS=2558580	? (TO) TT=1800.06	? (TO) TT=1800.07	UNSAT TT=20.834 CHECKS=1878100	UNSAT TT=30.634 ASSIGNMENTS=996 CHECKS=2564460	UNSAT TT=133.136 ASSIGNMENTS=5002 CHECKS=11662400	? (MO) TT=7.6	UNSAT TT=29.869 ASSIGNMENTS=1044 CHECKS=2653780	UNSAT TT=30.352 ASSIGNMENTS=993 CHECKS=2558580	? TT=87.929	UNSAT TT=2.027 ASSIGNMENTS=2094 CHECKS=258122	? TT=134.739	? TT=140.713
csp/bddLarge/ normalized-bdd-21-2713-15-79-17_ext.xml	UNSAT TT=17.901	UNSAT TT=77.791 CHECKS=2796 NODES=1534	UNSAT TT=78.278 CHECKS=2833 NODES=1546	? (problem) TT=535.58	? (TO) TT=1800.16	? (TO) TT=1800.15	UNSAT TT=665.002 ASSIGNMENTS=930	UNSAT TT=1087.51 ASSIGNMENTS=1353	UNSAT TT=126.499 CHECKS=121332000 NODES=3085	? (TO) TT=1800.01	? (TO) TT=1800.09	UNSAT TT=42.007 ASSIGNMENTS=918 CHECKS=2666600	UNSAT TT=38.352 ASSIGNMENTS=918 CHECKS=2666600	? (MO) TT=1774.34	? (MO) TT=1743.5	UNSAT TT=17.901 CHECKS=1868550	UNSAT TT=29.664 ASSIGNMENTS=936 CHECKS=2712710	UNSAT TT=134.993 ASSIGNMENTS=4888 CHECKS=11483900	? (MO) TT=7.01	UNSAT TT=33.097 ASSIGNMENTS=972 CHECKS=2785700	UNSAT TT=30.193 ASSIGNMENTS=918 CHECKS=2666600	? TT=92.833	UNSAT TT=1.544 ASSIGNMENTS=2103 CHECKS=147952	? TT=147.356	? TT=141.49
csp/bddLarge/ normalized-bdd-21-2713-15-79-18_ext.xml	UNSAT TT=17.426	UNSAT TT=73.469 CHECKS=3539 NODES=1698	UNSAT TT=80.459 CHECKS=4097 NODES=1805	? (problem) TT=363.737	? (TO) TT=1800.18	? (TO) TT=1800.17	UNSAT TT=501.132 ASSIGNMENTS=1001	UNSAT TT=911.154 ASSIGNMENTS=1349	UNSAT TT=100.868 CHECKS=90786100 NODES=2209	? (TO) TT=1800.01	? (TO) TT=1800.07	UNSAT TT=35.383 ASSIGNMENTS=931 CHECKS=2229850	UNSAT TT=36.148 ASSIGNMENTS=931 CHECKS=2229850	? (MO) TT=1786.03	? (TO) TT=1800	UNSAT TT=17.426 CHECKS=1998940	UNSAT TT=28.616 ASSIGNMENTS=997 CHECKS=2507850	UNSAT TT=143.498 ASSIGNMENTS=5280 CHECKS=12357500	? (MO) TT=7.4	UNSAT TT=29.391 ASSIGNMENTS=1048 CHECKS=2608780	UNSAT TT=25.151 ASSIGNMENTS=931 CHECKS=2229850	? TT=91.204	UNSAT TT=1.715 ASSIGNMENTS=2107 CHECKS=146214	? TT=112.317	? TT=107.073
csp/bddLarge/ normalized-bdd-21-2713-15-79-19_ext.xml	SAT TT=12.706	SAT TT=24.235 CHECKS=3185 NODES=321	SAT TT=28.389 CHECKS=3237 NODES=317	? (problem) TT=521.08	SAT TT=562.963 CHECKS=1050710	SAT TT=941.565 CHECKS=1870630	SAT TT=393.514 ASSIGNMENTS=876	SAT TT=1001.32 ASSIGNMENTS=1236	SAT TT=119.394 CHECKS=120110000 NODES=2190	SAT TT=13.143 CHECKS=17534700 NODES=318	SAT TT=39.935 CHECKS=66394700 NODES=1169	SAT TT=36.854 ASSIGNMENTS=884 CHECKS=2219740	SAT TT=36.295 ASSIGNMENTS=884 CHECKS=2219740	? (TO) TT=1800.01	? (MO) TT=1766.44	SAT TT=12.706 CHECKS=1298280	SAT TT=26.447 ASSIGNMENTS=905 CHECKS=2287370	SAT TT=143.58 ASSIGNMENTS=5223 CHECKS=12342200	? (MO) TT=6.89	SAT TT=31.06 ASSIGNMENTS=972 CHECKS=2451620	SAT TT=27.76 ASSIGNMENTS=884 CHECKS=2219740	? TT=95.267	SAT TT=1.799 ASSIGNMENTS=2053 CHECKS=183799	? TT=143.301	? TT=151.378
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddLarge/ normalized-bdd-21-2713-15-79-1_ext.xml	UNSAT TT=21.264	UNSAT TT=79.051 CHECKS=3188 NODES=1542	UNSAT TT=91.595 CHECKS=3075 NODES=1592	? (problem) TT=554.699	? (TO) TT=1800.13	? (TO) TT=1800.15	UNSAT TT=681.082 ASSIGNMENTS=988	UNSAT TT=986.172 ASSIGNMENTS=1474	UNSAT TT=107.558 CHECKS=98932400 NODES=2240	? (TO) TT=1800.01	? (TO) TT=1800.03	UNSAT TT=40.515 ASSIGNMENTS=975 CHECKS=2814450	UNSAT TT=42.407 ASSIGNMENTS=975 CHECKS=2814450	? (MO) TT=1708.4	? (TO) TT=1800.02	UNSAT TT=21.264 CHECKS=1971060	UNSAT TT=33.669 ASSIGNMENTS=985 CHECKS=2835870	UNSAT TT=140.165 ASSIGNMENTS=4787 CHECKS=11448900	? (MO) TT=7.18	UNSAT TT=34.457 ASSIGNMENTS=1095 CHECKS=3045100	UNSAT TT=33.91 ASSIGNMENTS=975 CHECKS=2814450	? TT=97.517	UNSAT TT=1.606 ASSIGNMENTS=2120 CHECKS=162036	? TT=151.893	? TT=162.59
csp/bddLarge/ normalized-bdd-21-2713-15-79-20_ext.xml	UNSAT TT=20.718	UNSAT TT=88.17 CHECKS=3545 NODES=1668	UNSAT TT=85.059 CHECKS=3548 NODES=1651	? (problem) TT=557.04	? (TO) TT=1800.18	? (TO) TT=1800.15	UNSAT TT=541.651 ASSIGNMENTS=1033	UNSAT TT=1151.26 ASSIGNMENTS=1430	UNSAT TT=105.423 CHECKS=102038000 NODES=2222	? (TO) TT=1800.07	? (TO) TT=1800.04	UNSAT TT=35.547 ASSIGNMENTS=1042 CHECKS=2604580	UNSAT TT=37.139 ASSIGNMENTS=1042 CHECKS=2604580	? (MO) TT=1753.71	? (MO) TT=1773.83	UNSAT TT=20.718 CHECKS=2070290	UNSAT TT=27.151 ASSIGNMENTS=1041 CHECKS=2600350	UNSAT TT=120.811 ASSIGNMENTS=4747 CHECKS=11343200	? (MO) TT=8.2	UNSAT TT=27.744 ASSIGNMENTS=1078 CHECKS=2664880	UNSAT TT=29.358 ASSIGNMENTS=1042 CHECKS=2604580	? TT=85.767	UNSAT TT=1.738 ASSIGNMENTS=2165 CHECKS=182874	? TT=199.88	? TT=194.024
csp/bddLarge/ normalized-bdd-21-2713-15-79-21_ext.xml	UNSAT TT=18.996	UNSAT TT=70.61 CHECKS=3717 NODES=1633	UNSAT TT=72.345 CHECKS=3676 NODES=1644	? (problem) TT=523.827	? (TO) TT=1800.2	? (TO) TT=1800.15	UNSAT TT=629.111 ASSIGNMENTS=998	UNSAT TT=1171.35 ASSIGNMENTS=1513	UNSAT TT=127.925 CHECKS=132136000 NODES=3066	? (TO) TT=1800.05	? (TO) TT=1800.05	UNSAT TT=38.647 ASSIGNMENTS=991 CHECKS=2586350	UNSAT TT=36.693 ASSIGNMENTS=991 CHECKS=2586350	? (MO) TT=1707.37	? (TO) TT=1800.05	UNSAT TT=18.996 CHECKS=2045950	UNSAT TT=32.226 ASSIGNMENTS=1026 CHECKS=2612970	UNSAT TT=128.656 ASSIGNMENTS=4740 CHECKS=11184900	? (MO) TT=7.41	UNSAT TT=26.622 ASSIGNMENTS=997 CHECKS=2244880	UNSAT TT=30.417 ASSIGNMENTS=991 CHECKS=2586350	? TT=107.501	UNSAT TT=1.98 ASSIGNMENTS=2248 CHECKS=238372	? TT=169.887	? TT=160.315
csp/bddLarge/ normalized-bdd-21-2713-15-79-22_ext.xml	SAT TT=1.674	SAT TT=20.597 CHECKS=2847 NODES=176	SAT TT=8.439 CHECKS=2713 NODES=0	? (problem) TT=523.495	SAT TT=1014.21 CHECKS=2266570	SAT TT=187.636 CHECKS=359718	SAT TT=12.879 ASSIGNMENTS=12	SAT TT=12.889 ASSIGNMENTS=16	SAT TT=29.545 CHECKS=141102 NODES=21	SAT TT=52.193 CHECKS=70578300 NODES=1297	SAT TT=176.532 CHECKS=84183200 NODES=1476	SAT TT=1.674 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.674 ASSIGNMENTS=11 CHECKS=40695	? (TO) TT=1800.01	? (MO) TT=1758.46	SAT TT=7.559 CHECKS=777147	SAT TT=1.676 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.785 ASSIGNMENTS=12 CHECKS=40695	? (MO) TT=6.81	SAT TT=1.706 ASSIGNMENTS=15 CHECKS=40695	SAT TT=1.675 ASSIGNMENTS=11 CHECKS=40695	? TT=104.144	SAT TT=0.38 ASSIGNMENTS=21 CHECKS=6883	? TT=168.226	? TT=170.135
csp/bddLarge/ normalized-bdd-21-2713-15-79-23_ext.xml	UNSAT TT=22.809	UNSAT TT=74.582 CHECKS=4326 NODES=1854	UNSAT TT=76.905 CHECKS=4041 NODES=1801	? (problem) TT=572.871	? (TO) TT=1800.17	? (TO) TT=1800.15	UNSAT TT=1093.63 ASSIGNMENTS=1162	UNSAT TT=1028.95 ASSIGNMENTS=1505	UNSAT TT=97.584 CHECKS=95862300 NODES=2202	? (TO) TT=1800.06	? (TO) TT=1800.05	UNSAT TT=44.808 ASSIGNMENTS=1124 CHECKS=3237550	UNSAT TT=48.073 ASSIGNMENTS=1124 CHECKS=3237550	? (MO) TT=1751.48	? (TO) TT=1800.02	UNSAT TT=22.809 CHECKS=2331460	UNSAT TT=44.149 ASSIGNMENTS=1235 CHECKS=3864730	UNSAT TT=138.79 ASSIGNMENTS=5229 CHECKS=12165000	? (MO) TT=7.2	UNSAT TT=34.917 ASSIGNMENTS=1113 CHECKS=3053670	UNSAT TT=39.371 ASSIGNMENTS=1124 CHECKS=3237550	? TT=94.747	UNSAT TT=2.113 ASSIGNMENTS=2360 CHECKS=280553	? TT=141.156	? TT=129.483
csp/bddLarge/ normalized-bdd-21-2713-15-79-24_ext.xml	SAT TT=1.521	SAT TT=63.096 CHECKS=4879 NODES=1171	SAT TT=7.305 CHECKS=2713 NODES=0	? (problem) TT=560.595	SAT TT=184.045 CHECKS=340856	? (TO) TT=1800.15	SAT TT=14.671 ASSIGNMENTS=11	SAT TT=13.35 ASSIGNMENTS=12	SAT TT=29.266 CHECKS=260494 NODES=21	SAT TT=207.989 CHECKS=367836000 NODES=6774	SAT TT=330.506 CHECKS=71950900 NODES=1247	SAT TT=1.64 ASSIGNMENTS=10 CHECKS=40695	SAT TT=1.654 ASSIGNMENTS=10 CHECKS=40695	? (MO) TT=1794.2	? (MO) TT=1765.67	SAT TT=7.031 CHECKS=700049	SAT TT=1.644 ASSIGNMENTS=10 CHECKS=40695	SAT TT=1.661 ASSIGNMENTS=10 CHECKS=40695	? (MO) TT=7.88	SAT TT=1.567 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.521 ASSIGNMENTS=10 CHECKS=40695	? TT=93.055	SAT TT=0.386 ASSIGNMENTS=21 CHECKS=5545	? TT=132.094	? TT=124.069
csp/bddLarge/ normalized-bdd-21-2713-15-79-25_ext.xml	UNSAT TT=21.108	UNSAT TT=81.523 CHECKS=1797 NODES=1652	UNSAT TT=96.102 CHECKS=1492 NODES=1642	? (problem) TT=571.979	? (TO) TT=1800.19	? (TO) TT=1800.2	UNSAT TT=421.713 ASSIGNMENTS=1085	UNSAT TT=931.27 ASSIGNMENTS=1341	UNSAT TT=97.406 CHECKS=93518300 NODES=2214	? (TO) TT=1800	? (TO) TT=1800.01	UNSAT TT=40.802 ASSIGNMENTS=1057 CHECKS=2495840	UNSAT TT=38.952 ASSIGNMENTS=1057 CHECKS=2495840	? (MO) TT=1797.16	? (TO) TT=1800.09	UNSAT TT=21.108 CHECKS=1966000	UNSAT TT=31.174 ASSIGNMENTS=1057 CHECKS=2539990	UNSAT TT=138.226 ASSIGNMENTS=4779 CHECKS=11388300	? (MO) TT=7.4	UNSAT TT=32.27 ASSIGNMENTS=1140 CHECKS=2708360	UNSAT TT=29.836 ASSIGNMENTS=1057 CHECKS=2495840	? TT=88.911	UNSAT TT=2.156 ASSIGNMENTS=2213 CHECKS=297305	? TT=160.342	? TT=156.877
csp/bddLarge/ normalized-bdd-21-2713-15-79-26_ext.xml	UNSAT TT=23.759	UNSAT TT=98.128 CHECKS=1368 NODES=1693	UNSAT TT=104.274 CHECKS=1370 NODES=1683	? (problem) TT=533.471	? (TO) TT=1800.22	? (TO) TT=1800.18	UNSAT TT=602.975 ASSIGNMENTS=956	UNSAT TT=1202.17 ASSIGNMENTS=1409	UNSAT TT=107.385 CHECKS=101914000 NODES=2214	? (TO) TT=1800.01	? (TO) TT=1800.04	UNSAT TT=42.613 ASSIGNMENTS=979 CHECKS=2764020	UNSAT TT=42.652 ASSIGNMENTS=979 CHECKS=2764020	? (MO) TT=1769.31	? (MO) TT=1771.59	UNSAT TT=23.759 CHECKS=2038140	UNSAT TT=31.94 ASSIGNMENTS=939 CHECKS=2658320	UNSAT TT=131.122 ASSIGNMENTS=4930 CHECKS=11677200	? (MO) TT=7.39	UNSAT TT=33.012 ASSIGNMENTS=999 CHECKS=2767690	UNSAT TT=33.678 ASSIGNMENTS=979 CHECKS=2764020	? TT=93.217	UNSAT TT=1.691 ASSIGNMENTS=2097 CHECKS=177503	? TT=167.076	? TT=166.835
csp/bddLarge/ normalized-bdd-21-2713-15-79-27_ext.xml	UNSAT TT=21.921	UNSAT TT=82.754 CHECKS=2387 NODES=1534	UNSAT TT=87.021 CHECKS=2342 NODES=1537	? (problem) TT=531.873	? (TO) TT=1800.24	? (TO) TT=1800.15	UNSAT TT=330.266 ASSIGNMENTS=973	UNSAT TT=1257.88 ASSIGNMENTS=1490	UNSAT TT=130.771 CHECKS=127027000 NODES=3086	? (TO) TT=1800.09	? (TO) TT=1800.01	UNSAT TT=35.764 ASSIGNMENTS=944 CHECKS=2194040	UNSAT TT=36.166 ASSIGNMENTS=944 CHECKS=2194040	? (TO) TT=1800.05	? (MO) TT=1672.59	UNSAT TT=21.921 CHECKS=1926410	UNSAT TT=27.884 ASSIGNMENTS=976 CHECKS=2287120	UNSAT TT=144.546 ASSIGNMENTS=5284 CHECKS=12327000	? (MO) TT=7.38	UNSAT TT=30.34 ASSIGNMENTS=1012 CHECKS=2360410	UNSAT TT=27.207 ASSIGNMENTS=944 CHECKS=2194040	? TT=90.256	UNSAT TT=1.77 ASSIGNMENTS=2113 CHECKS=227342	? TT=126.605	? TT=134.017
csp/bddLarge/ normalized-bdd-21-2713-15-79-28_ext.xml	UNSAT TT=20.517	UNSAT TT=77.034 CHECKS=4120 NODES=1576	UNSAT TT=82.767 CHECKS=4052 NODES=1582	? (problem) TT=531.038	? (TO) TT=1800.16	? (TO) TT=1800.22	UNSAT TT=413.635 ASSIGNMENTS=1004	UNSAT TT=1060.19 ASSIGNMENTS=1306	UNSAT TT=126.338 CHECKS=135596000 NODES=3184	? (TO) TT=1800.05	? (TO) TT=1800.04	UNSAT TT=35.451 ASSIGNMENTS=953 CHECKS=2253460	UNSAT TT=34.492 ASSIGNMENTS=953 CHECKS=2253460	? (TO) TT=1800	? (MO) TT=1787.75	UNSAT TT=20.517 CHECKS=2042390	UNSAT TT=26.304 ASSIGNMENTS=959 CHECKS=2307450	UNSAT TT=118.932 ASSIGNMENTS=4754 CHECKS=11379900	? (MO) TT=8.94	UNSAT TT=29.614 ASSIGNMENTS=1037 CHECKS=2460350	UNSAT TT=28.019 ASSIGNMENTS=953 CHECKS=2253460	? TT=97.051	UNSAT TT=1.818 ASSIGNMENTS=2236 CHECKS=223828	? TT=149.407	? TT=149.912
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddLarge/ normalized-bdd-21-2713-15-79-29_ext.xml	UNSAT TT=19.765	UNSAT TT=80.187 CHECKS=2747 NODES=1642	UNSAT TT=84.888 CHECKS=2547 NODES=1654	? (problem) TT=564.747	? (TO) TT=1800.22	? (TO) TT=1800.13	UNSAT TT=820.25 ASSIGNMENTS=1031	UNSAT TT=1099.31 ASSIGNMENTS=1459	UNSAT TT=119.048 CHECKS=124596000 NODES=3110	? (TO) TT=1800.05	? (TO) TT=1800.04	UNSAT TT=49.585 ASSIGNMENTS=1137 CHECKS=3521360	UNSAT TT=49.579 ASSIGNMENTS=1137 CHECKS=3521360	? (TO) TT=1800.02	? (MO) TT=1677	UNSAT TT=19.765 CHECKS=2134890	UNSAT TT=39.881 ASSIGNMENTS=1137 CHECKS=3490600	UNSAT TT=133.568 ASSIGNMENTS=4952 CHECKS=11626300	? (MO) TT=7.4	UNSAT TT=39.112 ASSIGNMENTS=1140 CHECKS=3542300	UNSAT TT=40.714 ASSIGNMENTS=1137 CHECKS=3521360	? TT=91.676	UNSAT TT=1.833 ASSIGNMENTS=2178 CHECKS=178302	? TT=84.678	? TT=85.312
csp/bddLarge/ normalized-bdd-21-2713-15-79-2_ext.xml	UNSAT TT=21.554	UNSAT TT=89.376 CHECKS=1339 NODES=1571	UNSAT TT=104.163 CHECKS=1345 NODES=1666	? (problem) TT=570.639	? (TO) TT=1800.18	? (TO) TT=1800.17	UNSAT TT=442.809 ASSIGNMENTS=962	UNSAT TT=1194.14 ASSIGNMENTS=1509	UNSAT TT=125.819 CHECKS=133972000 NODES=3131	? (TO) TT=1800.04	? (TO) TT=1800.03	UNSAT TT=37.628 ASSIGNMENTS=963 CHECKS=2328100	UNSAT TT=37.107 ASSIGNMENTS=963 CHECKS=2328100	? (TO) TT=1800.08	? (TO) TT=1800	UNSAT TT=21.554 CHECKS=1938450	UNSAT TT=29.372 ASSIGNMENTS=962 CHECKS=2328390	UNSAT TT=138.306 ASSIGNMENTS=4973 CHECKS=11741900	? (MO) TT=7.29	UNSAT TT=29.98 ASSIGNMENTS=1034 CHECKS=2295360	UNSAT TT=29.067 ASSIGNMENTS=963 CHECKS=2328100	? TT=90.104	UNSAT TT=1.831 ASSIGNMENTS=2103 CHECKS=202244	? TT=147.454	? TT=148.083
csp/bddLarge/ normalized-bdd-21-2713-15-79-30_ext.xml	UNSAT TT=21.331	UNSAT TT=83.927 CHECKS=2535 NODES=1457	UNSAT TT=96.291 CHECKS=3024 NODES=1471	? (problem) TT=560.038	? (TO) TT=1800.15	? (TO) TT=1800.2	UNSAT TT=225.84 ASSIGNMENTS=914	UNSAT TT=903.81 ASSIGNMENTS=1439	UNSAT TT=100.898 CHECKS=94215000 NODES=2171	? (TO) TT=1800.04	? (TO) TT=1800.03	UNSAT TT=34.278 ASSIGNMENTS=962 CHECKS=2120180	UNSAT TT=34.163 ASSIGNMENTS=962 CHECKS=2120180	? (TO) TT=1800	? (TO) TT=1800.07	UNSAT TT=21.331 CHECKS=1854620	UNSAT TT=26.104 ASSIGNMENTS=937 CHECKS=2167260	UNSAT TT=120.832 ASSIGNMENTS=4961 CHECKS=11627100	? (MO) TT=6.75	UNSAT TT=26.655 ASSIGNMENTS=992 CHECKS=2235430	UNSAT TT=27.932 ASSIGNMENTS=962 CHECKS=2120180	? TT=91.612	UNSAT TT=1.564 ASSIGNMENTS=2076 CHECKS=141327	? TT=144.477	? TT=149.962
csp/bddLarge/ normalized-bdd-21-2713-15-79-31_ext.xml	SAT TT=1.416	SAT TT=25.094 CHECKS=2948 NODES=295	SAT TT=7.568 CHECKS=2713 NODES=0	? (problem) TT=550.534	SAT TT=835.331 CHECKS=1404870	SAT TT=78.225 CHECKS=96014	SAT TT=14.484 ASSIGNMENTS=12	SAT TT=14.192 ASSIGNMENTS=13	SAT TT=29.282 CHECKS=219837 NODES=21	SAT TT=47.308 CHECKS=62951700 NODES=1156	SAT TT=171.452 CHECKS=266278000 NODES=4392	SAT TT=1.518 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.552 ASSIGNMENTS=11 CHECKS=40695	? (MO) TT=1775.42	? (TO) TT=1800.04	SAT TT=10.77 CHECKS=960413	SAT TT=1.416 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.521 ASSIGNMENTS=11 CHECKS=40695	? (MO) TT=7.6	SAT TT=1.512 ASSIGNMENTS=12 CHECKS=40695	SAT TT=1.519 ASSIGNMENTS=11 CHECKS=40695	? TT=95.87	SAT TT=0.378 ASSIGNMENTS=21 CHECKS=9428	? TT=139.997	? TT=151.922
csp/bddLarge/ normalized-bdd-21-2713-15-79-32_ext.xml	SAT TT=3.027	SAT TT=11.171 CHECKS=2856 NODES=71	SAT TT=17.687 CHECKS=2931 NODES=136	? (problem) TT=557.038	SAT TT=730.293 CHECKS=1490360	SAT TT=82.244 CHECKS=73361	SAT TT=626.32 ASSIGNMENTS=924	SAT TT=402.452 ASSIGNMENTS=636	SAT TT=116.036 CHECKS=118115000 NODES=2203	SAT TT=156.758 CHECKS=229957000 NODES=4233	SAT TT=5.284 CHECKS=954016 NODES=12	SAT TT=36.966 ASSIGNMENTS=836 CHECKS=2338640	SAT TT=34.601 ASSIGNMENTS=836 CHECKS=2338640	? (MO) TT=1684.51	? (MO) TT=1764.72	SAT TT=3.027 CHECKS=270372	SAT TT=27.435 ASSIGNMENTS=841 CHECKS=2364290	SAT TT=128.792 ASSIGNMENTS=4840 CHECKS=11519000	? (MO) TT=7.2	SAT TT=25.547 ASSIGNMENTS=813 CHECKS=2211900	SAT TT=27.928 ASSIGNMENTS=836 CHECKS=2338640	? TT=90.115	SAT TT=1.888 ASSIGNMENTS=2046 CHECKS=250096	? TT=151.479	? TT=136.284
csp/bddLarge/ normalized-bdd-21-2713-15-79-33_ext.xml	UNSAT TT=22.624	UNSAT TT=81.111 CHECKS=3946 NODES=1895	UNSAT TT=86.823 CHECKS=4783 NODES=2058	? (problem) TT=524.143	? (TO) TT=1800.17	? (TO) TT=1800.13	UNSAT TT=340.465 ASSIGNMENTS=1126	UNSAT TT=1023.77 ASSIGNMENTS=1344	UNSAT TT=97.739 CHECKS=95900300 NODES=2183	? (TO) TT=1800.03	? (TO) TT=1800.09	UNSAT TT=33.247 ASSIGNMENTS=970 CHECKS=2060490	UNSAT TT=34.564 ASSIGNMENTS=970 CHECKS=2060490	? (MO) TT=1761.8	? (MO) TT=1704.85	UNSAT TT=22.624 CHECKS=2236630	UNSAT TT=28.275 ASSIGNMENTS=1096 CHECKS=2374740	UNSAT TT=134.257 ASSIGNMENTS=4760 CHECKS=11402500	? (MO) TT=7.1	UNSAT TT=30.053 ASSIGNMENTS=1150 CHECKS=2472890	UNSAT TT=25.261 ASSIGNMENTS=970 CHECKS=2060490	? TT=90.335	UNSAT TT=2.036 ASSIGNMENTS=2238 CHECKS=299933	? TT=181.85	? TT=173.487
csp/bddLarge/ normalized-bdd-21-2713-15-79-34_ext.xml	SAT TT=2.241	SAT TT=67.185 CHECKS=4695 NODES=1123	SAT TT=68.939 CHECKS=4700 NODES=1124	? (problem) TT=517.224	? (TO) TT=1800.18	SAT TT=265.753 CHECKS=474798	SAT TT=609.566 ASSIGNMENTS=1066	SAT TT=946.836 ASSIGNMENTS=1070	SAT TT=95.502 CHECKS=92391700 NODES=2211	SAT TT=64.401 CHECKS=93727100 NODES=1722	SAT TT=783.315 CHECKS=1057450000 NODES=17108	SAT TT=48.646 ASSIGNMENTS=1105 CHECKS=3179260	SAT TT=48.076 ASSIGNMENTS=1105 CHECKS=3179260	? (MO) TT=1705.64	? (TO) TT=1800	SAT TT=13.906 CHECKS=1340990	SAT TT=36.212 ASSIGNMENTS=1069 CHECKS=3029320	SAT TT=137.616 ASSIGNMENTS=4952 CHECKS=11539100	? (MO) TT=6.91	SAT TT=2.241 ASSIGNMENTS=23 CHECKS=62038	SAT TT=38.703 ASSIGNMENTS=1105 CHECKS=3179260	? TT=90.337	SAT TT=1.93 ASSIGNMENTS=2218 CHECKS=198321	? TT=185.216	? TT=175.483
csp/bddLarge/ normalized-bdd-21-2713-15-79-35_ext.xml	SAT TT=1.302	SAT TT=69.451 CHECKS=5172 NODES=1378	SAT TT=74.186 CHECKS=4964 NODES=1347	? (problem) TT=527.731	SAT TT=631.137 CHECKS=1280510	SAT TT=675.649 CHECKS=1275640	SAT TT=553.814 ASSIGNMENTS=954	SAT TT=556.58 ASSIGNMENTS=868	SAT TT=96.699 CHECKS=94045200 NODES=2201	SAT TT=147.74 CHECKS=260319000 NODES=4793	? (TO) TT=1800.09	SAT TT=42.673 ASSIGNMENTS=996 CHECKS=2734450	SAT TT=44.302 ASSIGNMENTS=996 CHECKS=2734450	? (TO) TT=1800.02	? (MO) TT=1796.56	SAT TT=1.302 CHECKS=87052	SAT TT=33.955 ASSIGNMENTS=964 CHECKS=2680310	SAT TT=133.788 ASSIGNMENTS=5065 CHECKS=11840400	? (MO) TT=7.5	SAT TT=33.097 ASSIGNMENTS=997 CHECKS=2740220	SAT TT=34.676 ASSIGNMENTS=996 CHECKS=2734450	? TT=87.471	SAT TT=1.681 ASSIGNMENTS=2096 CHECKS=205316	? TT=141.829	? TT=141.975
csp/bddLarge/ normalized-bdd-21-2713-15-79-3_ext.xml	UNSAT TT=20.884	UNSAT TT=71.732 CHECKS=3499 NODES=1584	UNSAT TT=87.981 CHECKS=3032 NODES=1500	? (problem) TT=515.502	? (TO) TT=1800.21	? (TO) TT=1800.16	UNSAT TT=373.027 ASSIGNMENTS=945	UNSAT TT=1136.28 ASSIGNMENTS=1415	UNSAT TT=99.899 CHECKS=99449700 NODES=2183	? (TO) TT=1800.04	? (TO) TT=1800.03	UNSAT TT=35.265 ASSIGNMENTS=923 CHECKS=2101300	UNSAT TT=32.607 ASSIGNMENTS=923 CHECKS=2101300	? (TO) TT=1800	? (MO) TT=1757.63	UNSAT TT=20.884 CHECKS=1912900	UNSAT TT=27.83 ASSIGNMENTS=955 CHECKS=2188010	UNSAT TT=137.473 ASSIGNMENTS=5229 CHECKS=12318400	? (MO) TT=7.6	UNSAT TT=28.369 ASSIGNMENTS=1016 CHECKS=2276350	UNSAT TT=24.649 ASSIGNMENTS=923 CHECKS=2101300	? TT=89.025	UNSAT TT=1.856 ASSIGNMENTS=2093 CHECKS=192376	? TT=238.94	? TT=232.528
csp/bddLarge/ normalized-bdd-21-2713-15-79-4_ext.xml	SAT TT=1.586	SAT TT=21.646 CHECKS=2932 NODES=226	SAT TT=7.677 CHECKS=2713 NODES=0	? (problem) TT=525.085	SAT TT=772.201 CHECKS=1426610	SAT TT=1181.53 CHECKS=2432610	SAT TT=14.224 ASSIGNMENTS=13	SAT TT=13.406 ASSIGNMENTS=22	SAT TT=31.338 CHECKS=192623 NODES=21	SAT TT=16.492 CHECKS=14935400 NODES=271	SAT TT=123.605 CHECKS=79153300 NODES=1412	SAT TT=1.614 ASSIGNMENTS=12 CHECKS=40695	SAT TT=1.723 ASSIGNMENTS=12 CHECKS=40695	? (MO) TT=1745.33	? (MO) TT=1790.98	SAT TT=5.711 CHECKS=556387	SAT TT=1.706 ASSIGNMENTS=12 CHECKS=40695	SAT TT=1.721 ASSIGNMENTS=12 CHECKS=40695	? (MO) TT=7.07	SAT TT=1.613 ASSIGNMENTS=21 CHECKS=40695	SAT TT=1.586 ASSIGNMENTS=12 CHECKS=40695	? TT=97.744	SAT TT=0.384 ASSIGNMENTS=21 CHECKS=9939	? TT=150.631	? TT=142.071
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddLarge/ normalized-bdd-21-2713-15-79-5_ext.xml	SAT TT=1.335	SAT TT=56.391 CHECKS=3713 NODES=893	SAT TT=7.571 CHECKS=2713 NODES=0	? (problem) TT=523.602	SAT TT=666.73 CHECKS=1414240	SAT TT=881.401 CHECKS=1754550	SAT TT=13.055 ASSIGNMENTS=12	SAT TT=14.067 ASSIGNMENTS=12	SAT TT=29.538 CHECKS=255057 NODES=21	SAT TT=39.547 CHECKS=59021300 NODES=1082	SAT TT=1181.2 CHECKS=1191160000 NODES=18178	SAT TT=1.514 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.521 ASSIGNMENTS=11 CHECKS=40695	? (MO) TT=1750.97	? (MO) TT=1743.03	SAT TT=1.335 CHECKS=105438	SAT TT=1.524 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.497 ASSIGNMENTS=11 CHECKS=40695	? (MO) TT=7.1	SAT TT=1.465 ASSIGNMENTS=11 CHECKS=40695	SAT TT=1.47 ASSIGNMENTS=11 CHECKS=40695	? TT=98.243	SAT TT=0.372 ASSIGNMENTS=21 CHECKS=9361	? TT=217.428	? TT=198.989
csp/bddLarge/ normalized-bdd-21-2713-15-79-6_ext.xml	UNSAT TT=22.888	UNSAT TT=76.181 CHECKS=5316 NODES=2097	UNSAT TT=76.052 CHECKS=4113 NODES=1850	? (problem) TT=571.072	? (TO) TT=1800.12	? (TO) TT=1800.21	UNSAT TT=972.952 ASSIGNMENTS=1218	UNSAT TT=965.503 ASSIGNMENTS=1466	UNSAT TT=97.879 CHECKS=100176000 NODES=2195	? (TO) TT=1800.01	? (TO) TT=1800.04	UNSAT TT=38.699 ASSIGNMENTS=975 CHECKS=2554150	UNSAT TT=36.126 ASSIGNMENTS=975 CHECKS=2554150	? (MO) TT=1785.81	? (MO) TT=1695.23	UNSAT TT=22.888 CHECKS=2355680	UNSAT TT=33.071 ASSIGNMENTS=1071 CHECKS=2793970	UNSAT TT=151.94 ASSIGNMENTS=5177 CHECKS=12005300	? (MO) TT=7	UNSAT TT=32.418 ASSIGNMENTS=1089 CHECKS=2868430	UNSAT TT=29.988 ASSIGNMENTS=975 CHECKS=2554150	? TT=104.317	UNSAT TT=2.39 ASSIGNMENTS=2339 CHECKS=411939	? TT=150.56	? TT=152.108
csp/bddLarge/ normalized-bdd-21-2713-15-79-7_ext.xml	UNSAT TT=20.045	UNSAT TT=73.817 CHECKS=3979 NODES=1823	UNSAT TT=80.599 CHECKS=3864 NODES=1828	? (problem) TT=571.431	? (TO) TT=1800.21	? (TO) TT=1800.16	UNSAT TT=880.847 ASSIGNMENTS=1112	UNSAT TT=1022.82 ASSIGNMENTS=1421	UNSAT TT=101.629 CHECKS=92677300 NODES=2239	? (TO) TT=1800.02	? (TO) TT=1800.1	UNSAT TT=47.983 ASSIGNMENTS=1083 CHECKS=3176710	UNSAT TT=48.465 ASSIGNMENTS=1083 CHECKS=3176710	? (TO) TT=1800	? (MO) TT=1688.72	UNSAT TT=20.045 CHECKS=2500500	UNSAT TT=37.42 ASSIGNMENTS=1037 CHECKS=3267150	UNSAT TT=131.437 ASSIGNMENTS=4955 CHECKS=11741300	? (MO) TT=7.6	UNSAT TT=36.206 ASSIGNMENTS=1075 CHECKS=3342090	UNSAT TT=36.466 ASSIGNMENTS=1083 CHECKS=3176710	? TT=93.456	UNSAT TT=1.861 ASSIGNMENTS=2138 CHECKS=206666	? TT=80.34	? TT=80.275
csp/bddLarge/ normalized-bdd-21-2713-15-79-8_ext.xml	SAT TT=1.479	SAT TT=39.424 CHECKS=4801 NODES=526	SAT TT=7.722 CHECKS=2713 NODES=0	? (problem) TT=528.732	SAT TT=646.048 CHECKS=1134900	SAT TT=454.389 CHECKS=715287	SAT TT=12.961 ASSIGNMENTS=11	SAT TT=12.866 ASSIGNMENTS=14	SAT TT=31.814 CHECKS=210120 NODES=21	SAT TT=42.29 CHECKS=74186000 NODES=1363	SAT TT=619.153 CHECKS=821644000 NODES=12224	SAT TT=1.809 ASSIGNMENTS=10 CHECKS=40695	SAT TT=1.793 ASSIGNMENTS=10 CHECKS=40695	? (MO) TT=1769.14	? (MO) TT=1794.91	SAT TT=15.893 CHECKS=1356620	SAT TT=1.786 ASSIGNMENTS=10 CHECKS=40695	SAT TT=1.747 ASSIGNMENTS=10 CHECKS=40695	? (MO) TT=7.4	SAT TT=1.479 ASSIGNMENTS=13 CHECKS=40695	SAT TT=1.783 ASSIGNMENTS=10 CHECKS=40695	? TT=100.806	SAT TT=0.376 ASSIGNMENTS=21 CHECKS=6325	? TT=134.363	? TT=142.448
csp/bddLarge/ normalized-bdd-21-2713-15-79-9_ext.xml	UNSAT TT=19.999	UNSAT TT=83.219 CHECKS=3030 NODES=1531	UNSAT TT=91.779 CHECKS=3319 NODES=1552	? (problem) TT=571.444	? (TO) TT=1800.19	? (TO) TT=1800.13	UNSAT TT=556.598 ASSIGNMENTS=978	UNSAT TT=1237.96 ASSIGNMENTS=1447	UNSAT TT=117.461 CHECKS=125061000 NODES=2181	? (TO) TT=1800.09	? (TO) TT=1800.06	UNSAT TT=36.518 ASSIGNMENTS=1034 CHECKS=2599890	UNSAT TT=39.232 ASSIGNMENTS=1034 CHECKS=2599890	? (MO) TT=1779.2	? (MO) TT=1717.1	UNSAT TT=19.999 CHECKS=1966230	UNSAT TT=30.788 ASSIGNMENTS=997 CHECKS=2518740	UNSAT TT=141.407 ASSIGNMENTS=4864 CHECKS=11536200	? (MO) TT=7.4	UNSAT TT=31.853 ASSIGNMENTS=1044 CHECKS=2608730	UNSAT TT=30.368 ASSIGNMENTS=1034 CHECKS=2599890	? TT=86	UNSAT TT=1.665 ASSIGNMENTS=2129 CHECKS=177514	? TT=135.336	? TT=135.77
csp/bddSmall/ normalized-bdd-21-133-18-78-10_ext.xml	SAT TT=0.714	SAT TT=31.417 CHECKS=11568 NODES=4444	SAT TT=4.436 CHECKS=133 NODES=0	SAT TT=143.153	? (TO) TT=1800.03	? (TO) TT=1800.1	SAT TT=6.573 ASSIGNMENTS=17	SAT TT=6.797 ASSIGNMENTS=22	SAT TT=12.562 CHECKS=10640 NODES=21	SAT TT=9.581 CHECKS=17916700 NODES=4623	SAT TT=2 CHECKS=1295600 NODES=299	SAT TT=0.786 ASSIGNMENTS=16 CHECKS=2394	SAT TT=0.714 ASSIGNMENTS=16 CHECKS=2394	? (MO) TT=597.44	? (MO) TT=557.11	SAT TT=8.114 CHECKS=1016120	SAT TT=0.77 ASSIGNMENTS=16 CHECKS=2394	SAT TT=0.704 ASSIGNMENTS=16 CHECKS=2394	? (MO) TT=6.9	SAT TT=0.793 ASSIGNMENTS=21 CHECKS=2394	SAT TT=0.787 ASSIGNMENTS=16 CHECKS=2394	? TT=100.094	SAT TT=0.999 ASSIGNMENTS=21 CHECKS=775	? TT=95.111	? TT=108.712
csp/bddSmall/ normalized-bdd-21-133-18-78-11_ext.xml	SAT TT=0.801	SAT TT=39.633 CHECKS=26849 NODES=8913	SAT TT=51.35 CHECKS=28653 NODES=9183	SAT TT=227.687	SAT TT=910.008 CHECKS=181944	? (TO) TT=1800.05	? (TO) TT=1800.14	? (TO) TT=1800.13	SAT TT=74.59 CHECKS=66037600 NODES=24505	SAT TT=2.407 CHECKS=1448240 NODES=370	SAT TT=84.083 CHECKS=122010000 NODES=19217	SAT TT=24.1 ASSIGNMENTS=10143 CHECKS=1645510	SAT TT=22.712 ASSIGNMENTS=10143 CHECKS=1645510	? (MO) TT=581.37	? (MO) TT=541.75	SAT TT=3.245 CHECKS=268178	SAT TT=15.351 ASSIGNMENTS=9960 CHECKS=1647540	SAT TT=23.924 ASSIGNMENTS=16692 CHECKS=2346960	? (MO) TT=7.29	SAT TT=0.801 ASSIGNMENTS=29 CHECKS=4050	SAT TT=15.959 ASSIGNMENTS=10143 CHECKS=1645510	? TT=105.684	SAT TT=7.553 ASSIGNMENTS=18609 CHECKS=1471740	? TT=223.17	? TT=227.275
csp/bddSmall/ normalized-bdd-21-133-18-78-12_ext.xml	UNSAT TT=12.089	UNSAT TT=49.652 CHECKS=80918 NODES=21031	UNSAT TT=61.526 CHECKS=62322 NODES=16438	UNSAT TT=251.288	? (TO) TT=1800.03	? (TO) TT=1800.01	? (TO) TT=1800.11	? (TO) TT=1800.09	UNSAT TT=61.7 CHECKS=62633200 NODES=24042	? (TO) TT=1800.04	? (TO) TT=1800.07	UNSAT TT=25.876 ASSIGNMENTS=11516 CHECKS=1773070	UNSAT TT=27.005 ASSIGNMENTS=11516 CHECKS=1773070	? (MO) TT=592.97	? (MO) TT=557.54	UNSAT TT=12.089 CHECKS=1464290	UNSAT TT=18.785 ASSIGNMENTS=11986 CHECKS=1778540	UNSAT TT=21.749 ASSIGNMENTS=16635 CHECKS=2347980	? (MO) TT=7.1	UNSAT TT=16.496 ASSIGNMENTS=10496 CHECKS=1486790	UNSAT TT=18.511 ASSIGNMENTS=11516 CHECKS=1773070	? TT=106.065	UNSAT TT=5.743 ASSIGNMENTS=19103 CHECKS=976016	? TT=221.837	? TT=221.075
csp/bddSmall/ normalized-bdd-21-133-18-78-13_ext.xml	UNSAT TT=15.11	UNSAT TT=86.699 CHECKS=62729 NODES=17802	UNSAT TT=93.276 CHECKS=54869 NODES=16026	UNSAT TT=271.282	? (TO) TT=1800.04	? (TO) TT=1800.07	? (TO) TT=1800.13	? (TO) TT=1800.12	UNSAT TT=70.573 CHECKS=63872600 NODES=23912	? (TO) TT=1800.07	? (TO) TT=1800.05	UNSAT TT=23.505 ASSIGNMENTS=10062 CHECKS=1513130	UNSAT TT=23.537 ASSIGNMENTS=10062 CHECKS=1513130	? (MO) TT=543.38	? (MO) TT=552.42	UNSAT TT=19.475 CHECKS=1320750	UNSAT TT=16.015 ASSIGNMENTS=10652 CHECKS=1582110	UNSAT TT=21.701 ASSIGNMENTS=14811 CHECKS=2110990	? (MO) TT=7.3	UNSAT TT=16.736 ASSIGNMENTS=10564 CHECKS=1543190	UNSAT TT=15.11 ASSIGNMENTS=10062 CHECKS=1513130	? TT=90.725	UNSAT TT=9.541 ASSIGNMENTS=18943 CHECKS=1798090	? TT=125.03	? TT=126.479
csp/bddSmall/ normalized-bdd-21-133-18-78-14_ext.xml	UNSAT TT=12.937	UNSAT TT=55.155 CHECKS=71926 NODES=19555	UNSAT TT=70.867 CHECKS=71691 NODES=19487	UNSAT TT=242.868	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.1	UNSAT TT=89.287 CHECKS=82401600 NODES=34610	? (TO) TT=1800.03	? (TO) TT=1800	UNSAT TT=23.022 ASSIGNMENTS=10037 CHECKS=1516500	UNSAT TT=25.235 ASSIGNMENTS=10037 CHECKS=1516500	? (MO) TT=595.56	? (MO) TT=568.02	UNSAT TT=12.937 CHECKS=1282110	UNSAT TT=16.472 ASSIGNMENTS=10610 CHECKS=1474700	UNSAT TT=24.076 ASSIGNMENTS=15503 CHECKS=2217940	? (MO) TT=7.01	UNSAT TT=16.64 ASSIGNMENTS=11579 CHECKS=1536460	UNSAT TT=16.798 ASSIGNMENTS=10037 CHECKS=1516500	? TT=79.517	UNSAT TT=7.554 ASSIGNMENTS=19169 CHECKS=1444690	? TT=239.534	? TT=234.845
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddSmall/ normalized-bdd-21-133-18-78-15_ext.xml	UNSAT TT=13.111	UNSAT TT=60.022 CHECKS=56615 NODES=17053	UNSAT TT=68.782 CHECKS=63200 NODES=18124	UNSAT TT=248.323	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.12	? (TO) TT=1800.1	UNSAT TT=66.274 CHECKS=65794900 NODES=24053	? (TO) TT=1800.05	? (TO) TT=1800.01	UNSAT TT=24.985 ASSIGNMENTS=10147 CHECKS=1682160	UNSAT TT=23.332 ASSIGNMENTS=10147 CHECKS=1682160	? (MO) TT=555.51	? (MO) TT=558.69	UNSAT TT=13.111 CHECKS=1322890	UNSAT TT=17.438 ASSIGNMENTS=11469 CHECKS=1762530	UNSAT TT=21.585 ASSIGNMENTS=15567 CHECKS=2236000	? (MO) TT=7.78	UNSAT TT=17.222 ASSIGNMENTS=11661 CHECKS=1765920	UNSAT TT=16.425 ASSIGNMENTS=10147 CHECKS=1682160	? TT=95.284	UNSAT TT=7.372 ASSIGNMENTS=19762 CHECKS=1466590	? TT=122.463	? TT=122.985
csp/bddSmall/ normalized-bdd-21-133-18-78-16_ext.xml	UNSAT TT=15.544	UNSAT TT=58.602 CHECKS=67531 NODES=18434	UNSAT TT=66.19 CHECKS=70573 NODES=18531	UNSAT TT=275.808	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.15	? (TO) TT=1800.13	UNSAT TT=64.081 CHECKS=60196900 NODES=24854	? (TO) TT=1800.09	? (TO) TT=1800.03	UNSAT TT=24.869 ASSIGNMENTS=11172 CHECKS=1573320	UNSAT TT=25.689 ASSIGNMENTS=11172 CHECKS=1573320	? (MO) TT=555.69	? (MO) TT=547.18	UNSAT TT=15.591 CHECKS=1373920	UNSAT TT=17.417 ASSIGNMENTS=10548 CHECKS=1663600	UNSAT TT=23.438 ASSIGNMENTS=15901 CHECKS=2259680	? (MO) TT=7.32	UNSAT TT=15.83 ASSIGNMENTS=10231 CHECKS=1571370	UNSAT TT=15.544 ASSIGNMENTS=11172 CHECKS=1573320	? TT=95.583	UNSAT TT=9.907 ASSIGNMENTS=20868 CHECKS=2298450	? TT=147.18	? TT=142.222
csp/bddSmall/ normalized-bdd-21-133-18-78-17_ext.xml	SAT TT=0.725	SAT TT=33.241 CHECKS=10260 NODES=4826	SAT TT=4.408 CHECKS=133 NODES=0	SAT TT=153.943	? (TO) TT=1800.09	? (TO) TT=1800.11	SAT TT=6.975 ASSIGNMENTS=14	SAT TT=6.953 ASSIGNMENTS=15	SAT TT=12.487 CHECKS=15673 NODES=21	SAT TT=3.259 CHECKS=4355540 NODES=1120	SAT TT=70.471 CHECKS=105941000 NODES=17944	SAT TT=0.796 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.793 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=549.44	? (MO) TT=597.14	SAT TT=5.053 CHECKS=485363	SAT TT=0.79 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.779 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=7.43	SAT TT=0.725 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.772 ASSIGNMENTS=13 CHECKS=2394	? TT=99.236	SAT TT=0.993 ASSIGNMENTS=21 CHECKS=417	? TT=122.009	? TT=126.753
csp/bddSmall/ normalized-bdd-21-133-18-78-18_ext.xml	SAT TT=0.768	SAT TT=29.54 CHECKS=9000 NODES=4153	SAT TT=4.221 CHECKS=133 NODES=0	SAT TT=133.505	? (TO) TT=1800.03	? (TO) TT=1800.05	SAT TT=7.502 ASSIGNMENTS=15	SAT TT=6.72 ASSIGNMENTS=14	SAT TT=13.25 CHECKS=16276 NODES=21	SAT TT=32.463 CHECKS=77831800 NODES=20092	SAT TT=33.902 CHECKS=48272100 NODES=7918	SAT TT=0.774 ASSIGNMENTS=15 CHECKS=2394	SAT TT=0.769 ASSIGNMENTS=15 CHECKS=2394	? (MO) TT=594.57	? (MO) TT=588.16	SAT TT=2.428 CHECKS=174515	SAT TT=0.768 ASSIGNMENTS=15 CHECKS=2394	SAT TT=0.759 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=7.4	SAT TT=0.792 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.786 ASSIGNMENTS=15 CHECKS=2394	? TT=91.612	SAT TT=0.967 ASSIGNMENTS=21 CHECKS=717	? TT=217.324	? TT=227.368
csp/bddSmall/ normalized-bdd-21-133-18-78-19_ext.xml	UNSAT TT=11.976	UNSAT TT=55.298 CHECKS=85411 NODES=22034	UNSAT TT=62.258 CHECKS=70291 NODES=19050	UNSAT TT=242.671	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.13	? (TO) TT=1800.12	UNSAT TT=62.824 CHECKS=62000600 NODES=24023	? (TO) TT=1800.03	? (TO) TT=1800.03	UNSAT TT=24.023 ASSIGNMENTS=11246 CHECKS=1675490	UNSAT TT=23.837 ASSIGNMENTS=11246 CHECKS=1675490	? (MO) TT=587.62	? (MO) TT=602.08	UNSAT TT=11.976 CHECKS=1313320	UNSAT TT=12.724 ASSIGNMENTS=10054 CHECKS=1471910	UNSAT TT=22.387 ASSIGNMENTS=15374 CHECKS=2235730	? (MO) TT=7.18	UNSAT TT=14.695 ASSIGNMENTS=10442 CHECKS=1606760	UNSAT TT=16.127 ASSIGNMENTS=11246 CHECKS=1675490	? TT=85.245	UNSAT TT=6.562 ASSIGNMENTS=20063 CHECKS=1244730	? TT=231.844	? TT=254.803
csp/bddSmall/ normalized-bdd-21-133-18-78-1_ext.xml	SAT TT=0.719	SAT TT=33.089 CHECKS=18106 NODES=6241	SAT TT=4.345 CHECKS=133 NODES=0	SAT TT=146.005	? (TO) TT=1800.07	SAT TT=569.106 CHECKS=114108	SAT TT=7.389 ASSIGNMENTS=14	SAT TT=7.363 ASSIGNMENTS=16	SAT TT=13.338 CHECKS=16817 NODES=21	SAT TT=1.695 CHECKS=544442 NODES=136	SAT TT=229.192 CHECKS=257826000 NODES=19620	SAT TT=0.77 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.779 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=600.22	? (MO) TT=553.03	SAT TT=13.331 CHECKS=1161870	SAT TT=0.733 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.73 ASSIGNMENTS=15 CHECKS=2394	? (MO) TT=7.89	SAT TT=0.719 ASSIGNMENTS=15 CHECKS=2394	SAT TT=0.797 ASSIGNMENTS=13 CHECKS=2394	? TT=81.62	SAT TT=0.955 ASSIGNMENTS=21 CHECKS=590	? TT=98.121	? TT=100.159
csp/bddSmall/ normalized-bdd-21-133-18-78-20_ext.xml	SAT TT=0.731	SAT TT=41.671 CHECKS=38439 NODES=11735	SAT TT=3.951 CHECKS=133 NODES=0	SAT TT=144.413	? (TO) TT=1800.1	? (TO) TT=1800.01	SAT TT=6.598 ASSIGNMENTS=15	SAT TT=7.258 ASSIGNMENTS=16	SAT TT=12.38 CHECKS=10119 NODES=21	SAT TT=5.196 CHECKS=9473980 NODES=2442	SAT TT=103.362 CHECKS=131512000 NODES=14884	SAT TT=0.789 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.794 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=596.18	? (MO) TT=557.43	SAT TT=3.753 CHECKS=381201	SAT TT=0.785 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.724 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=7.54	SAT TT=0.802 ASSIGNMENTS=15 CHECKS=2394	SAT TT=0.731 ASSIGNMENTS=14 CHECKS=2394	? TT=84.077	SAT TT=1.051 ASSIGNMENTS=21 CHECKS=467	? TT=238.746	? TT=238.797
csp/bddSmall/ normalized-bdd-21-133-18-78-21_ext.xml	UNSAT TT=14.384	UNSAT TT=58.504 CHECKS=63268 NODES=18664	UNSAT TT=65.704 CHECKS=59131 NODES=17021	UNSAT TT=254.492	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.15	UNSAT TT=69.807 CHECKS=61116800 NODES=24456	? (TO) TT=1800.06	? (TO) TT=1800.01	UNSAT TT=24.71 ASSIGNMENTS=10706 CHECKS=1579960	UNSAT TT=22.846 ASSIGNMENTS=10706 CHECKS=1579960	? (MO) TT=557.11	? (MO) TT=574.39	UNSAT TT=14.384 CHECKS=1300580	UNSAT TT=15.93 ASSIGNMENTS=10747 CHECKS=1588220	UNSAT TT=22.185 ASSIGNMENTS=15139 CHECKS=2149850	? (MO) TT=6.91	UNSAT TT=16.467 ASSIGNMENTS=10738 CHECKS=1561600	UNSAT TT=14.944 ASSIGNMENTS=10706 CHECKS=1579960	? TT=105.625	UNSAT TT=7.115 ASSIGNMENTS=19450 CHECKS=1444360	? TT=229.884	? TT=227.686
csp/bddSmall/ normalized-bdd-21-133-18-78-22_ext.xml	UNSAT TT=12.404	UNSAT TT=60.515 CHECKS=69110 NODES=18827	UNSAT TT=67.062 CHECKS=62628 NODES=17039	UNSAT TT=235.217	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.11	? (TO) TT=1800.14	UNSAT TT=66.403 CHECKS=60842600 NODES=24631	? (TO) TT=1800.05	? (TO) TT=1800.03	UNSAT TT=23.446 ASSIGNMENTS=10793 CHECKS=1600220	UNSAT TT=23.872 ASSIGNMENTS=10793 CHECKS=1600220	? (MO) TT=548.73	? (MO) TT=549.57	UNSAT TT=12.404 CHECKS=1220260	UNSAT TT=15.949 ASSIGNMENTS=10733 CHECKS=1618230	UNSAT TT=25.586 ASSIGNMENTS=17896 CHECKS=2503990	? (MO) TT=7.99	UNSAT TT=18.216 ASSIGNMENTS=10580 CHECKS=1784720	UNSAT TT=14.781 ASSIGNMENTS=10793 CHECKS=1600220	? TT=97.181	UNSAT TT=6.423 ASSIGNMENTS=18262 CHECKS=1358800	? TT=99.573	? TT=93.276
csp/bddSmall/ normalized-bdd-21-133-18-78-23_ext.xml	SAT TT=0.754	SAT TT=32.66 CHECKS=12478 NODES=4701	SAT TT=4.176 CHECKS=133 NODES=0	SAT TT=149.269	SAT TT=69.429 CHECKS=8047	? (TO) TT=1800.04	SAT TT=6.864 ASSIGNMENTS=15	SAT TT=7.328 ASSIGNMENTS=14	SAT TT=12.74 CHECKS=12991 NODES=21	SAT TT=20.616 CHECKS=43141400 NODES=11130	SAT TT=13.393 CHECKS=20080300 NODES=4961	SAT TT=0.773 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.789 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=548.53	? (MO) TT=588.33	SAT TT=2.171 CHECKS=127107	SAT TT=0.772 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.778 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=7.3	SAT TT=0.754 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.756 ASSIGNMENTS=14 CHECKS=2394	? TT=86.479	SAT TT=1.072 ASSIGNMENTS=21 CHECKS=360	? TT=292.122	? TT=296.447
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddSmall/ normalized-bdd-21-133-18-78-24_ext.xml	UNSAT TT=13.798	UNSAT TT=54.411 CHECKS=68899 NODES=18430	UNSAT TT=67.062 CHECKS=57164 NODES=16641	UNSAT TT=272.993	? (TO) TT=1800.01	? (TO) TT=1800.12	? (TO) TT=1800.07	? (TO) TT=1800.04	UNSAT TT=64.678 CHECKS=56626400 NODES=24515	? (TO) TT=1800.09	? (TO) TT=1800.08	UNSAT TT=25.874 ASSIGNMENTS=10291 CHECKS=1883190	UNSAT TT=25.933 ASSIGNMENTS=10291 CHECKS=1883190	? (MO) TT=560.1	? (MO) TT=561.91	UNSAT TT=14.311 CHECKS=1442150	UNSAT TT=15.214 ASSIGNMENTS=9579 CHECKS=1579640	UNSAT TT=21.111 ASSIGNMENTS=14879 CHECKS=2140840	? (MO) TT=7.79	UNSAT TT=13.798 ASSIGNMENTS=9266 CHECKS=1466150	UNSAT TT=16.499 ASSIGNMENTS=10291 CHECKS=1883190	? TT=88.236	UNSAT TT=7.636 ASSIGNMENTS=18876 CHECKS=1555390	? TT=262.944	? TT=259.693
csp/bddSmall/ normalized-bdd-21-133-18-78-25_ext.xml	SAT TT=0.717	SAT TT=44.395 CHECKS=17754 NODES=6507	SAT TT=4.215 CHECKS=133 NODES=0	SAT TT=142.159	? (TO) TT=1800.07	? (TO) TT=1800.02	SAT TT=7.31 ASSIGNMENTS=15	SAT TT=7.26 ASSIGNMENTS=18	SAT TT=13.383 CHECKS=11014 NODES=21	SAT TT=12.931 CHECKS=25220700 NODES=6506	SAT TT=1451.47 CHECKS=1415560000 NODES=35730	SAT TT=0.793 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.807 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=561.47	? (MO) TT=556.14	SAT TT=16.122 CHECKS=1247380	SAT TT=0.717 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.778 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=7.93	SAT TT=0.77 ASSIGNMENTS=17 CHECKS=2394	SAT TT=0.727 ASSIGNMENTS=14 CHECKS=2394	? TT=87.429	SAT TT=0.952 ASSIGNMENTS=21 CHECKS=917	? TT=274.729	? TT=296.427
csp/bddSmall/ normalized-bdd-21-133-18-78-26_ext.xml	SAT TT=0.723	SAT TT=24.775 CHECKS=8361 NODES=3300	SAT TT=3.975 CHECKS=133 NODES=0	SAT TT=145.535	? (TO) TT=1800.01	? (TO) TT=1800.08	SAT TT=6.415 ASSIGNMENTS=15	SAT TT=7.21 ASSIGNMENTS=15	SAT TT=12.383 CHECKS=17186 NODES=21	SAT TT=7.71 CHECKS=15585400 NODES=4019	SAT TT=98.444 CHECKS=128900000 NODES=16365	SAT TT=0.737 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.768 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=590.36	? (MO) TT=574.77	SAT TT=5.341 CHECKS=502716	SAT TT=0.792 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.759 ASSIGNMENTS=15 CHECKS=2394	? (MO) TT=7.1	SAT TT=0.723 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.758 ASSIGNMENTS=14 CHECKS=2394	? TT=87.596	SAT TT=0.965 ASSIGNMENTS=21 CHECKS=607	? TT=264.571	? TT=239.915
csp/bddSmall/ normalized-bdd-21-133-18-78-27_ext.xml	SAT TT=0.713	SAT TT=26.707 CHECKS=7401 NODES=2951	SAT TT=4.369 CHECKS=133 NODES=0	SAT TT=134.465	? (TO) TT=1800.03	? (TO) TT=1800.03	SAT TT=7.487 ASSIGNMENTS=15	SAT TT=7.358 ASSIGNMENTS=16	SAT TT=12.62 CHECKS=8393 NODES=21	SAT TT=4.743 CHECKS=7221190 NODES=1861	SAT TT=249.066 CHECKS=263496000 NODES=17084	SAT TT=0.713 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.794 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=596.49	? (MO) TT=549.89	SAT TT=0.775 CHECKS=27164	SAT TT=0.783 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.777 ASSIGNMENTS=15 CHECKS=2394	? (MO) TT=6.78	SAT TT=0.798 ASSIGNMENTS=15 CHECKS=2394	SAT TT=0.787 ASSIGNMENTS=14 CHECKS=2394	? TT=91.623	SAT TT=1.051 ASSIGNMENTS=21 CHECKS=501	? TT=292.533	? TT=277.87
csp/bddSmall/ normalized-bdd-21-133-18-78-28_ext.xml	UNSAT TT=12.778	UNSAT TT=49.741 CHECKS=72626 NODES=20985	UNSAT TT=62.727 CHECKS=68387 NODES=20089	UNSAT TT=252.09	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.12	? (TO) TT=1800.18	UNSAT TT=61.839 CHECKS=60510700 NODES=23937	? (TO) TT=1800	? (TO) TT=1800.04	UNSAT TT=24.121 ASSIGNMENTS=10326 CHECKS=1530210	UNSAT TT=24.674 ASSIGNMENTS=10326 CHECKS=1530210	? (MO) TT=574.22	? (MO) TT=540.41	UNSAT TT=12.778 CHECKS=1366560	UNSAT TT=17.569 ASSIGNMENTS=11822 CHECKS=1778470	UNSAT TT=20.043 ASSIGNMENTS=14997 CHECKS=2178820	? (MO) TT=6.81	UNSAT TT=14.987 ASSIGNMENTS=10020 CHECKS=1443980	UNSAT TT=16.394 ASSIGNMENTS=10326 CHECKS=1530210	? TT=89.438	UNSAT TT=6.846 ASSIGNMENTS=19022 CHECKS=1332520	? TT=93.414	? TT=105.629
csp/bddSmall/ normalized-bdd-21-133-18-78-29_ext.xml	UNSAT TT=12.13	UNSAT TT=63.654 CHECKS=65648 NODES=17331	UNSAT TT=73.221 CHECKS=65288 NODES=16953	UNSAT TT=223.995	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.13	? (TO) TT=1800.12	UNSAT TT=68.696 CHECKS=61114500 NODES=24648	? (TO) TT=1800.07	? (TO) TT=1800.06	UNSAT TT=25.944 ASSIGNMENTS=11498 CHECKS=1641480	UNSAT TT=25.348 ASSIGNMENTS=11498 CHECKS=1641480	? (MO) TT=553.35	? (MO) TT=590.91	UNSAT TT=12.13 CHECKS=1386000	UNSAT TT=16.527 ASSIGNMENTS=11769 CHECKS=1721300	UNSAT TT=19.961 ASSIGNMENTS=14800 CHECKS=2093360	? (MO) TT=7.29	UNSAT TT=15.019 ASSIGNMENTS=10851 CHECKS=1499380	UNSAT TT=17.372 ASSIGNMENTS=11498 CHECKS=1641480	? TT=97.33	UNSAT TT=7.028 ASSIGNMENTS=18668 CHECKS=1419660	? TT=135.204	? TT=131.137
csp/bddSmall/ normalized-bdd-21-133-18-78-2_ext.xml	SAT TT=1.264	SAT TT=32.214 CHECKS=22972 NODES=7545	SAT TT=40.965 CHECKS=26351 NODES=8437	SAT TT=158.923	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.1	SAT TT=72.653 CHECKS=64942100 NODES=26049	SAT TT=1.774 CHECKS=663297 NODES=166	SAT TT=203.114 CHECKS=223101000 NODES=16656	SAT TT=20.768 ASSIGNMENTS=8312 CHECKS=1292560	SAT TT=22.05 ASSIGNMENTS=8312 CHECKS=1292560	? (MO) TT=555.84	? (MO) TT=572.55	SAT TT=1.264 CHECKS=89502	SAT TT=12.254 ASSIGNMENTS=8245 CHECKS=1242660	SAT TT=0.883 ASSIGNMENTS=94 CHECKS=14021	? (MO) TT=7.05	SAT TT=9.321 ASSIGNMENTS=5863 CHECKS=1049050	SAT TT=13.859 ASSIGNMENTS=8312 CHECKS=1292560	? TT=85.614	SAT TT=1.786 ASSIGNMENTS=1902 CHECKS=171788	? TT=98.403	? TT=98.761
csp/bddSmall/ normalized-bdd-21-133-18-78-30_ext.xml	UNSAT TT=16.415	UNSAT TT=61.912 CHECKS=54050 NODES=16986	UNSAT TT=71.147 CHECKS=58082 NODES=17559	UNSAT TT=250.307	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.13	? (TO) TT=1800.11	UNSAT TT=71.227 CHECKS=59540100 NODES=24841	? (TO) TT=1800.05	? (TO) TT=1800.07	UNSAT TT=25.377 ASSIGNMENTS=11079 CHECKS=1742230	UNSAT TT=25.54 ASSIGNMENTS=11079 CHECKS=1742230	? (MO) TT=559.07	? (MO) TT=583.01	UNSAT TT=18.758 CHECKS=1431650	UNSAT TT=16.415 ASSIGNMENTS=11699 CHECKS=1598120	UNSAT TT=21.7 ASSIGNMENTS=16896 CHECKS=2347340	? (MO) TT=7.46	UNSAT TT=16.997 ASSIGNMENTS=12139 CHECKS=1662080	UNSAT TT=16.925 ASSIGNMENTS=11079 CHECKS=1742230	? TT=80.288	UNSAT TT=10.509 ASSIGNMENTS=22003 CHECKS=1576020	? TT=224.487	? TT=247.41
csp/bddSmall/ normalized-bdd-21-133-18-78-31_ext.xml	UNSAT TT=14.791	UNSAT TT=95.062 CHECKS=59416 NODES=17473	UNSAT TT=106.809 CHECKS=55033 NODES=18090	UNSAT TT=242.139	? (TO) TT=1800.08	? (TO) TT=1800.06	UNSAT TT=1554.51 ASSIGNMENTS=10289	? (TO) TT=1800.05	UNSAT TT=68.224 CHECKS=58346500 NODES=24228	? (TO) TT=1800.07	? (TO) TT=1800.08	UNSAT TT=22.982 ASSIGNMENTS=10508 CHECKS=1470000	UNSAT TT=21.747 ASSIGNMENTS=10508 CHECKS=1470000	? (MO) TT=558.83	? (MO) TT=591.98	UNSAT TT=18.057 CHECKS=1374040	UNSAT TT=15.381 ASSIGNMENTS=10624 CHECKS=1470240	UNSAT TT=20.571 ASSIGNMENTS=14711 CHECKS=2073900	? (MO) TT=6.84	UNSAT TT=16.03 ASSIGNMENTS=10964 CHECKS=1515920	UNSAT TT=14.791 ASSIGNMENTS=10508 CHECKS=1470000	? TT=90.919	UNSAT TT=7.92 ASSIGNMENTS=17936 CHECKS=1426550	? TT=262.218	? TT=256.199
csp/bddSmall/ normalized-bdd-21-133-18-78-32_ext.xml	SAT TT=0.72	SAT TT=9.431 CHECKS=551 NODES=316	SAT TT=4.181 CHECKS=133 NODES=0	SAT TT=144.006	? (TO) TT=1800.08	SAT TT=137.203 CHECKS=24194	SAT TT=7.106 ASSIGNMENTS=15	SAT TT=7.295 ASSIGNMENTS=17	SAT TT=13.23 CHECKS=10919 NODES=21	SAT TT=12.541 CHECKS=25508400 NODES=6582	SAT TT=22.129 CHECKS=35857900 NODES=6831	SAT TT=0.779 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.785 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=593.4	? (MO) TT=589.35	SAT TT=3.804 CHECKS=396554	SAT TT=0.72 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.781 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=6.91	SAT TT=0.781 ASSIGNMENTS=16 CHECKS=2394	SAT TT=0.758 ASSIGNMENTS=14 CHECKS=2394	? TT=95.526	SAT TT=1.03 ASSIGNMENTS=21 CHECKS=579	? TT=100.247	? TT=99.489
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/bddSmall/ normalized-bdd-21-133-18-78-33_ext.xml	UNSAT TT=16.157	UNSAT TT=58.092 CHECKS=58833 NODES=19701	UNSAT TT=62.823 CHECKS=65779 NODES=19303	UNSAT TT=251.674	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.16	? (TO) TT=1800.08	UNSAT TT=69.614 CHECKS=65493200 NODES=24400	? (TO) TT=1800.01	? (TO) TT=1800.02	UNSAT TT=26.887 ASSIGNMENTS=12381 CHECKS=1850770	UNSAT TT=26.966 ASSIGNMENTS=12381 CHECKS=1850770	? (MO) TT=577.3	? (MO) TT=578.12	UNSAT TT=21.246 CHECKS=1332060	UNSAT TT=16.157 ASSIGNMENTS=11345 CHECKS=1605130	UNSAT TT=25.087 ASSIGNMENTS=16918 CHECKS=2396730	? (MO) TT=7.11	UNSAT TT=16.478 ASSIGNMENTS=11335 CHECKS=1520940	UNSAT TT=17.331 ASSIGNMENTS=12381 CHECKS=1850770	? TT=80.257	UNSAT TT=8.136 ASSIGNMENTS=20011 CHECKS=2016650	? TT=163.377	? TT=164.594
csp/bddSmall/ normalized-bdd-21-133-18-78-34_ext.xml	SAT TT=4.658	SAT TT=33.215 CHECKS=22442 NODES=7265	SAT TT=35.779 CHECKS=16011 NODES=5864	SAT TT=153.264	? (TO) TT=1800.07	? (TO) TT=1800.06	SAT TT=900.01 ASSIGNMENTS=6026	? (TO) TT=1800.08	SAT TT=55.574 CHECKS=53835200 NODES=22016	SAT TT=9.887 CHECKS=19526500 NODES=5036	SAT TT=180.709 CHECKS=226250000 NODES=22308	SAT TT=18.784 ASSIGNMENTS=7106 CHECKS=982989	SAT TT=18.697 ASSIGNMENTS=7106 CHECKS=982989	? (MO) TT=607.36	? (MO) TT=597.19	SAT TT=11.356 CHECKS=1107490	SAT TT=8.536 ASSIGNMENTS=6107 CHECKS=855725	SAT TT=9.679 ASSIGNMENTS=7298 CHECKS=1046620	? (MO) TT=7.31	SAT TT=4.658 ASSIGNMENTS=3425 CHECKS=483507	SAT TT=9.61 ASSIGNMENTS=7106 CHECKS=982989	? TT=93.958	SAT TT=7.008 ASSIGNMENTS=18307 CHECKS=1522910	? TT=119.179	? TT=109.337
csp/bddSmall/ normalized-bdd-21-133-18-78-35_ext.xml	UNSAT TT=13.553	UNSAT TT=71.316 CHECKS=59807 NODES=16866	UNSAT TT=75.633 CHECKS=56666 NODES=18545	UNSAT TT=196.748	? (TO) TT=1800.08	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.06	UNSAT TT=67.369 CHECKS=57562200 NODES=24814	? (TO) TT=1800.06	? (TO) TT=1800.06	UNSAT TT=22.404 ASSIGNMENTS=11159 CHECKS=1524050	UNSAT TT=23.573 ASSIGNMENTS=11159 CHECKS=1524050	? (MO) TT=548.18	? (MO) TT=564.11	UNSAT TT=13.553 CHECKS=1256280	UNSAT TT=15.873 ASSIGNMENTS=10981 CHECKS=1506470	UNSAT TT=19.84 ASSIGNMENTS=14907 CHECKS=2150450	? (MO) TT=8.09	UNSAT TT=14.543 ASSIGNMENTS=10842 CHECKS=1486900	UNSAT TT=15.578 ASSIGNMENTS=11159 CHECKS=1524050	? TT=90.875	UNSAT TT=5.961 ASSIGNMENTS=19155 CHECKS=1123370	? TT=216.29	? TT=216.235
csp/bddSmall/ normalized-bdd-21-133-18-78-3_ext.xml	SAT TT=0.709	SAT TT=25.143 CHECKS=18934 NODES=5967	SAT TT=4.178 CHECKS=133 NODES=0	SAT TT=148.908	SAT TT=1056.7 CHECKS=223126	SAT TT=367.975 CHECKS=80899	SAT TT=6.593 ASSIGNMENTS=15	SAT TT=7.377 ASSIGNMENTS=15	SAT TT=13.35 CHECKS=11371 NODES=21	SAT TT=35.732 CHECKS=78450700 NODES=20255	SAT TT=186.843 CHECKS=230954000 NODES=22708	SAT TT=0.782 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.797 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=586.49	? (MO) TT=567.8	SAT TT=2.902 CHECKS=296067	SAT TT=0.78 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.762 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=7.6	SAT TT=0.778 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.709 ASSIGNMENTS=14 CHECKS=2394	? TT=87.334	SAT TT=1.06 ASSIGNMENTS=21 CHECKS=965	? TT=313.581	? TT=311.655
csp/bddSmall/ normalized-bdd-21-133-18-78-4_ext.xml	SAT TT=0.815	SAT TT=28.103 CHECKS=8638 NODES=3826	SAT TT=32.354 CHECKS=13615 NODES=4494	SAT TT=259.118	? (TO) TT=1800.03	SAT TT=458.504 CHECKS=120760	? (TO) TT=1800.1	SAT TT=1640.61 ASSIGNMENTS=5576	SAT TT=66.172 CHECKS=65313800 NODES=24481	SAT TT=12.115 CHECKS=24136800 NODES=6228	SAT TT=113.595 CHECKS=141529000 NODES=18435	SAT TT=23.907 ASSIGNMENTS=10886 CHECKS=1704260	SAT TT=25.224 ASSIGNMENTS=10886 CHECKS=1704260	? (MO) TT=557.78	? (MO) TT=575.58	SAT TT=4.393 CHECKS=385744	SAT TT=11.574 ASSIGNMENTS=7985 CHECKS=1296240	SAT TT=22.367 ASSIGNMENTS=15321 CHECKS=2252060	? (MO) TT=7.6	SAT TT=0.815 ASSIGNMENTS=29 CHECKS=4102	SAT TT=15.737 ASSIGNMENTS=10886 CHECKS=1704260	? TT=88.466	SAT TT=5.102 ASSIGNMENTS=10422 CHECKS=1187820	? TT=239.902	? TT=240.824
csp/bddSmall/ normalized-bdd-21-133-18-78-5_ext.xml	SAT TT=0.723	SAT TT=31.826 CHECKS=23174 NODES=7261	SAT TT=4.335 CHECKS=133 NODES=0	SAT TT=146.667	? (TO) TT=1800.11	SAT TT=1210.76 CHECKS=264689	SAT TT=6.373 ASSIGNMENTS=14	SAT TT=6.91 ASSIGNMENTS=15	SAT TT=12.62 CHECKS=13070 NODES=21	SAT TT=5.479 CHECKS=8566250 NODES=2210	SAT TT=204.982 CHECKS=228402000 NODES=20969	SAT TT=0.742 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.723 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=558.1	? (MO) TT=581.74	SAT TT=4.416 CHECKS=376924	SAT TT=0.776 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.763 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=6.8	SAT TT=0.808 ASSIGNMENTS=14 CHECKS=2394	SAT TT=0.777 ASSIGNMENTS=13 CHECKS=2394	? TT=93.341	SAT TT=1.033 ASSIGNMENTS=21 CHECKS=411	? TT=238.441	? TT=246.13
csp/bddSmall/ normalized-bdd-21-133-18-78-6_ext.xml	UNSAT TT=14.88	UNSAT TT=58.165 CHECKS=39854 NODES=15245	UNSAT TT=75.614 CHECKS=36826 NODES=14492	UNSAT TT=284.58	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.15	? (TO) TT=1800.06	UNSAT TT=70.985 CHECKS=61066800 NODES=24384	? (TO) TT=1800.09	? (TO) TT=1800.09	UNSAT TT=24.373 ASSIGNMENTS=11325 CHECKS=1587550	UNSAT TT=24.91 ASSIGNMENTS=11325 CHECKS=1587550	? (MO) TT=548.39	? (MO) TT=578.82	UNSAT TT=18.274 CHECKS=1373470	UNSAT TT=15.86 ASSIGNMENTS=10771 CHECKS=1560070	UNSAT TT=21.542 ASSIGNMENTS=15723 CHECKS=2247850	? (MO) TT=6.8	UNSAT TT=14.88 ASSIGNMENTS=10514 CHECKS=1467170	UNSAT TT=15.696 ASSIGNMENTS=11325 CHECKS=1587550	? TT=81.781	UNSAT TT=7.522 ASSIGNMENTS=18438 CHECKS=1510260	? TT=212.353	? TT=221.729
csp/bddSmall/ normalized-bdd-21-133-18-78-7_ext.xml	SAT TT=0.813	SAT TT=24.206 CHECKS=18492 NODES=6060	SAT TT=30.109 CHECKS=16344 NODES=5691	SAT TT=215.53	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.06	SAT TT=71.042 CHECKS=64632500 NODES=24477	SAT TT=7.2 CHECKS=12071900 NODES=3113	SAT TT=58.298 CHECKS=81116600 NODES=13809	SAT TT=23.952 ASSIGNMENTS=11935 CHECKS=1731840	SAT TT=23.728 ASSIGNMENTS=11935 CHECKS=1731840	? (MO) TT=587.91	? (MO) TT=589.29	SAT TT=10.423 CHECKS=1170080	SAT TT=14.717 ASSIGNMENTS=10260 CHECKS=1590280	SAT TT=20.27 ASSIGNMENTS=14689 CHECKS=2231340	? (MO) TT=7.11	SAT TT=0.813 ASSIGNMENTS=31 CHECKS=4119	SAT TT=16.372 ASSIGNMENTS=11935 CHECKS=1731840	? TT=85.604	SAT TT=6.664 ASSIGNMENTS=19737 CHECKS=1193090	? TT=109.834	? TT=117.544
csp/bddSmall/ normalized-bdd-21-133-18-78-8_ext.xml	SAT TT=0.716	SAT TT=19.655 CHECKS=4897 NODES=1923	SAT TT=4.354 CHECKS=133 NODES=0	SAT TT=135.041	SAT TT=502.312 CHECKS=108116	? (TO) TT=1800.1	SAT TT=7.098 ASSIGNMENTS=14	SAT TT=6.61 ASSIGNMENTS=19	SAT TT=13.442 CHECKS=10029 NODES=21	SAT TT=50.773 CHECKS=115221000 NODES=29765	SAT TT=323.324 CHECKS=346201000 NODES=22271	SAT TT=0.729 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.771 ASSIGNMENTS=13 CHECKS=2394	? (MO) TT=587.22	? (MO) TT=546.07	SAT TT=9.759 CHECKS=922513	SAT TT=0.716 ASSIGNMENTS=13 CHECKS=2394	SAT TT=0.72 ASSIGNMENTS=14 CHECKS=2394	? (MO) TT=7.12	SAT TT=0.772 ASSIGNMENTS=18 CHECKS=2394	SAT TT=0.766 ASSIGNMENTS=13 CHECKS=2394	? TT=81.822	SAT TT=1.031 ASSIGNMENTS=21 CHECKS=342	? TT=93.153	? TT=102.929
csp/bddSmall/ normalized-bdd-21-133-18-78-9_ext.xml	SAT TT=0.744	SAT TT=18.886 CHECKS=5197 NODES=2236	SAT TT=4.171 CHECKS=133 NODES=0	SAT TT=221.13	SAT TT=406.132 CHECKS=99353	? (TO) TT=1800.06	? (TO) TT=1800.14	SAT TT=6.751 ASSIGNMENTS=17	SAT TT=12.602 CHECKS=13421 NODES=21	SAT TT=3.132 CHECKS=4255100 NODES=1094	SAT TT=115.992 CHECKS=148509000 NODES=16458	SAT TT=20.033 ASSIGNMENTS=7765 CHECKS=1113110	SAT TT=19.065 ASSIGNMENTS=7765 CHECKS=1113110	? (MO) TT=598.82	? (MO) TT=576.94	SAT TT=3.339 CHECKS=376742	SAT TT=13.69 ASSIGNMENTS=9668 CHECKS=1345280	SAT TT=0.967 ASSIGNMENTS=215 CHECKS=33232	? (MO) TT=7.39	SAT TT=0.744 ASSIGNMENTS=16 CHECKS=2394	SAT TT=11.142 ASSIGNMENTS=7765 CHECKS=1113110	? TT=95.04	SAT TT=1.187 ASSIGNMENTS=222 CHECKS=27801	? TT=104.127	? TT=105.206
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/dag-half/ normalized-half-n25-d5-e56-r7-11.xml	SAT TT=308.316	? (TO) TT=1800.09	? (TO) TT=1800.08	? (problem) TT=162.158	? (TO) TT=1800.13	? (TO) TT=1800.1	? (TO) TT=1800.21	? (TO) TT=1800.18	SAT TT=1582.53 CHECKS=2785180000 NODES=3281760	SAT TT=676.028 CHECKS=3008380000 NODES=6609950	? (TO) TT=1800.06	SAT TT=513.531 ASSIGNMENTS=1487580 CHECKS=186631000	SAT TT=781.989 ASSIGNMENTS=1487580 CHECKS=186631000	? (TO) TT=1800.01	? (TO) TT=1800.03	SAT TT=308.316 CHECKS=81926400	SAT TT=509.482 ASSIGNMENTS=1493520 CHECKS=187453000	SAT TT=484.025 ASSIGNMENTS=1486420 CHECKS=186770000	? (MO) TT=42.83	SAT TT=520.44 ASSIGNMENTS=1493730 CHECKS=187439000	SAT TT=505.038 ASSIGNMENTS=1487580 CHECKS=186631000	? (TO) TT=1800.24	? (TO) TT=1800.07	? (MO) TT=12.94	? (MO) TT=13.901
csp/dag-half/ normalized-half-n25-d5-e56-r7-12.xml	SAT TT=75.674	? (TO) TT=1800.1	? (TO) TT=1800.17	? (problem) TT=167.205	? (TO) TT=1800.12	? (TO) TT=1800.1	? (TO) TT=1800.15	? (TO) TT=1800.49	? (TO) TT=1800.02	SAT TT=371.047 CHECKS=1538590000 NODES=3347100	? (TO) TT=1800.01	SAT TT=91.848 ASSIGNMENTS=245954 CHECKS=30807900	SAT TT=378.19 ASSIGNMENTS=245954 CHECKS=30807900	? (TO) TT=1800.08	? (TO) TT=1800.1	SAT TT=124.697 CHECKS=34281000	SAT TT=83.672 ASSIGNMENTS=248086 CHECKS=31128400	SAT TT=107.342 ASSIGNMENTS=337027 CHECKS=42427400	? (MO) TT=45.71	SAT TT=239.786 ASSIGNMENTS=774673 CHECKS=97773500	SAT TT=75.674 ASSIGNMENTS=245954 CHECKS=30807900	? (TO) TT=1800.22	? (TO) TT=1800.11	? (MO) TT=13.861	? (MO) TT=13.659
csp/dag-half/ normalized-half-n25-d5-e56-r7-13.xml	SAT TT=728.091	? (TO) TT=1800.16	? (TO) TT=1800.18	? (problem) TT=153.185	? (TO) TT=1800.1	? (TO) TT=1800.14	? (TO) TT=1800.19	? (TO) TT=1800.21	? (TO) TT=1800.01	SAT TT=728.091 CHECKS=3214990000 NODES=7037520	? (TO) TT=1800.04	SAT TT=929.588 ASSIGNMENTS=2585780 CHECKS=324727000	? (TO) TT=1800.08	? (TO) TT=1800.04	? (TO) TT=1800.07	SAT TT=729.686 CHECKS=207623000	SAT TT=837.047 ASSIGNMENTS=2595970 CHECKS=326054000	SAT TT=845.154 ASSIGNMENTS=2487240 CHECKS=312766000	? (MO) TT=48.69	SAT TT=848.883 ASSIGNMENTS=2490630 CHECKS=313309000	SAT TT=858.97 ASSIGNMENTS=2585780 CHECKS=324727000	? (TO) TT=1800.11	? (TO) TT=1800.03	? (MO) TT=14.54	? (MO) TT=13.679
csp/dag-half/ normalized-half-n25-d5-e56-r7-16.xml	SAT TT=98.032	SAT TT=1125.88 CHECKS=2742230 NODES=213114	SAT TT=1595.18 CHECKS=3878370 NODES=294866	? (problem) TT=150.389	? (TO) TT=1800.07	? (TO) TT=1800.11	? (TO) TT=1800.16	? (TO) TT=1800.12	? (TO) TT=1800	? (TO) TT=1800.03	? (TO) TT=1800.02	SAT TT=480.028 ASSIGNMENTS=1365190 CHECKS=165398000	SAT TT=740.208 ASSIGNMENTS=1365190 CHECKS=165398000	? (TO) TT=1800.09	? (TO) TT=1800.07	SAT TT=98.032 CHECKS=25152800	SAT TT=438.337 ASSIGNMENTS=1400520 CHECKS=169671000	SAT TT=336.319 ASSIGNMENTS=1032600 CHECKS=123553000	? (MO) TT=49.26	SAT TT=316.721 ASSIGNMENTS=939626 CHECKS=113901000	SAT TT=443.519 ASSIGNMENTS=1365190 CHECKS=165398000	? (TO) TT=1800.18	SAT TT=1043.04 ASSIGNMENTS=334688 CHECKS=170989000	? (MO) TT=12.979	? (MO) TT=14.83
csp/dag-half/ normalized-half-n25-d5-e56-r7-17.xml	SAT TT=40.662	? (TO) TT=1800.13	? (TO) TT=1800.16	? (problem) TT=164.588	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.21	? (TO) TT=1800.12	SAT TT=806.289 CHECKS=1220860000 NODES=1572910	SAT TT=965.371 CHECKS=4392430000 NODES=9746820	? (TO) TT=1800.04	SAT TT=210.071 ASSIGNMENTS=668650 CHECKS=81029900	SAT TT=503.902 ASSIGNMENTS=668650 CHECKS=81029900	? (TO) TT=1800.02	? (TO) TT=1800.02	SAT TT=495.351 CHECKS=148480000	SAT TT=214.568 ASSIGNMENTS=674529 CHECKS=81883600	SAT TT=35.315 ASSIGNMENTS=95803 CHECKS=11482000	? (MO) TT=45.87	SAT TT=40.662 ASSIGNMENTS=111780 CHECKS=13725100	SAT TT=211.764 ASSIGNMENTS=668650 CHECKS=81029900	? (TO) TT=1800.16	SAT TT=539.288 ASSIGNMENTS=178568 CHECKS=94965000	? (MO) TT=13.88	? (MO) TT=12.87
csp/dag-half/ normalized-half-n25-d5-e56-r7-1.xml	SAT TT=40.572	? (TO) TT=1800.14	? (TO) TT=1800.16	? (problem) TT=167.782	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.17	? (TO) TT=1800.16	SAT TT=338.648 CHECKS=498633000 NODES=608460	SAT TT=126.456 CHECKS=447761000 NODES=988037	? (TO) TT=1800.01	SAT TT=55.235 ASSIGNMENTS=117290 CHECKS=14562500	SAT TT=346.12 ASSIGNMENTS=117290 CHECKS=14562500	? (TO) TT=1800.05	? (TO) TT=1800.04	SAT TT=91.178 CHECKS=24388400	SAT TT=40.572 ASSIGNMENTS=119570 CHECKS=14862700	SAT TT=110.799 ASSIGNMENTS=324908 CHECKS=39793400	? (MO) TT=46.59	SAT TT=215.803 ASSIGNMENTS=649684 CHECKS=78876200	SAT TT=42.893 ASSIGNMENTS=117290 CHECKS=14562500	? (TO) TT=1800.2	? (TO) TT=1800.07	? (MO) TT=13.73	? (MO) TT=14.86
csp/dag-half/ normalized-half-n25-d5-e56-r7-21.xml	SAT TT=532.791	? (TO) TT=1800.17	? (TO) TT=1800.14	? (problem) TT=164.338	? (TO) TT=1800.12	? (TO) TT=1800.06	? (TO) TT=1800.13	? (TO) TT=1800.13	? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.02	SAT TT=532.791 CHECKS=153461000	? (TO) TT=1800.05	? (TO) TT=1800.05	? (MO) TT=48.32	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.22	? (TO) TT=1800.08	? (MO) TT=12.86	? (MO) TT=13.893
csp/dag-half/ normalized-half-n25-d5-e56-r7-24.xml	SAT TT=149.569	SAT TT=864.025 CHECKS=6857370 NODES=479120	SAT TT=870.579 CHECKS=6642060 NODES=463249	? (problem) TT=164.349	? (TO) TT=1800.12	? (TO) TT=1800.11	? (TO) TT=1800.2	? (TO) TT=1800.19	? (TO) TT=1800.02	SAT TT=173.449 CHECKS=668821000 NODES=1465980	? (TO) TT=1800.01	SAT TT=310.33 ASSIGNMENTS=995888 CHECKS=123714000	SAT TT=624.338 ASSIGNMENTS=995888 CHECKS=123714000	? (TO) TT=1800.08	? (TO) TT=1800.04	SAT TT=398.592 CHECKS=128278000	SAT TT=301.341 ASSIGNMENTS=940269 CHECKS=116803000	SAT TT=118.905 ASSIGNMENTS=362312 CHECKS=44125800	? (MO) TT=48.92	SAT TT=149.569 ASSIGNMENTS=454518 CHECKS=56459400	SAT TT=319.349 ASSIGNMENTS=995888 CHECKS=123714000	? (TO) TT=1800.16	? (TO) TT=1800.06	? (MO) TT=13.87	? (MO) TT=14.74
csp/dag-half/ normalized-half-n25-d5-e56-r7-25.xml	SAT TT=91.25	? (TO) TT=1800.14	? (TO) TT=1800.12	? (problem) TT=168.468	? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.2	? (TO) TT=1800.18	? (TO) TT=1800.09	SAT TT=324.933 CHECKS=1313900000 NODES=2897410	? (TO) TT=1800.01	SAT TT=391.368 ASSIGNMENTS=1258110 CHECKS=150317000	SAT TT=703.673 ASSIGNMENTS=1258110 CHECKS=150317000	? (TO) TT=1800.06	? (TO) TT=1800.08	SAT TT=91.25 CHECKS=24102800	SAT TT=391.885 ASSIGNMENTS=1263600 CHECKS=150946000	SAT TT=67.162 ASSIGNMENTS=208507 CHECKS=24540600	? (MO) TT=46.7	SAT TT=386.507 ASSIGNMENTS=1181160 CHECKS=144400000	SAT TT=395.637 ASSIGNMENTS=1258110 CHECKS=150317000	SAT TT=480.212	? (TO) TT=1800.05	? (MO) TT=13.685	? (MO) TT=14.66
csp/dag-half/ normalized-half-n25-d5-e56-r7-2.xml	SAT TT=13.322	? (TO) TT=1800.13	? (TO) TT=1800.16	? (problem) TT=166.396	? (TO) TT=1800.09	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800.12	SAT TT=1700.02 CHECKS=2699870000 NODES=3259260	SAT TT=243.163 CHECKS=942138000 NODES=2071290	? (TO) TT=1800.02	SAT TT=36.611 ASSIGNMENTS=60234 CHECKS=7283950	SAT TT=323.933 ASSIGNMENTS=60234 CHECKS=7283950	SAT TT=868.906	? (TO) TT=1800.05	SAT TT=133.317 CHECKS=37314000	SAT TT=24.809 ASSIGNMENTS=59760 CHECKS=7246990	SAT TT=23.793 ASSIGNMENTS=63491 CHECKS=7515460	? (MO) TT=43.51	SAT TT=13.322 ASSIGNMENTS=25618 CHECKS=3190790	SAT TT=25.007 ASSIGNMENTS=60234 CHECKS=7283950	? (TO) TT=1800.14	? (TO) TT=1800.09	? (MO) TT=13.84	? (MO) TT=13.884
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/dag-half/ normalized-half-n25-d5-e56-r7-3.xml	SAT TT=21.553	? (TO) TT=1800.12	? (TO) TT=1800.13	? (problem) TT=155.583	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.43	? (TO) TT=1800.17	SAT TT=851.618 CHECKS=1344230000 NODES=1582130	SAT TT=21.553 CHECKS=20907000 NODES=45278	? (TO) TT=1800.09	SAT TT=125.836 ASSIGNMENTS=311651 CHECKS=38911900	SAT TT=414.325 ASSIGNMENTS=311651 CHECKS=38911900	? (TO) TT=1800.06	? (TO) TT=1800.09	SAT TT=308.051 CHECKS=83887100	SAT TT=113.148 ASSIGNMENTS=317479 CHECKS=39682400	SAT TT=37.646 ASSIGNMENTS=96354 CHECKS=11823700	? (MO) TT=46.16	SAT TT=111.296 ASSIGNMENTS=312882 CHECKS=39097100	SAT TT=112.675 ASSIGNMENTS=311651 CHECKS=38911900	? (TO) TT=1800.22	? (TO) TT=1800.05	? (MO) TT=13.78	? (MO) TT=13.898
csp/dag-half/ normalized-half-n25-d5-e56-r7-4.xml	SAT TT=32.435	? (TO) TT=1800.18	? (TO) TT=1800.16	? (problem) TT=165.867	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.18	? (TO) TT=1800.19	? (TO) TT=1800	SAT TT=32.435 CHECKS=68955400 NODES=150490	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.04	SAT TT=671.545 CHECKS=200360000	? (TO) TT=1800.08	? (TO) TT=1800.05	? (MO) TT=45.15	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.18	? (TO) TT=1800.1	? (MO) TT=14.74	? (MO) TT=14.145
csp/dag-half/ normalized-half-n25-d5-e56-r7-5.xml	SAT TT=20.642	? (TO) TT=1800.16	? (TO) TT=1800.17	? (problem) TT=154.276	? (TO) TT=1800.12	? (TO) TT=1800.2	? (TO) TT=1800.11	? (TO) TT=1800.17	? (TO) TT=1800.03	SAT TT=702.247 CHECKS=2894530000 NODES=6277310	? (TO) TT=1800.06	SAT TT=1607.25 ASSIGNMENTS=4647660 CHECKS=566484000	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.02	SAT TT=202.792 CHECKS=59796600	SAT TT=1604.78 ASSIGNMENTS=4764390 CHECKS=581201000	SAT TT=1520.69 ASSIGNMENTS=4804690 CHECKS=586695000	? (MO) TT=48.7	SAT TT=20.642 ASSIGNMENTS=47615 CHECKS=5848230	SAT TT=1534.42 ASSIGNMENTS=4647660 CHECKS=566484000	? (TO) TT=1800.18	? (TO) TT=1800.08	? (MO) TT=13.79	? (MO) TT=13.34
csp/dag-half/ normalized-half-n25-d5-e56-r7-7.xml	SAT TT=38.111	? (TO) TT=1800.17	? (TO) TT=1800.15	? (problem) TT=165.277	? (TO) TT=1800.04	? (TO) TT=1800.08	? (TO) TT=1800.2	? (TO) TT=1800.21	SAT TT=1220.55 CHECKS=1913640000 NODES=2239300	SAT TT=91.084 CHECKS=298906000 NODES=653865	? (TO) TT=1800.06	SAT TT=52.797 ASSIGNMENTS=105448 CHECKS=13239700	SAT TT=342.595 ASSIGNMENTS=105448 CHECKS=13239700	? (TO) TT=1800.01	? (TO) TT=1800.09	SAT TT=370.15 CHECKS=100969000	SAT TT=38.111 ASSIGNMENTS=105880 CHECKS=13341200	SAT TT=142.834 ASSIGNMENTS=402193 CHECKS=50033200	? (MO) TT=49.51	SAT TT=653.118 ASSIGNMENTS=2027500 CHECKS=253531000	SAT TT=40.529 ASSIGNMENTS=105448 CHECKS=13239700	? (TO) TT=1800.22	? (TO) TT=1800.1	? (MO) TT=12.91	? (MO) TT=14.74
csp/dag-half/ normalized-half-n25-d5-e56-r7-8.xml	SAT TT=22.895	? (TO) TT=1800.2	? (TO) TT=1800.14	? (problem) TT=158.144	? (TO) TT=1800.21	? (TO) TT=1800.18	? (TO) TT=1800.16	? (TO) TT=1800.19	SAT TT=1119.14 CHECKS=1825350000 NODES=2213770	SAT TT=22.895 CHECKS=25425600 NODES=55535	? (TO) TT=1800.06	SAT TT=60.514 ASSIGNMENTS=136187 CHECKS=16975900	SAT TT=346.682 ASSIGNMENTS=136187 CHECKS=16975900	? (TO) TT=1800.04	? (TO) TT=1800.06	SAT TT=94.212 CHECKS=25235400	SAT TT=45.256 ASSIGNMENTS=137424 CHECKS=17186200	SAT TT=208.007 ASSIGNMENTS=667386 CHECKS=84586800	? (MO) TT=46.41	SAT TT=58.478 ASSIGNMENTS=166061 CHECKS=20684500	SAT TT=47.651 ASSIGNMENTS=136187 CHECKS=16975900	? (TO) TT=1800.15	SAT TT=1131.7 ASSIGNMENTS=373697 CHECKS=187595000	? (MO) TT=13.876	? (MO) TT=13.911
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-11.xml	UNSAT TT=26.248	UNSAT TT=84.097 CHECKS=43072 NODES=35306	UNSAT TT=99.048 CHECKS=91831 NODES=44218	? (problem) TT=291.472	? (TO) TT=1800.21	? (TO) TT=1800.14	? (TO) TT=1800.21	? (TO) TT=1800.18	UNSAT TT=372.16 CHECKS=1090440000 NODES=64596	? (TO) TT=1800.08	? (TO) TT=1800.01	UNSAT TT=99.476 ASSIGNMENTS=35306 CHECKS=43072	UNSAT TT=104.444 ASSIGNMENTS=35306 CHECKS=43072	? (problem) TT=116.444	? (problem) TT=100.596	UNSAT TT=26.248 CHECKS=440322	? (TO) TT=1800.12	? (TO) TT=1800.16	? (exit code) TT=39.201	? (TO) TT=1800.08	? (TO) TT=1800.13	? TT=150.002	UNSAT TT=80.913 ASSIGNMENTS=113957 CHECKS=9644100	? (MO) TT=24.31	? (MO) TT=25.83
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-12.xml	UNSAT TT=40.991	UNSAT TT=168.954 CHECKS=1939 NODES=44725	UNSAT TT=150.607 CHECKS=7248 NODES=51402	? (problem) TT=314.009	? (TO) TT=1800.14	? (TO) TT=1800.14	? (TO) TT=1800.28	? (TO) TT=1800.21	UNSAT TT=342.23 CHECKS=1068820000 NODES=67455	? (TO) TT=1800.01	? (TO) TT=1800.04	UNSAT TT=193.372 ASSIGNMENTS=44725 CHECKS=1939	UNSAT TT=196.009 ASSIGNMENTS=44725 CHECKS=1939	? (problem) TT=114.985	? (problem) TT=108.174	UNSAT TT=40.991 CHECKS=595917	? (TO) TT=1800.16	? (TO) TT=1800.08	? (exit code) TT=43.495	? (TO) TT=1800.06	? (TO) TT=1800.08	? TT=140.419	UNSAT TT=72.772 ASSIGNMENTS=141156 CHECKS=9895740	? (MO) TT=24.861	? (MO) TT=23.828
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-13.xml	UNSAT TT=42.289	UNSAT TT=98.188 CHECKS=3035 NODES=52121	UNSAT TT=97.817 CHECKS=3092 NODES=51952	? (problem) TT=314.873	? (TO) TT=1800.16	? (TO) TT=1800.15	? (TO) TT=1800.16	? (TO) TT=1800.23	UNSAT TT=329.211 CHECKS=943073000 NODES=61799	? (TO) TT=1800.05	? (TO) TT=1800.01	UNSAT TT=111.386 ASSIGNMENTS=52121 CHECKS=3035	UNSAT TT=105.743 ASSIGNMENTS=52121 CHECKS=3035	? (problem) TT=113.4	? (problem) TT=115.568	UNSAT TT=42.289 CHECKS=595170	? (TO) TT=1800.13	? (TO) TT=1800.08	? (exit code) TT=41.299	? (TO) TT=1800.09	? (TO) TT=1800.13	? TT=84.314	UNSAT TT=80.471 ASSIGNMENTS=106634 CHECKS=7878100	? (MO) TT=21.92	? (MO) TT=23.736
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-14.xml	UNSAT TT=40.138	UNSAT TT=142.676 CHECKS=24466 NODES=39065	UNSAT TT=133.081 CHECKS=23902 NODES=39173	? (problem) TT=292.923	? (TO) TT=1800.14	? (TO) TT=1800.13	? (TO) TT=1800.18	? (TO) TT=1800.24	UNSAT TT=377.463 CHECKS=1295460000 NODES=98824	? (TO) TT=1800.01	? (TO) TT=1800.08	UNSAT TT=140.811 ASSIGNMENTS=39065 CHECKS=24466	UNSAT TT=155.302 ASSIGNMENTS=39065 CHECKS=24466	? (problem) TT=110.735	? (problem) TT=116.889	UNSAT TT=40.138 CHECKS=602486	? (TO) TT=1800.12	? (TO) TT=1800.16	? (exit code) TT=45.386	? (TO) TT=1800.16	? (TO) TT=1800.11	? TT=164.268	UNSAT TT=55.471 ASSIGNMENTS=136554 CHECKS=5550630	? (MO) TT=23.392	? (MO) TT=23.49
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-15.xml	UNSAT TT=32.026	UNSAT TT=187.075 CHECKS=14847 NODES=37943	UNSAT TT=189.538 CHECKS=14853 NODES=38039	? (problem) TT=314.616	? (TO) TT=1800.14	? (TO) TT=1800.13	? (TO) TT=1800.21	? (TO) TT=1800.17	UNSAT TT=271.456 CHECKS=755958000 NODES=48617	? (TO) TT=1800.03	? (TO) TT=1800.03	UNSAT TT=214.958 ASSIGNMENTS=37943 CHECKS=14847	UNSAT TT=220.46 ASSIGNMENTS=37943 CHECKS=14847	? (problem) TT=105.065	? (problem) TT=114.16	UNSAT TT=32.026 CHECKS=458932	? (TO) TT=1800.08	? (TO) TT=1800.09	? (exit code) TT=43.744	? (TO) TT=1800.08	? (TO) TT=1800.13	? TT=141.129	UNSAT TT=73.836 ASSIGNMENTS=96898 CHECKS=7291470	? (MO) TT=24.746	? (MO) TT=23.77
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-17.xml	UNSAT TT=36.904	UNSAT TT=159.117 CHECKS=525 NODES=65819	UNSAT TT=134.654 CHECKS=464 NODES=65328	? (problem) TT=315.635	? (TO) TT=1800.17	? (TO) TT=1800.13	? (TO) TT=1800.21	? (TO) TT=1800.2	UNSAT TT=276.685 CHECKS=752824000 NODES=61313	? (TO) TT=1800.09	? (TO) TT=1800.06	UNSAT TT=157.531 ASSIGNMENTS=65819 CHECKS=525	UNSAT TT=170.128 ASSIGNMENTS=65819 CHECKS=525	? (problem) TT=119.791	? (problem) TT=120.89	UNSAT TT=36.904 CHECKS=447780	? (TO) TT=1800.15	? (TO) TT=1800.07	? (exit code) TT=46.763	? (TO) TT=1800.07	? (TO) TT=1800.11	? TT=146.021	UNSAT TT=87.69 ASSIGNMENTS=96119 CHECKS=11052700	? (MO) TT=25.939	? (MO) TT=27.014
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-19.xml	UNSAT TT=36.565	UNSAT TT=105.387 CHECKS=524 NODES=35244	UNSAT TT=108.507 CHECKS=547 NODES=35109	? (problem) TT=295.752	? (TO) TT=1800.18	? (TO) TT=1800.19	? (TO) TT=1800.22	? (TO) TT=1800.22	UNSAT TT=326.445 CHECKS=1034200000 NODES=61385	? (TO) TT=1800.09	? (TO) TT=1800.04	UNSAT TT=122.198 ASSIGNMENTS=35244 CHECKS=524	UNSAT TT=110.163 ASSIGNMENTS=35244 CHECKS=524	? (problem) TT=123.65	? (problem) TT=124.006	UNSAT TT=36.565 CHECKS=572657	? (TO) TT=1800.1	? (TO) TT=1800.13	? (exit code) TT=47.85	? (TO) TT=1800.09	? (TO) TT=1800.06	? TT=154.334	UNSAT TT=71.357 ASSIGNMENTS=92625 CHECKS=8583460	? (MO) TT=26.931	? (MO) TT=22.703
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-21.xml	UNSAT TT=43.103	UNSAT TT=164.095 CHECKS=334 NODES=82832	UNSAT TT=154.458 CHECKS=15291 NODES=50521	? (problem) TT=313.317	? (TO) TT=1800.17	? (TO) TT=1800.21	? (TO) TT=1800.24	? (TO) TT=1800.18	UNSAT TT=340.667 CHECKS=938490000 NODES=71885	? (TO) TT=1800.01	? (TO) TT=1800.05	UNSAT TT=175.041 ASSIGNMENTS=82832 CHECKS=334	UNSAT TT=171.401 ASSIGNMENTS=82832 CHECKS=334	? (problem) TT=116.192	? (problem) TT=100.488	UNSAT TT=43.103 CHECKS=412254	? (TO) TT=1800.08	? (TO) TT=1800.07	? (exit code) TT=38.878	? (TO) TT=1800.14	? (TO) TT=1800.15	? TT=71.575	UNSAT TT=83.86 ASSIGNMENTS=98877 CHECKS=10610800	? (MO) TT=23.33	? (MO) TT=25.782
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-22.xml	UNSAT TT=33.142	UNSAT TT=99.585 CHECKS=21188 NODES=54102	UNSAT TT=111.883 CHECKS=21318 NODES=54089	? (problem) TT=296.923	? (TO) TT=1800.16	? (TO) TT=1800.13	? (TO) TT=1800.25	? (TO) TT=1800.21	UNSAT TT=257.175 CHECKS=661532000 NODES=48494	? (TO) TT=1800.09	? (TO) TT=1800.02	UNSAT TT=110.938 ASSIGNMENTS=54102 CHECKS=21188	UNSAT TT=114.369 ASSIGNMENTS=54102 CHECKS=21188	? (problem) TT=112.427	? (problem) TT=110.798	UNSAT TT=33.142 CHECKS=396876	? (TO) TT=1800.13	? (TO) TT=1800.09	? (exit code) TT=46.648	? (TO) TT=1800.11	? (TO) TT=1800.11	? TT=151.591	UNSAT TT=63.67 ASSIGNMENTS=93084 CHECKS=8020060	? (MO) TT=27.71	? (MO) TT=21.808
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-25.xml	UNSAT TT=30.962	UNSAT TT=174.968 CHECKS=743 NODES=51744	UNSAT TT=170.964 CHECKS=2752 NODES=38066	? (problem) TT=295.8	? (TO) TT=1800.21	? (TO) TT=1800.18	? (TO) TT=1800.26	? (TO) TT=1800.17	UNSAT TT=258.619 CHECKS=680658000 NODES=42210	? (TO) TT=1800.09	? (TO) TT=1800.05	UNSAT TT=186.189 ASSIGNMENTS=51744 CHECKS=743	UNSAT TT=191.05 ASSIGNMENTS=51744 CHECKS=743	? (problem) TT=122.852	? (problem) TT=123.578	UNSAT TT=30.962 CHECKS=375539	? (TO) TT=1800.12	? (TO) TT=1800.1	? (exit code) TT=40.223	? (TO) TT=1800.08	? (TO) TT=1800.15	? TT=143.664	UNSAT TT=68.329 ASSIGNMENTS=88677 CHECKS=9643050	? (MO) TT=25.855	? (MO) TT=26.938
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-3.xml	UNSAT TT=34.77	UNSAT TT=145.116 CHECKS=7730 NODES=42442	UNSAT TT=243.391 CHECKS=1771 NODES=64539	? (problem) TT=309.471	? (TO) TT=1800.17	? (TO) TT=1800.02	? (TO) TT=1800.25	? (TO) TT=1801.01	UNSAT TT=447.26 CHECKS=1453470000 NODES=91478	? (TO) TT=1800.05	? (TO) TT=1800.04	UNSAT TT=161.713 ASSIGNMENTS=42442 CHECKS=7730	UNSAT TT=159.339 ASSIGNMENTS=42442 CHECKS=7730	? (problem) TT=113.395	? (problem) TT=107.255	UNSAT TT=34.77 CHECKS=491762	? (TO) TT=1800.1	? (TO) TT=1800.06	? (exit code) TT=45.697	? (TO) TT=1800.12	? (TO) TT=1800.16	? TT=148.956	UNSAT TT=104.672 ASSIGNMENTS=146016 CHECKS=19048600	? (MO) TT=26.71	? (MO) TT=26.69
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-5.xml	UNSAT TT=41.771	UNSAT TT=140.941 CHECKS=7386 NODES=42965	UNSAT TT=124.656 CHECKS=9076 NODES=49567	? (problem) TT=318.34	? (TO) TT=1800.21	? (TO) TT=1800.17	? (TO) TT=1800.27	? (TO) TT=1800.19	UNSAT TT=346.724 CHECKS=1039470000 NODES=65873	? (TO) TT=1800.08	? (TO) TT=1800.01	UNSAT TT=148.858 ASSIGNMENTS=42965 CHECKS=7386	UNSAT TT=152.611 ASSIGNMENTS=42965 CHECKS=7386	? (problem) TT=105.114	? (problem) TT=117.179	UNSAT TT=41.771 CHECKS=440207	? (TO) TT=1800.11	? (TO) TT=1800.07	? (exit code) TT=48.074	? (TO) TT=1800.06	? (TO) TT=1800.11	? TT=113.475	UNSAT TT=101.267 ASSIGNMENTS=99292 CHECKS=15823800	? (MO) TT=21.654	? (MO) TT=25.825
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-6.xml	UNSAT TT=43.342	UNSAT TT=163.938 CHECKS=4563 NODES=44412	UNSAT TT=139.875 CHECKS=1811 NODES=44103	? (problem) TT=293.97	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.16	? (TO) TT=1800.26	UNSAT TT=381.615 CHECKS=1148620000 NODES=64665	? (TO) TT=1800.02	? (TO) TT=1800.01	UNSAT TT=173.683 ASSIGNMENTS=44412 CHECKS=4563	UNSAT TT=178.124 ASSIGNMENTS=44412 CHECKS=4563	? (problem) TT=107.012	? (problem) TT=116.452	UNSAT TT=43.342 CHECKS=571807	? (TO) TT=1800.07	? (TO) TT=1800.15	? (exit code) TT=45.312	? (TO) TT=1800.09	? (TO) TT=1800.08	? TT=142.929	UNSAT TT=87.611 ASSIGNMENTS=124538 CHECKS=11166200	? (MO) TT=21.96	? (MO) TT=25.43
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-7.xml	UNSAT TT=36.219	UNSAT TT=75.778 CHECKS=1638 NODES=42749	UNSAT TT=82.4 CHECKS=1072 NODES=42741	? (problem) TT=298.247	? (TO) TT=1800.18	? (TO) TT=1800.12	? (TO) TT=1800.18	? (TO) TT=1800.2	UNSAT TT=242.082 CHECKS=681372000 NODES=48488	? (TO) TT=1800	? (TO) TT=1800	UNSAT TT=89.998 ASSIGNMENTS=42749 CHECKS=1638	UNSAT TT=90.669 ASSIGNMENTS=42749 CHECKS=1638	? (problem) TT=110.354	? (problem) TT=116.81	UNSAT TT=36.219 CHECKS=446156	? (TO) TT=1800.1	? (TO) TT=1800.12	? (exit code) TT=47.56	? (TO) TT=1800.09	? (TO) TT=1800.12	? TT=153.632	UNSAT TT=102.179 ASSIGNMENTS=95555 CHECKS=14460300	? (MO) TT=27.05	? (MO) TT=21.784
csp/dag-rand/ normalized-rand-n23-d3-e16-r15-t150000-9.xml	UNSAT TT=33.623	UNSAT TT=182.033 CHECKS=19851 NODES=54803	UNSAT TT=180.858 CHECKS=4973 NODES=66193	? (problem) TT=292.156	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) TT=1800.18	? (TO) TT=1800.19	UNSAT TT=297.49 CHECKS=794206000 NODES=48260	? (TO) TT=1800.04	? (TO) TT=1800.07	UNSAT TT=181.054 ASSIGNMENTS=54803 CHECKS=19851	UNSAT TT=195.753 ASSIGNMENTS=54803 CHECKS=19851	? (problem) TT=102.21	? (problem) TT=98.322	UNSAT TT=33.623 CHECKS=463124	? (TO) TT=1800.06	? (TO) TT=1800.05	? (exit code) TT=45.493	? (TO) TT=1800.1	? (TO) TT=1800.11	? TT=125.187	UNSAT TT=115.401 ASSIGNMENTS=124609 CHECKS=18470600	? (MO) TT=23.675	? (MO) TT=23.31

Some statistics...

	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13
Number of times the solver is able to give the best known answer	87	86	35	15	15	50	50	92	43	29	98	96	1	100	83	83	83	1
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)	87	86	35	15	15	50	50	92	43	29	98	96	1	100	83	83	83	1
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	0	0	0	0	0	0	0	0	4	0	2	3	0	57	9	18	8	0
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0