CSP 2008 Competition: results by benchmark

Results by benchmark for category binary constraints in intension (2-ARY-INT), subcategory academic problems

This page displays the results of the different solvers for each benchmark for category binary constraints in intension (2-ARY-INT), subcategory academic problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...	Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 112v4 AC
Abscon 112v4 ESAC
bpsolver 2008-06-27
casper zao
casper zito
choco2_dwdeg 2008-06-26
choco2_impwdeg 2008-06-26
Concrete + CPS4J 2008-05-30
Concrete + CSP4J - Tabu Engine 2008-05-30
Concrete + CSP4J - WMC Engine 2008-05-30
cpHydra k_10
cpHydra k_40
galac E
galac M
MDG-noprobe 2008-06-27
MDG-probe 2008-06-27
Minion/Tailor 2008-07-04
Mistral-option 1.314
Mistral-prime 1.313
SAT4J CSP 2008-06-13
Sugar v1.13+minisat
Sugar v1.13+picosat

Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/domino/ normalized-domino-1000-100.xml	SAT TT=0.052	? (MO) TT=0.897	SAT TT=1.999 CHECKS=8219800 NODES=1000	SAT TT=0.247	SAT TT=2.452 CHECKS=396013	SAT TT=0.188 CHECKS=362367	SAT TT=1.486 ASSIGNMENTS=1	SAT TT=1.33 ASSIGNMENTS=1	SAT TT=4.175 CHECKS=607204 NODES=1000	SAT TT=2.623 CHECKS=813003 NODES=0	SAT TT=2.503 CHECKS=813003 NODES=0	SAT TT=0.067 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.065 ASSIGNMENTS=1 CHECKS=14	SAT TT=102	SAT TT=99.986	SAT TT=0.052 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.057 ASSIGNMENTS=4 CHECKS=254	SAT TT=1.003	SAT TT=0.057 ASSIGNMENTS=99 CHECKS=1295	SAT TT=0.056 ASSIGNMENTS=1 CHECKS=14	SAT TT=10.788	SAT TT=2.613	SAT TT=4.09
csp/domino/ normalized-domino-1000-300.xml	SAT TT=0.051	SAT TT=4.121 CHECKS=165489000 NODES=1000	SAT TT=4.216 CHECKS=165489000 NODES=1000	SAT TT=0.283	SAT TT=3.251 CHECKS=1678790	SAT TT=0.786 CHECKS=1572070	SAT TT=1.528 ASSIGNMENTS=1	SAT TT=1.24 ASSIGNMENTS=1	SAT TT=21.158 CHECKS=4560550 NODES=1000	SAT TT=5.881 CHECKS=6111000 NODES=0	SAT TT=5.853 CHECKS=6111000 NODES=0	SAT TT=0.068 ASSIGNMENTS=1 CHECKS=622	SAT TT=0.067 ASSIGNMENTS=1 CHECKS=622	? (MO) TT=788.1	? (MO) TT=819.22	SAT TT=0.051 ASSIGNMENTS=1 CHECKS=622	SAT TT=0.051 ASSIGNMENTS=2 CHECKS=632	SAT TT=0.964	SAT TT=0.057 ASSIGNMENTS=299 CHECKS=3895	SAT TT=0.054 ASSIGNMENTS=1 CHECKS=14	SAT TT=63.341	SAT TT=3.985	SAT TT=9.66
csp/domino/ normalized-domino-1000-800.xml	SAT TT=0.052	SAT TT=10.148 CHECKS=21070700 NODES=1000	SAT TT=11.117 CHECKS=21070700 NODES=1000	SAT TT=0.414	SAT TT=4.678 CHECKS=4245970	SAT TT=2.275 CHECKS=4216470	SAT TT=1.22 ASSIGNMENTS=1	SAT TT=1.568 ASSIGNMENTS=1	SAT TT=82.357 CHECKS=34870200 NODES=1000	SAT TT=22.054 CHECKS=40776000 NODES=0	SAT TT=23.046 CHECKS=40776000 NODES=0	SAT TT=0.067 ASSIGNMENTS=1 CHECKS=1622	SAT TT=0.068 ASSIGNMENTS=1 CHECKS=1622	? (TO) TT=1800.01	? (TO) TT=1800.04	SAT TT=0.056 ASSIGNMENTS=1 CHECKS=1622	SAT TT=0.057 ASSIGNMENTS=1 CHECKS=1617	SAT TT=1.05	SAT TT=0.056 ASSIGNMENTS=799 CHECKS=10395	SAT TT=0.052 ASSIGNMENTS=1 CHECKS=14	SAT TT=417.584	SAT TT=9.211	SAT TT=23.433
csp/domino/ normalized-domino-100-100.xml	SAT TT=0.009	SAT TT=0.94 CHECKS=821805 NODES=100	SAT TT=0.899 CHECKS=821805 NODES=0	SAT TT=0.045	SAT TT=0.265 CHECKS=46790	SAT TT=0.042 CHECKS=50761	SAT TT=0.885 ASSIGNMENTS=1	SAT TT=0.689 ASSIGNMENTS=1	SAT TT=0.928 CHECKS=60904 NODES=100	SAT TT=0.775 CHECKS=81303 NODES=0	SAT TT=0.699 CHECKS=81303 NODES=0	SAT TT=0.021 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.022 ASSIGNMENTS=1 CHECKS=14	SAT TT=11.538	SAT TT=10.498	SAT TT=0.009 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.01 ASSIGNMENTS=2 CHECKS=234	SAT TT=0.383	SAT TT=0.013 ASSIGNMENTS=99 CHECKS=1295	SAT TT=0.01 ASSIGNMENTS=1 CHECKS=14	SAT TT=3.444	SAT TT=0.828	SAT TT=1.094
csp/domino/ normalized-domino-100-300.xml	SAT TT=0.009	SAT TT=1.066 CHECKS=16547500 NODES=100	SAT TT=1.056 CHECKS=16547500 NODES=0	SAT TT=0.051	SAT TT=0.356 CHECKS=148256	SAT TT=0.119 CHECKS=146065	SAT TT=0.677 ASSIGNMENTS=1	SAT TT=0.665 ASSIGNMENTS=1	SAT TT=2.498 CHECKS=457450 NODES=100	SAT TT=1.035 CHECKS=611103 NODES=0	SAT TT=1.041 CHECKS=611103 NODES=0	SAT TT=0.021 ASSIGNMENTS=1 CHECKS=622	SAT TT=0.021 ASSIGNMENTS=1 CHECKS=622	SAT TT=73.72	SAT TT=79.692	SAT TT=0.01 ASSIGNMENTS=1 CHECKS=622	SAT TT=0.011 ASSIGNMENTS=2 CHECKS=27	SAT TT=0.383	SAT TT=0.009 ASSIGNMENTS=299 CHECKS=3895	SAT TT=0.009 ASSIGNMENTS=1 CHECKS=14	SAT TT=8.27	SAT TT=1.053	SAT TT=1.646
csp/domino/ normalized-domino-2000-2000.xml	SAT TT=0.127	SAT TT=95.796 CHECKS=256674000 NODES=2000	SAT TT=101.963 CHECKS=256674000 NODES=2000	SAT TT=1.312	SAT TT=19.056 CHECKS=24541300	SAT TT=9.623 CHECKS=13508000	SAT TT=2.063 ASSIGNMENTS=1	SAT TT=2.297 ASSIGNMENTS=1	SAT TT=309.85 CHECKS=500240000 NODES=2000	SAT TT=227.705 CHECKS=505892000 NODES=0	SAT TT=238.123 CHECKS=505892000 NODES=0	SAT TT=0.135 ASSIGNMENTS=1 CHECKS=4022	SAT TT=0.133 ASSIGNMENTS=1 CHECKS=4022	? (TO) TT=1800.06	? (TO) TT=1800.02	SAT TT=0.136 ASSIGNMENTS=1 CHECKS=4022	SAT TT=0.132 ASSIGNMENTS=1 CHECKS=4017	SAT TT=1.587	SAT TT=0.142 ASSIGNMENTS=1999 CHECKS=25995	SAT TT=0.127 ASSIGNMENTS=1 CHECKS=14	? TT=69.951	SAT TT=45.803	SAT TT=120.777
csp/domino/ normalized-domino-500-100.xml	SAT TT=0.025	SAT TT=1.276 CHECKS=4109800 NODES=500	SAT TT=1.324 CHECKS=4109800 NODES=0	SAT TT=0.143	SAT TT=1.307 CHECKS=226885	SAT TT=0.121 CHECKS=218536	SAT TT=1.345 ASSIGNMENTS=1	SAT TT=1.212 ASSIGNMENTS=1	SAT TT=2.319 CHECKS=303704 NODES=500	SAT TT=1.649 CHECKS=406503 NODES=0	SAT TT=1.614 CHECKS=406503 NODES=0	SAT TT=0.039 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.039 ASSIGNMENTS=1 CHECKS=14	SAT TT=51.756	SAT TT=48.275	SAT TT=0.027 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.025 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.769	SAT TT=0.028 ASSIGNMENTS=99 CHECKS=1295	SAT TT=0.025 ASSIGNMENTS=1 CHECKS=14	SAT TT=6.918	SAT TT=1.568	SAT TT=2.602
csp/domino/ normalized-domino-500-200.xml	SAT TT=0.026	SAT TT=1.683 CHECKS=26167200 NODES=500	SAT TT=1.717 CHECKS=26167200 NODES=500	SAT TT=0.132	SAT TT=1.398 CHECKS=498405	SAT TT=0.254 CHECKS=493533	SAT TT=0.956 ASSIGNMENTS=1	SAT TT=1.088 ASSIGNMENTS=1	SAT TT=5.524 CHECKS=1064220 NODES=500	SAT TT=2.247 CHECKS=1425000 NODES=0	SAT TT=2.235 CHECKS=1425000 NODES=0	SAT TT=0.039 ASSIGNMENTS=1 CHECKS=422	SAT TT=0.039 ASSIGNMENTS=1 CHECKS=422	SAT TT=187.595	SAT TT=182.769	SAT TT=0.028 ASSIGNMENTS=1 CHECKS=422	SAT TT=0.031 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.893	SAT TT=0.028 ASSIGNMENTS=199 CHECKS=2595	SAT TT=0.026 ASSIGNMENTS=1 CHECKS=14	SAT TT=16.147	SAT TT=2.235	SAT TT=3.959
csp/domino/ normalized-domino-500-500.xml	SAT TT=0.027	SAT TT=3.314 CHECKS=4138360 NODES=500	SAT TT=3.355 CHECKS=4138360 NODES=500	SAT TT=0.186	SAT TT=1.926 CHECKS=1348840	SAT TT=0.755 CHECKS=1320220	SAT TT=1.197 ASSIGNMENTS=1	SAT TT=1.04 ASSIGNMENTS=1	SAT TT=26.181 CHECKS=6083640 NODES=500	SAT TT=5.727 CHECKS=8152500 NODES=0	SAT TT=5.784 CHECKS=8152500 NODES=0	SAT TT=0.04 ASSIGNMENTS=1 CHECKS=1022	SAT TT=0.039 ASSIGNMENTS=1 CHECKS=1022	? (MO) TT=1055.97	? (MO) TT=1046.51	SAT TT=0.027 ASSIGNMENTS=1 CHECKS=1022	SAT TT=0.029 ASSIGNMENTS=1 CHECKS=1022	SAT TT=0.757	SAT TT=0.03 ASSIGNMENTS=499 CHECKS=6495	SAT TT=0.027 ASSIGNMENTS=1 CHECKS=14	SAT TT=86.609	SAT TT=3.373	SAT TT=7.977
csp/domino/ normalized-domino-800-100.xml	SAT TT=0.042	SAT TT=1.679 CHECKS=6575800 NODES=800	SAT TT=1.681 CHECKS=6575800 NODES=800	SAT TT=0.203	SAT TT=2.099 CHECKS=337227	SAT TT=0.15 CHECKS=306659	SAT TT=1.329 ASSIGNMENTS=1	SAT TT=1.33 ASSIGNMENTS=1	SAT TT=3.538 CHECKS=485804 NODES=800	SAT TT=2.145 CHECKS=650403 NODES=0	SAT TT=2.228 CHECKS=650403 NODES=0	SAT TT=0.056 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.055 ASSIGNMENTS=1 CHECKS=14	SAT TT=79.289	SAT TT=78.386	SAT TT=0.044 ASSIGNMENTS=1 CHECKS=14	SAT TT=0.042 ASSIGNMENTS=1 CHECKS=222	SAT TT=1.106	SAT TT=0.044 ASSIGNMENTS=99 CHECKS=1295	SAT TT=0.044 ASSIGNMENTS=1 CHECKS=14	SAT TT=9.478	SAT TT=2.183	SAT TT=3.658
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/knights/ normalized-knights-100-25.xml		? (TO) TT=1800.14	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.15	? (TO) TT=1800.01	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.11	? (TO) TT=1800.06	? (TO) TT=1800.04	? TT=1199.4	? (TO) TT=1800.12	? (TO) TT=1800.01
csp/knights/ normalized-knights-12-5.xml	UNSAT TT=0.075	UNSAT TT=0.815 CHECKS=232842 NODES=128	UNSAT TT=0.883 CHECKS=355092 NODES=0	UNSAT TT=0.075	UNSAT TT=2.229 CHECKS=66548	UNSAT TT=0.237 CHECKS=67390	UNSAT TT=0.838 ASSIGNMENTS=0	UNSAT TT=0.789 ASSIGNMENTS=0	UNSAT TT=0.648 CHECKS=435922 NODES=215	? (TO) TT=1800.05	? (TO) TT=1800.06	UNSAT TT=8.414 ASSIGNMENTS=140 CHECKS=349494	UNSAT TT=8.292 ASSIGNMENTS=140 CHECKS=349494	UNSAT TT=2.351	UNSAT TT=1.997	UNSAT TT=2.488 ASSIGNMENTS=213 CHECKS=3285	UNSAT TT=3.115 ASSIGNMENTS=638 CHECKS=7558	UNSAT TT=4.103	UNSAT TT=2.575 ASSIGNMENTS=143 CHECKS=2758	UNSAT TT=2.483 ASSIGNMENTS=214 CHECKS=3508	UNSAT TT=4.308	UNSAT TT=0.644	UNSAT TT=1.038
csp/knights/ normalized-knights-12-9.xml	UNSAT TT=0.184	UNSAT TT=0.887 CHECKS=586217 NODES=128	UNSAT TT=0.882 CHECKS=757398 NODES=0	UNSAT TT=0.184	UNSAT TT=3.6 CHECKS=116971	UNSAT TT=0.366 CHECKS=179528	UNSAT TT=1.856 ASSIGNMENTS=0	UNSAT TT=2 ASSIGNMENTS=0	UNSAT TT=0.81 CHECKS=899040 NODES=230	? (TO) TT=1800.03	? (TO) TT=1800.08	UNSAT TT=8.62 ASSIGNMENTS=143 CHECKS=733962	UNSAT TT=8.114 ASSIGNMENTS=143 CHECKS=733962	UNSAT TT=3.752	UNSAT TT=2.776	UNSAT TT=9.02 ASSIGNMENTS=211 CHECKS=7975	UNSAT TT=19.281 ASSIGNMENTS=1160 CHECKS=32779	? (TO) TT=1800.09	UNSAT TT=8.93 ASSIGNMENTS=143 CHECKS=5984	UNSAT TT=10.606 ASSIGNMENTS=214 CHECKS=8089	UNSAT TT=6.936	UNSAT TT=55.213	UNSAT TT=179.107
csp/knights/ normalized-knights-15-9.xml	UNSAT TT=0.548	UNSAT TT=1.07 CHECKS=1911490 NODES=206	UNSAT TT=1.076 CHECKS=2497140 NODES=0	UNSAT TT=0.548	UNSAT TT=10.306 CHECKS=355527	UNSAT TT=1.146 CHECKS=541157	UNSAT TT=4.343 ASSIGNMENTS=0	UNSAT TT=4.254 ASSIGNMENTS=0	UNSAT TT=1.182 CHECKS=3263140 NODES=323	? (TO) TT=1800.09	? (TO) TT=1800.02	UNSAT TT=8.772 ASSIGNMENTS=221 CHECKS=2399880	UNSAT TT=8.693 ASSIGNMENTS=221 CHECKS=2399880	UNSAT TT=9.313	UNSAT TT=5.422	UNSAT TT=35.351 ASSIGNMENTS=332 CHECKS=12778	UNSAT TT=65.411 ASSIGNMENTS=1842 CHECKS=51955	? (TO) TT=1800.11	UNSAT TT=31.187 ASSIGNMENTS=224 CHECKS=9674	UNSAT TT=35.137 ASSIGNMENTS=335 CHECKS=12893	UNSAT TT=12.105	UNSAT TT=141.26	UNSAT TT=404.132
csp/knights/ normalized-knights-25-5.xml	UNSAT TT=0.826	UNSAT TT=1.808 CHECKS=6742540 NODES=596	UNSAT TT=2.236 CHECKS=10878900 NODES=0	UNSAT TT=0.898	UNSAT TT=52.552 CHECKS=325771	UNSAT TT=5.436 CHECKS=1980860	UNSAT TT=1694.94 ASSIGNMENTS=678385	UNSAT TT=1614.63 ASSIGNMENTS=678385	UNSAT TT=2.647 CHECKS=30628900 NODES=936	? (TO) TT=1800.06	? (TO) TT=1800.05	UNSAT TT=9.108 ASSIGNMENTS=621 CHECKS=11640700	UNSAT TT=9.284 ASSIGNMENTS=621 CHECKS=11640700	UNSAT TT=57.029	UNSAT TT=18.795	UNSAT TT=198.457 ASSIGNMENTS=934 CHECKS=15114	UNSAT TT=220.292 ASSIGNMENTS=3011 CHECKS=35606	UNSAT TT=25.84	UNSAT TT=211.27 ASSIGNMENTS=624 CHECKS=12872	UNSAT TT=193.105 ASSIGNMENTS=935 CHECKS=16058	UNSAT TT=36.581	UNSAT TT=0.826	UNSAT TT=1.246
csp/knights/ normalized-knights-25-9.xml	UNSAT TT=2.453	UNSAT TT=2.453 CHECKS=16875000 NODES=596	UNSAT TT=2.79 CHECKS=23068500 NODES=0	UNSAT TT=6.765	UNSAT TT=94.589 CHECKS=1187040	UNSAT TT=6.369 CHECKS=3397030	? (TO) TT=1800.05	? (TO) TT=1800.1	UNSAT TT=5.968 CHECKS=66977300 NODES=969	? (TO) TT=1800.08	? (TO) TT=1800.07	UNSAT TT=9.928 ASSIGNMENTS=621 CHECKS=25531000	UNSAT TT=9.413 ASSIGNMENTS=621 CHECKS=25531000	UNSAT TT=115.98	UNSAT TT=33.158	UNSAT TT=654.639 ASSIGNMENTS=932 CHECKS=37204	UNSAT TT=884.118 ASSIGNMENTS=1765 CHECKS=57225	? (TO) TT=1800.11	UNSAT TT=604.695 ASSIGNMENTS=624 CHECKS=28474	UNSAT TT=619.106 ASSIGNMENTS=935 CHECKS=37388	UNSAT TT=85.684	UNSAT TT=575.577	? (TO) TT=1800.13
csp/knights/ normalized-knights-50-9.xml	UNSAT TT=27.038	UNSAT TT=27.038 CHECKS=1005120000 NODES=2446	UNSAT TT=36.346 CHECKS=1408930000 NODES=0	UNSAT TT=106.089	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.14	? (TO) TT=1800.06	UNSAT TT=190.691 CHECKS=4506580000 NODES=4307	? (TO) TT=1800.05	? (TO) TT=1800.07	UNSAT TT=63.244 ASSIGNMENTS=2494 CHECKS=1440480000	UNSAT TT=63.37 ASSIGNMENTS=2494 CHECKS=1440480000	? (MO) TT=404.03	? (MO) TT=408.5	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.03	? TT=314.444	? (TO) TT=1800.03	? (TO) TT=1800.12
csp/knights/ normalized-knights-80-25.xml		? (TO) TT=1800.11	? (TO) TT=1800.18	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.14	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.08	? TT=494.963	? (TO) TT=1800.08	? (TO) TT=1800.06
csp/knights/ normalized-knights-80-9.xml		? (TO) TT=1800.1	? (TO) TT=1800.16	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.04	? (TO) TT=1800.18	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.02	? TT=541.08	? (TO) TT=1800.07	? (TO) TT=1800.04
csp/knights/ normalized-knights-8-5.xml	UNSAT TT=0.036	UNSAT TT=0.763 CHECKS=24208 NODES=52	UNSAT TT=0.762 CHECKS=33573 NODES=0	UNSAT TT=0.036	UNSAT TT=0.42 CHECKS=13194	UNSAT TT=2.587 CHECKS=923473	UNSAT TT=0.841 ASSIGNMENTS=0	UNSAT TT=0.844 ASSIGNMENTS=0	UNSAT TT=0.567 CHECKS=35956 NODES=95	? (TO) TT=1800	? (TO) TT=1800.09	UNSAT TT=0.276 ASSIGNMENTS=94 CHECKS=1396	UNSAT TT=0.271 ASSIGNMENTS=94 CHECKS=1396	UNSAT TT=1.069	UNSAT TT=1.182	UNSAT TT=0.261 ASSIGNMENTS=93 CHECKS=1385	UNSAT TT=0.344 ASSIGNMENTS=265 CHECKS=3127	UNSAT TT=1.403	UNSAT TT=0.27 ASSIGNMENTS=63 CHECKS=1142	UNSAT TT=0.265 ASSIGNMENTS=94 CHECKS=1488	UNSAT TT=3.355	UNSAT TT=0.75	UNSAT TT=0.64
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/langford2/ normalized-langford-2-10.xml	UNSAT TT=1.192	UNSAT TT=7.253 CHECKS=8302060 NODES=95782	UNSAT TT=7.393 CHECKS=8309230 NODES=95666	UNSAT TT=1.192	UNSAT TT=82.834 CHECKS=490285000	UNSAT TT=83.114 CHECKS=475605000	UNSAT TT=14.365 ASSIGNMENTS=112954	UNSAT TT=11.686 ASSIGNMENTS=67618	UNSAT TT=7.855 CHECKS=1446730 NODES=61492	? (TO) TT=1800.02	? (TO) TT=1800.06	UNSAT TT=1.39 ASSIGNMENTS=36820 CHECKS=2299550	UNSAT TT=1.404 ASSIGNMENTS=36820 CHECKS=2299550	UNSAT TT=68.53	UNSAT TT=17.098	UNSAT TT=1.247 ASSIGNMENTS=37026 CHECKS=2342630	UNSAT TT=1.338 ASSIGNMENTS=40020 CHECKS=2480990	UNSAT TT=6.199	UNSAT TT=2.498 ASSIGNMENTS=73647 CHECKS=4600430	UNSAT TT=2.141 ASSIGNMENTS=61169 CHECKS=3777720	UNSAT TT=364.274	UNSAT TT=11.273	UNSAT TT=19.705
csp/langford2/ normalized-langford-2-11.xml	SAT TT=0.013	SAT TT=0.867 CHECKS=6587 NODES=41	SAT TT=0.943 CHECKS=16536 NODES=41	SAT TT=0.055	SAT TT=0.04 CHECKS=29414	SAT TT=0.018 CHECKS=27499	SAT TT=0.966 ASSIGNMENTS=14	SAT TT=1.001 ASSIGNMENTS=29	SAT TT=0.956 CHECKS=6975 NODES=68	SAT TT=1.222 CHECKS=49450 NODES=99	SAT TT=0.959 CHECKS=627046 NODES=788	SAT TT=0.028 ASSIGNMENTS=16 CHECKS=1342	SAT TT=0.024 ASSIGNMENTS=16 CHECKS=1342	SAT TT=1.899	SAT TT=2.082	SAT TT=0.013 ASSIGNMENTS=13 CHECKS=1082	SAT TT=0.016 ASSIGNMENTS=41 CHECKS=3199	SAT TT=0.499	SAT TT=0.016 ASSIGNMENTS=14 CHECKS=1091	SAT TT=0.013 ASSIGNMENTS=16 CHECKS=1351	SAT TT=3.739	SAT TT=0.856	SAT TT=0.917
csp/langford2/ normalized-langford-2-13.xml	UNSAT TT=364.714	? (TO) TT=1800.14	? (TO) TT=1800.11	UNSAT TT=364.714	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.08	UNSAT TT=1082.78 ASSIGNMENTS=15456400 CHECKS=1072250000	UNSAT TT=1084.54 ASSIGNMENTS=15456400 CHECKS=1072250000	? (TO) TT=1800	? (TO) TT=1800.11	UNSAT TT=433.005 ASSIGNMENTS=10878000 CHECKS=756156000	UNSAT TT=393.132 ASSIGNMENTS=10682900 CHECKS=736722000	UNSAT TT=1152.95	UNSAT TT=928.323 ASSIGNMENTS=22184000 CHECKS=1616560000	UNSAT TT=582.537 ASSIGNMENTS=15456400 CHECKS=1072250000	? (TO) TT=1800.17	? (TO) TT=1800.05	? (TO) TT=1800.14
csp/langford2/ normalized-langford-2-14.xml		? (TO) TT=1800.13	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.02	? (TO) TT=1800.16	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.01	? (TO) TT=1800.13	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.06	? (TO) TT=1800.01	? (TO) TT=1800.13	? (TO) TT=1800.08	? (TO) TT=1800.1
csp/langford2/ normalized-langford-2-17.xml		? (TO) TT=1800.11	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.18	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.01	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.11	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.14	? (TO) TT=1800.11	? (TO) TT=1800.07
csp/langford2/ normalized-langford-2-19.xml	SAT TT=0.031	SAT TT=1.169 CHECKS=47249 NODES=122	SAT TT=1.139 CHECKS=86218 NODES=121	SAT TT=0.119	SAT TT=1.099 CHECKS=6400640	SAT TT=1.152 CHECKS=6094220	SAT TT=1.502 ASSIGNMENTS=28	SAT TT=3.062 ASSIGNMENTS=7880	SAT TT=1.563 CHECKS=34374 NODES=86	SAT TT=1.61 CHECKS=1244670 NODES=1126	SAT TT=1.469 CHECKS=349415 NODES=184	SAT TT=0.043 ASSIGNMENTS=34 CHECKS=3346	SAT TT=0.044 ASSIGNMENTS=34 CHECKS=3346	SAT TT=3.806	SAT TT=3.677	SAT TT=0.031 ASSIGNMENTS=36 CHECKS=3650	SAT TT=0.032 ASSIGNMENTS=15 CHECKS=2143	SAT TT=9.495	SAT TT=0.098 ASSIGNMENTS=521 CHECKS=54440	SAT TT=0.039 ASSIGNMENTS=193 CHECKS=16430	SAT TT=5.648	SAT TT=1.384	SAT TT=1.464
csp/langford2/ normalized-langford-2-20.xml	SAT TT=0.035	SAT TT=1.233 CHECKS=92424 NODES=252	SAT TT=1.321 CHECKS=137728 NODES=252	SAT TT=0.122	SAT TT=2.203 CHECKS=13292700	SAT TT=2.538 CHECKS=13317300	SAT TT=1.557 ASSIGNMENTS=55	SAT TT=2.37 ASSIGNMENTS=2721	SAT TT=1.849 CHECKS=49899 NODES=135	SAT TT=1.895 CHECKS=2458250 NODES=2065	SAT TT=1.61 CHECKS=457625 NODES=230	SAT TT=0.047 ASSIGNMENTS=40 CHECKS=4384	SAT TT=0.049 ASSIGNMENTS=40 CHECKS=4384	SAT TT=4.323	SAT TT=4.355	SAT TT=0.036 ASSIGNMENTS=40 CHECKS=4384	SAT TT=0.035 ASSIGNMENTS=23 CHECKS=2870	SAT TT=17.167	SAT TT=0.134 ASSIGNMENTS=713 CHECKS=76941	SAT TT=0.038 ASSIGNMENTS=76 CHECKS=7851	SAT TT=6.8	SAT TT=1.261	SAT TT=1.551
csp/langford2/ normalized-langford-2-25.xml		? (TO) TT=1800.1	? (TO) TT=1800.14	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.02	? (TO) TT=1800.16	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.01	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.2	? (TO) TT=1800.01	? (TO) TT=1800.09
csp/langford2/ normalized-langford-2-2.xml	UNSAT TT=0.006	UNSAT TT=0.676 CHECKS=25 NODES=0	UNSAT TT=0.68 CHECKS=25 NODES=0	UNSAT TT=0.025	UNSAT TT=0.008 CHECKS=21	UNSAT TT=0.008 CHECKS=21	UNSAT TT=0.45 ASSIGNMENTS=0	UNSAT TT=0.481 ASSIGNMENTS=0	UNSAT TT=0.421 CHECKS=26 NODES=0	UNSAT TT=0.407 CHECKS=34 NODES=0	UNSAT TT=0.424 CHECKS=34 NODES=0	UNSAT TT=0.018 ASSIGNMENTS=0 CHECKS=17	UNSAT TT=0.018 ASSIGNMENTS=0 CHECKS=17	UNSAT TT=0.975	UNSAT TT=0.692	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=17	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=17	UNSAT TT=0.285	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=17	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=17	UNSAT TT=0.883	UNSAT TT=0.453	UNSAT TT=0.452
csp/langford2/ normalized-langford-2-30.xml		? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.02	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.16	? (TO) TT=1800.08	? (TO) TT=1800.07
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/langford2/ normalized-langford-2-4.xml	SAT TT=0.007	SAT TT=0.714 CHECKS=240 NODES=8	SAT TT=0.694 CHECKS=240 NODES=0	SAT TT=0.029	SAT TT=0.011 CHECKS=1735	SAT TT=0.008 CHECKS=7133	SAT TT=0.529 ASSIGNMENTS=2	SAT TT=1.153 ASSIGNMENTS=2	SAT TT=0.482 CHECKS=263 NODES=10	SAT TT=0.509 CHECKS=4535 NODES=105	SAT TT=0.54 CHECKS=217772 NODES=2381	SAT TT=0.019 ASSIGNMENTS=2 CHECKS=96	SAT TT=0.019 ASSIGNMENTS=2 CHECKS=96	SAT TT=1.207	SAT TT=1.058	SAT TT=0.007 ASSIGNMENTS=2 CHECKS=96	SAT TT=0.008 ASSIGNMENTS=1 CHECKS=95	SAT TT=0.309	SAT TT=0.007 ASSIGNMENTS=2 CHECKS=133	SAT TT=0.008 ASSIGNMENTS=2 CHECKS=96	SAT TT=0.976	SAT TT=0.518	SAT TT=0.506
csp/langford2/ normalized-langford-2-50.xml		? (TO) TT=1800.17	? (TO) TT=1800.06	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.13	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.03	? (MO) TT=1483.11	? (MO) TT=1064.28	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.12	? (TO) TT=1800.02	? (TO) TT=1800.06	? (TO) TT=1800.24	? (TO) TT=1800.03	? (TO) TT=1800.09
csp/langford2/ normalized-langford-2-5.xml	UNSAT TT=0.006	UNSAT TT=0.732 CHECKS=2511 NODES=36	UNSAT TT=0.716 CHECKS=3275 NODES=36	UNSAT TT=0.03	UNSAT TT=0.016 CHECKS=7536	UNSAT TT=0.009 CHECKS=6652	UNSAT TT=0.617 ASSIGNMENTS=8	UNSAT TT=0.642 ASSIGNMENTS=9	UNSAT TT=0.675 CHECKS=1307 NODES=35	? (TO) TT=1800.09	? (TO) TT=1800.08	UNSAT TT=0.02 ASSIGNMENTS=17 CHECKS=836	UNSAT TT=0.02 ASSIGNMENTS=17 CHECKS=836	UNSAT TT=0.884	UNSAT TT=0.897	UNSAT TT=0.006 ASSIGNMENTS=15 CHECKS=779	UNSAT TT=0.007 ASSIGNMENTS=16 CHECKS=852	UNSAT TT=0.334	UNSAT TT=0.009 ASSIGNMENTS=46 CHECKS=1755	UNSAT TT=0.008 ASSIGNMENTS=20 CHECKS=980	UNSAT TT=1.437	UNSAT TT=0.592	UNSAT TT=0.533
csp/langford2/ normalized-langford-2-7.xml	SAT TT=0.008	SAT TT=0.807 CHECKS=1221 NODES=15	SAT TT=0.754 CHECKS=2687 NODES=0	SAT TT=0.036	SAT TT=0.02 CHECKS=7154	SAT TT=0.011 CHECKS=4875	SAT TT=0.792 ASSIGNMENTS=4	SAT TT=0.711 ASSIGNMENTS=8	SAT TT=0.671 CHECKS=1172 NODES=19	SAT TT=0.67 CHECKS=32731 NODES=226	SAT TT=1.069 CHECKS=2759890 NODES=6201	SAT TT=0.021 ASSIGNMENTS=3 CHECKS=284	SAT TT=0.021 ASSIGNMENTS=3 CHECKS=284	SAT TT=1.543	SAT TT=1.283	SAT TT=0.008 ASSIGNMENTS=3 CHECKS=284	SAT TT=0.01 ASSIGNMENTS=8 CHECKS=503	SAT TT=0.356	SAT TT=0.012 ASSIGNMENTS=30 CHECKS=1484	SAT TT=0.01 ASSIGNMENTS=5 CHECKS=374	SAT TT=1.624	SAT TT=0.619	SAT TT=0.661
csp/langford2/ normalized-langford-2-8.xml	SAT TT=0.01	SAT TT=0.787 CHECKS=5268 NODES=63	SAT TT=0.868 CHECKS=8832 NODES=63	SAT TT=0.039	SAT TT=0.026 CHECKS=15286	SAT TT=0.013 CHECKS=14851	SAT TT=0.742 ASSIGNMENTS=13	SAT TT=0.788 ASSIGNMENTS=8	SAT TT=0.799 CHECKS=2225 NODES=30	SAT TT=0.781 CHECKS=53755 NODES=282	SAT TT=0.864 CHECKS=12710 NODES=27	SAT TT=0.022 ASSIGNMENTS=13 CHECKS=918	SAT TT=0.022 ASSIGNMENTS=13 CHECKS=918	SAT TT=1.84	SAT TT=1.511	SAT TT=0.011 ASSIGNMENTS=13 CHECKS=946	SAT TT=0.01 ASSIGNMENTS=5 CHECKS=396	SAT TT=0.393	SAT TT=0.012 ASSIGNMENTS=67 CHECKS=3406	SAT TT=0.011 ASSIGNMENTS=13 CHECKS=914	SAT TT=2.131	SAT TT=0.673	SAT TT=0.681
csp/langford3/ normalized-langford-3-10.xml	SAT TT=0.026	SAT TT=1.327 CHECKS=430675 NODES=1480	SAT TT=1.355 CHECKS=455334 NODES=1508	SAT TT=0.109	SAT TT=0.959 CHECKS=5165700	SAT TT=0.7 CHECKS=4143060	SAT TT=1.32 ASSIGNMENTS=378	SAT TT=1.248 ASSIGNMENTS=105	SAT TT=1.172 CHECKS=21923 NODES=146	SAT TT=5.262 CHECKS=29411500 NODES=57184	? (TO) TT=1800.09	SAT TT=0.067 ASSIGNMENTS=336 CHECKS=63773	SAT TT=0.066 ASSIGNMENTS=336 CHECKS=63773	SAT TT=2.654	SAT TT=2.763	SAT TT=0.053 ASSIGNMENTS=335 CHECKS=62877	SAT TT=0.026 ASSIGNMENTS=56 CHECKS=11058	SAT TT=0.808	SAT TT=0.078 ASSIGNMENTS=491 CHECKS=89677	SAT TT=0.143 ASSIGNMENTS=1245 CHECKS=237305	SAT TT=5.346	SAT TT=1.208	SAT TT=1.46
csp/langford3/ normalized-langford-3-11.xml	UNSAT TT=0.88	UNSAT TT=7.225 CHECKS=9951240 NODES=34279	UNSAT TT=7.398 CHECKS=9985230 NODES=34349	UNSAT TT=0.945	UNSAT TT=92.396 CHECKS=525283000	UNSAT TT=102.815 CHECKS=567343000	UNSAT TT=7.949 ASSIGNMENTS=23027	UNSAT TT=8.918 ASSIGNMENTS=23267	UNSAT TT=9.446 CHECKS=3169520 NODES=19411	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=0.88 ASSIGNMENTS=8186 CHECKS=1577810	UNSAT TT=0.888 ASSIGNMENTS=8186 CHECKS=1577810	UNSAT TT=13.608	UNSAT TT=13.103	UNSAT TT=0.925 ASSIGNMENTS=8218 CHECKS=1587330	UNSAT TT=1.289 ASSIGNMENTS=11503 CHECKS=2184580	UNSAT TT=3.984	UNSAT TT=2.99 ASSIGNMENTS=27411 CHECKS=5212290	UNSAT TT=1.534 ASSIGNMENTS=13775 CHECKS=2652040	UNSAT TT=155.457	UNSAT TT=11.445	UNSAT TT=12.662
csp/langford3/ normalized-langford-3-12.xml	UNSAT TT=3.945	UNSAT TT=31.016 CHECKS=46124200 NODES=153737	UNSAT TT=30.531 CHECKS=46169400 NODES=153762	UNSAT TT=3.945	UNSAT TT=375.462 CHECKS=2321330000	UNSAT TT=407.061 CHECKS=2245260000	UNSAT TT=26.657 ASSIGNMENTS=90137	UNSAT TT=28.396 ASSIGNMENTS=74336	UNSAT TT=66.393 CHECKS=20081000 NODES=81778	? (TO) TT=1800.02	? (TO) TT=1800.02	UNSAT TT=35.393 ASSIGNMENTS=153737 CHECKS=46124200	UNSAT TT=35.031 ASSIGNMENTS=153737 CHECKS=46124200	UNSAT TT=209.475	UNSAT TT=92.37	UNSAT TT=4.134 ASSIGNMENTS=34675 CHECKS=7114670	UNSAT TT=4.471 ASSIGNMENTS=38052 CHECKS=7767450	UNSAT TT=18.353	UNSAT TT=13.007 ASSIGNMENTS=105689 CHECKS=22446300	UNSAT TT=6.199 ASSIGNMENTS=51872 CHECKS=10659200	UNSAT TT=1502.82	UNSAT TT=85.49	UNSAT TT=82.731
csp/langford3/ normalized-langford-3-13.xml	UNSAT TT=18.507	UNSAT TT=156.822 CHECKS=237252000 NODES=750737	UNSAT TT=153.211 CHECKS=237232000 NODES=750563	UNSAT TT=18.507	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=117.058 ASSIGNMENTS=400170	UNSAT TT=146.249 ASSIGNMENTS=360818	UNSAT TT=209.299 CHECKS=119771000 NODES=382480	? (TO) TT=1800	? (TO) TT=1800.06	UNSAT TT=149.085 ASSIGNMENTS=750737 CHECKS=237252000	UNSAT TT=149.671 ASSIGNMENTS=750737 CHECKS=237252000	? (TO) TT=1800.08	UNSAT TT=827.471	UNSAT TT=20.403 ASSIGNMENTS=164322 CHECKS=35437500	UNSAT TT=20.677 ASSIGNMENTS=167158 CHECKS=36001200	UNSAT TT=95.162	UNSAT TT=77.834 ASSIGNMENTS=623437 CHECKS=135623000	UNSAT TT=30.852 ASSIGNMENTS=250177 CHECKS=53971200	? (TO) TT=1800.11	UNSAT TT=788.596	UNSAT TT=1059.88
csp/langford3/ normalized-langford-3-16.xml		? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.15	? (TO) TT=1800.13	? (TO) TT=1800.11
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/langford3/ normalized-langford-3-18.xml	SAT TT=0.52	SAT TT=28.567 CHECKS=40139700 NODES=86840	SAT TT=26.4 CHECKS=40176700 NODES=86651	SAT TT=3.72	SAT TT=10.542 CHECKS=65004100	SAT TT=51.46 CHECKS=272826000	SAT TT=11.796 ASSIGNMENTS=27332	SAT TT=74.74 ASSIGNMENTS=130122	SAT TT=45.722 CHECKS=19529500 NODES=36825	SAT TT=701.332 CHECKS=5319120000 NODES=2973570	? (TO) TT=1800.04	SAT TT=33.607 ASSIGNMENTS=86840 CHECKS=40139700	SAT TT=33.548 ASSIGNMENTS=86840 CHECKS=40139700	SAT TT=7.946	SAT TT=25.245	SAT TT=4.882 ASSIGNMENTS=30291 CHECKS=8635890	SAT TT=0.52 ASSIGNMENTS=2237 CHECKS=598521	SAT TT=13.07	SAT TT=0.76 ASSIGNMENTS=3630 CHECKS=997705	SAT TT=2.589 ASSIGNMENTS=17428 CHECKS=4831400	SAT TT=300.078	SAT TT=12.513	SAT TT=4.98
csp/langford3/ normalized-langford-3-20.xml		? (TO) TT=1800.13	? (TO) TT=1800.14	? (TO) TT=1800.11	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.12	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.38	? (TO) TT=1800.09	? (TO) TT=1800.16
csp/langford3/ normalized-langford-3-21.xml		? (TO) TT=1800.13	? (TO) TT=1800.15	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.01	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.07
csp/langford3/ normalized-langford-3-2.xml	UNSAT TT=0.006	UNSAT TT=0.649 CHECKS=44 NODES=0	UNSAT TT=0.726 CHECKS=44 NODES=0	UNSAT TT=0.028	UNSAT TT=0.008 CHECKS=18	UNSAT TT=0.008 CHECKS=18	UNSAT TT=0.488 ASSIGNMENTS=0	UNSAT TT=0.48 ASSIGNMENTS=0	UNSAT TT=0.427 CHECKS=15 NODES=0	UNSAT TT=0.444 CHECKS=78 NODES=0	UNSAT TT=0.418 CHECKS=78 NODES=0	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=12	UNSAT TT=0.022 ASSIGNMENTS=0 CHECKS=12	UNSAT TT=1.027	UNSAT TT=0.76	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=12	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=12	UNSAT TT=0.288	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=12	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=12	UNSAT TT=0.925	UNSAT TT=0.48	UNSAT TT=0.499
csp/langford3/ normalized-langford-3-3.xml	UNSAT TT=0.006	UNSAT TT=0.676 CHECKS=189 NODES=0	UNSAT TT=0.721 CHECKS=189 NODES=0	UNSAT TT=0.029	UNSAT TT=0.012 CHECKS=203	UNSAT TT=0.007 CHECKS=203	UNSAT TT=0.518 ASSIGNMENTS=0	UNSAT TT=0.613 ASSIGNMENTS=0	UNSAT TT=0.522 CHECKS=200 NODES=0	UNSAT TT=0.515 CHECKS=358 NODES=0	UNSAT TT=0.542 CHECKS=358 NODES=0	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=82	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=82	UNSAT TT=1.082	UNSAT TT=0.919	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=82	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=82	UNSAT TT=0.31	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=82	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=82	UNSAT TT=1.275	UNSAT TT=0.564	UNSAT TT=0.531
csp/langford3/ normalized-langford-3-40.xml		? (TO) TT=1800.2	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800	? (TO) TT=1800.01	? (MO) TT=1182.78	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1800.09	? TT=561.598	? (TO) TT=1800.02	? (TO) TT=1800.08
csp/langford3/ normalized-langford-3-5.xml	UNSAT TT=0.009	UNSAT TT=0.768 CHECKS=3135 NODES=12	UNSAT TT=0.863 CHECKS=7462 NODES=0	UNSAT TT=0.038	UNSAT TT=0.025 CHECKS=9334	UNSAT TT=0.011 CHECKS=12337	UNSAT TT=0.669 ASSIGNMENTS=0	UNSAT TT=0.72 ASSIGNMENTS=0	UNSAT TT=0.746 CHECKS=1927 NODES=10	? (TO) TT=1800.1	? (TO) TT=1800.08	UNSAT TT=0.021 ASSIGNMENTS=5 CHECKS=804	UNSAT TT=0.023 ASSIGNMENTS=5 CHECKS=804	UNSAT TT=1.142	UNSAT TT=0.975	UNSAT TT=0.009 ASSIGNMENTS=5 CHECKS=804	UNSAT TT=0.012 ASSIGNMENTS=6 CHECKS=920	UNSAT TT=0.413	UNSAT TT=0.009 ASSIGNMENTS=18 CHECKS=1802	UNSAT TT=0.01 ASSIGNMENTS=5 CHECKS=804	UNSAT TT=1.718	UNSAT TT=0.693	UNSAT TT=0.687
csp/langford3/ normalized-langford-3-6.xml	UNSAT TT=0.012	UNSAT TT=0.854 CHECKS=10812 NODES=43	UNSAT TT=0.877 CHECKS=16154 NODES=43	UNSAT TT=0.044	UNSAT TT=0.038 CHECKS=37526	UNSAT TT=0.02 CHECKS=42296	UNSAT TT=0.901 ASSIGNMENTS=0	UNSAT TT=0.804 ASSIGNMENTS=0	UNSAT TT=0.966 CHECKS=4712 NODES=35	? (TO) TT=1800.05	? (TO) TT=1800.06	UNSAT TT=0.024 ASSIGNMENTS=15 CHECKS=2190	UNSAT TT=0.024 ASSIGNMENTS=15 CHECKS=2190	UNSAT TT=1.228	UNSAT TT=1.241	UNSAT TT=0.012 ASSIGNMENTS=15 CHECKS=2257	UNSAT TT=0.012 ASSIGNMENTS=17 CHECKS=2629	UNSAT TT=0.406	UNSAT TT=0.015 ASSIGNMENTS=72 CHECKS=8304	UNSAT TT=0.013 ASSIGNMENTS=15 CHECKS=2190	UNSAT TT=2.042	UNSAT TT=0.909	UNSAT TT=0.77
csp/langford3/ normalized-langford-3-9.xml	SAT TT=0.029	SAT TT=1.116 CHECKS=122460 NODES=439	SAT TT=1.188 CHECKS=145475 NODES=464	SAT TT=0.082	SAT TT=0.273 CHECKS=1254320	SAT TT=0.239 CHECKS=1231570	SAT TT=1.27 ASSIGNMENTS=47	SAT TT=1.204 ASSIGNMENTS=148	SAT TT=1.112 CHECKS=4944 NODES=32	SAT TT=8.558 CHECKS=50477600 NODES=123551	? (TO) TT=1800.01	SAT TT=0.042 ASSIGNMENTS=110 CHECKS=19284	SAT TT=0.039 ASSIGNMENTS=110 CHECKS=19284	SAT TT=2.425	SAT TT=2.392	SAT TT=0.029 ASSIGNMENTS=110 CHECKS=18737	SAT TT=0.033 ASSIGNMENTS=157 CHECKS=24694	SAT TT=0.773	SAT TT=0.044 ASSIGNMENTS=247 CHECKS=39094	SAT TT=0.051 ASSIGNMENTS=329 CHECKS=56553	SAT TT=3.996	SAT TT=1.167	SAT TT=1.043
csp/langford4/ normalized-langford-4-11.xml	UNSAT TT=0.287	UNSAT TT=3.274 CHECKS=3540950 NODES=4874	UNSAT TT=3.273 CHECKS=3608040 NODES=4824	UNSAT TT=0.42	UNSAT TT=76.405 CHECKS=438714000	UNSAT TT=74.117 CHECKS=439312000	UNSAT TT=3.329 ASSIGNMENTS=1997	UNSAT TT=3.849 ASSIGNMENTS=2297	UNSAT TT=4.388 CHECKS=1797160 NODES=2660	? (TO) TT=1800.06	? (TO) TT=1800.08	UNSAT TT=0.308 ASSIGNMENTS=1085 CHECKS=421132	UNSAT TT=0.321 ASSIGNMENTS=1085 CHECKS=421132	UNSAT TT=4.763	UNSAT TT=5.556	UNSAT TT=0.287 ASSIGNMENTS=1085 CHECKS=420374	UNSAT TT=1.045 ASSIGNMENTS=4370 CHECKS=1610720	UNSAT TT=1.614	UNSAT TT=0.687 ASSIGNMENTS=2547 CHECKS=927686	UNSAT TT=0.457 ASSIGNMENTS=1694 CHECKS=653829	UNSAT TT=50.933	UNSAT TT=3.633	UNSAT TT=3.885
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/langford4/ normalized-langford-4-12.xml	UNSAT TT=0.924	UNSAT TT=8.389 CHECKS=12397300 NODES=17043	UNSAT TT=8.346 CHECKS=12474400 NODES=17039	UNSAT TT=1.056	UNSAT TT=111.334 CHECKS=644339000	UNSAT TT=107.688 CHECKS=594234000	UNSAT TT=6.429 ASSIGNMENTS=6711	UNSAT TT=12.201 ASSIGNMENTS=11501	UNSAT TT=12.282 CHECKS=6315260 NODES=7914	? (TO) TT=1800.02	? (TO) TT=1800.04	UNSAT TT=0.939 ASSIGNMENTS=3231 CHECKS=1339270	UNSAT TT=0.934 ASSIGNMENTS=3231 CHECKS=1339270	UNSAT TT=11.216	UNSAT TT=10.849	UNSAT TT=0.924 ASSIGNMENTS=3269 CHECKS=1353120	UNSAT TT=1.823 ASSIGNMENTS=6462 CHECKS=2611750	UNSAT TT=3.891	UNSAT TT=2.996 ASSIGNMENTS=10575 CHECKS=4253700	UNSAT TT=1.487 ASSIGNMENTS=5190 CHECKS=2134220	UNSAT TT=188.915	UNSAT TT=11.746	UNSAT TT=10.733
csp/langford4/ normalized-langford-4-14.xml	UNSAT TT=11.54	UNSAT TT=104.153 CHECKS=180272000 NODES=225947	UNSAT TT=102.354 CHECKS=180438000 NODES=226034	UNSAT TT=11.54	? (TO) TT=1800.09	? (TO) TT=1800.04	UNSAT TT=60.717 ASSIGNMENTS=86778	UNSAT TT=113.189 ASSIGNMENTS=127155	UNSAT TT=184.683 CHECKS=99379800 NODES=101231	? (TO) TT=1800.04	? (TO) TT=1800.04	UNSAT TT=110.935 ASSIGNMENTS=225947 CHECKS=180272000	UNSAT TT=110.463 ASSIGNMENTS=225947 CHECKS=180272000	UNSAT TT=502.476	UNSAT TT=523.083	UNSAT TT=12.444 ASSIGNMENTS=41400 CHECKS=19024500	UNSAT TT=13.608 ASSIGNMENTS=44886 CHECKS=20579500	UNSAT TT=61.888	UNSAT TT=55.108 ASSIGNMENTS=204109 CHECKS=87042100	UNSAT TT=19.061 ASSIGNMENTS=64780 CHECKS=29731500	? (TO) TT=1800.15	UNSAT TT=499.393	UNSAT TT=377.969
csp/langford4/ normalized-langford-4-17.xml	UNSAT TT=759.747	? (TO) TT=1800.11	? (TO) TT=1800.09	UNSAT TT=759.747	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.04	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.01	UNSAT TT=985.394 ASSIGNMENTS=2930680 CHECKS=1539890000	UNSAT TT=981.587 ASSIGNMENTS=2934720 CHECKS=1542030000	? (TO) TT=1800.12	? (TO) TT=1800.04	UNSAT TT=1259.87 ASSIGNMENTS=4010500 CHECKS=2108980000	? (TO) TT=1800.1	? (TO) TT=1800	? (TO) TT=1800.11
csp/langford4/ normalized-langford-4-18.xml		? (TO) TT=1800.11	? (TO) TT=1800.17	? (TO) TT=1800.01	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.12	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.01	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.12	? (TO) TT=1800.15	? (TO) TT=1800.06
csp/langford4/ normalized-langford-4-19.xml		? (TO) TT=1800.1	? (TO) TT=1800.15	? (TO) TT=1800.1	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.11	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.01	? (TO) TT=1800.2	? (TO) TT=1800.04	? (TO) TT=1800.07
csp/langford4/ normalized-langford-4-20.xml		? (TO) TT=1800.09	? (TO) TT=1800.12	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800	? (TO) TT=1800.08	? (TO) TT=1800	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.14	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.2	? (TO) TT=1800	? (TO) TT=1800.09
csp/langford4/ normalized-langford-4-2.xml	UNSAT TT=0.007	UNSAT TT=0.671 CHECKS=89 NODES=0	UNSAT TT=0.7 CHECKS=89 NODES=0	UNSAT TT=0.028	UNSAT TT=0.011 CHECKS=33	UNSAT TT=0.008 CHECKS=33	UNSAT TT=0.59 ASSIGNMENTS=0	UNSAT TT=0.514 ASSIGNMENTS=0	UNSAT TT=0.477 CHECKS=34 NODES=0	UNSAT TT=0.472 CHECKS=215 NODES=0	UNSAT TT=0.527 CHECKS=215 NODES=0	UNSAT TT=0.018 ASSIGNMENTS=0 CHECKS=25	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=25	UNSAT TT=0.931	UNSAT TT=0.823	UNSAT TT=0.009 ASSIGNMENTS=0 CHECKS=25	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=25	UNSAT TT=0.308	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=25	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=25	UNSAT TT=1.044	UNSAT TT=0.497	UNSAT TT=0.517
csp/langford4/ normalized-langford-4-30.xml		? (TO) TT=1800.1	? (TO) TT=1800.17	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.11	? (TO) TT=1800.11	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.01	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.1	? (TO) TT=1800.04	? (TO) TT=1800.04	? TT=349.357	? (TO) TT=1800.07	? (TO) TT=1800.03
csp/langford4/ normalized-langford-4-3.xml	UNSAT TT=0.006	UNSAT TT=0.737 CHECKS=218 NODES=0	UNSAT TT=0.742 CHECKS=218 NODES=0	UNSAT TT=0.032	UNSAT TT=0.02 CHECKS=72	UNSAT TT=0.011 CHECKS=72	UNSAT TT=0.607 ASSIGNMENTS=0	UNSAT TT=0.666 ASSIGNMENTS=0	UNSAT TT=0.593 CHECKS=52 NODES=0	UNSAT TT=0.625 CHECKS=540 NODES=0	UNSAT TT=0.62 CHECKS=540 NODES=0	UNSAT TT=0.02 ASSIGNMENTS=0 CHECKS=50	UNSAT TT=0.018 ASSIGNMENTS=0 CHECKS=50	UNSAT TT=1.169	UNSAT TT=0.829	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=50	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=50	UNSAT TT=0.362	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=50	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=50	UNSAT TT=1.657	UNSAT TT=0.597	UNSAT TT=0.66
csp/langford4/ normalized-langford-4-40.xml		? (TO) TT=1800.14	? (TO) TT=1800.14	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.15	? (TO) TT=1800.01	? (TO) TT=1800.01	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.06	? (MO) TT=1262.45	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.07	? TT=118.917	? (TO) TT=1800.08	? (TO) TT=1800
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/langford4/ normalized-langford-4-4.xml	UNSAT TT=0.01	UNSAT TT=0.753 CHECKS=875 NODES=0	UNSAT TT=0.775 CHECKS=875 NODES=0	UNSAT TT=0.04	UNSAT TT=0.027 CHECKS=815	UNSAT TT=0.01 CHECKS=815	UNSAT TT=0.771 ASSIGNMENTS=0	UNSAT TT=0.731 ASSIGNMENTS=0	UNSAT TT=0.756 CHECKS=857 NODES=0	UNSAT TT=0.833 CHECKS=1773 NODES=0	UNSAT TT=0.706 CHECKS=1773 NODES=0	UNSAT TT=0.023 ASSIGNMENTS=0 CHECKS=224	UNSAT TT=0.022 ASSIGNMENTS=0 CHECKS=224	UNSAT TT=1.569	UNSAT TT=1.006	UNSAT TT=0.01 ASSIGNMENTS=0 CHECKS=224	UNSAT TT=0.011 ASSIGNMENTS=0 CHECKS=224	UNSAT TT=0.438	UNSAT TT=0.01 ASSIGNMENTS=0 CHECKS=224	UNSAT TT=0.012 ASSIGNMENTS=0 CHECKS=224	UNSAT TT=1.968	UNSAT TT=0.688	UNSAT TT=0.694
csp/langford4/ normalized-langford-4-5.xml	UNSAT TT=0.011	UNSAT TT=0.831 CHECKS=3255 NODES=1	UNSAT TT=0.816 CHECKS=3285 NODES=0	UNSAT TT=0.05	UNSAT TT=0.035 CHECKS=6858	UNSAT TT=0.014 CHECKS=6934	UNSAT TT=0.784 ASSIGNMENTS=0	UNSAT TT=0.88 ASSIGNMENTS=0	UNSAT TT=0.88 CHECKS=3147 NODES=4	? (TO) TT=1800.08	? (TO) TT=1800.06	UNSAT TT=0.024 ASSIGNMENTS=1 CHECKS=806	UNSAT TT=0.025 ASSIGNMENTS=1 CHECKS=806	UNSAT TT=1.191	UNSAT TT=1.175	UNSAT TT=0.012 ASSIGNMENTS=1 CHECKS=806	UNSAT TT=0.013 ASSIGNMENTS=2 CHECKS=891	UNSAT TT=0.528	UNSAT TT=0.011 ASSIGNMENTS=12 CHECKS=2217	UNSAT TT=0.014 ASSIGNMENTS=1 CHECKS=806	UNSAT TT=2.085	UNSAT TT=0.776	UNSAT TT=0.809
csp/langford4/ normalized-langford-4-8.xml	UNSAT TT=0.029	UNSAT TT=1.144 CHECKS=107341 NODES=172	UNSAT TT=1.149 CHECKS=139296 NODES=172	UNSAT TT=0.096	UNSAT TT=0.147 CHECKS=469450	UNSAT TT=0.1 CHECKS=456513	UNSAT TT=1.355 ASSIGNMENTS=19	UNSAT TT=1.276 ASSIGNMENTS=22	UNSAT TT=1.283 CHECKS=28080 NODES=113	? (TO) TT=1800.08	? (TO) TT=1800.06	UNSAT TT=0.042 ASSIGNMENTS=38 CHECKS=12609	UNSAT TT=0.041 ASSIGNMENTS=38 CHECKS=12609	UNSAT TT=1.974	UNSAT TT=2.272	UNSAT TT=0.029 ASSIGNMENTS=38 CHECKS=12605	UNSAT TT=0.038 ASSIGNMENTS=107 CHECKS=29706	UNSAT TT=0.633	UNSAT TT=0.055 ASSIGNMENTS=176 CHECKS=44518	UNSAT TT=0.032 ASSIGNMENTS=38 CHECKS=12642	UNSAT TT=4.686	UNSAT TT=1.131	UNSAT TT=1.164
csp/langford4/ normalized-langford-4-9.xml	UNSAT TT=0.048	UNSAT TT=1.36 CHECKS=310918 NODES=384	UNSAT TT=1.292 CHECKS=350777 NODES=396	UNSAT TT=0.119	UNSAT TT=0.741 CHECKS=4143060	UNSAT TT=0.802 CHECKS=4812750	UNSAT TT=1.383 ASSIGNMENTS=40	UNSAT TT=1.595 ASSIGNMENTS=205	UNSAT TT=1.51 CHECKS=116307 NODES=194	? (TO) TT=1800.09	? (TO) TT=1800.07	UNSAT TT=0.063 ASSIGNMENTS=107 CHECKS=38109	UNSAT TT=0.061 ASSIGNMENTS=107 CHECKS=38109	UNSAT TT=2.5	UNSAT TT=2.281	UNSAT TT=0.048 ASSIGNMENTS=90 CHECKS=33757	UNSAT TT=0.077 ASSIGNMENTS=247 CHECKS=78361	UNSAT TT=0.728	UNSAT TT=0.103 ASSIGNMENTS=360 CHECKS=109655	UNSAT TT=0.052 ASSIGNMENTS=128 CHECKS=45891	UNSAT TT=6.969	UNSAT TT=1.173	UNSAT TT=1.277
csp/largeQueens/ normalized-queens-1000.xml		? (MO) TT=195.063	? (MO) TT=190.497	? (problem) TT=114.741	? TT=41.426	? TT=1408.4	? (TO) TT=1802.18	? (TO) TT=1802.18	? (TO) TT=1800.06	? (TO) TT=1800.02	? (TO) TT=1800.1	? (TO) TT=1800.06	? (problem) TT=1651.78	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.17	? (TO) TT=1800.09	? (exit code) TT=879.604	? (TO) TT=1800.16	? (TO) TT=1800.16	? TT=214.851	? TT=329.86	? TT=349.209
csp/largeQueens/ normalized-queens-300.xml	SAT TT=10.921	SAT TT=47.667 CHECKS=60271000 NODES=764	SAT TT=48.538 CHECKS=60271000 NODES=764	SAT TT=10.921	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) TT=1800.16	? (TO) TT=1800.14	? TT=1094.35	? (TO) TT=1078.44	? (TO) TT=1095.6	? (TO) TT=1800.04	SAT TT=96.78 ASSIGNMENTS=3857 CHECKS=192532000	? (MO) TT=1293.4	? (MO) TT=1278.05	? (TO) TT=1800.09	SAT TT=381.094 ASSIGNMENTS=283 CHECKS=8146680	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.04	? TT=538.817	? (MO) TT=148.413	? (MO) TT=335.366
csp/largeQueens/ normalized-queens-400.xml	SAT TT=20.094	? (TO) TT=1800.15	? (TO) TT=1800.15	SAT TT=20.094	? (TO) TT=1800.08	? (TO) TT=1800.13	? (TO) TT=1800.25	? (TO) TT=1800.25	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.01	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.01	SAT TT=1163.14 ASSIGNMENTS=386 CHECKS=19469000	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.11	? (TO) TT=1800.1	? TT=217.91	? TT=198.9	? TT=182.026
csp/largeQueens/ normalized-queens-500.xml	SAT TT=47.463	? (TO) TT=1800.18	? (TO) TT=1800.17	SAT TT=47.463	? TT=798.022	? (TO) TT=1800.13	? (MO) TT=147.012	? (MO) TT=168.129	? (TO) TT=1800	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.03	? TT=217.806	? TT=211.551	? TT=197.264
csp/largeQueens/ normalized-queens-800.xml		? (MO) TT=58.382	? (MO) TT=57.752	? (TO) TT=1800.15	Signal TT=44.892	? TT=568.455	? (TO) TT=1802.22	? (TO) TT=1802.19	? (TO) TT=1800	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.08	? (exit code) TT=207.864	? (TO) TT=1800.14	? (TO) TT=1800.06	? TT=243.976	? (MO) TT=6.299	? TT=139.476
csp/nengfa/ normalized-color.xml	UNSAT TT=1.574	UNSAT TT=30.112 CHECKS=16358100 NODES=15419	UNSAT TT=20.121 CHECKS=8515120 NODES=8855	UNSAT TT=56.097	UNSAT TT=1038.52 CHECKS=2277000000	? (TO) TT=1800.04	UNSAT TT=18.273 ASSIGNMENTS=471	UNSAT TT=14.323 ASSIGNMENTS=91	UNSAT TT=124.562 CHECKS=2886830 NODES=7605	? (TO) TT=1800.02	? (TO) TT=1800.09	UNSAT TT=1.594 ASSIGNMENTS=1460 CHECKS=1007930	UNSAT TT=1.574 ASSIGNMENTS=1460 CHECKS=1007930	UNSAT TT=1525.69	UNSAT TT=1428.69	UNSAT TT=1.575 ASSIGNMENTS=1500 CHECKS=967643	UNSAT TT=12.455 ASSIGNMENTS=20302 CHECKS=15248200	UNSAT TT=3.252	UNSAT TT=2.427 ASSIGNMENTS=4242 CHECKS=1827390	UNSAT TT=1.761 ASSIGNMENTS=1993 CHECKS=1329870	UNSAT TT=174.167	UNSAT TT=8.212	UNSAT TT=8.352
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/nengfa/ normalized-queens1.xml	SAT TT=2.198	SAT TT=4.196 CHECKS=734619 NODES=171	SAT TT=4.677 CHECKS=1226290 NODES=171	SAT TT=64.045	SAT TT=24.803 CHECKS=50564700	SAT TT=51.429 CHECKS=119303000	SAT TT=57.435 ASSIGNMENTS=538	SAT TT=56.573 ASSIGNMENTS=179	SAT TT=53.104 CHECKS=4287610 NODES=872	SAT TT=21.312 CHECKS=7264080 NODES=72	SAT TT=22.38 CHECKS=8987760 NODES=100	SAT TT=51.329 ASSIGNMENTS=538	SAT TT=12.959 ASSIGNMENTS=192 CHECKS=752993	? (TO) TT=1800	? (TO) TT=1800.09	SAT TT=249.191 ASSIGNMENTS=50386 CHECKS=222703000	SAT TT=133.192 ASSIGNMENTS=10209 CHECKS=103484000	? (TO) TT=1800.08	SAT TT=141.931 ASSIGNMENTS=7459 CHECKS=99992400	SAT TT=2.198 ASSIGNMENTS=103 CHECKS=965177	? TT=605.589	SAT TT=12.047	SAT TT=35.547
csp/nengfa/ normalized-route.xml	SAT TT=4.692	SAT TT=7.963 CHECKS=75471 NODES=81	SAT TT=8.606 CHECKS=202929 NODES=0	SAT TT=257.539	SAT TT=21.234 CHECKS=20942800	SAT TT=17.992 CHECKS=14367400	SAT TT=17.059 ASSIGNMENTS=33	SAT TT=16.142 ASSIGNMENTS=36	SAT TT=21.759 CHECKS=115846 NODES=117	SAT TT=138.084 CHECKS=49755100 NODES=9057	SAT TT=28.152 CHECKS=8875040 NODES=437	SAT TT=14.706 ASSIGNMENTS=115 CHECKS=3601910	SAT TT=16.973 ASSIGNMENTS=81 CHECKS=75471	? (TO) TT=1800.07	? (TO) TT=1800.09	SAT TT=4.692 ASSIGNMENTS=116 CHECKS=3594490	SAT TT=6.223 ASSIGNMENTS=389 CHECKS=4617750	SAT TT=6.323	SAT TT=8.757 ASSIGNMENTS=271 CHECKS=7017180	SAT TT=5.609 ASSIGNMENTS=115 CHECKS=3601910	? (exit code) TT=98.927	SAT TT=6.839	SAT TT=7.342
csp/pigeons/ normalized-pigeons-11.xml	UNSAT TT=0.005	UNSAT TT=0.68 CHECKS=20 NODES=0	UNSAT TT=0.72 CHECKS=20 NODES=0	UNSAT TT=0.031	UNSAT TT=0.011 CHECKS=65	UNSAT TT=0.011 CHECKS=65	UNSAT TT=0.598 ASSIGNMENTS=0	UNSAT TT=0.564 ASSIGNMENTS=0	UNSAT TT=75.331 CHECKS=21797200 NODES=3977560	? (TO) TT=1800.09	? (TO) TT=1800.09	UNSAT TT=0.02 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.02 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.08	UNSAT TT=59.16	UNSAT TT=0.009 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=23.999	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.005 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=580.843	UNSAT TT=3.213	UNSAT TT=9.989
csp/pigeons/ normalized-pigeons-13.xml	UNSAT TT=0.006	UNSAT TT=0.78 CHECKS=24 NODES=0	UNSAT TT=0.792 CHECKS=24 NODES=0	UNSAT TT=0.033	UNSAT TT=0.012 CHECKS=90	UNSAT TT=0.008 CHECKS=90	UNSAT TT=0.734 ASSIGNMENTS=0	UNSAT TT=0.634 ASSIGNMENTS=0	? (TO) TT=1800.08	? (TO) TT=1800.01	? (TO) TT=1800.05	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.02 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.01	? (TO) TT=1800.07	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.009 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.05	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.11	UNSAT TT=127.239	UNSAT TT=421.563
csp/pigeons/ normalized-pigeons-14.xml	UNSAT TT=0.008	UNSAT TT=0.806 CHECKS=26 NODES=0	UNSAT TT=0.716 CHECKS=26 NODES=0	UNSAT TT=0.035	UNSAT TT=0.013 CHECKS=104	UNSAT TT=0.009 CHECKS=104	UNSAT TT=0.589 ASSIGNMENTS=0	UNSAT TT=0.623 ASSIGNMENTS=0	? (TO) TT=1800.01	? (TO) TT=1800.01	? (TO) TT=1800.03	UNSAT TT=0.02 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.09	? (TO) TT=1800.07	UNSAT TT=0.009 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.009 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.04	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.009 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.17	UNSAT TT=721.198	? (TO) TT=1800.05
csp/pigeons/ normalized-pigeons-16.xml	UNSAT TT=0.007	UNSAT TT=0.816 CHECKS=30 NODES=0	UNSAT TT=0.768 CHECKS=30 NODES=0	UNSAT TT=0.039	UNSAT TT=0.016 CHECKS=135	UNSAT TT=0.012 CHECKS=135	UNSAT TT=0.589 ASSIGNMENTS=0	UNSAT TT=0.59 ASSIGNMENTS=0	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.03	UNSAT TT=0.021 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.021 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.008 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.1	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.011 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.18	? (TO) TT=1800.15	? (TO) TT=1800.06
csp/pigeons/ normalized-pigeons-20.xml	UNSAT TT=0.011	UNSAT TT=0.827 CHECKS=38 NODES=0	UNSAT TT=0.838 CHECKS=38 NODES=0	UNSAT TT=0.044	UNSAT TT=0.021 CHECKS=209	UNSAT TT=0.015 CHECKS=209	UNSAT TT=0.671 ASSIGNMENTS=0	UNSAT TT=0.758 ASSIGNMENTS=0	? (TO) TT=1800.02	? (TO) TT=1800.03	? (TO) TT=1800.04	UNSAT TT=0.023 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.022 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800	? (TO) TT=1800.08	UNSAT TT=0.012 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.011 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.11	UNSAT TT=0.013 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.011 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.16	? (TO) TT=1800.11	? (TO) TT=1800.11
csp/pigeons/ normalized-pigeons-30-ord.xml	UNSAT TT=0.032	UNSAT TT=1.049 CHECKS=1654 NODES=0	UNSAT TT=1.077 CHECKS=1654 NODES=0	UNSAT TT=0.128	UNSAT TT=0.292 CHECKS=1363450	UNSAT TT=0.27 CHECKS=1468320	UNSAT TT=1.225 ASSIGNMENTS=0	UNSAT TT=1.358 ASSIGNMENTS=0	UNSAT TT=2.289 CHECKS=96750 NODES=1234	? (TO) TT=1800.03	? (TO) TT=1800	UNSAT TT=0.047 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.046 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=14.976	UNSAT TT=5.758	UNSAT TT=0.032 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.032 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.712	UNSAT TT=0.034 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.034 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=4.861	UNSAT TT=1.912	UNSAT TT=1.611
csp/pigeons/ normalized-pigeons-30.xml	UNSAT TT=0.018	UNSAT TT=1.033 CHECKS=58 NODES=0	UNSAT TT=0.947 CHECKS=58 NODES=0	UNSAT TT=0.076	UNSAT TT=0.043 CHECKS=464	UNSAT TT=0.024 CHECKS=464	UNSAT TT=0.938 ASSIGNMENTS=0	UNSAT TT=1.058 ASSIGNMENTS=0	? (TO) TT=1800.05	? (TO) TT=1800.04	? (TO) TT=1800.01	UNSAT TT=0.03 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.031 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800	? (TO) TT=1800.08	UNSAT TT=0.02 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.03	UNSAT TT=0.019 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.018 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.23	? (TO) TT=1800.08	? (TO) TT=1800.08
csp/pigeons/ normalized-pigeons-35.xml	UNSAT TT=0.024	UNSAT TT=1.015 CHECKS=68 NODES=0	UNSAT TT=1.062 CHECKS=68 NODES=0	UNSAT TT=0.096	UNSAT TT=0.054 CHECKS=629	UNSAT TT=0.034 CHECKS=629	UNSAT TT=1.194 ASSIGNMENTS=0	UNSAT TT=1.087 ASSIGNMENTS=0	? (TO) TT=1800.02	? (TO) TT=1800.03	? (TO) TT=1800.09	UNSAT TT=0.037 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.036 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.03	? (TO) TT=1800.02	UNSAT TT=0.024 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.024 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.11	UNSAT TT=0.025 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.026 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.1	? (TO) TT=1800.05	? (TO) TT=1800.14
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/pigeons/ normalized-pigeons-40.xml	UNSAT TT=0.029	UNSAT TT=1.224 CHECKS=78 NODES=0	UNSAT TT=1.216 CHECKS=78 NODES=0	UNSAT TT=0.117	UNSAT TT=0.076 CHECKS=819	UNSAT TT=0.043 CHECKS=819	UNSAT TT=1.389 ASSIGNMENTS=0	UNSAT TT=1.102 ASSIGNMENTS=0	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.01	UNSAT TT=0.042 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.043 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.01	? (TO) TT=1800.02	UNSAT TT=0.03 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.03 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.09	UNSAT TT=0.029 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.03 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.22	? (TO) TT=1800.04	? (TO) TT=1800.12
csp/pigeons/ normalized-pigeons-45.xml	UNSAT TT=0.035	UNSAT TT=1.168 CHECKS=88 NODES=0	UNSAT TT=1.219 CHECKS=88 NODES=0	UNSAT TT=0.142	UNSAT TT=0.095 CHECKS=1034	UNSAT TT=0.055 CHECKS=1034	UNSAT TT=1.19 ASSIGNMENTS=0	UNSAT TT=1.425 ASSIGNMENTS=0	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.02	UNSAT TT=0.047 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.05 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.02	? (TO) TT=1800.06	UNSAT TT=0.035 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.037 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.06	UNSAT TT=0.037 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.037 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.16	? (TO) TT=1800.07	? (TO) TT=1800.07
csp/pigeons/ normalized-pigeons-50.xml	UNSAT TT=0.04	UNSAT TT=1.25 CHECKS=98 NODES=0	UNSAT TT=1.246 CHECKS=98 NODES=0	UNSAT TT=0.159	UNSAT TT=0.115 CHECKS=1274	UNSAT TT=0.063 CHECKS=1274	UNSAT TT=1.244 ASSIGNMENTS=0	UNSAT TT=1.442 ASSIGNMENTS=0	? (TO) TT=1800.01	? (TO) TT=1800.03	? (TO) TT=1800.05	UNSAT TT=0.056 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.057 ASSIGNMENTS=0 CHECKS=1	? (MO) TT=1629.72	? (TO) TT=1800.07	UNSAT TT=0.042 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.043 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.06	UNSAT TT=0.043 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.04 ASSIGNMENTS=0 CHECKS=1	? (TO) TT=1800.33	? (TO) TT=1800.01	? (TO) TT=1800.1
csp/pigeons/ normalized-pigeons-8.xml	UNSAT TT=0.006	UNSAT TT=0.643 CHECKS=14 NODES=0	UNSAT TT=0.663 CHECKS=14 NODES=0	UNSAT TT=0.025	UNSAT TT=0.008 CHECKS=35	UNSAT TT=0.008 CHECKS=35	UNSAT TT=0.559 ASSIGNMENTS=0	UNSAT TT=0.513 ASSIGNMENTS=0	UNSAT TT=0.607 CHECKS=33553 NODES=5913	? (TO) TT=1800.04	? (TO) TT=1800.01	UNSAT TT=0.018 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.016 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.838	UNSAT TT=0.65	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.007 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.303	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=0.006 ASSIGNMENTS=0 CHECKS=1	UNSAT TT=1.598	UNSAT TT=0.56	UNSAT TT=0.555
csp/queenAttacking/ normalized-queenAttacking-10.xml		? (TO) TT=1800.13	? (TO) TT=1800.05	? (TO) TT=1800.02	UNSAT TT=61.888 CHECKS=73816900	UNSAT TT=63.604 CHECKS=99104600	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) TT=1800.02	? (TO) TT=1800	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.03	? TT=898.494	? (TO) TT=1800.05	? (TO) TT=1800.09
csp/queenAttacking/ normalized-queenAttacking-11.xml	UNSAT TT=368.848	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.03	UNSAT TT=82.413 CHECKS=83432700	UNSAT TT=71.69 CHECKS=117741000	? (TO) TT=1800.13	UNSAT TT=368.848 ASSIGNMENTS=66	? (TO) TT=1800.09	? (TO) TT=1800	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.11	? TT=150.712	? (TO) TT=1800.09	? (TO) TT=1800.05
csp/queenAttacking/ normalized-queenAttacking-12.xml	UNSAT TT=748.425	? (TO) TT=1800.13	? (TO) TT=1800.11	? (TO) TT=1800.1	UNSAT TT=142.411 CHECKS=141295000	UNSAT TT=78.293 CHECKS=105624000	UNSAT TT=1539.86 ASSIGNMENTS=1520	UNSAT TT=748.425 ASSIGNMENTS=22	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.08	? (TO) TT=1800.02	? (TO) TT=1800	? (TO) TT=1800.06	? (MO) TT=842.09	? (TO) TT=1800.01	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.05	? TT=256.536	? (TO) TT=1800.1	? (TO) TT=1800.07
csp/queenAttacking/ normalized-queenAttacking-3.xml	UNSAT TT=0.009	UNSAT TT=0.679 CHECKS=137 NODES=0	UNSAT TT=0.74 CHECKS=137 NODES=0	UNSAT TT=0.034	UNSAT TT=0.024 CHECKS=1239	UNSAT TT=0.011 CHECKS=1377	UNSAT TT=0.704 ASSIGNMENTS=0	UNSAT TT=0.701 ASSIGNMENTS=0	UNSAT TT=0.583 CHECKS=1406 NODES=30	? (TO) TT=1800.02	? (TO) TT=1800	UNSAT TT=0.022 ASSIGNMENTS=31 CHECKS=1205	UNSAT TT=0.021 ASSIGNMENTS=31 CHECKS=1205	UNSAT TT=0.779	UNSAT TT=0.772	UNSAT TT=0.009 ASSIGNMENTS=31 CHECKS=1133	UNSAT TT=0.012 ASSIGNMENTS=28 CHECKS=1338	UNSAT TT=2.968	UNSAT TT=0.016 ASSIGNMENTS=89 CHECKS=3000	UNSAT TT=0.011 ASSIGNMENTS=24 CHECKS=1243	UNSAT TT=1.754	UNSAT TT=0.661	UNSAT TT=0.646
csp/queenAttacking/ normalized-queenAttacking-4.xml	UNSAT TT=0.05	UNSAT TT=1.099 CHECKS=271816 NODES=1889	UNSAT TT=1.066 CHECKS=278200 NODES=1885	UNSAT TT=0.076	UNSAT TT=21.554 CHECKS=17419400	UNSAT TT=32.073 CHECKS=26245100	UNSAT TT=1.435 ASSIGNMENTS=388	UNSAT TT=1.413 ASSIGNMENTS=343	UNSAT TT=0.858 CHECKS=20669 NODES=440	? (TO) TT=1800.06	? (TO) TT=1800.02	UNSAT TT=0.071 ASSIGNMENTS=335 CHECKS=23413	UNSAT TT=0.071 ASSIGNMENTS=335 CHECKS=23413	UNSAT TT=1.01	UNSAT TT=1.173	UNSAT TT=0.05 ASSIGNMENTS=298 CHECKS=21117	UNSAT TT=0.286 ASSIGNMENTS=3326 CHECKS=188227	? (TO) TT=1800.11	UNSAT TT=0.156 ASSIGNMENTS=928 CHECKS=57155	UNSAT TT=0.058 ASSIGNMENTS=380 CHECKS=25928	UNSAT TT=3.275	UNSAT TT=1.278	UNSAT TT=1.483
csp/queenAttacking/ normalized-queenAttacking-5.xml	SAT TT=0.049	SAT TT=1.417 CHECKS=796459 NODES=3289	SAT TT=1.531 CHECKS=811169 NODES=3289	SAT TT=5.81	Wrong UNSAT TT=24.873 CHECKS=27090400	Wrong UNSAT TT=40.65 CHECKS=44647500	SAT TT=20.396 ASSIGNMENTS=38676	SAT TT=4.237 ASSIGNMENTS=2481	SAT TT=2.359 CHECKS=259545 NODES=5730	SAT TT=2.389 CHECKS=9788200 NODES=16496	SAT TT=178.158 CHECKS=1345730000 NODES=495407	SAT TT=0.986 ASSIGNMENTS=7096 CHECKS=644160	SAT TT=0.987 ASSIGNMENTS=7096 CHECKS=644160	SAT TT=2.224	SAT TT=2.275	SAT TT=1.658 ASSIGNMENTS=12809 CHECKS=1104960	SAT TT=0.049 ASSIGNMENTS=130 CHECKS=10334	? (TO) TT=1800.07	SAT TT=1.2 ASSIGNMENTS=5299 CHECKS=511608	SAT TT=0.108 ASSIGNMENTS=533 CHECKS=48236	SAT TT=7.266	SAT TT=1.629	SAT TT=61.253
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/queenAttacking/ normalized-queenAttacking-6.xml	SAT TT=4.23	? (TO) TT=1800.09	? (TO) TT=1800.17	SAT TT=285.777	Wrong UNSAT TT=23.896 CHECKS=31886300	Wrong UNSAT TT=32.401 CHECKS=48609900	SAT TT=9.904 ASSIGNMENTS=647	SAT TT=94.433 ASSIGNMENTS=114209	SAT TT=4.23 CHECKS=1503520 NODES=7137	SAT TT=19.538 CHECKS=152673000 NODES=124555	? (TO) TT=1800.07	SAT TT=29.634 ASSIGNMENTS=34451 CHECKS=3923430	SAT TT=299.917 ASSIGNMENTS=34451 CHECKS=3923430	SAT TT=15.85	SAT TT=11.322	SAT TT=54.742 ASSIGNMENTS=97561 CHECKS=11313400	SAT TT=21.415 ASSIGNMENTS=26424 CHECKS=3197780	? (TO) TT=1800.1	SAT TT=58.59 ASSIGNMENTS=89665 CHECKS=13315300	SAT TT=22.209 ASSIGNMENTS=34451 CHECKS=3923430	SAT TT=103.475	? (TO) TT=1800.1	SAT TT=154.158
csp/queenAttacking/ normalized-queenAttacking-7.xml	SAT TT=450.699	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) TT=1800.07	Wrong UNSAT TT=37.204 CHECKS=58035700	Wrong UNSAT TT=52.559 CHECKS=87594000	SAT TT=491.793 ASSIGNMENTS=597366	SAT TT=450.699 ASSIGNMENTS=328995	SAT TT=511.645 CHECKS=337241000 NODES=792499	SAT TT=624.23 CHECKS=5777620000 NODES=2511800	? (TO) TT=1800.07	SAT TT=1059.03 ASSIGNMENTS=1727560 CHECKS=212930000	SAT TT=1619.54 ASSIGNMENTS=597366	? (TO) TT=1800.04	SAT TT=909.859	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.03	SAT TT=1212.58 ASSIGNMENTS=1780320 CHECKS=320460000	SAT TT=1091.67 ASSIGNMENTS=1727560 CHECKS=212930000	? (TO) TT=1800.09	SAT TT=1010.82	? (TO) TT=1800.13
csp/queenAttacking/ normalized-queenAttacking-8.xml		? (TO) TT=1800.11	? (TO) TT=1800.13	? (TO) TT=1800.07	Wrong UNSAT TT=30.106 CHECKS=41352500	Wrong UNSAT TT=37.672 CHECKS=60161000	? (TO) TT=1800.12	? (TO) TT=1800.16	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.06	? (TO) TT=1800	? (TO) TT=1800.09	? (TO) TT=1800.05	SAT TT=1090.66	? (TO) TT=1800.01	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.17	? (TO) TT=1800.09	? (TO) TT=1800.1
csp/queenAttacking/ normalized-queenAttacking-9.xml		? (TO) TT=1800.1	? (TO) TT=1800.18	? (TO) TT=1800.09	UNSAT TT=35.756 CHECKS=40447900	UNSAT TT=35.827 CHECKS=55227200	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.01	? (TO) TT=1800.07	? (TO) TT=1800.1	? (MO) TT=1261.81	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.18	? (TO) TT=1800.08	? (TO) TT=1800.01
csp/queensKnights/ normalized-queensKnights-100-5-add.xml	UNSAT TT=21.342	UNSAT TT=1252.2 CHECKS=1355730000 NODES=9898	UNSAT TT=1282.62 CHECKS=1355730000 NODES=9898	? (TO) TT=1800.05	? (TO) TT=1800.11	? (TO) TT=1800.01	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800	? (TO) TT=1800.06	? (TO) TT=1800.06	? TT=1216.81	UNSAT TT=21.342	UNSAT TT=43.567
csp/queensKnights/ normalized-queensKnights-100-5-mul.xml	UNSAT TT=30.76	? (TO) TT=1800.1	? (TO) TT=1800.19	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.12	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800	? (TO) TT=1800.04	? (TO) TT=1800.04	? (TO) TT=1800.11	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.12	? TT=292.326	UNSAT TT=30.76	UNSAT TT=45.023
csp/queensKnights/ normalized-queensKnights-10-5-mul.xml	UNSAT TT=0.864	UNSAT TT=0.864 CHECKS=101151 NODES=85	UNSAT TT=0.87 CHECKS=141037 NODES=0	UNSAT TT=16.858	UNSAT TT=24.47 CHECKS=11796600	UNSAT TT=34.952 CHECKS=15679800	UNSAT TT=1.429 ASSIGNMENTS=0	UNSAT TT=1.442 ASSIGNMENTS=0	UNSAT TT=0.93 CHECKS=369064 NODES=326	? (TO) TT=1800.01	? (TO) TT=1800.02	UNSAT TT=8.243 ASSIGNMENTS=0	UNSAT TT=7.933 ASSIGNMENTS=99 CHECKS=141102	UNSAT TT=2.205	UNSAT TT=2.038	UNSAT TT=13.386 ASSIGNMENTS=1681 CHECKS=78255	UNSAT TT=287.272 ASSIGNMENTS=37844 CHECKS=1590670	? (TO) TT=1800.06	UNSAT TT=4.387 ASSIGNMENTS=700 CHECKS=103869	UNSAT TT=3.463 ASSIGNMENTS=441 CHECKS=25257	UNSAT TT=4.628	UNSAT TT=0.882	UNSAT TT=0.979
csp/queensKnights/ normalized-queensKnights-12-5-mul.xml	UNSAT TT=1.005	UNSAT TT=1.191 CHECKS=272609 NODES=128	UNSAT TT=1.005 CHECKS=403318 NODES=0	UNSAT TT=762.433	UNSAT TT=27.736 CHECKS=13912600	UNSAT TT=35.239 CHECKS=22822700	UNSAT TT=1.371 ASSIGNMENTS=0	UNSAT TT=1.519 ASSIGNMENTS=0	UNSAT TT=1.203 CHECKS=783353 NODES=366	? (TO) TT=1800.02	? (TO) TT=1800.07	UNSAT TT=19.973 ASSIGNMENTS=765 CHECKS=44331	UNSAT TT=8.745 ASSIGNMENTS=140 CHECKS=383522	UNSAT TT=3.232	UNSAT TT=2.828	UNSAT TT=85.812 ASSIGNMENTS=5339 CHECKS=323229	UNSAT TT=495.219 ASSIGNMENTS=35037 CHECKS=1501340	? (TO) TT=1800.05	UNSAT TT=9.68 ASSIGNMENTS=1326 CHECKS=250870	UNSAT TT=11.284 ASSIGNMENTS=765 CHECKS=44331	UNSAT TT=5.108	UNSAT TT=1.362	UNSAT TT=1.368
csp/queensKnights/ normalized-queensKnights-20-5-mul.xml	UNSAT TT=1.338	UNSAT TT=1.637 CHECKS=4575410 NODES=376	UNSAT TT=1.661 CHECKS=6397900 NODES=0	? (TO) TT=1800.02	UNSAT TT=56.733 CHECKS=45808700	UNSAT TT=49.01 CHECKS=62842700	? (TO) TT=1800.15	? (TO) TT=1800.08	UNSAT TT=3.511 CHECKS=18966100 NODES=1384	? (TO) TT=1800	? (TO) TT=1800.02	UNSAT TT=9.031 ASSIGNMENTS=396 CHECKS=6534760	UNSAT TT=9.07 ASSIGNMENTS=396 CHECKS=6534760	UNSAT TT=24.121	UNSAT TT=10.496	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.09	UNSAT TT=417.029 ASSIGNMENTS=4077 CHECKS=2007810	UNSAT TT=235.402 ASSIGNMENTS=2469 CHECKS=199335	UNSAT TT=18.619	UNSAT TT=1.338	UNSAT TT=1.73
csp/queensKnights/ normalized-queensKnights-50-5-add.xml	UNSAT TT=4.381	UNSAT TT=20.094 CHECKS=406092000 NODES=2446	UNSAT TT=21.751 CHECKS=516515000 NODES=0	? (TO) TT=1800.02	? (TO) TT=1800.1	UNSAT TT=721.046 CHECKS=2919320000	? (TO) TT=1800.12	? (TO) TT=1800.12	UNSAT TT=257.721 CHECKS=5736140000 NODES=10946	? (TO) TT=1800.08	? (TO) TT=1800.08	UNSAT TT=38.47 ASSIGNMENTS=2491 CHECKS=649643000	UNSAT TT=38.218 ASSIGNMENTS=2491 CHECKS=649643000	UNSAT TT=386.723	UNSAT TT=305.295	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.1	UNSAT TT=697.537	UNSAT TT=4.381	UNSAT TT=9.212
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/queensKnights/ normalized-queensKnights-80-5-add.xml	UNSAT TT=11.313	UNSAT TT=436.192 CHECKS=555846000 NODES=6318	UNSAT TT=480.142 CHECKS=575061000 NODES=0	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.18	? (TO) TT=1800.16	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.04	UNSAT TT=661.188 ASSIGNMENTS=6399 CHECKS=788057000	UNSAT TT=662.202 ASSIGNMENTS=6399 CHECKS=788057000	? (MO) TT=1604.53	? (MO) TT=1625.51	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.08	? TT=1505.07	UNSAT TT=11.313	UNSAT TT=25.91
csp/queensKnights/ normalized-queensKnights-80-5-mul.xml	UNSAT TT=13.082	UNSAT TT=452.789 CHECKS=636801000 NODES=6318	UNSAT TT=632.74 CHECKS=665022000 NODES=0	? (TO) TT=1800.03	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.14	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.02	UNSAT TT=698.007 ASSIGNMENTS=6399 CHECKS=821018000	UNSAT TT=668.806 ASSIGNMENTS=6399 CHECKS=821018000	? (TO) TT=1800.03	? (MO) TT=1692.69	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.05	? TT=347.817	UNSAT TT=13.082	UNSAT TT=26.406
csp/queensKnights/ normalized-queensKnights-8-5-add.xml	UNSAT TT=0.273	UNSAT TT=0.849 CHECKS=24172 NODES=52	UNSAT TT=0.82 CHECKS=29251 NODES=0	UNSAT TT=0.682	UNSAT TT=0.547 CHECKS=62972	UNSAT TT=8.467 CHECKS=3006820	UNSAT TT=1.059 ASSIGNMENTS=0	UNSAT TT=1.259 ASSIGNMENTS=0	UNSAT TT=0.678 CHECKS=60111 NODES=151	? (TO) TT=1800.04	? (TO) TT=1800.03	UNSAT TT=8.238 ASSIGNMENTS=62 CHECKS=33864	UNSAT TT=8.575 ASSIGNMENTS=302 CHECKS=6823	UNSAT TT=1.443	UNSAT TT=1.28	UNSAT TT=3.23 ASSIGNMENTS=925 CHECKS=24534	UNSAT TT=22.899 ASSIGNMENTS=9175 CHECKS=294156	UNSAT TT=151.053	UNSAT TT=0.273 ASSIGNMENTS=369 CHECKS=22897	UNSAT TT=0.946 ASSIGNMENTS=302 CHECKS=6823	UNSAT TT=3.126	UNSAT TT=0.689	UNSAT TT=0.739
csp/queensKnights/ normalized-queensKnights-8-5-mul.xml	UNSAT TT=0.757	UNSAT TT=0.845 CHECKS=28925 NODES=52	UNSAT TT=0.834 CHECKS=42313 NODES=0	UNSAT TT=0.776	UNSAT TT=0.711 CHECKS=118455	UNSAT TT=34.149 CHECKS=14475700	UNSAT TT=1.151 ASSIGNMENTS=0	UNSAT TT=1.366 ASSIGNMENTS=0	UNSAT TT=0.801 CHECKS=64541 NODES=159	? (TO) TT=1800.06	? (TO) TT=1800	UNSAT TT=8.666 ASSIGNMENTS=0	UNSAT TT=8.497 ASSIGNMENTS=62 CHECKS=37832	UNSAT TT=1.816	UNSAT TT=1.701	UNSAT TT=2.879 ASSIGNMENTS=821 CHECKS=34332	UNSAT TT=21.81 ASSIGNMENTS=8568 CHECKS=378893	UNSAT TT=199.287	UNSAT TT=1.017 ASSIGNMENTS=437 CHECKS=42121	UNSAT TT=0.934 ASSIGNMENTS=273 CHECKS=13609	UNSAT TT=4.608	UNSAT TT=0.757	UNSAT TT=0.822
csp/queens/ normalized-queens-100.xml	SAT TT=3.19	SAT TT=3.356 CHECKS=837102 NODES=179	SAT TT=3.19 CHECKS=551867 NODES=0	SAT TT=11.176	SAT TT=48.216 CHECKS=78137000	SAT TT=47.671 CHECKS=113701000	SAT TT=295.168 ASSIGNMENTS=4904	SAT TT=316.581 ASSIGNMENTS=4904	SAT TT=18.631 CHECKS=1501120 NODES=601	SAT TT=9.146 CHECKS=3004650 NODES=99	SAT TT=8.634 CHECKS=2965050 NODES=69	SAT TT=23.115 ASSIGNMENTS=92 CHECKS=1143800	SAT TT=10.575 ASSIGNMENTS=188 CHECKS=841630	SAT TT=258.006	SAT TT=45.711	SAT TT=81.429 ASSIGNMENTS=6467 CHECKS=27832200	SAT TT=614.944 ASSIGNMENTS=22425 CHECKS=38872600	? (TO) TT=1800	? (TO) TT=1800.11	SAT TT=14.489 ASSIGNMENTS=92 CHECKS=1143800	SAT TT=202.366	SAT TT=16.345	SAT TT=30.622
csp/queens/ normalized-queens-120.xml	SAT TT=4.492	SAT TT=4.492 CHECKS=1410660 NODES=214	SAT TT=5.379 CHECKS=2384650 NODES=214	SAT TT=8.252	SAT TT=110.367 CHECKS=216012000	SAT TT=22.086 CHECKS=44000700	SAT TT=505.86 ASSIGNMENTS=4210	SAT TT=556.302 ASSIGNMENTS=4210	SAT TT=38.366 CHECKS=2715810 NODES=691	SAT TT=17.191 CHECKS=4419660 NODES=65	SAT TT=17.37 CHECKS=5383560 NODES=99	SAT TT=13.286 ASSIGNMENTS=231 CHECKS=1468090	SAT TT=12.532 ASSIGNMENTS=231 CHECKS=1468090	SAT TT=608.25	SAT TT=74.058	SAT TT=26.46 ASSIGNMENTS=112 CHECKS=1758840	SAT TT=91.796 ASSIGNMENTS=372 CHECKS=5970600	? (TO) TT=1800.05	? (TO) TT=1800.06	SAT TT=96.061 ASSIGNMENTS=634 CHECKS=7477280	? TT=609.795	SAT TT=24.088	SAT TT=52.465
csp/queens/ normalized-queens-12.xml	SAT TT=0.012	SAT TT=0.758 CHECKS=2093 NODES=26	SAT TT=0.775 CHECKS=3733 NODES=26	SAT TT=0.037	SAT TT=0.104 CHECKS=9682	SAT TT=0.018 CHECKS=33946	SAT TT=1.086 ASSIGNMENTS=22	SAT TT=1.165 ASSIGNMENTS=15	SAT TT=0.671 CHECKS=1273 NODES=28	SAT TT=0.676 CHECKS=7326 NODES=30	SAT TT=0.655 CHECKS=7656 NODES=22	SAT TT=0.023 ASSIGNMENTS=7 CHECKS=2131	SAT TT=0.023 ASSIGNMENTS=7 CHECKS=2131	SAT TT=1.455	SAT TT=1.449	SAT TT=0.012 ASSIGNMENTS=7 CHECKS=2131	SAT TT=0.027 ASSIGNMENTS=134 CHECKS=19038	SAT TT=0.439	SAT TT=0.174 ASSIGNMENTS=868 CHECKS=131036	SAT TT=0.013 ASSIGNMENTS=8 CHECKS=2362	SAT TT=1.755	SAT TT=0.684	SAT TT=0.844
csp/queens/ normalized-queens-150.xml	SAT TT=2.322	SAT TT=8.572 CHECKS=5298730 NODES=459	SAT TT=10.79 CHECKS=7248470 NODES=458	SAT TT=2.322	? (TO) TT=1800.12	SAT TT=96.065 CHECKS=187828000	SAT TT=1798.36 ASSIGNMENTS=29374	? (TO) TT=1800.08	SAT TT=101.628 CHECKS=8904790 NODES=1314	SAT TT=41.586 CHECKS=8906480 NODES=93	SAT TT=42.046 CHECKS=9565800 NODES=89	SAT TT=163.466 ASSIGNMENTS=440 CHECKS=8583920	SAT TT=17.033 ASSIGNMENTS=416 CHECKS=5075110	SAT TT=149.563	SAT TT=156.854	SAT TT=71.854 ASSIGNMENTS=142 CHECKS=3434780	SAT TT=162.182 ASSIGNMENTS=302 CHECKS=7704080	? (TO) TT=1800.05	? (TO) TT=1800.12	SAT TT=162.049 ASSIGNMENTS=440 CHECKS=8583920	? TT=650.379	SAT TT=28.431	SAT TT=106.835
csp/queens/ normalized-queens-180.xml	SAT TT=10.492	SAT TT=12.593 CHECKS=9178230 NODES=479	SAT TT=10.492 CHECKS=3764840 NODES=0	? (TO) TT=1800.1	? (TO) TT=1800.1	SAT TT=103.073 CHECKS=192000000	? (TO) TT=1800.18	? (TO) TT=1800.14	SAT TT=213.927 CHECKS=24891500 NODES=1642	SAT TT=92.91 CHECKS=16029400 NODES=131	SAT TT=92.254 CHECKS=16915500 NODES=116	SAT TT=198.963 ASSIGNMENTS=179 CHECKS=6618400	SAT TT=26.43 ASSIGNMENTS=668 CHECKS=12254900	SAT TT=278.703	SAT TT=266.598	? (TO) TT=1800.06	SAT TT=364.084 ASSIGNMENTS=361 CHECKS=13342200	? (TO) TT=1800.03	? (TO) TT=1800.04	SAT TT=180.927 ASSIGNMENTS=179 CHECKS=6618400	? TT=536.207	SAT TT=62.149	SAT TT=171.105
csp/queens/ normalized-queens-200.xml	SAT TT=4.379	SAT TT=15.969 CHECKS=13451000 NODES=528	SAT TT=19.605 CHECKS=18804900 NODES=528	SAT TT=4.379	SAT TT=218.419 CHECKS=367484000	SAT TT=39.141 CHECKS=39135100	? (TO) TT=1800.14	? (TO) TT=1800.16	SAT TT=376.822 CHECKS=48927100 NODES=1826	SAT TT=157.937 CHECKS=21591500 NODES=135	SAT TT=155.093 CHECKS=24278000 NODES=146	SAT TT=313.411 ASSIGNMENTS=223 CHECKS=8980140	SAT TT=29.586 ASSIGNMENTS=534 CHECKS=13549300	SAT TT=390.46	SAT TT=378.24	SAT TT=268.442 ASSIGNMENTS=211 CHECKS=8081850	SAT TT=301.51 ASSIGNMENTS=186 CHECKS=8961030	? (TO) TT=1800.03	? (TO) TT=1800.11	SAT TT=293.91 ASSIGNMENTS=223 CHECKS=8980140	? TT=220.443	SAT TT=132.042	SAT TT=214.313
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/queens/ normalized-queens-30.xml	SAT TT=0.113	SAT TT=1.041 CHECKS=21234 NODES=72	SAT TT=1.072 CHECKS=36094 NODES=72	SAT TT=0.113	SAT TT=12.038 CHECKS=39991700	SAT TT=17.102 CHECKS=58695600	SAT TT=7.44 ASSIGNMENTS=328	SAT TT=740.767 ASSIGNMENTS=278790	SAT TT=1.281 CHECKS=12399 NODES=60	SAT TT=1.231 CHECKS=75255 NODES=25	SAT TT=1.222 CHECKS=75255 NODES=18	SAT TT=0.208 ASSIGNMENTS=132 CHECKS=130103	SAT TT=0.21 ASSIGNMENTS=132 CHECKS=130103	SAT TT=4.372	SAT TT=3.776	SAT TT=0.147 ASSIGNMENTS=54 CHECKS=63529	SAT TT=2.283 ASSIGNMENTS=1633 CHECKS=1002980	? (TO) TT=1800.05	SAT TT=1.236 ASSIGNMENTS=347 CHECKS=416733	SAT TT=0.129 ASSIGNMENTS=31 CHECKS=37038	SAT TT=4.172	SAT TT=1.468	SAT TT=1.935
csp/queens/ normalized-queens-4.xml	SAT TT=0.005	SAT TT=0.744 CHECKS=90 NODES=5	SAT TT=0.638 CHECKS=95 NODES=0	SAT TT=0.024	SAT TT=0.009 CHECKS=299	SAT TT=0.008 CHECKS=390	SAT TT=0.518 ASSIGNMENTS=2	SAT TT=0.589 ASSIGNMENTS=4	SAT TT=0.391 CHECKS=42 NODES=5	SAT TT=0.397 CHECKS=150 NODES=3	SAT TT=0.392 CHECKS=150 NODES=2	SAT TT=0.019 ASSIGNMENTS=2 CHECKS=33	SAT TT=0.018 ASSIGNMENTS=2 CHECKS=33	SAT TT=0.865	SAT TT=0.865	SAT TT=0.007 ASSIGNMENTS=2 CHECKS=33	SAT TT=0.006 ASSIGNMENTS=2 CHECKS=33	SAT TT=0.261	SAT TT=0.008 ASSIGNMENTS=2 CHECKS=33	SAT TT=0.005 ASSIGNMENTS=1 CHECKS=19	SAT TT=0.844	SAT TT=0.451	SAT TT=0.444
csp/queens/ normalized-queens-80.xml	SAT TT=0.691	SAT TT=2.353 CHECKS=618851 NODES=246	SAT TT=2.543 CHECKS=895075 NODES=246	SAT TT=0.691	SAT TT=35.251 CHECKS=75887800	SAT TT=31.869 CHECKS=85717600	SAT TT=161.24 ASSIGNMENTS=4968	SAT TT=162.086 ASSIGNMENTS=4968	SAT TT=9.814 CHECKS=854665 NODES=593	SAT TT=5.014 CHECKS=1494680 NODES=73	SAT TT=4.922 CHECKS=1646360 NODES=71	SAT TT=24.143 ASSIGNMENTS=392 CHECKS=2348980	SAT TT=10.249 ASSIGNMENTS=263 CHECKS=640082	SAT TT=109.284	SAT TT=25.122	SAT TT=101.104 ASSIGNMENTS=12252 CHECKS=43934300	SAT TT=335.144 ASSIGNMENTS=25931 CHECKS=27388000	? (TO) TT=1800.11	SAT TT=577.353 ASSIGNMENTS=23058 CHECKS=49633000	SAT TT=16.772 ASSIGNMENTS=392 CHECKS=2348980	SAT TT=97.318	SAT TT=7.301	SAT TT=15.316
csp/queens/ normalized-queens-8.xml	SAT TT=0.009	SAT TT=0.694 CHECKS=1044 NODES=18	SAT TT=0.729 CHECKS=558 NODES=0	SAT TT=0.028	SAT TT=0.047 CHECKS=4337	SAT TT=0.012 CHECKS=9304	SAT TT=0.882 ASSIGNMENTS=24	SAT TT=0.802 ASSIGNMENTS=9	SAT TT=0.529 CHECKS=324 NODES=12	SAT TT=0.527 CHECKS=2156 NODES=21	SAT TT=0.507 CHECKS=4956 NODES=48	SAT TT=0.021 ASSIGNMENTS=8 CHECKS=1268	SAT TT=0.021 ASSIGNMENTS=8 CHECKS=1268	SAT TT=1.196	SAT TT=1.173	SAT TT=0.011 ASSIGNMENTS=19 CHECKS=2489	SAT TT=0.009 ASSIGNMENTS=15 CHECKS=1286	SAT TT=0.321	SAT TT=0.023 ASSIGNMENTS=204 CHECKS=14571	SAT TT=0.009 ASSIGNMENTS=11 CHECKS=1309	SAT TT=1.431	SAT TT=0.556	SAT TT=0.548

Some statistics...

	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
Number of times the solver is able to give the best known answer	82	83	82	76	76	72	40	35	84	85	79	80	75	81	61	76	74
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)	82	83	82	76	76	72	40	35	84	85	79	80	75	81	61	76	74
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	4	3	17	0	3	1	0	0	1	1	25	17	12	15	0	8	0
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0