CSP 2008 Competition: results by benchmark

Results by benchmark for category n-ary constraints in extension (N-ARY-EXT), subcategory problems with patterns

This page displays the results of the different solvers for each benchmark for category n-ary constraints in extension (N-ARY-EXT), subcategory problems with patterns

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...	Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 112v4 AC
Abscon 112v4 ESAC
bpsolver 2008-06-27
casper zao
casper zito
choco2_dwdeg 2008-06-26
choco2_impwdeg 2008-06-26
Concrete + CPS4J 2008-05-30
Concrete + CSP4J - Tabu Engine 2008-05-30
Concrete + CSP4J - WMC Engine 2008-05-30
cpHydra k_10
cpHydra k_40
galac E
galac M
mddc-solv 2008-06-09
MDG-noprobe 2008-06-27
MDG-probe 2008-06-27
Minion/Tailor 2008-07-04
Mistral-option 1.314
Mistral-prime 1.313
SAT4J CSP 2008-06-13
spider 2008-06-14
Sugar v1.13+minisat
Sugar v1.13+picosat

Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/travellingSalesman-20/ normalized-tsp-20-142_ext.xml	SAT TT=10.242	SAT TT=10.242 CHECKS=9218830 NODES=15329	SAT TT=20.578 CHECKS=21333800 NODES=34447	SAT TT=53.443	? (TO) TT=1800.02	SAT TT=675.219 CHECKS=5830190	SAT TT=75.984 ASSIGNMENTS=6158	SAT TT=21.665 ASSIGNMENTS=99	SAT TT=94.388 CHECKS=416661000 NODES=9376	? (TO) TT=1800.04	? (TO) TT=1800.02	SAT TT=25.065 ASSIGNMENTS=13319 CHECKS=7040200	SAT TT=16.777 ASSIGNMENTS=15329 CHECKS=9218830	? (TO) TT=1800.02	? (TO) TT=1800.05	SAT TT=32.703 CHECKS=6168390	SAT TT=23.967 ASSIGNMENTS=19287 CHECKS=10293700	SAT TT=8.719 ASSIGNMENTS=7688 CHECKS=3634910	? (problem) TT=3.36	SAT TT=77.149 ASSIGNMENTS=53190 CHECKS=28084900	SAT TT=17.69 ASSIGNMENTS=13319 CHECKS=7040200	? TT=324.053	SAT TT=975.861 ASSIGNMENTS=9951 CHECKS=897751000	SAT TT=185.403	SAT TT=489.51
csp/travellingSalesman-20/ normalized-tsp-20-190_ext.xml	SAT TT=1.151	SAT TT=3.439 CHECKS=2188490 NODES=5756	SAT TT=4.076 CHECKS=2730900 NODES=6575	SAT TT=9.297	SAT TT=60.96 CHECKS=1787300	SAT TT=73.666 CHECKS=1937770	SAT TT=11.547 ASSIGNMENTS=1959	SAT TT=9.963 ASSIGNMENTS=1679	SAT TT=59.944 CHECKS=9677140 NODES=305	? (TO) TT=1800.09	? (TO) TT=1800.04	SAT TT=1.236 ASSIGNMENTS=2099 CHECKS=739623	SAT TT=1.151 ASSIGNMENTS=2099 CHECKS=739623	SAT TT=664.347	SAT TT=887.381	SAT TT=3.369 CHECKS=801590	SAT TT=1.424 ASSIGNMENTS=2362 CHECKS=865825	SAT TT=4.023 ASSIGNMENTS=7277 CHECKS=2487280	? (problem) TT=2.491	SAT TT=8.117 ASSIGNMENTS=14641 CHECKS=5384060	SAT TT=1.216 ASSIGNMENTS=2099 CHECKS=739623	? TT=709.602	SAT TT=363.049 ASSIGNMENTS=10233 CHECKS=444469000	SAT TT=73.262	SAT TT=91.892
csp/travellingSalesman-20/ normalized-tsp-20-193_ext.xml	SAT TT=2.552	SAT TT=54.53 CHECKS=62746000 NODES=67406	SAT TT=2.552 CHECKS=1511900 NODES=1281	SAT TT=540.98	SAT TT=1712.83 CHECKS=27330400	? (TO) TT=1800.05	SAT TT=33.015 ASSIGNMENTS=71	SAT TT=46.048 ASSIGNMENTS=978	SAT TT=126.635 CHECKS=855654000 NODES=10435	? (TO) TT=1800.05	? (TO) TT=1800.07	SAT TT=60.142 ASSIGNMENTS=67406 CHECKS=62746000	SAT TT=60.494 ASSIGNMENTS=67406 CHECKS=62746000	? (TO) TT=1800.06	? (TO) TT=1800.01	SAT TT=7.362 CHECKS=1140660	SAT TT=4.821 ASSIGNMENTS=2295 CHECKS=1368850	SAT TT=14.805 ASSIGNMENTS=8925 CHECKS=4463300	? (exit code) TT=4.655	SAT TT=140.605 ASSIGNMENTS=68223 CHECKS=33556500	SAT TT=46.212 ASSIGNMENTS=22866 CHECKS=12811200	? TT=389.161	? (TO) TT=1800.1	SAT TT=194.551	SAT TT=205.286
csp/travellingSalesman-20/ normalized-tsp-20-1_ext.xml	SAT TT=0.181	SAT TT=1.236 CHECKS=150571 NODES=313	SAT TT=1.698 CHECKS=728854 NODES=300	SAT TT=1.819	SAT TT=51.398 CHECKS=1149590	SAT TT=9.095 CHECKS=54257	SAT TT=4.438 ASSIGNMENTS=24	SAT TT=4.799 ASSIGNMENTS=77	SAT TT=61.6 CHECKS=9497300 NODES=209	? (TO) TT=1800.07	? (TO) TT=1800.04	SAT TT=0.225 ASSIGNMENTS=193 CHECKS=101838	SAT TT=0.226 ASSIGNMENTS=193 CHECKS=101838	SAT TT=497.808	SAT TT=391.411	SAT TT=0.498 CHECKS=65472	SAT TT=0.181 ASSIGNMENTS=163 CHECKS=81350	SAT TT=1.14 ASSIGNMENTS=1617 CHECKS=584545	? (problem) TT=2.523	SAT TT=1.65 ASSIGNMENTS=1666 CHECKS=798510	SAT TT=0.213 ASSIGNMENTS=193 CHECKS=101838	? TT=429.417	SAT TT=335.035 ASSIGNMENTS=310 CHECKS=467160000	SAT TT=51.467	SAT TT=75.115
csp/travellingSalesman-20/ normalized-tsp-20-29_ext.xml	SAT TT=0.061	SAT TT=1.07 CHECKS=55615 NODES=114	SAT TT=1.571 CHECKS=415924 NODES=82	SAT TT=1.018	SAT TT=21.622 CHECKS=697377	SAT TT=4.491 CHECKS=37365	SAT TT=3.057 ASSIGNMENTS=8	SAT TT=3.24 ASSIGNMENTS=14	SAT TT=61.109 CHECKS=7274290 NODES=106	? (TO) TT=1800.01	? (TO) TT=1800.09	SAT TT=0.078 ASSIGNMENTS=35 CHECKS=22156	SAT TT=0.077 ASSIGNMENTS=35 CHECKS=22156	SAT TT=308.944	SAT TT=400.892	SAT TT=0.26 CHECKS=28746	SAT TT=0.061 ASSIGNMENTS=36 CHECKS=22435	SAT TT=0.403 ASSIGNMENTS=716 CHECKS=274860	? (problem) TT=1.851	SAT TT=1.482 ASSIGNMENTS=2706 CHECKS=1013960	SAT TT=0.066 ASSIGNMENTS=35 CHECKS=22156	? TT=926.379	SAT TT=136.002 ASSIGNMENTS=227 CHECKS=277677000	SAT TT=30.113	SAT TT=51.681
csp/travellingSalesman-20/ normalized-tsp-20-2_ext.xml	SAT TT=0.498	SAT TT=2.875 CHECKS=1411860 NODES=2851	SAT TT=4.368 CHECKS=3093010 NODES=6051	SAT TT=6.885	SAT TT=244.535 CHECKS=6732120	SAT TT=53.079 CHECKS=603545	SAT TT=10.327 ASSIGNMENTS=41	SAT TT=32.294 ASSIGNMENTS=5829	SAT TT=68.168 CHECKS=81058300 NODES=3196	? (TO) TT=1800.05	? (TO) TT=1800.08	SAT TT=0.512 ASSIGNMENTS=412 CHECKS=202419	SAT TT=0.508 ASSIGNMENTS=412 CHECKS=202419	SAT TT=1579.05	SAT TT=971.699	SAT TT=8.669 CHECKS=1915480	SAT TT=2.898 ASSIGNMENTS=2591 CHECKS=1192980	SAT TT=3.556 ASSIGNMENTS=4241 CHECKS=1467260	? (problem) TT=3.457	SAT TT=19.667 ASSIGNMENTS=18942 CHECKS=8008830	SAT TT=0.498 ASSIGNMENTS=412 CHECKS=202419	? TT=318.289	SAT TT=747.122 ASSIGNMENTS=3799 CHECKS=600519000	SAT TT=79.588	SAT TT=186.015
csp/travellingSalesman-20/ normalized-tsp-20-30_ext.xml	SAT TT=0.084	SAT TT=1.02 CHECKS=87176 NODES=128	SAT TT=1.382 CHECKS=579593 NODES=106	SAT TT=1.557	SAT TT=52.268 CHECKS=899367	SAT TT=8.865 CHECKS=40889	SAT TT=7.596 ASSIGNMENTS=155	SAT TT=7.515 ASSIGNMENTS=65	SAT TT=61.734 CHECKS=7791010 NODES=90	? (TO) TT=1800.06	? (TO) TT=1800.09	SAT TT=0.14 ASSIGNMENTS=90 CHECKS=47102	SAT TT=0.136 ASSIGNMENTS=90 CHECKS=47102	SAT TT=527.82	SAT TT=606.612	SAT TT=0.31 CHECKS=26705	SAT TT=0.084 ASSIGNMENTS=48 CHECKS=27579	SAT TT=2.179 ASSIGNMENTS=3453 CHECKS=1222830	? (problem) TT=2.248	SAT TT=10.869 ASSIGNMENTS=13649 CHECKS=5999530	SAT TT=0.124 ASSIGNMENTS=90 CHECKS=47102	? (exit code) TT=689.05	SAT TT=165.595 ASSIGNMENTS=237 CHECKS=342440000	SAT TT=43.26	SAT TT=67.13
csp/travellingSalesman-20/ normalized-tsp-20-366_ext.xml	SAT TT=0.847	SAT TT=14.939 CHECKS=12285300 NODES=26549	SAT TT=11.497 CHECKS=8946030 NODES=19124	SAT TT=49.665	SAT TT=303.318 CHECKS=8828180	SAT TT=104.221 CHECKS=1367690	SAT TT=14.989 ASSIGNMENTS=75	SAT TT=21.281 ASSIGNMENTS=1455	SAT TT=65.125 CHECKS=47412300 NODES=1804	? (TO) TT=1800.01	? (TO) TT=1800.05	SAT TT=0.847 ASSIGNMENTS=777 CHECKS=353892	SAT TT=0.849 ASSIGNMENTS=777 CHECKS=353892	SAT TT=1618.04	? (TO) TT=1800.05	SAT TT=9.699 CHECKS=2002110	SAT TT=1.03 ASSIGNMENTS=928 CHECKS=433979	SAT TT=0.464 ASSIGNMENTS=553 CHECKS=179499	? (problem) TT=3.335	SAT TT=9.532 ASSIGNMENTS=6967 CHECKS=3323480	SAT TT=0.849 ASSIGNMENTS=777 CHECKS=353892	? TT=384.203	SAT TT=508.933 ASSIGNMENTS=1495 CHECKS=25274100	SAT TT=87.488	SAT TT=125.738
csp/travellingSalesman-20/ normalized-tsp-20-453_ext.xml	SAT TT=4.835	SAT TT=23.8 CHECKS=22762500 NODES=43000	SAT TT=12.865 CHECKS=11750700 NODES=23906	SAT TT=4.835	SAT TT=1769.17 CHECKS=57999700	SAT TT=222.626 CHECKS=2756500	SAT TT=24.083 ASSIGNMENTS=925	SAT TT=20.584 ASSIGNMENTS=77	SAT TT=125.594 CHECKS=763309000 NODES=28370	? (TO) TT=1800.06	? (TO) TT=1800.06	SAT TT=42.632 ASSIGNMENTS=30270 CHECKS=14294500	SAT TT=31.129 ASSIGNMENTS=43000 CHECKS=22762500	? (TO) TT=1800.02	? (TO) TT=1800.05	SAT TT=101.894 CHECKS=20811900	SAT TT=12.007 ASSIGNMENTS=11596 CHECKS=5647870	SAT TT=28.356 ASSIGNMENTS=26938 CHECKS=12274800	? (problem) TT=3.347	SAT TT=52.416 ASSIGNMENTS=37820 CHECKS=18129100	SAT TT=35.201 ASSIGNMENTS=30270 CHECKS=14294500	? TT=539.362	SAT TT=1562.8 ASSIGNMENTS=24862 CHECKS=822640000	SAT TT=107.608	SAT TT=403.189
csp/travellingSalesman-20/ normalized-tsp-20-68_ext.xml	SAT TT=0.037	SAT TT=0.915 CHECKS=27423 NODES=63	SAT TT=1.022 CHECKS=25468 NODES=0	SAT TT=0.465	SAT TT=0.953 CHECKS=43123	SAT TT=0.644 CHECKS=9589	SAT TT=0.726 ASSIGNMENTS=2	SAT TT=0.831 ASSIGNMENTS=2	SAT TT=60.859 CHECKS=3907940 NODES=67	SAT TT=35.719 CHECKS=29836800 NODES=76501000	? (TO) TT=1800.07	SAT TT=0.045 ASSIGNMENTS=8 CHECKS=5214	SAT TT=0.049 ASSIGNMENTS=8 CHECKS=5214	SAT TT=222.764	SAT TT=221.204	SAT TT=0.159 CHECKS=7000	SAT TT=0.037 ASSIGNMENTS=8 CHECKS=5214	SAT TT=0.036 ASSIGNMENTS=14 CHECKS=5487	? (problem) TT=1.533	SAT TT=0.041 ASSIGNMENTS=18 CHECKS=10103	SAT TT=0.038 ASSIGNMENTS=8 CHECKS=5214	SAT TT=848.183	SAT TT=31.856 ASSIGNMENTS=76 CHECKS=120644000	SAT TT=16.538	SAT TT=20.333
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/travellingSalesman-20/ normalized-tsp-20-727_ext.xml	SAT TT=0.534	SAT TT=1.569 CHECKS=455443 NODES=842	SAT TT=1.674 CHECKS=502572 NODES=844	SAT TT=5.058	SAT TT=49.545 CHECKS=543202	SAT TT=70.492 CHECKS=564513	SAT TT=14.12 ASSIGNMENTS=260	SAT TT=31.844 ASSIGNMENTS=4049	SAT TT=69.235 CHECKS=93199500 NODES=2674	? (TO) TT=1800.05	? (TO) TT=1800.06	SAT TT=1.116 ASSIGNMENTS=831 CHECKS=387852	SAT TT=1.128 ASSIGNMENTS=831 CHECKS=387852	? (TO) TT=1800.03	SAT TT=600.775	SAT TT=2.555 CHECKS=438262	SAT TT=0.534 ASSIGNMENTS=368 CHECKS=189577	SAT TT=4.221 ASSIGNMENTS=4028 CHECKS=1499020	? (exit code) TT=5.079	SAT TT=28.301 ASSIGNMENTS=21220 CHECKS=9339350	SAT TT=1.106 ASSIGNMENTS=831 CHECKS=387852	? TT=358.587	SAT TT=1034.99 ASSIGNMENTS=7769 CHECKS=854726000	SAT TT=152.243	SAT TT=299.586
csp/travellingSalesman-20/ normalized-tsp-20-75_ext.xml	SAT TT=2.21	SAT TT=3.759 CHECKS=2698900 NODES=3331	SAT TT=4.407 CHECKS=3738990 NODES=4646	SAT TT=73.535	SAT TT=601.043 CHECKS=14335900	SAT TT=719.874 CHECKS=6059460	SAT TT=21.146 ASSIGNMENTS=691	SAT TT=87.787 ASSIGNMENTS=12780	SAT TT=106.194 CHECKS=595741000 NODES=9695	? (TO) TT=1800.08	? (TO) TT=1800.03	SAT TT=9.507 ASSIGNMENTS=1817 CHECKS=952951	SAT TT=10.817 ASSIGNMENTS=3331 CHECKS=2698900	SAT TT=1421.2	? (TO) TT=1800	SAT TT=2.306 CHECKS=386199	SAT TT=3.225 ASSIGNMENTS=2465 CHECKS=1300540	SAT TT=6.744 ASSIGNMENTS=6131 CHECKS=2685810	? (problem) TT=3.385	SAT TT=57.926 ASSIGNMENTS=42050 CHECKS=20724000	SAT TT=2.21 ASSIGNMENTS=1817 CHECKS=952951	? TT=375.49	SAT TT=913.522 ASSIGNMENTS=2278 CHECKS=742750000	SAT TT=226.043	SAT TT=258.672
csp/travellingSalesman-20/ normalized-tsp-20-76_ext.xml	SAT TT=1.483	SAT TT=2.685 CHECKS=1311540 NODES=3635	SAT TT=2.225 CHECKS=988345 NODES=1822	SAT TT=3.659	SAT TT=96.672 CHECKS=2429350	SAT TT=76.205 CHECKS=1074660	SAT TT=16.031 ASSIGNMENTS=1399	SAT TT=18.445 ASSIGNMENTS=1622	SAT TT=69.064 CHECKS=80899200 NODES=4671	? (TO) TT=1800.02	? (TO) TT=1800.07	SAT TT=10.923 ASSIGNMENTS=3539 CHECKS=1619760	SAT TT=10.034 ASSIGNMENTS=3635 CHECKS=1311540	? (TO) TT=1800.02	? (TO) TT=1800.04	SAT TT=17.342 CHECKS=3696110	SAT TT=1.483 ASSIGNMENTS=1551 CHECKS=646622	SAT TT=4.069 ASSIGNMENTS=4828 CHECKS=1736260	? (problem) TT=3.435	SAT TT=18.76 ASSIGNMENTS=17805 CHECKS=8010070	SAT TT=3.617 ASSIGNMENTS=3539 CHECKS=1619760	? (TO) TT=1802.16	SAT TT=839.087 ASSIGNMENTS=5073 CHECKS=513133000	SAT TT=97.944	SAT TT=453.966
csp/travellingSalesman-20/ normalized-tsp-20-8_ext.xml	SAT TT=0.062	SAT TT=1.267 CHECKS=137754 NODES=286	SAT TT=1.117 CHECKS=273165 NODES=0	SAT TT=0.792	SAT TT=40.93 CHECKS=1095950	SAT TT=8.458 CHECKS=126197	SAT TT=4.603 ASSIGNMENTS=16	SAT TT=4.909 ASSIGNMENTS=40	SAT TT=61.489 CHECKS=6093430 NODES=78	? (TO) TT=1800.04	? (TO) TT=1800.02	SAT TT=0.08 ASSIGNMENTS=42 CHECKS=25297	SAT TT=0.076 ASSIGNMENTS=42 CHECKS=25297	SAT TT=319.669	SAT TT=350.046	SAT TT=0.315 CHECKS=34882	SAT TT=0.062 ASSIGNMENTS=40 CHECKS=24174	SAT TT=0.604 ASSIGNMENTS=1201 CHECKS=414812	? (problem) TT=1.953	SAT TT=2.04 ASSIGNMENTS=2881 CHECKS=1225080	SAT TT=0.067 ASSIGNMENTS=42 CHECKS=25297	? (TO) TT=1800.18	SAT TT=129.223 ASSIGNMENTS=183 CHECKS=292563000	SAT TT=30.283	SAT TT=60.079
csp/travellingSalesman-20/ normalized-tsp-20-901_ext.xml	SAT TT=4.18	SAT TT=7.178 CHECKS=5921700 NODES=9633	SAT TT=7.123 CHECKS=5920580 NODES=8752	SAT TT=16.594	SAT TT=567.789 CHECKS=15155300	SAT TT=50.539 CHECKS=432921	SAT TT=21.538 ASSIGNMENTS=691	SAT TT=44.929 ASSIGNMENTS=2222	SAT TT=81.998 CHECKS=257332000 NODES=7478	? (TO) TT=1800.02	? (TO) TT=1800.01	SAT TT=12.038 ASSIGNMENTS=3548 CHECKS=1892880	SAT TT=14.475 ASSIGNMENTS=9633 CHECKS=5921700	? (TO) TT=1800.01	? (TO) TT=1800.02	SAT TT=16.911 CHECKS=2928260	SAT TT=5.582 ASSIGNMENTS=4510 CHECKS=2307920	SAT TT=9.234 ASSIGNMENTS=8237 CHECKS=3818210	? (problem) TT=3.85	SAT TT=40.4 ASSIGNMENTS=27731 CHECKS=13489700	SAT TT=4.18 ASSIGNMENTS=3548 CHECKS=1892880	? TT=386.497	SAT TT=1755.52 ASSIGNMENTS=23498 CHECKS=1111920000	SAT TT=162.054	SAT TT=170.213
csp/travellingSalesman-25/ normalized-tsp-25-13_ext.xml	SAT TT=0.601	SAT TT=2.191 CHECKS=1079220 NODES=1423	SAT TT=2.539 CHECKS=1798740 NODES=1559	SAT TT=5.87	SAT TT=53.293 CHECKS=2379720	SAT TT=22.963 CHECKS=556130	SAT TT=9.261 ASSIGNMENTS=457	SAT TT=7.441 ASSIGNMENTS=88	SAT TT=63.501 CHECKS=30946800 NODES=996	? (TO) TT=1800.01	? (TO) TT=1800.03	SAT TT=0.64 ASSIGNMENTS=586 CHECKS=507534	SAT TT=0.649 ASSIGNMENTS=586 CHECKS=507534	SAT TT=525.698	SAT TT=982.817	SAT TT=3.929 CHECKS=1044070	SAT TT=0.601 ASSIGNMENTS=537 CHECKS=498017	SAT TT=1.718 ASSIGNMENTS=2726 CHECKS=1477440	? (problem) TT=2.535	SAT TT=12.748 ASSIGNMENTS=17546 CHECKS=10955700	SAT TT=0.637 ASSIGNMENTS=586 CHECKS=507534	? TT=277.68	SAT TT=515.728 ASSIGNMENTS=2167 CHECKS=427624000	SAT TT=101.243	SAT TT=204.16
csp/travellingSalesman-25/ normalized-tsp-25-163_ext.xml	SAT TT=30.541	SAT TT=233.006 CHECKS=253863000 NODES=278834	SAT TT=225.334 CHECKS=245694000 NODES=276748	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.02	SAT TT=199.114 ASSIGNMENTS=15783	SAT TT=1219.7 ASSIGNMENTS=127815	SAT TT=69.97 CHECKS=101620000 NODES=1822	? (TO) TT=1800.05	? (TO) TT=1800.05	SAT TT=232.574 ASSIGNMENTS=278834 CHECKS=253863000	SAT TT=227.297 ASSIGNMENTS=278834 CHECKS=253863000	? (TO) TT=1800.02	? (TO) TT=1800.02	SAT TT=30.541 CHECKS=6163350	SAT TT=102.723 ASSIGNMENTS=54066 CHECKS=37747700	SAT TT=117.664 ASSIGNMENTS=69322 CHECKS=47735500	? (exit code) TT=6.382	SAT TT=238.5 ASSIGNMENTS=130859 CHECKS=93535300	SAT TT=71.373 ASSIGNMENTS=39660 CHECKS=28357600	? (TO) TT=1802.19	? (TO) TT=1800.06	SAT TT=974.008	SAT TT=674.203
csp/travellingSalesman-25/ normalized-tsp-25-190_ext.xml	SAT TT=8.242	SAT TT=84.627 CHECKS=92426700 NODES=133713	SAT TT=78.199 CHECKS=91213500 NODES=130982	SAT TT=305.656	? (TO) TT=1800.09	? (TO) TT=1800.03	SAT TT=220.4 ASSIGNMENTS=34096	SAT TT=31.949 ASSIGNMENTS=1055	SAT TT=97.439 CHECKS=379382000 NODES=14056	? (TO) TT=1800.07	? (TO) TT=1800.08	SAT TT=15.967 ASSIGNMENTS=7136 CHECKS=4386280	SAT TT=85.524 ASSIGNMENTS=133713 CHECKS=92426700	? (TO) TT=1800.04	? (TO) TT=1800.1	SAT TT=251.611 CHECKS=51777300	SAT TT=48.065 ASSIGNMENTS=40657 CHECKS=25303500	SAT TT=53.75 ASSIGNMENTS=51168 CHECKS=32026800	? (problem) TT=3.697	SAT TT=124.375 ASSIGNMENTS=103237 CHECKS=65835200	SAT TT=8.242 ASSIGNMENTS=7136 CHECKS=4386280	? TT=300.691	? (TO) TT=1800.07	SAT TT=1152.68	SAT TT=1001.65
csp/travellingSalesman-25/ normalized-tsp-25-312_ext.xml	SAT TT=28.665	SAT TT=34.638 CHECKS=33646900 NODES=46902	SAT TT=28.665 CHECKS=27625300 NODES=41092	SAT TT=113.687	? (TO) TT=1800.01	? (TO) TT=1800.03	SAT TT=343.335 ASSIGNMENTS=48477	SAT TT=65.812 ASSIGNMENTS=3518	SAT TT=110.582 CHECKS=512206000 NODES=17119	? (TO) TT=1800.06	? (TO) TT=1800.01	SAT TT=40.883 ASSIGNMENTS=28169 CHECKS=18077500	SAT TT=38.304 ASSIGNMENTS=46902 CHECKS=33646900	? (TO) TT=1800.03	? (TO) TT=1800.07	SAT TT=84.161 CHECKS=18173900	SAT TT=77.782 ASSIGNMENTS=59683 CHECKS=39108800	SAT TT=104.706 ASSIGNMENTS=82081 CHECKS=53555400	? (problem) TT=4.45	SAT TT=108.21 ASSIGNMENTS=81072 CHECKS=54531000	SAT TT=36.418 ASSIGNMENTS=28169 CHECKS=18077500	? TT=310.35	? (TO) TT=1800.1	SAT TT=919.528	SAT TT=553.232
csp/travellingSalesman-25/ normalized-tsp-25-38_ext.xml	SAT TT=1.25	SAT TT=4.636 CHECKS=4367910 NODES=4658	SAT TT=2.077 CHECKS=1372170 NODES=759	SAT TT=8.603	SAT TT=48.842 CHECKS=1211410	SAT TT=87.543 CHECKS=1725220	SAT TT=15.036 ASSIGNMENTS=585	SAT TT=18.07 ASSIGNMENTS=1200	SAT TT=66.748 CHECKS=60173900 NODES=2439	? (TO) TT=1800.1	? (TO) TT=1800.05	SAT TT=1.34 ASSIGNMENTS=1363 CHECKS=1044830	SAT TT=1.355 ASSIGNMENTS=1363 CHECKS=1044830	SAT TT=622.083	SAT TT=956.936	SAT TT=12.118 CHECKS=2881900	SAT TT=1.487 ASSIGNMENTS=1471 CHECKS=1137290	SAT TT=2.857 ASSIGNMENTS=4006 CHECKS=2111720	? (problem) TT=2.59	SAT TT=20.493 ASSIGNMENTS=25748 CHECKS=16179300	SAT TT=1.25 ASSIGNMENTS=1363 CHECKS=1044830	? TT=379.382	SAT TT=630 ASSIGNMENTS=3563 CHECKS=523083000	SAT TT=108.604	SAT TT=111.123
Bench name	Best results on this instance	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Concrete + CSP4J - WMC Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	MDG-probe 2008-06-27	Minion/Tailor 2008-07-04	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	spider 2008-06-14	Sugar v1.13+minisat	Sugar v1.13+picosat
csp/travellingSalesman-25/ normalized-tsp-25-3_ext.xml	SAT TT=0.957	SAT TT=2.566 CHECKS=1555160 NODES=2427	SAT TT=2.754 CHECKS=1630700 NODES=2176	SAT TT=19.438	SAT TT=169.158 CHECKS=6890250	SAT TT=29.659 CHECKS=545588	SAT TT=11.818 ASSIGNMENTS=315	SAT TT=11.903 ASSIGNMENTS=306	SAT TT=62.491 CHECKS=17013400 NODES=369	? (TO) TT=1800.04	? (TO) TT=1800.06	SAT TT=1.306 ASSIGNMENTS=1138 CHECKS=958470	SAT TT=1.318 ASSIGNMENTS=1138 CHECKS=958470	SAT TT=1083.22	SAT TT=995.618	SAT TT=7.618 CHECKS=1912290	SAT TT=0.957 ASSIGNMENTS=812 CHECKS=640423	SAT TT=3.629 ASSIGNMENTS=4239 CHECKS=2406170	? (problem) TT=2.91	SAT TT=22.268 ASSIGNMENTS=23031 CHECKS=15955700	SAT TT=1.392 ASSIGNMENTS=1138 CHECKS=958470	? TT=290.914	SAT TT=603.501 ASSIGNMENTS=3637 CHECKS=412463000	SAT TT=103.758	SAT TT=65.753
csp/travellingSalesman-25/ normalized-tsp-25-456_ext.xml	SAT TT=46.001	SAT TT=479.281 CHECKS=416315000 NODES=786780	SAT TT=119.892 CHECKS=103716000 NODES=201812	SAT TT=48.808	? (TO) TT=1800.09	? (TO) TT=1800.11	SAT TT=227.317 ASSIGNMENTS=23374	SAT TT=501.047 ASSIGNMENTS=56787	SAT TT=119.883 CHECKS=633761000 NODES=18153	? (TO) TT=1800.05	? (TO) TT=1800.07	SAT TT=53.805 ASSIGNMENTS=27953 CHECKS=19009000	SAT TT=288.549 ASSIGNMENTS=27953 CHECKS=19009000	? (TO) TT=1800.08	SAT TT=1316.55	SAT TT=189.929 CHECKS=34704300	SAT TT=57.267 ASSIGNMENTS=36279 CHECKS=23174800	SAT TT=98.152 ASSIGNMENTS=67970 CHECKS=41768600	? (problem) TT=4.198	SAT TT=88.537 ASSIGNMENTS=54605 CHECKS=36191600	SAT TT=46.001 ASSIGNMENTS=27953 CHECKS=19009000	? TT=498.581	? (TO) TT=1800.07	SAT TT=677.635	SAT TT=491.277
csp/travellingSalesman-25/ normalized-tsp-25-48_ext.xml	SAT TT=6.998	SAT TT=150.584 CHECKS=166718000 NODES=200370	SAT TT=297.661 CHECKS=324799000 NODES=412625	? (TO) TT=1800.1	SAT TT=974.08 CHECKS=28519400	? (TO) TT=1800.08	SAT TT=745.284 ASSIGNMENTS=113458	SAT TT=685.334 ASSIGNMENTS=79258	SAT TT=231.719 CHECKS=1649490000 NODES=66348	? (TO) TT=1800.06	? (TO) TT=1800.04	SAT TT=154.565 ASSIGNMENTS=200370 CHECKS=166718000	SAT TT=145.812 ASSIGNMENTS=200370 CHECKS=166718000	? (TO) TT=1800.07	? (TO) TT=1800.07	SAT TT=288.794 CHECKS=55720800	SAT TT=15.667 ASSIGNMENTS=13915 CHECKS=8292820	SAT TT=226.105 ASSIGNMENTS=166815 CHECKS=97254200	? (problem) TT=4.309	SAT TT=158.234 ASSIGNMENTS=119858 CHECKS=75254300	SAT TT=6.998 ASSIGNMENTS=5528 CHECKS=3262660	? TT=297.374	? (TO) TT=1800.1	SAT TT=541.317	SAT TT=780.969
csp/travellingSalesman-25/ normalized-tsp-25-523_ext.xml	SAT TT=3.199	SAT TT=42.131 CHECKS=41220300 NODES=70107	SAT TT=112.781 CHECKS=111180000 NODES=186041	SAT TT=57.901	? (TO) TT=1800.04	? (TO) TT=1800.06	SAT TT=202.175 ASSIGNMENTS=34627	SAT TT=135.822 ASSIGNMENTS=20459	SAT TT=712.675 CHECKS=6284450000 NODES=369813	? (TO) TT=1800.04	? (TO) TT=1800.04	SAT TT=34.09 ASSIGNMENTS=24701 CHECKS=14158200	SAT TT=49.232 ASSIGNMENTS=70107 CHECKS=41220300	? (TO) TT=1800.09	? (TO) TT=1800.06	SAT TT=257.594 CHECKS=58240800	SAT TT=3.199 ASSIGNMENTS=2781 CHECKS=1685930	SAT TT=19.931 ASSIGNMENTS=19492 CHECKS=11381800	? (problem) TT=3.977	SAT TT=107.332 ASSIGNMENTS=99763 CHECKS=59854400	SAT TT=26.292 ASSIGNMENTS=24701 CHECKS=14158200	? TT=344.602	? (TO) TT=1800.12	SAT TT=1655.33	SAT TT=908.703
csp/travellingSalesman-25/ normalized-tsp-25-54_ext.xml	SAT TT=0.512	SAT TT=1.416 CHECKS=412071 NODES=432	SAT TT=1.883 CHECKS=947880 NODES=134	SAT TT=4.494	SAT TT=38.083 CHECKS=1432860	SAT TT=19.789 CHECKS=315283	SAT TT=5.535 ASSIGNMENTS=24	SAT TT=5.212 ASSIGNMENTS=55	SAT TT=63.363 CHECKS=26943500 NODES=1326	? (TO) TT=1800.02	? (TO) TT=1800.05	SAT TT=0.664 ASSIGNMENTS=640 CHECKS=417477	SAT TT=0.657 ASSIGNMENTS=640 CHECKS=417477	SAT TT=471.178	SAT TT=714.616	SAT TT=2.542 CHECKS=623993	SAT TT=0.52 ASSIGNMENTS=486 CHECKS=364152	SAT TT=0.442 ASSIGNMENTS=658 CHECKS=330094	? (problem) TT=2.534	SAT TT=6.606 ASSIGNMENTS=10828 CHECKS=5522150	SAT TT=0.597 ASSIGNMENTS=640 CHECKS=417477	? (exit code) TT=303.238	SAT TT=188.524 ASSIGNMENTS=108 CHECKS=5415000	SAT TT=67.067	SAT TT=67.766
csp/travellingSalesman-25/ normalized-tsp-25-561_ext.xml	SAT TT=4.074	SAT TT=4.074 CHECKS=3005660 NODES=4342	SAT TT=21.122 CHECKS=26034100 NODES=31070	SAT TT=90.067	? (TO) TT=1800.02	SAT TT=832.061 CHECKS=11624000	SAT TT=43.493 ASSIGNMENTS=1756	SAT TT=75.279 ASSIGNMENTS=7125	SAT TT=70.478 CHECKS=104823000 NODES=3032	? (TO) TT=1800.03	? (TO) TT=1800.04	SAT TT=15.15 ASSIGNMENTS=5831 CHECKS=3568410	SAT TT=11.491 ASSIGNMENTS=4342 CHECKS=3005660	? (TO) TT=1800.03	? (TO) TT=1800.02	SAT TT=52.256 CHECKS=11261500	SAT TT=5.045 ASSIGNMENTS=3879 CHECKS=2625400	SAT TT=17.359 ASSIGNMENTS=13591 CHECKS=9336340	? (problem) TT=4.063	SAT TT=49.235 ASSIGNMENTS=28744 CHECKS=21876400	SAT TT=7.169 ASSIGNMENTS=5831 CHECKS=3568410	? TT=360.515	SAT TT=1688.72 ASSIGNMENTS=1895 CHECKS=680034000	SAT TT=329.202	SAT TT=213.395
csp/travellingSalesman-25/ normalized-tsp-25-66_ext.xml	SAT TT=0.317	SAT TT=1.966 CHECKS=720062 NODES=1185	SAT TT=1.626 CHECKS=434240 NODES=361	SAT TT=3.146	SAT TT=54.164 CHECKS=2249880	SAT TT=28.646 CHECKS=871493	SAT TT=10.624 ASSIGNMENTS=530	SAT TT=10.395 ASSIGNMENTS=105	SAT TT=62.007 CHECKS=14287800 NODES=520	? (TO) TT=1800.01	? (TO) TT=1800.05	SAT TT=0.509 ASSIGNMENTS=625 CHECKS=392033	SAT TT=0.511 ASSIGNMENTS=625 CHECKS=392033	SAT TT=357.282	SAT TT=344.788	SAT TT=3.495 CHECKS=1070320	SAT TT=2.46 ASSIGNMENTS=3110 CHECKS=2093820	SAT TT=1.552 ASSIGNMENTS=2587 CHECKS=1249550	? (problem) TT=2.003	SAT TT=16.845 ASSIGNMENTS=22087 CHECKS=12976700	SAT TT=0.495 ASSIGNMENTS=625 CHECKS=392033	? (exit code) TT=479.173	SAT TT=172.768 ASSIGNMENTS=264 CHECKS=375958000	SAT TT=32.277	SAT TT=71.388
csp/travellingSalesman-25/ normalized-tsp-25-681_ext.xml	SAT TT=45.928	SAT TT=45.928 CHECKS=43862700 NODES=66094	SAT TT=79.278 CHECKS=77594100 NODES=116988	SAT TT=276.753	SAT TT=569.302 CHECKS=23597100	? (TO) TT=1800.05	SAT TT=141.997 ASSIGNMENTS=17903	SAT TT=82.457 ASSIGNMENTS=7372	SAT TT=322.927 CHECKS=2663680000 NODES=118028	? (TO) TT=1800.08	? (TO) TT=1800.02	SAT TT=75.263 ASSIGNMENTS=57891 CHECKS=40450000	SAT TT=49.947 ASSIGNMENTS=66094 CHECKS=43862700	? (TO) TT=1800.08	? (TO) TT=1800.1	SAT TT=115.048 CHECKS=24357600	SAT TT=61.643 ASSIGNMENTS=48393 CHECKS=34319000	SAT TT=65.293 ASSIGNMENTS=56215 CHECKS=38340000	? (problem) TT=4.539	SAT TT=84.272 ASSIGNMENTS=67197 CHECKS=45101800	SAT TT=66.883 ASSIGNMENTS=57891 CHECKS=40450000	? TT=500.417	? (TO) TT=1800.05	SAT TT=518.846	SAT TT=1333.29
csp/travellingSalesman-25/ normalized-tsp-25-715_ext.xml	SAT TT=184.488	SAT TT=446.391 CHECKS=454181000 NODES=568001	SAT TT=271.525 CHECKS=284879000 NODES=338824	SAT TT=568.085	? (TO) TT=1800.02	? (TO) TT=1800.06	SAT TT=682.113 ASSIGNMENTS=65222	? (TO) TT=1800.13	SAT TT=227.11 CHECKS=1920630000 NODES=41304	? (TO) TT=1800.01	? (TO) TT=1800.07	SAT TT=196.048 ASSIGNMENTS=114025 CHECKS=85295500	SAT TT=417.469 ASSIGNMENTS=114025 CHECKS=85295500	? (TO) TT=1800.07	? (TO) TT=1800.01	SAT TT=398.841 CHECKS=77697700	SAT TT=346.066 ASSIGNMENTS=222639 CHECKS=158463000	SAT TT=40.029 ASSIGNMENTS=25261 CHECKS=19380400	? (problem) TT=3.81	SAT TT=248.577 ASSIGNMENTS=144009 CHECKS=113010000	SAT TT=184.488 ASSIGNMENTS=114025 CHECKS=85295500	? TT=384.098	? (TO) TT=1800.08	SAT TT=1113.77	SAT TT=480.475
csp/travellingSalesman-25/ normalized-tsp-25-843_ext.xml	SAT TT=17.836	SAT TT=28.987 CHECKS=27667000 NODES=40479	SAT TT=33.106 CHECKS=31827200 NODES=46706	SAT TT=17.836	SAT TT=503.377 CHECKS=23112300	SAT TT=841.778 CHECKS=15113300	SAT TT=41.96 ASSIGNMENTS=3536	SAT TT=71.354 ASSIGNMENTS=4920	SAT TT=79.303 CHECKS=189322000 NODES=5868	? (TO) TT=1800.08	? (TO) TT=1800.08	SAT TT=39.324 ASSIGNMENTS=27437 CHECKS=18693900	SAT TT=34.453 ASSIGNMENTS=40479 CHECKS=27667000	SAT TT=1750.43	SAT TT=1573.32	SAT TT=187.681 CHECKS=37074400	SAT TT=20.125 ASSIGNMENTS=16360 CHECKS=11354100	SAT TT=36.722 ASSIGNMENTS=32898 CHECKS=21889700	? (problem) TT=2.789	SAT TT=73.339 ASSIGNMENTS=56871 CHECKS=40028800	SAT TT=29.816 ASSIGNMENTS=27437 CHECKS=18693900	? TT=304.725	SAT TT=960.188 ASSIGNMENTS=10516 CHECKS=774436000	SAT TT=205.76	SAT TT=133.742

Some statistics...

	Abscon 112v4 AC	Abscon 112v4 ESAC	bpsolver 2008-06-27	casper zao	casper zito	choco2_dwdeg 2008-06-26	choco2_impwdeg 2008-06-26	Concrete + CPS4J 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	cpHydra k_10	cpHydra k_40	galac E	galac M	mddc-solv 2008-06-09	MDG-noprobe 2008-06-27	Mistral-option 1.314	Mistral-prime 1.313	SAT4J CSP 2008-06-13	Sugar v1.13+minisat	Sugar v1.13+picosat
Number of times the solver is able to give the best known answer	30	30	28	22	21	30	29	30	1	30	30	15	15	30	30	30	30	1	30	30
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)	30	30	28	22	21	30	29	30	1	30	30	15	15	30	30	30	30	1	30	30
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	3	2	2	0	0	0	0	0	0	1	1	0	0	1	10	0	8	0	0	0
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0