CSP 2009 Competition: results by benchmark

Results by benchmark for category alldiff (Alldiff), subcategory academic problems

This page displays the results of the different solvers for each benchmark for category alldiff (Alldiff), subcategory academic problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 112v4 AC (complete)
Abscon 112v4 ESAC (complete)
bpsolver 09 (complete)
Choco2.1.1 2009-06-10 (complete)
Choco2.1.1b 2009-07-16 (complete)
Concrete 2009-07-14 (complete)
Concrete DC 2009-07-14 (complete)
Mistral 1.545 (complete)
SAT4J CSP 2.1.1 (complete)
Sugar v1.14.6+minisat (complete)
Sugar v1.14.6+picosat (complete)

Bench name	Best results on this instance	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/costasArray/ normalized-costasArray-10.xml	SAT TT=0.009	SAT TT=0.831	SAT TT=0.945	SAT TT=0.036	SAT TT=1.137	SAT TT=1.212	SAT TT=1.284	SAT TT=4.314	SAT TT=0.009	SAT TT=2.908	SAT TT=0.953	SAT TT=1.021
csp/costasArray/ normalized-costasArray-11.xml	SAT TT=0.019	SAT TT=1.041	SAT TT=1.044	SAT TT=0.062	SAT TT=1.421	SAT TT=1.54	SAT TT=1.327	SAT TT=7.711	SAT TT=0.019	SAT TT=3.595	SAT TT=1.13	SAT TT=1.314
csp/costasArray/ normalized-costasArray-12.xml	SAT TT=0.013	SAT TT=1.395	SAT TT=1.176	SAT TT=0.05	SAT TT=1.989	SAT TT=2.093	SAT TT=1.666	SAT TT=19.738	SAT TT=0.013	SAT TT=6.479	SAT TT=1.19	SAT TT=2.138
csp/costasArray/ normalized-costasArray-13.xml	SAT TT=0.086	SAT TT=3.522	SAT TT=1.679	SAT TT=0.086	SAT TT=3.483	SAT TT=3.518	SAT TT=3.055	SAT TT=28.64	SAT TT=0.113	SAT TT=12.822	SAT TT=1.904	SAT TT=1.83
csp/costasArray/ normalized-costasArray-14.xml	SAT TT=0.196	SAT TT=1.358	SAT TT=1.953	SAT TT=0.196	SAT TT=6.571	SAT TT=8.943	SAT TT=2.942	SAT TT=127.593	SAT TT=0.663	SAT TT=11.229	SAT TT=2.372	SAT TT=7.703
csp/costasArray/ normalized-costasArray-15.xml	SAT TT=2.038	SAT TT=101.212	SAT TT=17.552	SAT TT=6.654	SAT TT=10.139	SAT TT=11.177	SAT TT=17.508	SAT TT=178.954	SAT TT=2.038	SAT TT=70.869	SAT TT=16.212	SAT TT=15.108
csp/costasArray/ normalized-costasArray-16.xml	SAT TT=3.112	SAT TT=40.859	SAT TT=329.336	SAT TT=3.112	SAT TT=24.099	SAT TT=6.11	SAT TT=61.591	SAT TT=892.701	SAT TT=7.567	SAT TT=530.6	SAT TT=381.815	SAT TT=36.012
csp/costasArray/ normalized-costasArray-17.xml	SAT TT=35.778	SAT TT=596.264	SAT TT=1450.62	SAT TT=145.97	SAT TT=216.625	SAT TT=162.049	SAT TT=131.831	? (TO) TT=1800.01	SAT TT=35.778	SAT TT=301.253	SAT TT=1430.55	SAT TT=520.327
csp/costasArray/ normalized-costasArray-18.xml	SAT TT=63.832	? (TO) TT=1800.13	? (TO) TT=1800.16	SAT TT=63.832	SAT TT=1409.68	? (TO) TT=1800.04	SAT TT=201.55	? (TO) TT=1800.08	SAT TT=384.884	? (TO) TT=1800.44	? (TO) TT=1800.06	? (TO) TT=1800.05
csp/costasArray/ normalized-costasArray-19.xml		? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.11	? (TO) TT=1800.35	? (TO) TT=1800.04	? (TO) TT=1800.14
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/costasArray/ normalized-costasArray-20.xml		? (TO) TT=1800.14	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.18	? (TO) TT=1800.12	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.41	? (TO) TT=1800.07	? (TO) TT=1800.04
csp/latinSquare/ normalized-latinSquare-dg-10_all.xml	UNSAT TT=1.163	? (TO) TT=1800.16	? (TO) TT=1800.13	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.15	? (TO) TT=1800.02	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.31	UNSAT TT=1.163	UNSAT TT=1.528
csp/latinSquare/ normalized-latinSquare-dg-11_all.xml	SAT TT=1.349	? (TO) TT=1800.13	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.14	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.39	SAT TT=1.349	? (TO) TT=1800.05
csp/latinSquare/ normalized-latinSquare-dg-12_all.xml	UNSAT TT=1.66	? (TO) TT=1800.13	? (TO) TT=1800.06	? (TO) TT=1800.1	? (TO) TT=1800.13	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.39	UNSAT TT=1.66	UNSAT TT=2.477
csp/latinSquare/ normalized-latinSquare-dg-3_all.xml	UNSAT TT=0.007	UNSAT TT=0.658	UNSAT TT=0.639	UNSAT TT=0.026	UNSAT TT=0.408	UNSAT TT=0.393	UNSAT TT=0.859	UNSAT TT=0.835	UNSAT TT=0.007	UNSAT TT=0.526	UNSAT TT=0.565	UNSAT TT=0.58
csp/latinSquare/ normalized-latinSquare-dg-4_all.xml	UNSAT TT=0.007	UNSAT TT=0.625	UNSAT TT=0.67	UNSAT TT=0.027	UNSAT TT=0.407	UNSAT TT=0.444	UNSAT TT=0.87	UNSAT TT=0.875	UNSAT TT=0.007	UNSAT TT=0.525	UNSAT TT=0.592	UNSAT TT=0.593
csp/latinSquare/ normalized-latinSquare-dg-5_all.xml	SAT TT=0.006	SAT TT=0.681	SAT TT=0.635	SAT TT=0.028	SAT TT=0.575	SAT TT=0.537	SAT TT=0.85	SAT TT=1.022	SAT TT=0.006	SAT TT=0.521	SAT TT=0.73	SAT TT=0.889
csp/latinSquare/ normalized-latinSquare-dg-6_all.xml	UNSAT TT=0.034	UNSAT TT=0.823	UNSAT TT=0.807	UNSAT TT=0.174	UNSAT TT=0.731	UNSAT TT=0.742	UNSAT TT=1.017	UNSAT TT=1.347	UNSAT TT=0.034	UNSAT TT=1.044	UNSAT TT=0.787	UNSAT TT=0.798
csp/latinSquare/ normalized-latinSquare-dg-7_all.xml	SAT TT=0.01	SAT TT=0.781	SAT TT=0.812	SAT TT=0.037	SAT TT=0.766	SAT TT=0.791	SAT TT=0.916	SAT TT=1.334	SAT TT=0.01	SAT TT=0.744	SAT TT=0.936	SAT TT=1.015
csp/latinSquare/ normalized-latinSquare-dg-8_all.xml	UNSAT TT=1.014	UNSAT TT=676.746	UNSAT TT=727.935	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.15	? (TO) TT=1800.09	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800.3	UNSAT TT=1.014	UNSAT TT=1.044
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/latinSquare/ normalized-latinSquare-dg-9_all.xml	UNSAT TT=1.077	? (TO) TT=1800.06	? (TO) TT=1800.15	? (TO) TT=1800.03	? (TO) TT=1800.22	? (TO) TT=1800.11	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.28	UNSAT TT=1.077	UNSAT TT=1.291
csp/nengfa/ normalized-protein.xml	SAT TT=6.158	SAT TT=1144.29	SAT TT=566.633	SAT TT=1518.17	SAT TT=15.139	SAT TT=83.87	SAT TT=306.502	? (TO) TT=1800.04	SAT TT=140.489	SAT TT=548.303	SAT TT=6.158	SAT TT=9.074
csp/nengfa/ normalized-queens2.xml	SAT TT=0.119	SAT TT=3.809	SAT TT=5.726	SAT TT=0.508	SAT TT=24.782	SAT TT=27.549	SAT TT=11.964	? (TO) TT=1800.09	SAT TT=0.119	SAT TT=17.821	SAT TT=18.349	SAT TT=37.224
csp/nengfa/ normalized-sudoku368.xml	SAT TT=0.23	SAT TT=2.873	SAT TT=16.489	SAT TT=9.315	SAT TT=1.405	SAT TT=1.294	SAT TT=8.309	SAT TT=31.425	SAT TT=0.23	SAT TT=2.593	SAT TT=3.78	SAT TT=4.337
csp/ortholatin/ normalized-ortholatin-10.xml		? (TO) TT=1800.08	? (TO) TT=1800.13	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.01	? (TO) TT=1800.02	? (TO) TT=1800.31	? (TO) TT=1800.09	? (TO) TT=1800.06
csp/ortholatin/ normalized-ortholatin-15.xml		? (TO) TT=1800.12	? (TO) TT=1800.14	? (TO) TT=1800.07	? (TO) TT=1800.12	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.07	? (exit code) TT=105.992	? (TO) TT=1800.16	? (TO) TT=1800.05
csp/ortholatin/ normalized-ortholatin-20.xml		? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.04	? (TO) TT=1800.16	? (TO) TT=1800.08	? (TO) TT=1800.09	? TT=769.744	? (TO) TT=1800.09	? TT=103.831	? (MO) TT=157.479	? (MO) TT=350.66
csp/ortholatin/ normalized-ortholatin-2.xml	UNSAT TT=0.007	UNSAT TT=0.624	UNSAT TT=0.632	UNSAT TT=0.026	UNSAT TT=0.433	UNSAT TT=0.419	UNSAT TT=0.833	UNSAT TT=0.857	UNSAT TT=0.007	UNSAT TT=0.507	UNSAT TT=0.515	UNSAT TT=0.551
csp/ortholatin/ normalized-ortholatin-5.xml	SAT TT=0.013	SAT TT=2.865	SAT TT=0.758	SAT TT=1.69	SAT TT=1.269	SAT TT=1.183	SAT TT=0.958	SAT TT=2.629	SAT TT=0.013	SAT TT=1.571	SAT TT=1.235	SAT TT=1.213
csp/ortholatin/ normalized-ortholatin-6.xml		? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.5	? (TO) TT=1800.13	? (TO) TT=1800.08
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/ortholatin/ normalized-ortholatin-7.xml	SAT TT=9.426	SAT TT=81.926	SAT TT=9.426	? (TO) TT=1800.04	? (TO) TT=1800.04	? (TO) TT=1800.12	SAT TT=32.843	SAT TT=43.202	SAT TT=1207.39	SAT TT=734.884	SAT TT=57.359	SAT TT=435.392
csp/ortholatin/ normalized-ortholatin-8.xml	SAT TT=8.902	SAT TT=859.944	? (TO) TT=1800.11	SAT TT=8.902	? (TO) TT=1800.11	? (TO) TT=1800.11	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.08	? (TO) TT=1800.34	? (TO) TT=1800.02	? (TO) TT=1800.1
csp/ortholatin/ normalized-ortholatin-9.xml		? (TO) TT=1800.16	? (TO) TT=1800.15	? (TO) TT=1800.07	? (TO) TT=1800.15	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.4	? (TO) TT=1800.04	? (TO) TT=1800.1
csp/pigeons_glb/ normalized-pigeons-10_ext.xml	UNSAT TT=0.008	UNSAT TT=0.591	UNSAT TT=0.593	UNSAT TT=0.023	UNSAT TT=0.392	UNSAT TT=0.351	UNSAT TT=0.77	UNSAT TT=0.863	UNSAT TT=0.008	UNSAT TT=6.656	UNSAT TT=0.365	UNSAT TT=0.385
csp/pigeons_glb/ normalized-pigeons-11_ext.xml	UNSAT TT=0.007	UNSAT TT=0.592	UNSAT TT=0.583	UNSAT TT=0.025	UNSAT TT=0.38	UNSAT TT=0.353	UNSAT TT=0.864	UNSAT TT=0.844	UNSAT TT=0.007	UNSAT TT=69.245	UNSAT TT=0.362	UNSAT TT=0.36
csp/pigeons_glb/ normalized-pigeons-12_ext.xml	UNSAT TT=0.006	UNSAT TT=0.594	UNSAT TT=0.582	UNSAT TT=0.025	UNSAT TT=0.346	UNSAT TT=0.356	UNSAT TT=0.777	UNSAT TT=0.875	UNSAT TT=0.006	UNSAT TT=1119.68	UNSAT TT=0.398	UNSAT TT=0.392
csp/pigeons_glb/ normalized-pigeons-13_ext.xml	UNSAT TT=0.005	UNSAT TT=0.587	UNSAT TT=0.621	UNSAT TT=0.026	UNSAT TT=0.353	UNSAT TT=0.372	UNSAT TT=0.838	UNSAT TT=0.861	UNSAT TT=0.005	? (TO) TT=1800.33	UNSAT TT=0.391	UNSAT TT=0.402
csp/pigeons_glb/ normalized-pigeons-14_ext.xml	UNSAT TT=0.006	UNSAT TT=0.648	UNSAT TT=0.616	UNSAT TT=0.026	UNSAT TT=0.35	UNSAT TT=0.358	UNSAT TT=0.856	UNSAT TT=0.783	UNSAT TT=0.006	? (TO) TT=1800.33	UNSAT TT=0.381	UNSAT TT=0.402
csp/pigeons_glb/ normalized-pigeons-15_ext.xml	UNSAT TT=0.007	UNSAT TT=0.581	UNSAT TT=0.628	UNSAT TT=0.026	UNSAT TT=0.408	UNSAT TT=0.352	UNSAT TT=0.865	UNSAT TT=0.868	UNSAT TT=0.007	? (TO) TT=1800.37	UNSAT TT=0.412	UNSAT TT=0.403
csp/pigeons_glb/ normalized-pigeons-18_ext.xml	UNSAT TT=0.007	UNSAT TT=0.625	UNSAT TT=0.602	UNSAT TT=0.025	UNSAT TT=0.347	UNSAT TT=0.402	UNSAT TT=0.778	UNSAT TT=0.843	UNSAT TT=0.007	? (TO) TT=1800.27	UNSAT TT=0.407	UNSAT TT=0.465
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pigeons_glb/ normalized-pigeons-20_ext.xml	UNSAT TT=0.006	UNSAT TT=0.605	UNSAT TT=0.637	UNSAT TT=0.025	UNSAT TT=0.368	UNSAT TT=0.38	UNSAT TT=0.814	UNSAT TT=0.781	UNSAT TT=0.006	? (TO) TT=1800.39	UNSAT TT=0.505	UNSAT TT=0.417
csp/pigeons_glb/ normalized-pigeons-25_ext.xml	UNSAT TT=0.009	UNSAT TT=0.615	UNSAT TT=0.63	UNSAT TT=0.028	UNSAT TT=0.368	UNSAT TT=0.362	UNSAT TT=0.807	UNSAT TT=0.856	UNSAT TT=0.009	? (TO) TT=1800.32	UNSAT TT=0.579	UNSAT TT=0.621
csp/pigeons_glb/ normalized-pigeons-30_ext.xml	UNSAT TT=0.007	UNSAT TT=0.633	UNSAT TT=0.621	UNSAT TT=0.025	UNSAT TT=0.359	UNSAT TT=0.363	UNSAT TT=0.844	UNSAT TT=0.817	UNSAT TT=0.007	? (TO) TT=1800.27	UNSAT TT=0.55	UNSAT TT=0.527
csp/pigeons_glb/ normalized-pigeons-35_ext.xml	UNSAT TT=0.008	UNSAT TT=0.644	UNSAT TT=0.657	UNSAT TT=0.027	UNSAT TT=0.361	UNSAT TT=0.39	UNSAT TT=0.943	UNSAT TT=0.866	UNSAT TT=0.008	? (TO) TT=1800.32	UNSAT TT=0.58	UNSAT TT=0.735
csp/pigeons_glb/ normalized-pigeons-40_ext.xml	UNSAT TT=0.008	UNSAT TT=0.632	UNSAT TT=0.642	UNSAT TT=0.026	UNSAT TT=0.42	UNSAT TT=0.396	UNSAT TT=0.8	UNSAT TT=0.904	UNSAT TT=0.008	? (TO) TT=1800.36	UNSAT TT=0.745	UNSAT TT=0.594
csp/pigeons_glb/ normalized-pigeons-45_ext.xml	UNSAT TT=0.01	UNSAT TT=0.624	UNSAT TT=0.64	UNSAT TT=0.026	UNSAT TT=0.364	UNSAT TT=0.368	UNSAT TT=0.87	UNSAT TT=0.85	UNSAT TT=0.01	? (TO) TT=1800.45	UNSAT TT=0.563	UNSAT TT=0.746
csp/pigeons_glb/ normalized-pigeons-50_ext.xml	UNSAT TT=0.008	UNSAT TT=0.646	UNSAT TT=0.644	UNSAT TT=0.028	UNSAT TT=0.371	UNSAT TT=0.373	UNSAT TT=0.825	UNSAT TT=0.831	UNSAT TT=0.008	? (TO) TT=1800.32	UNSAT TT=0.611	UNSAT TT=0.599
csp/pigeons_glb/ normalized-pigeons-5_ext.xml	UNSAT TT=0.021	UNSAT TT=0.594	UNSAT TT=0.607	UNSAT TT=0.026	UNSAT TT=0.378	UNSAT TT=0.397	UNSAT TT=0.806	UNSAT TT=0.866	UNSAT TT=0.021	UNSAT TT=0.513	UNSAT TT=0.354	UNSAT TT=0.35
csp/pigeons_glb/ normalized-pigeons-6_ext.xml	UNSAT TT=0.006	UNSAT TT=0.596	UNSAT TT=0.621	UNSAT TT=0.026	UNSAT TT=0.348	UNSAT TT=0.371	UNSAT TT=0.824	UNSAT TT=0.855	UNSAT TT=0.006	UNSAT TT=0.544	UNSAT TT=0.339	UNSAT TT=0.369
csp/pigeons_glb/ normalized-pigeons-7_ext.xml	UNSAT TT=0.008	UNSAT TT=0.575	UNSAT TT=0.614	UNSAT TT=0.023	UNSAT TT=0.351	UNSAT TT=0.349	UNSAT TT=0.829	UNSAT TT=0.842	UNSAT TT=0.008	UNSAT TT=0.803	UNSAT TT=0.338	UNSAT TT=0.37
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pigeons_glb/ normalized-pigeons-8_ext.xml	UNSAT TT=0.006	UNSAT TT=0.593	UNSAT TT=0.647	UNSAT TT=0.025	UNSAT TT=0.341	UNSAT TT=0.384	UNSAT TT=0.776	UNSAT TT=0.798	UNSAT TT=0.006	UNSAT TT=1.047	UNSAT TT=0.378	UNSAT TT=0.348
csp/pigeons_glb/ normalized-pigeons-9_ext.xml	UNSAT TT=0.006	UNSAT TT=0.577	UNSAT TT=0.588	UNSAT TT=0.025	UNSAT TT=0.373	UNSAT TT=0.405	UNSAT TT=0.769	UNSAT TT=0.874	UNSAT TT=0.006	UNSAT TT=2.071	UNSAT TT=0.378	UNSAT TT=0.382

Statistics		Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
Number of times the solver is able to give the best known answer		40	39	39	38	37	39	35	39	27	43	42
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	1	5	0	0	0	0	33	0	6	0