CSP 2006 Competition (second round): results by benchmark

Results by benchmark for category binary constraints in intension (2-ARY-INT), subcategory quasi random

This page displays the results of the different solvers for each benchmark for category binary constraints in intension (2-ARY-INT), subcategory quasi random

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 109 ESAC
Abscon 109 AC
BPrologCSPSolver70a 2006-12-13
buggy_2_5 2007-01-08
buggy_2_5_s 2007-01-08
CSP4J - Combo 2006-12-19
CSP4J - MAC 2006-12-19
CSP4J - MAC 2007-01-16
CSPtoSAT+minisat 0.3
galac 1
galacJ beta 1
Mistral 2006-12-04
sat4jCSP 1.7 RC BF3
SAT4JCSP-CACHED 1.7 RC BF3
sugar 0.40
Tramontane 2006-12-04
VALCSP 3.0
VALCSP 3.1

Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
haystacks/ haystacks_04.xml	UNSAT TT=0.017	UNSAT TT=0.725 CHECKS=2263 NODES=75	UNSAT TT=0.733 CHECKS=3471 NODES=148	UNSAT TT=0.025	UNSAT TT=0.063	UNSAT TT=0.06	UNSAT TT=0.486 NODES=78	UNSAT TT=0.491 NODES=224	UNSAT TT=0.502 CHECKS=3122 NODES=224	UNSAT TT=0.017	UNSAT TT=0.667	UNSAT TT=0.653	Signal TT=0.016	UNSAT TT=0.935	UNSAT TT=0.887	? TT=0.421	Signal TT=0.015	Wrong Cert. TT=0.004 CHECKS=598 NODES=16	UNSAT TT=0.003 CHECKS=4883 NODES=95
haystacks/ haystacks_05.xml	UNSAT TT=0.023	UNSAT TT=1.425 CHECKS=366081 NODES=32345	UNSAT TT=1.495 CHECKS=449118 NODES=43555	UNSAT TT=1.085	UNSAT TT=0.07	UNSAT TT=0.069	UNSAT TT=0.997 NODES=2974	UNSAT TT=16.241 NODES=294983	UNSAT TT=13.637 CHECKS=3408150 NODES=242892	UNSAT TT=0.023	UNSAT TT=0.784	UNSAT TT=0.789	Signal TT=0.015	UNSAT TT=1.22	UNSAT TT=1.191	? TT=0.466	Signal TT=0.015	Wrong Cert. TT=0.004 CHECKS=1648 NODES=25	UNSAT TT=0.216 CHECKS=2997920 NODES=43105
haystacks/ haystacks_06.xml	UNSAT TT=0.048	? (TO) TT=1800.18	UNSAT TT=110.442 CHECKS=58712272 NODES=6765759	UNSAT TT=144.24	UNSAT TT=0.083	UNSAT TT=0.083	UNSAT TT=11.688 NODES=86871	? (TO) TT=1800.03	? (TO) TT=1800.04	UNSAT TT=0.048	UNSAT TT=0.871	UNSAT TT=0.84	Signal TT=0.015	UNSAT TT=1.716	UNSAT TT=1.516	? TT=0.245	Signal TT=0.014	Wrong Cert. TT=0.004 CHECKS=3785 NODES=36	? (TO) TT=1800.02
haystacks/ haystacks_07.xml	UNSAT TT=0.093	? (TO) TT=1800.16	? (TO) TT=1800.19	UNSAT TT=12.001	UNSAT TT=0.098	UNSAT TT=0.093	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.04	UNSAT TT=0.142	UNSAT TT=0.966	UNSAT TT=0.91	Signal TT=0.015	UNSAT TT=2.748	UNSAT TT=2.055	? TT=0.048	Signal TT=0.015	Wrong Cert. TT=0.006 CHECKS=7670 NODES=49	? (TO) TT=1800.01
haystacks/ haystacks_08.xml	UNSAT TT=0.132	? (TO) TT=1800.23	? (TO) TT=1800.14	? (TO) TT=1800.12	UNSAT TT=0.137	UNSAT TT=0.132	? (TO) TT=1800.03	? (TO) TT=1800.01	? (TO) TT=1800.02	UNSAT TT=0.827	UNSAT TT=1.28	UNSAT TT=1.314	Signal TT=0.016	UNSAT TT=3.207	UNSAT TT=2.839	? TT=0.27	Signal TT=0.016	Wrong Cert. TT=0.009 CHECKS=14182 NODES=64	? (TO) TT=1800.09
haystacks/ haystacks_09.xml	UNSAT TT=0.167	? (TO) TT=1800.14	? (TO) TT=1800.17	? (TO) TT=1800.03	UNSAT TT=0.167	UNSAT TT=0.174	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.05	UNSAT TT=5.874	UNSAT TT=6.174	UNSAT TT=3.364	Signal TT=0.019	UNSAT TT=7.233	UNSAT TT=6.4	? TT=0.712	Signal TT=0.018	Wrong Cert. TT=0.015 CHECKS=24448 NODES=81	? (TO) TT=1800.1
haystacks/ haystacks_10.xml	UNSAT TT=0.209	? (TO) TT=1800.2	? (TO) TT=1800.14	? (TO) TT=1800.11	UNSAT TT=0.209	UNSAT TT=0.213	? (TO) TT=1800.02	? (TO) TT=1800.06	? (TO) TT=1800.1	UNSAT TT=35.728	UNSAT TT=471.855	UNSAT TT=41.744	Signal TT=0.016	UNSAT TT=157.996	UNSAT TT=157.114	? TT=0.72	Signal TT=0.016	Wrong Cert. TT=0.019 CHECKS=39873 NODES=100	? (TO) TT=1800.01
haystacks/ haystacks_11.xml	UNSAT TT=0.35	? (TO) TT=1800.14	? (TO) TT=1800.54	? (TO) TT=1800.07	UNSAT TT=0.35	UNSAT TT=0.355	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.04	UNSAT TT=211.773	? (TO) TT=1800.07	UNSAT TT=1174.8	Signal TT=0.017	? (TO) TT=1800.16	? (TO) TT=1800.11	? TT=0.736	Signal TT=0.019	Wrong Cert. TT=0.025 CHECKS=62170 NODES=121	? (TO) TT=1800.07
haystacks/ haystacks_12.xml	UNSAT TT=0.548	? (TO) TT=1800.18	? (TO) TT=1800.18	? (TO) TT=1800.06	UNSAT TT=0.549	UNSAT TT=0.548	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.09	Signal TT=0.019	? (TO) TT=1800.11	? (TO) TT=1800.19	? TT=0.789	Signal TT=0.017	Wrong Cert. TT=0.035 CHECKS=93390 NODES=144	? (TO) TT=1800.1
haystacks/ haystacks_13.xml	UNSAT TT=0.541	? (TO) TT=1800.24	? (TO) TT=1800.16	? (TO) TT=1800.11	UNSAT TT=0.545	UNSAT TT=0.541	? (TO) TT=1800.01	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.05	Signal TT=0.019	? (TO) TT=1800.14	? (TO) TT=1800.2	? TT=0.56	Signal TT=0.018	Wrong Cert. TT=0.047 CHECKS=135952 NODES=169	? (TO) TT=1800.04
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
haystacks/ haystacks_14.xml	UNSAT TT=0.959	? (TO) TT=1800.15	? (TO) TT=1800.17	? (TO) TT=1800.04	UNSAT TT=0.959	UNSAT TT=1.011	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.1	? (TO) TT=1800.04	? (TO) TT=1800	Signal TT=0.019	? (TO) TT=1800.12	? (TO) TT=1800.21	? TT=0.314	Signal TT=0.019	Wrong Cert. TT=0.071 CHECKS=192673 NODES=196	? (TO) TT=1800.05
haystacks/ haystacks_15.xml	UNSAT TT=1.423	? (TO) TT=1800.22	? (TO) TT=1800.17	? (TO) TT=1800.04	UNSAT TT=1.423	UNSAT TT=1.482	? (TO) TT=1800.1	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.03	Signal TT=0.022	? (TO) TT=1800.19	? (TO) TT=1800.1	? TT=0.324	Signal TT=0.023	Wrong Cert. TT=0.097 CHECKS=266798 NODES=225	? (TO) TT=1800.08
haystacks/ haystacks_16.xml	UNSAT TT=2.387	? (TO) TT=1800.15	? (TO) TT=1800.18	? (TO) TT=1800.07	UNSAT TT=2.387	UNSAT TT=2.418	? (TO) TT=1800.01	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.06	Signal TT=0.022	? (TO) TT=1800.14	? (TO) TT=1800.38	? TT=0.63	Signal TT=0.025	Wrong Cert. TT=0.119 CHECKS=362030 NODES=256	? (TO) TT=1800.11
haystacks/ haystacks_17.xml	UNSAT TT=1.489	? (TO) TT=1800.17	? (TO) TT=1800.24	? (TO) TT=1800.08	UNSAT TT=1.489	UNSAT TT=1.607	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.06	Signal TT=0.021	? (TO) TT=1800.16	? (TO) TT=1800.12	? TT=0.343	Signal TT=0.02	Wrong Cert. TT=0.164 CHECKS=482560 NODES=289	? (TO) TT=1800.02
haystacks/ haystacks_18.xml	UNSAT TT=2.333	? (TO) TT=1800.24	? (TO) TT=1800.14	? (TO) TT=1800.06	UNSAT TT=2.333	UNSAT TT=2.353	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.01	? (TO) TT=1800.04	Signal TT=0.022	? (TO) TT=1800.17	? (TO) TT=1800.18	? TT=0.628	Signal TT=0.022	Wrong Cert. TT=0.21 CHECKS=633097 NODES=324	? (TO) TT=1800.06
haystacks/ haystacks_19.xml	UNSAT TT=2.775	? (TO) TT=1800.19	? (TO) TT=1800.24	? (TO) TT=1800.06	UNSAT TT=2.775	UNSAT TT=2.843	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.1	Signal TT=0.022	? (TO) TT=1800.2	? (TO) TT=1800.16	? TT=0.695	Signal TT=0.022	Wrong Cert. TT=0.256 CHECKS=818898 NODES=361	? (TO) TT=1800.05
haystacks/ haystacks_20.xml	UNSAT TT=7.192	? (TO) TT=1800.18	? (TO) TT=1800.17	? (TO) TT=1800.05	UNSAT TT=7.192	UNSAT TT=7.424	? (TO) TT=1800.09	? (TO) TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.03	Signal TT=0.022	? (TO) TT=1800.2	? (TO) TT=1800.58	? TT=1.07	Signal TT=0.023	Wrong Cert. TT=0.341 CHECKS=1045800 NODES=400	? (TO) TT=1800.02
haystacks/ haystacks_21.xml	UNSAT TT=7.446	? (TO) TT=1800.17	? (TO) TT=1800.24	? (TO) TT=1800.03	UNSAT TT=7.446	UNSAT TT=7.643	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800	? (TO) TT=1800.01	Signal TT=0.025	? (TO) TT=1800.23	? (TO) TT=1800.17	? TT=1.08	Signal TT=0.022	Wrong Cert. TT=0.41 CHECKS=1320240 NODES=441	? (TO) TT=1800.02
haystacks/ haystacks_22.xml	UNSAT TT=12.867	? (TO) TT=1800.16	? (TO) TT=1800.16	? (TO) TT=1800.04	UNSAT TT=13.171	UNSAT TT=12.867	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.05	? (TO) TT=1800.05	? (TO) TT=1800.08	Signal TT=0.023	? (TO) TT=1800.2	? (TO) TT=1800.18	? TT=0.838	Signal TT=0.026	Wrong Cert. TT=0.525 CHECKS=1649300 NODES=484	? (TO) TT=1800.07
haystacks/ haystacks_23.xml	UNSAT TT=10.451	? (TO) TT=1800.21	? (TO) TT=1800.15	? (TO) TT=1800.07	UNSAT TT=10.513	UNSAT TT=10.451	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=0.033	Signal TT=0.023	? (TO) TT=1800.22	? (TO) TT=1800.22	? TT=1.28	Signal TT=0.026	Wrong Cert. TT=0.635 CHECKS=2040740 NODES=529	? (TO) TT=1800.08
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
haystacks/ haystacks_24.xml	UNSAT TT=11.423	? (TO) TT=1800.2	? (TO) TT=1800.18	? (TO) TT=1800.02	UNSAT TT=11.423	UNSAT TT=11.633	? (TO) TT=1800.03	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=0.03	Signal TT=0.027	? (TO) TT=1800.24	? (TO) TT=1800.25	? TT=1.33	Signal TT=0.035	Wrong Cert. TT=0.797 CHECKS=2503000 NODES=576	? (TO) TT=1800.05
haystacks/ haystacks_25.xml	UNSAT TT=34.143	? (TO) TT=1800.23	? (TO) TT=1800.16	? (TO) TT=1800.03	UNSAT TT=34.143	UNSAT TT=34.588	? (TO) TT=1800.05	? (TO) TT=1800	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.02	? (TO) TT=1800.05	Signal TT=0.029	? (TO) TT=1800.21	? (TO) TT=1800.27	? TT=1.45	Signal TT=0.028	Wrong Cert. TT=0.936 CHECKS=3045250 NODES=625	? (TO) TT=1800.05
haystacks/ haystacks_26.xml	UNSAT TT=31.523	? (TO) TT=1800.21	? (TO) TT=1800.2	? (TO) TT=1800.11	UNSAT TT=31.68	UNSAT TT=31.523	? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=14.25	? (TO) TT=0.029	Signal TT=0.037	? (TO) TT=1800.15	? (TO) TT=1800.22	? TT=1.51	Signal TT=0.031	Wrong Cert. TT=1.16 CHECKS=3677420 NODES=676	? (TO) TT=1800.04
haystacks/ haystacks_27.xml	UNSAT TT=40.124	? (TO) TT=1800.17	? (TO) TT=1800.2	? (TO) TT=1800.11	UNSAT TT=40.124	UNSAT TT=40.538	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=16.136	? (TO) TT=1800	Signal TT=0.031	? (TO) TT=1800.24	? (TO) TT=1800.18	? TT=1.092	Signal TT=0.031	Wrong Cert. TT=1.368 CHECKS=4410250 NODES=729	? (TO) TT=1800.08
haystacks/ haystacks_28.xml	UNSAT TT=77.523	? (TO) TT=1800.22	? (TO) TT=1800.16	? (TO) TT=1800.06	UNSAT TT=78.601	UNSAT TT=77.523	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=17.416	? (TO) TT=1800.07	Signal TT=0.031	? (TO) TT=1800.23	? (TO) TT=1800.2	? TT=1.63	Signal TT=0.029	Wrong Cert. TT=1.672 CHECKS=5255260 NODES=784	? (TO) TT=1800.09
haystacks/ haystacks_29.xml	UNSAT TT=55.581	? (TO) TT=1800.23	? (TO) TT=1800.13	? (TO) TT=1800.02	UNSAT TT=55.581	UNSAT TT=56.057	? (TO) TT=1800.08	? (TO) TT=1800.05	? (TO) TT=1800	? (TO) TT=1800.04	? (TO) TT=1800.02	? (TO) TT=1800.1	Signal TT=0.03	? (TO) TT=1800.23	? (TO) TT=1800.18	? TT=1.849	Signal TT=0.032	Wrong Cert. TT=1.967 CHECKS=6224850 NODES=841	? (TO) TT=1800.07
haystacks/ haystacks_30.xml	UNSAT TT=63.761	? (TO) TT=1800.23	? (TO) TT=1800.14	? (TO) TT=1800.04	UNSAT TT=65.449	UNSAT TT=63.761	? (TO) TT=1800.07	? (TO) TT=1800	? (TO) TT=1800.05	? (TO) TT=1800.09	? (MO) TT=22.114	? (TO) TT=0.028	Signal TT=0.035	? (TO) TT=1800.2	? (TO) TT=1800.21	? TT=1.27	Signal TT=0.034	Wrong Cert. TT=2.196 CHECKS=7332270 NODES=900	? (TO) TT=1800.09
haystacks/ haystacks_31.xml	UNSAT TT=115.557	? (TO) TT=1800.23	? (TO) TT=1800.17	? (TO) TT=1800.03	UNSAT TT=115.557	UNSAT TT=120.199	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=26.902	? (TO) TT=1800.04	Signal TT=0.034	? (TO) TT=1800.22	? (TO) TT=1800.34	? TT=1.32	Signal TT=0.035	Wrong Cert. TT=2.638 CHECKS=8591710 NODES=961	? (TO) TT=1800.07
haystacks/ haystacks_32.xml	UNSAT TT=141.815	? (TO) TT=1800.19	? (TO) TT=1800.21	? (TO) TT=1800.11	UNSAT TT=141.815	UNSAT TT=145.243	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.06	? (MO) TT=30.264	? (TO) TT=0.027	Signal TT=0.036	? (TO) TT=1800.22	? (TO) TT=1800.22	? TT=2.13	Signal TT=0.036	Wrong Cert. TT=3.071 CHECKS=10018300 NODES=1024	? (TO) TT=1800.1
haystacks/ haystacks_33.xml	UNSAT TT=186.924	? (TO) TT=1800.22	? (TO) TT=1800.2	? (TO) TT=1800.05	UNSAT TT=189.854	UNSAT TT=186.924	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.02	? (MO) TT=34.521	? (TO) TT=1800.07	Signal TT=0.038	? (TO) TT=1800.25	? (MO) TT=1111.03	? TT=2.17	Signal TT=0.038	Wrong Cert. TT=4.044 CHECKS=11628000 NODES=1089	? (TO) TT=1800.03
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
haystacks/ haystacks_34.xml	UNSAT TT=71.238	? (TO) TT=1800.22	? (TO) TT=1800.19	? (TO) TT=1800.08	UNSAT TT=71.482	UNSAT TT=71.238	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.04	? (TO) TT=1800.06	? (MO) TT=36.62	? (TO) TT=0.029	Signal TT=0.04	? (MO) TT=1106.95	? (MO) TT=530.801	? TT=2.41	Signal TT=0.037	Wrong Cert. TT=4.157 CHECKS=13438100 NODES=1156	? (TO) TT=1800.11
haystacks/ haystacks_35.xml	UNSAT TT=221.558	? (TO) TT=1800.23	? (TO) TT=1800.22	? (TO) TT=1800.09	UNSAT TT=225.851	UNSAT TT=221.558	? (TO) TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.11	? (MO) TT=40.816	? (TO) TT=0.033	Signal TT=0.04	? (TO) TT=1800.19	? (TO) TT=1800.23	? TT=2.517	Signal TT=0.04	Wrong Cert. TT=4.988 CHECKS=15466500 NODES=1225	? (TO) TT=1800.07
haystacks/ haystacks_36.xml	UNSAT TT=542.367	? (TO) TT=1800.23	? (TO) TT=1800.13	? (TO) TT=1800.09	UNSAT TT=542.367	UNSAT TT=544.907	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800	? (TO) TT=1800.04	? (MO) TT=47.072	? (TO) TT=0.029	Signal TT=0.044	? (TO) TT=1800.21	? (MO) TT=202.226	? TT=2.64	Signal TT=0.043	Wrong Cert. TT=5.869 CHECKS=17732500 NODES=1296	? (TO) TT=1800.11
haystacks/ haystacks_37.xml	UNSAT TT=205.687	? (TO) TT=1800.23	? (TO) TT=1800.25	? (TO) TT=1800.08	UNSAT TT=206.637	UNSAT TT=205.687	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.05	? (MO) TT=53.539	? (TO) TT=0.028	Signal TT=0.043	? TT=288.357	? (MO) TT=1346.21	? TT=3.015	Signal TT=0.043	Wrong Cert. TT=7.146 CHECKS=20256200 NODES=1369	? (TO) TT=1800.04
haystacks/ haystacks_38.xml	UNSAT TT=538.005	? (TO) TT=1800.13	? (TO) TT=1800.17	? (TO) TT=1800.11	UNSAT TT=538.005	UNSAT TT=554.282	? (TO) TT=1800	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.11	? (MO) TT=57.412	? (TO) TT=0.029	Signal TT=0.046	? (exit code) TT=194.343	? (MO) TT=133.328	? TT=3.27	Signal TT=0.045	Wrong Cert. TT=7.703 CHECKS=23059200 NODES=1444	? (TO) TT=1800.09
haystacks/ haystacks_39.xml	UNSAT TT=333.583	? (TO) TT=1800.24	? (TO) TT=1800.22	? (TO) TT=1800.05	UNSAT TT=349.272	UNSAT TT=333.583	? (TO) TT=1800.04	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.07	? (MO) TT=0	? (TO) TT=0.025	Signal TT=0.046	? (MO) TT=170.378	? (exit code) TT=97.32	? TT=3.201	Signal TT=0.045	Wrong Cert. TT=8.725 CHECKS=26163800 NODES=1521	? (TO) TT=1800.13
haystacks/ haystacks_40.xml	UNSAT TT=641.278	? (TO) TT=1800.21	? (TO) TT=1800.13	? (TO) TT=1800.07	UNSAT TT=677.444	UNSAT TT=641.278	? (TO) TT=1800	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.07	? (MO) TT=0	? (TO) TT=0.026	Signal TT=0.049	? (exit code) TT=170.816	? (exit code) TT=98.966	? TT=3.33	Signal TT=0.047	Wrong Cert. TT=9.442 CHECKS=29593800 NODES=1600	? (TO) TT=1800.11
haystacks/ haystacks_41.xml	UNSAT TT=440.199	? (TO) TT=1800.19	? (TO) TT=1800.24	? (TO) TT=1800.1	UNSAT TT=440.199	UNSAT TT=442.897	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.04	? (MO) TT=0	? (TO) TT=0.031	Signal TT=0.048	? (exit code) TT=174.794	? (exit code) TT=103.337	? TT=3.674	Signal TT=0.05	Wrong Cert. TT=11.271 CHECKS=33374100 NODES=1681	? (TO) TT=1800.12
haystacks/ haystacks_42.xml	UNSAT TT=1252.18	? (TO) TT=1800.15	? (TO) TT=1800.16	? (TO) TT=1800.12	UNSAT TT=1288.63	UNSAT TT=1252.18	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.09	? (TO) TT=1800.03	? (MO) TT=0	? (TO) TT=0.033	Signal TT=0.052	? (MO) TT=180.722	? (MO) TT=66.869	? TT=4.1	Signal TT=0.053	Wrong Cert. TT=12.852 CHECKS=37530800 NODES=1764	? (TO) TT=1800.1
haystacks/ haystacks_43.xml	UNSAT TT=1102.67	? (TO) TT=1800.22	? (TO) TT=1800.18	? (TO) TT=1800.07	UNSAT TT=1102.67	UNSAT TT=1113.68	? (TO) TT=1800	? (TO) TT=1800.1	? (TO) TT=0.007	? (TO) TT=1800.03	? (MO) TT=0	? (TO) TT=0.027	Signal TT=0.052	? (MO) TT=129.319	? (MO) TT=94.323	? TT=4.125	Signal TT=0.055	Wrong Cert. TT=15.284 CHECKS=42091300 NODES=1849	? (TO) TT=1800.15
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
haystacks/ haystacks_44.xml	UNSAT TT=1560.29	? (TO) TT=1800.14	? (TO) TT=1800.24	? (TO) TT=1800.07	UNSAT TT=1560.29	UNSAT TT=1629.93	? (TO) TT=1800.04	? (TO) TT=1800.01	? (TO) TT=1800.01	? (TO) TT=1800.12	? (MO) TT=0	? (TO) TT=0.028	Signal TT=0.055	? (MO) TT=225.151	? (MO) TT=77.517	? TT=4.23	Signal TT=0.057	Wrong Cert. TT=17.255 CHECKS=47084400 NODES=1936	? (TO) TT=1800.08
haystacks/ haystacks_45.xml	UNSAT TT=1658.19	? (TO) TT=1800.17	? (TO) TT=1800.19	? (TO) TT=1800.04	UNSAT TT=1771.17	UNSAT TT=1658.19	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.02	? (TO) TT=1800.03	? (MO) TT=0	? (TO) TT=0.032	Signal TT=0.059	? (MO) TT=354.814	? (MO) TT=59.083	? TT=4.62	Signal TT=0.057	Wrong Cert. TT=19.245 CHECKS=52540000 NODES=2025	? (TO) TT=1800.11
haystacks/ haystacks_46.xml		? (TO) TT=1800.16	? (TO) TT=1800.21	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.08	? TT=658.198	? TT=704.611	? TT=625.792	? (TO) TT=1800.11	? (MO) TT=0	? (TO) TT=0.026	Signal TT=0.059	? (MO) TT=204.07	? (MO) TT=73.827	? TT=4.994	Signal TT=0.058	Wrong Cert. TT=20.581 CHECKS=58489700 NODES=2116	? (TO) TT=1800.09
haystacks/ haystacks_47.xml	UNSAT TT=1061.69	? (TO) TT=1800.22	? (TO) TT=1800.16	? (TO) TT=1800.05	UNSAT TT=1061.69	UNSAT TT=1154.81	? TT=653.764	? TT=637.111	? TT=618.517	? (TO) TT=1800.04	? (MO) TT=0	? (TO) TT=0.027	Signal TT=0.063	? (MO) TT=125.744	? (MO) TT=72.979	? TT=5.245	Signal TT=0.062	Wrong Cert. TT=24.099 CHECKS=64966000 NODES=2209	? (TO) TT=1800.15
haystacks/ haystacks_48.xml	UNSAT TT=1303.77	? (TO) TT=1800.24	? (TO) TT=1800.14	? (TO) TT=1800.08	UNSAT TT=1396.15	UNSAT TT=1303.77	? TT=667.774	? TT=661.729	? TT=616.776	? (TO) TT=1800.12	? (MO) TT=0	? (TO) TT=0.029	Signal TT=0.064	? (MO) TT=156.147	? (MO) TT=91.312	? TT=5.521	Signal TT=0.066	Wrong Cert. TT=24.807 CHECKS=72003300 NODES=2304	? (TO) TT=1800.21
haystacks/ haystacks_49.xml		? (TO) TT=1800.19	? (TO) TT=1800.23	? (TO) TT=1800.12	? (TO) TT=1800.02	? (TO) TT=1800.08	? TT=661.255	? TT=690.675	? TT=617.66	? (TO) TT=1800.07	? (MO) TT=0	? (TO) TT=0.031	Signal TT=0.072	? (MO) TT=158.063	? (MO) TT=69.471	? TT=5.45	Signal TT=0.065	Wrong Cert. TT=30.686 CHECKS=79637200 NODES=2401	? (TO) TT=1800.21
haystacks/ haystacks_50.xml		? (TO) TT=1800.16	? (TO) TT=1800.18	? (TO) TT=1800.14	? (TO) TT=1800.05	? (TO) TT=1800.1	? TT=651.342	? TT=665.473	? TT=645.601	? (TO) TT=1800.1	? (MO) TT=0	? (TO) TT=0.029	Signal TT=0.07	? (MO) TT=9.485	? (MO) TT=68.278	? TT=6.07	Signal TT=0.067	Wrong Cert. TT=29.279 CHECKS=87904900 NODES=2500	? (TO) TT=1800.15
haystacks/ haystacks_51.xml	UNSAT TT=922.179	? (TO) TT=1800.21	? (TO) TT=1800.11	? (TO) TT=1800.09	UNSAT TT=922.179	UNSAT TT=925.091	? TT=671.368	? TT=676.359	? TT=649.297	? (TO) TT=1800.05	? (MO) TT=0	? (TO) TT=0.026	Signal TT=0.072	? TT=228.771	? TT=119.696	? TT=6.445	Signal TT=0.072	Wrong Cert. TT=36.459 CHECKS=96844800 NODES=2601	? (TO) TT=1800.15
haystacks/ haystacks_52.xml		? (TO) TT=1800.22	? (TO) TT=1800.16	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.1	? TT=637.379	? TT=641.448	? TT=656.286	? (TO) TT=1800.12	? (MO) TT=0	? (TO) TT=0.031	Signal TT=0.07	? (MO) TT=135.432	? (MO) TT=73.617	? TT=6.47	Signal TT=0.074	? (MO) TT=28.364	? (MO) TT=27.123
haystacks/ haystacks_53.xml		? (TO) TT=1800.22	? (TO) TT=1800.16	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.1	? TT=649.49	? TT=682.766	? TT=649.593	? (TO) TT=1800.11	? (MO) TT=0	? (TO) TT=0.03	Signal TT=0.073	? (MO) TT=145.612	? (MO) TT=74.299	? TT=7.025	Signal TT=0.075	? (MO) TT=28.352	? (MO) TT=27.937
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
haystacks/ haystacks_54.xml		? (TO) TT=1800.17	? (TO) TT=1800.24	? (TO) TT=1800.12	? (TO) TT=1800.07	? (TO) TT=1800.08	? TT=665.228	? TT=650.314	? TT=642.5	? (TO) TT=1800.06	? (MO) TT=0	? (TO) TT=0.028	Signal TT=0.077	? (MO) TT=164.949	? (MO) TT=104.358	? TT=7.495	Signal TT=0.077	? (MO) TT=29.669	? (MO) TT=27.424

Some statistics...

	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	buggy_2_5 2007-01-08	buggy_2_5_s 2007-01-08	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04	VALCSP 3.0	VALCSP 3.1
Number of times the solver is able to give the best known answer	2	3	4	45	45	3	2	2	8	7	8	0	7	7	0	0	0	0
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	0	0	0	24	18	0	0	0	3	0	0	0	0	0	0	0	0	0