CSP 2006 Competition (second round): results by benchmark

Results by benchmark for category global constraints (GLOBAL), subcategory academic problems

This page displays the results of the different solvers for each benchmark for category global constraints (GLOBAL), subcategory academic problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 109 ESAC
Abscon 109 AC
BPrologCSPSolver70a 2006-12-13
CSP4J - Combo 2006-12-19
CSP4J - MAC 2006-12-19
CSP4J - MAC 2007-01-16
CSPtoSAT+minisat 0.3
galac 1
galacJ beta 1
Mistral 2006-12-04
sat4jCSP 1.7 RC BF3
SAT4JCSP-CACHED 1.7 RC BF3
sugar 0.40
Tramontane 2006-12-04

Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04
latinSquare/ latinSquare-dg-10_all.xml		? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.06	? (TO) TT=1800.03	? (TO) TT=1800.59	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.08
latinSquare/ latinSquare-dg-11_all.xml		? (TO) TT=1800.12	? (TO) TT=1800.14	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.08	? (TO) TT=1800.08
latinSquare/ latinSquare-dg-12_all.xml		? (TO) TT=1800.1	? (TO) TT=1800.17	? (TO) TT=1800.01	? (TO) TT=1800.1	? (TO) TT=1800.01	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.01	? (TO) TT=1800.15	? (TO) TT=1800.12	? (TO) TT=1800.11	? (TO) TT=1800.03
latinSquare/ latinSquare-dg-3_all.xml	UNSAT TT=0.012	UNSAT TT=0.649 CHECKS=0 NODES=0	UNSAT TT=0.6 CHECKS=0 NODES=2	UNSAT TT=0.017	UNSAT TT=0.291 NODES=11	UNSAT TT=0.289 NODES=2	UNSAT TT=0.269 CHECKS=0 NODES=2	UNSAT TT=0.013	UNSAT TT=0.687	UNSAT TT=0.618	UNSAT TT=0.012	UNSAT TT=0.6	UNSAT TT=0.582	UNSAT TT=0.05	UNSAT TT=0.012
latinSquare/ latinSquare-dg-4_all.xml	UNSAT TT=0.012	UNSAT TT=0.65 CHECKS=0 NODES=0	UNSAT TT=0.657 CHECKS=0 NODES=3	UNSAT TT=0.02	UNSAT TT=0.311 NODES=11	UNSAT TT=0.309 NODES=3	UNSAT TT=0.312 CHECKS=0 NODES=3	UNSAT TT=0.013	UNSAT TT=0.757	UNSAT TT=0.701	UNSAT TT=0.014	UNSAT TT=0.59	UNSAT TT=0.612	UNSAT TT=0.319	UNSAT TT=0.012
latinSquare/ latinSquare-dg-5_all.xml	SAT TT=0.013	SAT TT=0.668 CHECKS=20 NODES=0	SAT TT=0.624 CHECKS=20 NODES=25	SAT TT=0.018	SAT TT=0.318 NODES=33	SAT TT=0.282 NODES=25	SAT TT=0.308 CHECKS=0 NODES=25	SAT TT=0.013	SAT TT=1.385	SAT TT=1.241	SAT TT=0.015	SAT TT=0.606	SAT TT=0.571	SAT TT=0.796	SAT TT=0.014
latinSquare/ latinSquare-dg-6_all.xml	UNSAT TT=0.048	UNSAT TT=1.221 CHECKS=0 NODES=3429	UNSAT TT=1.213 CHECKS=0 NODES=3429	UNSAT TT=0.226	UNSAT TT=0.612 NODES=727	UNSAT TT=0.651 NODES=1049	UNSAT TT=0.513 CHECKS=0 NODES=1049	UNSAT TT=0.059	UNSAT TT=1.773	UNSAT TT=1.601	UNSAT TT=0.048	UNSAT TT=1.071	UNSAT TT=1.071	UNSAT TT=0.08	UNSAT TT=0.062
latinSquare/ latinSquare-dg-7_all.xml	SAT TT=0.015	SAT TT=0.741 CHECKS=28 NODES=93	SAT TT=0.733 CHECKS=28 NODES=93	SAT TT=0.021	SAT TT=0.645 NODES=138	SAT TT=0.627 NODES=122	SAT TT=0.609 CHECKS=0 NODES=127	SAT TT=0.019	SAT TT=5.176	SAT TT=6.697	SAT TT=0.017	SAT TT=0.779	SAT TT=0.841	SAT TT=0.761	SAT TT=0.015
latinSquare/ latinSquare-dg-8_all.xml		? (TO) TT=1800.07	? (TO) TT=1800.14	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.03	? (TO) TT=0.005	? (TO) TT=1800.09	? (TO) TT=1800.08	UNSAT TT=883.55	? (TO) TT=1800.08	? (TO) TT=1800.18	? (TO) TT=1800.15	? (TO) TT=1800.05	? (TO) TT=1800.11
latinSquare/ latinSquare-dg-9_all.xml		? (TO) TT=1800.11	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.03	? (TO) TT=1800.06	? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) TT=1800.1	? (TO) TT=1800.02
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04
pigeons_glb/ pigeons-10_ext.xml	UNSAT TT=0.011	UNSAT TT=0.604 CHECKS=0 NODES=0	UNSAT TT=0.592 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.278 NODES=0	UNSAT TT=0.261 NODES=0	UNSAT TT=0.259 CHECKS=0 NODES=0	UNSAT TT=4.038	? (TO) TT=1800.06	? (TO) TT=1800.09	UNSAT TT=0.014	UNSAT TT=24.483	UNSAT TT=25.689	UNSAT TT=0.715	UNSAT TT=0.011
pigeons_glb/ pigeons-11_ext.xml	UNSAT TT=0.01	UNSAT TT=0.631 CHECKS=0 NODES=0	UNSAT TT=0.601 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.292 NODES=0	UNSAT TT=0.262 NODES=0	UNSAT TT=0.27 CHECKS=0 NODES=0	UNSAT TT=47.055	? (TO) TT=1800.07	? (TO) TT=1800.1	UNSAT TT=0.012	UNSAT TT=910.415	UNSAT TT=1018.04	UNSAT TT=2.33	UNSAT TT=0.01
pigeons_glb/ pigeons-12_ext.xml	UNSAT TT=0.011	UNSAT TT=0.648 CHECKS=0 NODES=0	UNSAT TT=0.588 CHECKS=0 NODES=0	UNSAT TT=0.02	UNSAT TT=0.282 NODES=0	UNSAT TT=0.256 NODES=0	UNSAT TT=0.277 CHECKS=0 NODES=0	UNSAT TT=590.142	? (TO) TT=1800.05	? (TO) TT=1800.1	UNSAT TT=0.012	? (TO) TT=1800.12	? (TO) TT=1800.07	UNSAT TT=13.28	UNSAT TT=0.011
pigeons_glb/ pigeons-13_ext.xml	UNSAT TT=0.011	UNSAT TT=0.638 CHECKS=0 NODES=0	UNSAT TT=0.604 CHECKS=0 NODES=0	UNSAT TT=0.019	UNSAT TT=0.265 NODES=0	UNSAT TT=0.264 NODES=0	UNSAT TT=0.258 CHECKS=0 NODES=0	? (TO) TT=1800.03	? (TO) TT=1800	? (TO) TT=1800.06	UNSAT TT=0.012	? (TO) TT=1800.74	? (TO) TT=1800.78	UNSAT TT=71.07	UNSAT TT=0.011
pigeons_glb/ pigeons-14_ext.xml	UNSAT TT=0.011	UNSAT TT=0.614 CHECKS=0 NODES=0	UNSAT TT=0.633 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.278 NODES=0	UNSAT TT=0.262 NODES=0	UNSAT TT=0.282 CHECKS=0 NODES=0	? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.06	UNSAT TT=0.011	? (TO) TT=1800.14	? (TO) TT=1800.09	UNSAT TT=646.748	UNSAT TT=0.011
pigeons_glb/ pigeons-15_ext.xml	UNSAT TT=0.012	UNSAT TT=0.619 CHECKS=0 NODES=0	UNSAT TT=0.632 CHECKS=0 NODES=0	UNSAT TT=0.02	UNSAT TT=0.272 NODES=0	UNSAT TT=0.265 NODES=0	UNSAT TT=0.259 CHECKS=0 NODES=0	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.08	UNSAT TT=0.013	? (TO) TT=1800.11	? (TO) TT=1800.2	? (TO) TT=1800.02	UNSAT TT=0.012
pigeons_glb/ pigeons-18_ext.xml	UNSAT TT=0.011	UNSAT TT=0.617 CHECKS=0 NODES=0	UNSAT TT=0.601 CHECKS=0 NODES=0	UNSAT TT=0.019	UNSAT TT=0.269 NODES=0	UNSAT TT=0.267 NODES=0	UNSAT TT=0.282 CHECKS=0 NODES=0	? (TO) TT=1800.05	? (TO) TT=1800.06	? (TO) TT=1800.03	UNSAT TT=0.013	? (TO) TT=1800.12	? (TO) TT=1800.14	? (TO) TT=1800.07	UNSAT TT=0.011
pigeons_glb/ pigeons-20_ext.xml	UNSAT TT=0.01	UNSAT TT=0.622 CHECKS=0 NODES=0	UNSAT TT=0.619 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.3 NODES=0	UNSAT TT=0.269 NODES=0	UNSAT TT=0.283 CHECKS=0 NODES=0	? (TO) TT=1800.04	? (MO) TT=0.028	? (MO) TT=0.029	UNSAT TT=0.01	? (TO) TT=1800.18	? (TO) TT=1800.14	? (TO) TT=1800	UNSAT TT=0.011
pigeons_glb/ pigeons-25_ext.xml	UNSAT TT=0.013	UNSAT TT=0.617 CHECKS=0 NODES=0	UNSAT TT=0.602 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.304 NODES=0	UNSAT TT=0.301 NODES=0	UNSAT TT=0.288 CHECKS=0 NODES=0	? (TO) TT=1800.05	? (MO) TT=0.033	? (MO) TT=0.03	UNSAT TT=0.013	? (TO) TT=1800.2	? (TO) TT=1800.11	? (TO) TT=1800.01	UNSAT TT=0.014
pigeons_glb/ pigeons-30_ext.xml	UNSAT TT=0.012	UNSAT TT=0.666 CHECKS=0 NODES=0	UNSAT TT=0.61 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.32 NODES=0	UNSAT TT=0.304 NODES=0	UNSAT TT=0.323 CHECKS=0 NODES=0	? (TO) TT=1800.05	? (MO) TT=0.027	? (MO) TT=0.029	UNSAT TT=0.012	? (TO) TT=1800.61	? (TO) TT=1800.17	? (TO) TT=1800.14	UNSAT TT=0.014
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04
pigeons_glb/ pigeons-35_ext.xml	UNSAT TT=0.011	UNSAT TT=0.658 CHECKS=0 NODES=0	UNSAT TT=0.606 CHECKS=0 NODES=0	UNSAT TT=0.019	UNSAT TT=0.329 NODES=0	UNSAT TT=0.322 NODES=0	UNSAT TT=0.315 CHECKS=0 NODES=0	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.1	UNSAT TT=0.012	? (TO) TT=1800.16	? (TO) TT=1800.17	? (TO) TT=1800.08	UNSAT TT=0.011
pigeons_glb/ pigeons-40_ext.xml	UNSAT TT=0.013	UNSAT TT=0.676 CHECKS=0 NODES=0	UNSAT TT=0.663 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.353 NODES=0	UNSAT TT=0.352 NODES=0	UNSAT TT=0.318 CHECKS=0 NODES=0	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.05	UNSAT TT=0.013	? (TO) TT=1800.19	? (TO) TT=1800.15	? (TO) TT=1800.1	UNSAT TT=0.014
pigeons_glb/ pigeons-45_ext.xml	UNSAT TT=0.012	UNSAT TT=0.676 CHECKS=0 NODES=0	UNSAT TT=0.655 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.378 NODES=0	UNSAT TT=0.378 NODES=0	UNSAT TT=0.308 CHECKS=0 NODES=0	? (TO) TT=1800.1	? (MO) TT=0.029	? (MO) TT=0.028	UNSAT TT=0.016	? (TO) TT=1800.12	? (TO) TT=1800.18	? (TO) TT=1800.08	UNSAT TT=0.012
pigeons_glb/ pigeons-50_ext.xml	UNSAT TT=0.014	UNSAT TT=0.649 CHECKS=0 NODES=0	UNSAT TT=0.622 CHECKS=0 NODES=0	UNSAT TT=0.018	UNSAT TT=0.406 NODES=0	UNSAT TT=0.392 NODES=0	UNSAT TT=0.32 CHECKS=0 NODES=0	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.01	UNSAT TT=0.014	? (TO) TT=1800.15	? (TO) TT=1800.26	? (TO) TT=1800.07	UNSAT TT=0.014
pigeons_glb/ pigeons-5_ext.xml	UNSAT TT=0.011	UNSAT TT=0.603 CHECKS=0 NODES=0	UNSAT TT=0.606 CHECKS=0 NODES=0	UNSAT TT=0.017	UNSAT TT=0.257 NODES=0	UNSAT TT=0.277 NODES=0	UNSAT TT=0.265 CHECKS=0 NODES=0	UNSAT TT=0.012	UNSAT TT=0.488	Wrong Cert. TT=0.891	UNSAT TT=0.012	UNSAT TT=0.534	UNSAT TT=0.532	UNSAT TT=0.22	UNSAT TT=0.011
pigeons_glb/ pigeons-6_ext.xml	UNSAT TT=0.011	UNSAT TT=0.64 CHECKS=0 NODES=0	UNSAT TT=0.629 CHECKS=0 NODES=0	UNSAT TT=0.017	UNSAT TT=0.267 NODES=0	UNSAT TT=0.266 NODES=0	UNSAT TT=0.273 CHECKS=0 NODES=0	UNSAT TT=0.013	UNSAT TT=0.523	Wrong Cert. TT=0.933	UNSAT TT=0.014	UNSAT TT=0.621	UNSAT TT=0.635	UNSAT TT=0.237	UNSAT TT=0.011
pigeons_glb/ pigeons-7_ext.xml	UNSAT TT=0.013	UNSAT TT=0.656 CHECKS=0 NODES=0	UNSAT TT=0.614 CHECKS=0 NODES=0	UNSAT TT=0.019	UNSAT TT=0.27 NODES=0	UNSAT TT=0.271 NODES=0	UNSAT TT=0.273 CHECKS=0 NODES=0	UNSAT TT=0.027	UNSAT TT=0.741	Wrong Cert. TT=1.055	UNSAT TT=0.013	UNSAT TT=0.808	UNSAT TT=0.786	UNSAT TT=0.057	UNSAT TT=0.013
pigeons_glb/ pigeons-8_ext.xml	UNSAT TT=0.011	UNSAT TT=0.597 CHECKS=0 NODES=0	UNSAT TT=0.598 CHECKS=0 NODES=0	UNSAT TT=0.017	UNSAT TT=0.307 NODES=0	UNSAT TT=0.287 NODES=0	UNSAT TT=0.289 CHECKS=0 NODES=0	UNSAT TT=0.069	UNSAT TT=4.047	Wrong Cert. TT=4.459	UNSAT TT=0.011	UNSAT TT=1.026	UNSAT TT=1.042	UNSAT TT=0.274	UNSAT TT=0.011
pigeons_glb/ pigeons-9_ext.xml	UNSAT TT=0.011	UNSAT TT=0.63 CHECKS=0 NODES=0	UNSAT TT=0.624 CHECKS=0 NODES=0	UNSAT TT=0.017	UNSAT TT=0.791 NODES=0	UNSAT TT=0.838 NODES=0	UNSAT TT=0.816 CHECKS=0 NODES=0	UNSAT TT=0.388	UNSAT TT=89.927	Wrong Cert. TT=95.416	UNSAT TT=0.011	UNSAT TT=2.875	UNSAT TT=2.845	UNSAT TT=0.166	UNSAT TT=0.012
zhou/ protein.xml	SAT TT=10.836	? (TO) TT=1800.13	SAT TT=639.803 CHECKS=756011370 NODES=518928	SAT TT=10.836	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.03	? (TO) TT=1800.08	? (TO) TT=1800.09	Signal TT=0.02	SAT TT=1255.82	? TT=17.208	SAT TT=32.81	Signal TT=0.021
Bench name	Best results on this instance	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04
zhou/ queens2.xml	SAT TT=0.084	SAT TT=7.705 CHECKS=7301938 NODES=1215	SAT TT=4.928 CHECKS=4881221 NODES=1165	SAT TT=0.084	SAT TT=191.614 NODES=342	SAT TT=193.608 NODES=310	SAT TT=84.812 CHECKS=63769 NODES=310	? (TO) TT=1800.1	? (MO) TT=0.027	? (MO) TT=0.036	Signal TT=0.044	SAT TT=17.454	SAT TT=18.515	SAT TT=9.08	Signal TT=0.044
zhou/ sudoku368.xml	SAT TT=0.112	SAT TT=15.468 CHECKS=75 NODES=3512	SAT TT=3.532 CHECKS=75 NODES=5254	SAT TT=0.112	SAT TT=18.764 NODES=4489	SAT TT=298.25 NODES=64991	? (TO) TT=1800.07	SAT TT=0.129	? (TO) TT=1800.09	? (TO) TT=1800.01	SAT TT=0.227	SAT TT=3.091	SAT TT=3.009	SAT TT=3.33	SAT TT=0.227

Some statistics...

	Abscon 109 ESAC	Abscon 109 AC	BPrologCSPSolver70a 2006-12-13	CSP4J - Combo 2006-12-19	CSP4J - MAC 2006-12-19	CSP4J - MAC 2007-01-16	CSPtoSAT+minisat 0.3	galac 1	galacJ beta 1	Mistral 2006-12-04	sat4jCSP 1.7 RC BF3	SAT4JCSP-CACHED 1.7 RC BF3	sugar 0.40	Tramontane 2006-12-04
Number of times the solver is able to give the best known answer	26	27	27	26	26	25	14	10	0	25	15	14	18	25
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	0	0	3	0	0	0	1	0	0	11	0	0	0	17