CSP 2009 Competition: results by benchmark

Results by benchmark for category alldiff (Alldiff), subcategory problems with patterns

This page displays the results of the different solvers for each benchmark for category alldiff (Alldiff), subcategory problems with patterns

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 112v4 AC (complete)
Abscon 112v4 ESAC (complete)
bpsolver 09 (complete)
Choco2.1.1 2009-06-10 (complete)
Choco2.1.1b 2009-07-16 (complete)
Concrete 2009-07-14 (complete)
Concrete DC 2009-07-14 (complete)
Mistral 1.545 (complete)
SAT4J CSP 2.1.1 (complete)
Sugar v1.14.6+minisat (complete)
Sugar v1.14.6+picosat (complete)

Bench name	Best results on this instance	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-17_glb.xml	SAT TT=0.013	SAT TT=0.792	SAT TT=0.869	SAT TT=0.05	SAT TT=0.919	SAT TT=0.873	SAT TT=0.949	SAT TT=6.287	SAT TT=0.013	SAT TT=1.068	SAT TT=1.108	SAT TT=1.148
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-35_glb.xml	SAT TT=0.022	SAT TT=0.766	SAT TT=0.8	SAT TT=0.042	SAT TT=0.892	SAT TT=0.831	SAT TT=0.943	SAT TT=1.473	SAT TT=0.022	SAT TT=0.808	SAT TT=1.136	SAT TT=1.236
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-52_glb.xml	SAT TT=0.018	SAT TT=0.813	SAT TT=0.884	SAT TT=0.079	SAT TT=0.949	SAT TT=0.895	SAT TT=0.944	SAT TT=1.69	SAT TT=0.018	SAT TT=1.035	SAT TT=1.169	SAT TT=1.273
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-56_glb.xml	SAT TT=0.042	SAT TT=0.855	SAT TT=0.848	SAT TT=0.047	SAT TT=0.948	SAT TT=0.989	SAT TT=1.005	SAT TT=1.583	SAT TT=0.042	SAT TT=1.13	SAT TT=1.163	SAT TT=1.229
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-57_glb.xml	SAT TT=0.032	SAT TT=0.818	SAT TT=0.926	SAT TT=0.047	SAT TT=0.772	SAT TT=0.794	SAT TT=0.982	SAT TT=2.124	SAT TT=0.032	SAT TT=0.902	SAT TT=1.17	SAT TT=1.433
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-63_glb.xml	SAT TT=0.014	SAT TT=0.871	SAT TT=0.955	SAT TT=0.045	SAT TT=0.707	SAT TT=0.731	SAT TT=1.016	SAT TT=2.493	SAT TT=0.014	SAT TT=0.917	SAT TT=1.134	SAT TT=1.203
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-78_glb.xml	SAT TT=0.024	SAT TT=0.827	SAT TT=0.809	SAT TT=0.044	SAT TT=0.524	SAT TT=0.558	SAT TT=0.961	SAT TT=1.225	SAT TT=0.024	SAT TT=1.217	SAT TT=1.143	SAT TT=1.181
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-79_glb.xml	SAT TT=0.03	SAT TT=0.811	SAT TT=0.891	SAT TT=0.045	SAT TT=0.819	SAT TT=0.846	SAT TT=0.885	SAT TT=3.237	SAT TT=0.03	SAT TT=0.867	SAT TT=1.272	SAT TT=1.154
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-83_glb.xml	SAT TT=0.023	SAT TT=0.864	SAT TT=0.892	SAT TT=0.045	SAT TT=0.843	SAT TT=0.879	SAT TT=0.925	SAT TT=1.515	SAT TT=0.023	SAT TT=1.233	SAT TT=1.172	SAT TT=1.225
csp/bqwh-15-106_glb/ normalized-bqwh-15-106-90_glb.xml	SAT TT=0.034	SAT TT=0.852	SAT TT=0.873	SAT TT=0.044	SAT TT=0.836	SAT TT=0.797	SAT TT=0.974	SAT TT=2.422	SAT TT=0.034	SAT TT=0.85	SAT TT=1.087	SAT TT=1.128
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-17_glb.xml	SAT TT=0.064	SAT TT=1.115	SAT TT=1.089	SAT TT=0.07	SAT TT=0.712	SAT TT=0.746	SAT TT=1.721	SAT TT=5.679	SAT TT=0.064	SAT TT=1.345	SAT TT=1.254	SAT TT=1.326
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-21_glb.xml	SAT TT=0.145	SAT TT=1.119	SAT TT=0.887	SAT TT=0.668	SAT TT=0.904	SAT TT=0.891	SAT TT=1.155	SAT TT=5.472	SAT TT=0.145	SAT TT=1.25	SAT TT=1.279	SAT TT=1.307
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-23_glb.xml	SAT TT=0.056	SAT TT=1.086	SAT TT=1.004	SAT TT=0.056	SAT TT=0.715	SAT TT=0.688	SAT TT=1.063	SAT TT=2.698	SAT TT=0.374	SAT TT=1.201	SAT TT=1.488	SAT TT=1.347
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-38_glb.xml	SAT TT=0.272	SAT TT=1.076	SAT TT=1.458	SAT TT=0.272	SAT TT=1.099	SAT TT=1.182	SAT TT=1.013	SAT TT=2.792	SAT TT=0.804	SAT TT=1.328	SAT TT=1.223	SAT TT=1.363
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-5_glb.xml	SAT TT=0.117	SAT TT=2.199	SAT TT=1.016	SAT TT=0.322	SAT TT=1.12	SAT TT=1.131	SAT TT=1.521	SAT TT=2.718	SAT TT=0.117	SAT TT=1.412	SAT TT=1.272	SAT TT=1.602
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-62_glb.xml	SAT TT=0.058	SAT TT=1.389	SAT TT=0.991	SAT TT=0.058	SAT TT=1.14	SAT TT=1.071	SAT TT=1.228	SAT TT=2.525	SAT TT=0.095	SAT TT=1.18	SAT TT=1.392	SAT TT=1.355
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-65_glb.xml	SAT TT=0.135	SAT TT=0.836	SAT TT=1.224	SAT TT=0.135	SAT TT=0.735	SAT TT=0.716	SAT TT=2.233	SAT TT=4.695	SAT TT=0.684	SAT TT=1.188	SAT TT=1.417	SAT TT=1.337
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-73_glb.xml	SAT TT=0.218	SAT TT=1.394	SAT TT=1.452	SAT TT=0.711	SAT TT=1.244	SAT TT=1.224	SAT TT=2.032	SAT TT=3.432	SAT TT=0.218	SAT TT=1.782	SAT TT=1.372	SAT TT=1.598
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-74_glb.xml	SAT TT=0.027	SAT TT=1.111	SAT TT=0.994	SAT TT=4.763	SAT TT=1.123	SAT TT=1.036	SAT TT=1.378	SAT TT=2.394	SAT TT=0.027	SAT TT=1.422	SAT TT=1.284	SAT TT=1.327
csp/bqwh-18-141_glb/ normalized-bqwh-18-141-76_glb.xml	SAT TT=0.181	SAT TT=0.979	SAT TT=1.611	SAT TT=0.181	SAT TT=1.165	SAT TT=1.106	SAT TT=1.007	SAT TT=4.259	SAT TT=0.545	SAT TT=1.24	SAT TT=1.303	SAT TT=1.568

Statistics		Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Concrete 2009-07-14 (complete)	Concrete DC 2009-07-14 (complete)	Mistral 1.545 (complete)	SAT4J CSP 2.1.1 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
Number of times the solver is able to give the best known answer		20	20	20	20	20	20	20	20	20	20	20
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	0	5	0	0	0	0	15	0	0	0