CSP 2009 Competition: results by benchmark

Results by benchmark for category alldiff,element,weightedsum (Alldiff+Elt+WSum), subcategory boolean problems

This page displays the results of the different solvers for each benchmark for category alldiff,element,weightedsum (Alldiff+Elt+WSum), subcategory boolean problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver gave a definitive answer (SAT or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Abscon 112v4 AC (complete)
Abscon 112v4 ESAC (complete)
bpsolver 09 (complete)
Choco2.1.1 2009-06-10 (complete)
Choco2.1.1b 2009-07-16 (complete)
Mistral 1.545 (complete)
Sugar v1.14.6+minisat (complete)
Sugar v1.14.6+picosat (complete)

Bench name	Best results on this instance	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-37s-smv.xml	UNSAT TT=3.002	? TT=0.81	? TT=0.796	UNSAT TT=206.444	UNSAT TT=69.072	? (TO) TT=1800.11	UNSAT TT=491.771	UNSAT TT=3.002	UNSAT TT=3.569
csp/pseudoGLB/ normalized-army8-9bt.xml	SAT TT=0.574	? TT=0.577	? TT=0.605	SAT TT=0.574	SAT TT=2.4	SAT TT=2.441	SAT TT=0.758	SAT TT=2.278	SAT TT=3.358
csp/pseudoGLB/ normalized-cache-inv14-ucl.xml	UNSAT TT=1725.03	? TT=2.527	? TT=2.506	? (TO) TT=1800.1	? (problem) TT=40.836	? (problem) TT=39.23	? (TO) TT=1800.09	UNSAT TT=1725.03	? (MO) TT=19.29
csp/pseudoGLB/ normalized-chnl10-15-pb-cnf-cr.xml	UNSAT TT=2.3	UNSAT TT=2.3	UNSAT TT=2.415	? (TO) TT=1800.04	? (TO) TT=1800.09	? (TO) TT=1800.15	? (TO) TT=1800.02	UNSAT TT=9.823	UNSAT TT=16.16
csp/pseudoGLB/ normalized-chnl20-30-pb-cnf-cr.xml		? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) TT=1800.1	? (TO) TT=1800.08	? (TO) TT=1800.04
csp/pseudoGLB/ normalized-chnl30-31-pb-cnf-cr.xml		? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) TT=1800.17	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.09	? (TO) TT=1800.03
csp/pseudoGLB/ normalized-chnl30-35-pb-cnf-cr.xml		? (TO) TT=1800.13	? (TO) TT=1800.14	? (TO) TT=1800.07	? (TO) TT=1800.15	? (TO) TT=1800.13	? (TO) TT=1800.06	? (TO) TT=1800.1	? (TO) TT=1800.08
csp/pseudoGLB/ normalized-chnl35-36-pb-cnf-cr.xml		? (TO) TT=1800.13	? (TO) TT=1800.13	? (TO) TT=1800.05	? (TO) TT=1800.14	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.1
csp/pseudoGLB/ normalized-chnl35-45-pb-cnf-cr.xml		? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.04	? (TO) TT=1800.01	? (TO) TT=1800.06
csp/pseudoGLB/ normalized-chnl40-45-pb-cnf-cr.xml		? (TO) TT=1800.07	? (TO) TT=1800.16	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.12	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.08
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-dbst-v30-e350-d15-mw10-2.PB06.xml	SAT TT=24.969	? (MO) TT=22.169	? (MO) TT=21.684	SAT TT=202.118	? (problem) TT=47.65	? (problem) TT=53.111	SAT TT=24.969	? (MO) TT=215.047	? (MO) TT=6.308
csp/pseudoGLB/ normalized-dbst-v30-e350-d15-mw10-3.PB06.xml	SAT TT=25.054	? (MO) TT=22.46	? (MO) TT=22.758	? (TO) TT=1800.08	? (problem) TT=47.894	? (problem) TT=52.172	SAT TT=25.054	? (MO) TT=207.06	? (MO) TT=5.97
csp/pseudoGLB/ normalized-dlx1c-rwmem-ucl.xml	UNSAT TT=11.197	? TT=1.024	? TT=0.996	? (TO) TT=1800.05	? (TO) TT=1800.1	? (TO) TT=1800.14	? (TO) TT=1800.05	UNSAT TT=11.197	UNSAT TT=11.901
csp/pseudoGLB/ normalized-elf-rf7-ucl.xml	UNSAT TT=0.313	? TT=0.757	? TT=0.71	? (TO) TT=1800.1	UNSAT TT=3.626	UNSAT TT=4.999	UNSAT TT=0.313	UNSAT TT=2.706	UNSAT TT=2.934
csp/pseudoGLB/ normalized-fpga10-8-sat-pb-cnf-cr.xml	SAT TT=0.918	SAT TT=7.551	SAT TT=7.449	SAT TT=1332.21	SAT TT=34.03	SAT TT=5.379	SAT TT=13.374	SAT TT=0.918	SAT TT=0.958
csp/pseudoGLB/ normalized-fpga12-12-sat-pb-cnf-cr.xml	SAT TT=1.123	SAT TT=723.719	SAT TT=55.715	SAT TT=895.711	? (TO) TT=1800.1	? (TO) TT=1800.09	SAT TT=246.517	SAT TT=5.458	SAT TT=1.123
csp/pseudoGLB/ normalized-fpga15-15-sat-pb-cnf-cr.xml	SAT TT=1.677	? (TO) TT=1800.08	? (TO) TT=1800.05	SAT TT=903.454	? (TO) TT=1800.06	? (TO) TT=1800.15	? (TO) TT=1800.02	? (TO) TT=1800.05	SAT TT=1.677
csp/pseudoGLB/ normalized-fpga20-20-sat-pb-cnf-cr.xml	SAT TT=12.243	? (TO) TT=1800.11	? (TO) TT=1800.14	SAT TT=896.36	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.03	? (TO) TT=1800.05	SAT TT=12.243
csp/pseudoGLB/ normalized-fpga25-23-sat-pb-cnf-cr.xml	SAT TT=4.397	? (TO) TT=1800.09	? (TO) TT=1800.12	? (TO) TT=1800.09	? (TO) TT=1800.04	? (TO) TT=1800.06	? (TO) TT=1800.01	? (TO) TT=1800.01	SAT TT=4.397
csp/pseudoGLB/ normalized-fpga25-24-sat-pb-cnf-cr.xml	SAT TT=5.562	? (TO) TT=1800.11	? (TO) TT=1800.14	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.14	? (TO) TT=1800.08	? (TO) TT=1800.02	SAT TT=5.562
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-fpga25-25-sat-pb-cnf-cr.xml	SAT TT=5.686	? (TO) TT=1800.15	? (TO) TT=1800.14	SAT TT=1346.99	? (TO) TT=1800.13	? (TO) TT=1800.14	? (TO) TT=1800.02	? (TO) TT=1800.06	SAT TT=5.686
csp/pseudoGLB/ normalized-fpga35-35-sat-pb-cnf-cr.xml	SAT TT=16.278	? (TO) TT=1800.11	? (TO) TT=1800.06	SAT TT=1340.83	? (TO) TT=1800.11	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.02	SAT TT=16.278
csp/pseudoGLB/ normalized-fpga40-38-sat-pb-cnf-cr.xml	SAT TT=31.385	? (TO) TT=1800.12	? (TO) TT=1800.15	? (TO) TT=1800.11	? (TO) TT=1800.14	? (TO) TT=1800.12	? (TO) TT=1800.08	? (TO) TT=1800.05	SAT TT=31.385
csp/pseudoGLB/ normalized-fpga40-40-sat-pb-cnf-cr.xml	SAT TT=37.467	? (TO) TT=1800.09	? (TO) TT=1800.14	SAT TT=1349.78	? (TO) TT=1800.14	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.2	SAT TT=37.467
csp/pseudoGLB/ normalized-fpga45-45-sat-pb-cnf-cr.xml	SAT TT=104.213	? (TO) TT=1800.13	? (TO) TT=1800.08	SAT TT=1348.34	? (TO) TT=1800.05	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.15	SAT TT=104.213
csp/pseudoGLB/ normalized-ooo-burch-dill-3-accl-ucl.xml	UNSAT TT=10.377	? TT=0.94	? TT=0.867	? (TO) TT=1800.09	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.04	UNSAT TT=10.377	UNSAT TT=12.515
csp/pseudoGLB/ normalized-ooo-rf6-ucl.xml	UNSAT TT=1.527	? TT=0.769	? TT=0.774	? (TO) TT=1800.09	UNSAT TT=7.336	UNSAT TT=23.642	UNSAT TT=1.527	UNSAT TT=2.67	UNSAT TT=2.7
csp/pseudoGLB/ normalized-ooo-tag8-ucl.xml	UNSAT TT=5.947	? TT=0.9	? TT=0.766	? (TO) TT=1800.03	? (TO) TT=1800.15	? (TO) TT=1800.09	? (TO) TT=1800.1	UNSAT TT=5.947	UNSAT TT=6.05
csp/pseudoGLB/ normalized-pigeon-cardinality-41-40.xml		? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.15	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.09
csp/pseudoGLB/ normalized-pigeon-cardinality-65-60.xml		? (TO) TT=1800.13	? (TO) TT=1800.13	? (TO) TT=1800.04	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) TT=1800.07	? (MO) TT=1244.22	? (TO) TT=1800.08
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-pigeon-cardinality-71-70.xml		? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.06	? (TO) TT=1800.14	? (TO) TT=1800.05	? (TO) TT=1800.1	? (TO) TT=1800	? (TO) TT=1800.05
csp/pseudoGLB/ normalized-pigeon-clauses-15-10.xml	UNSAT TT=1.283	UNSAT TT=1.283	UNSAT TT=1.37	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.15	UNSAT TT=973.081	UNSAT TT=214.948	UNSAT TT=716.099
csp/pseudoGLB/ normalized-pigeon-clauses-25-20.xml	UNSAT TT=54.054	UNSAT TT=902.964	UNSAT TT=54.054	? (TO) TT=1800.05	? (TO) TT=1800.14	? (TO) TT=1800.23	? (TO) TT=1800.03	? (TO) TT=1800.01	? (TO) TT=1800.06
csp/pseudoGLB/ normalized-pigeon-clauses-31-30.xml		? (TO) TT=1800.08	? (TO) TT=1800.14	? (TO) TT=1800.01	? (TO) TT=1800.08	? (TO) TT=1800.14	? (TO) TT=1800.07	? (TO) TT=1800.05	? (TO) TT=1800.04
csp/pseudoGLB/ normalized-pigeon-clauses-51-50.xml		? (TO) TT=1800.39	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.13	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.1
csp/pseudoGLB/ normalized-pigeon-clauses-71-70.xml		? (MO) TT=11.741	? (MO) TT=9.424	? (TO) TT=1800.08	? (TO) TT=1800.22	? (TO) TT=1800.13	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.03
csp/pseudoGLB/ normalized-pigeon-clauses-91-90.xml		? (MO) TT=19.033	? (MO) TT=20.137	? (TO) TT=1800.06	? (problem) TT=92.013	? (problem) TT=87.672	? (TO) TT=1800.08	? (MO) TT=6	? (MO) TT=6.26
csp/pseudoGLB/ normalized-pigeon-clauses-95-90.xml		? (MO) TT=20.487	? (MO) TT=21.909	? (TO) TT=1800.14	? (problem) TT=96.745	? (problem) TT=56.501	? (TO) TT=1800.08	? (MO) TT=6.302	? (MO) TT=6.26
csp/pseudoGLB/ normalized-ppp:1-3-13-19.xml	SAT TT=5.007	? (TO) TT=1800.15	? (TO) TT=1800.12	? (TO) TT=1800.06	SAT TT=941.64	SAT TT=805.788	SAT TT=5.007	SAT TT=73.461	SAT TT=12.179
csp/pseudoGLB/ normalized-reduced-prob2.xml	SAT TT=0.009	? (TO) TT=1800.09	? (TO) TT=1800.06	SAT TT=0.037	SAT TT=0.527	SAT TT=0.679	SAT TT=0.009	? TT=1596.2	? (TO) TT=359.458
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-reduced-prob3.xml	SAT TT=0.008	? (TO) TT=1800.16	? (TO) TT=1800.11	SAT TT=0.041	SAT TT=0.553	SAT TT=0.671	SAT TT=0.008	? (TO) TT=798.985	? TT=1454.29
csp/pseudoGLB/ normalized-reduced-prob5.xml	SAT TT=0.012	? (TO) TT=1800.07	? (TO) TT=1800.16	SAT TT=0.037	SAT TT=0.535	SAT TT=0.662	SAT TT=0.012	? (TO) TT=1801.94	? TT=1534.77
csp/pseudoGLB/ normalized-reduced-prob6.xml	SAT TT=0.012	? (TO) TT=1800.09	? (TO) TT=1800.15	SAT TT=0.036	SAT TT=0.49	SAT TT=0.513	SAT TT=0.012	? TT=971.827	? TT=997.527
csp/pseudoGLB/ normalized-reduced-prob7.xml	SAT TT=0.009	? (TO) TT=1800.06	? (TO) TT=1800.07	SAT TT=0.045	SAT TT=0.521	SAT TT=0.602	SAT TT=0.009	? TT=1152.17	? TT=1121.39
csp/pseudoGLB/ normalized-robin14.xml	SAT TT=518.286	? (TO) TT=1800.13	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.17	? (TO) TT=1800.06	? (TO) TT=1800.04	SAT TT=518.286	? (TO) TT=1800.03
csp/pseudoGLB/ normalized-robin18.xml		? (TO) TT=1800.15	? (TO) TT=1800.2	? (TO) TT=1800.09	? (TO) TT=1800.14	? (TO) TT=1800.13	? (TO) TT=1800.1	? (MO) TT=1525.74	? (TO) TT=1800.01
csp/pseudoGLB/ normalized-robin8.xml	SAT TT=0.116	SAT TT=581.382	? (TO) TT=1800.14	SAT TT=0.328	SAT TT=1.575	SAT TT=2.208	SAT TT=0.116	SAT TT=17.015	SAT TT=24.688
csp/pseudoGLB/ normalized-t2001-13queen13-1110966688.xml	UNSAT TT=6.02	UNSAT TT=1383.29	UNSAT TT=268.663	UNSAT TT=6.02	UNSAT TT=23.075	UNSAT TT=87.778	UNSAT TT=20.93	UNSAT TT=12.257	UNSAT TT=16.305
csp/pseudoGLB/ normalized-t2001-13queen13-1110967142.xml	UNSAT TT=10.273	UNSAT TT=654.008	UNSAT TT=655.655	UNSAT TT=10.273	UNSAT TT=44.812	UNSAT TT=206.05	UNSAT TT=47.315	UNSAT TT=38.117	UNSAT TT=53.097
csp/pseudoGLB/ normalized-t2001-13queen13-1110967523.xml	UNSAT TT=8.216	UNSAT TT=487	UNSAT TT=1217.57	UNSAT TT=8.216	UNSAT TT=63.96	UNSAT TT=264.159	UNSAT TT=44.174	UNSAT TT=24.91	UNSAT TT=36.959
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-t2001-13queen13-1110967729.xml	UNSAT TT=3.991	UNSAT TT=416.898	UNSAT TT=804.889	UNSAT TT=3.991	UNSAT TT=30.628	UNSAT TT=120.308	UNSAT TT=15.292	UNSAT TT=9.749	UNSAT TT=11.971
csp/pseudoGLB/ normalized-t2001-13queen13-1110968094.xml	UNSAT TT=7.676	UNSAT TT=465.009	UNSAT TT=898.954	UNSAT TT=7.676	UNSAT TT=24.073	UNSAT TT=143.584	UNSAT TT=27.335	UNSAT TT=15.959	UNSAT TT=19.967
csp/pseudoGLB/ normalized-t2001-13queen13-1110968898.xml	UNSAT TT=7.981	UNSAT TT=899.886	UNSAT TT=323.409	UNSAT TT=7.981	UNSAT TT=27.968	UNSAT TT=135.086	UNSAT TT=26.813	UNSAT TT=19.306	UNSAT TT=27.028
csp/pseudoGLB/ normalized-t2001-13queen13-1110969577.xml	UNSAT TT=5.249	UNSAT TT=633.235	UNSAT TT=788.988	UNSAT TT=5.249	UNSAT TT=40.054	UNSAT TT=120.918	UNSAT TT=15.562	UNSAT TT=14.61	UNSAT TT=18.454
csp/pseudoGLB/ normalized-t2001-13queen13-1110969933.xml	SAT TT=0.8	SAT TT=164.05	SAT TT=187.247	SAT TT=2.091	SAT TT=0.8	SAT TT=2.071	SAT TT=2.077	SAT TT=4.168	SAT TT=12.571
csp/pseudoGLB/ normalized-t2001-13queen13-1110971363.xml	UNSAT TT=7.571	UNSAT TT=354.4	UNSAT TT=672.242	UNSAT TT=7.571	UNSAT TT=24.012	UNSAT TT=184.136	UNSAT TT=35.819	UNSAT TT=13.692	UNSAT TT=18.495
csp/pseudoGLB/ normalized-t2001-13queen13-1110972763.xml	UNSAT TT=7.819	UNSAT TT=990.462	UNSAT TT=964.143	UNSAT TT=7.819	UNSAT TT=21.963	UNSAT TT=111.092	UNSAT TT=27.836	UNSAT TT=16.736	UNSAT TT=24.324
csp/pseudoGLB/ normalized-t2001-13queen13-1110973670.xml	UNSAT TT=8.277	UNSAT TT=1244.7	UNSAT TT=1042.82	UNSAT TT=8.277	UNSAT TT=34.822	UNSAT TT=131.663	UNSAT TT=14.932	UNSAT TT=20.309	UNSAT TT=20.56
csp/pseudoGLB/ normalized-t2001-13queen13-1110975965.xml	UNSAT TT=11.035	UNSAT TT=1187.99	UNSAT TT=679.219	UNSAT TT=11.035	UNSAT TT=56.798	UNSAT TT=219.178	UNSAT TT=55.541	UNSAT TT=32.053	UNSAT TT=45.915
csp/pseudoGLB/ normalized-t2001-13queen13-1110976730.xml	UNSAT TT=7.239	UNSAT TT=899.952	UNSAT TT=745.423	UNSAT TT=7.239	UNSAT TT=35.267	UNSAT TT=209.612	UNSAT TT=72.04	UNSAT TT=30.369	UNSAT TT=38.23
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-t2001-13queen13-1110977464.xml	SAT TT=1.416	SAT TT=186.578	SAT TT=811.883	SAT TT=1.416	SAT TT=23.89	SAT TT=50.241	SAT TT=1.999	SAT TT=9.716	SAT TT=12.963
csp/pseudoGLB/ normalized-t2001-13queen13-1111212272.xml	UNSAT TT=5.94	UNSAT TT=1505.28	UNSAT TT=1144.69	UNSAT TT=5.94	UNSAT TT=33.333	UNSAT TT=172.174	UNSAT TT=56.576	UNSAT TT=19.956	UNSAT TT=26.862
csp/pseudoGLB/ normalized-t2001-13queen13-1111213670.xml	UNSAT TT=5.775	UNSAT TT=442.826	? (TO) TT=1800.06	UNSAT TT=5.775	UNSAT TT=22.648	UNSAT TT=113.734	UNSAT TT=15.344	UNSAT TT=15.12	UNSAT TT=15.674
csp/pseudoGLB/ normalized-t2001-13queen13-1111216411.xml	UNSAT TT=10.244	UNSAT TT=701.123	? (TO) TT=1800.14	UNSAT TT=10.244	UNSAT TT=28.159	UNSAT TT=169.219	UNSAT TT=43.93	UNSAT TT=22.679	UNSAT TT=32.434
csp/pseudoGLB/ normalized-t2001-13queen13-1111216823.xml	UNSAT TT=8.744	UNSAT TT=1672.4	? (TO) TT=1800.18	UNSAT TT=8.744	UNSAT TT=46.854	UNSAT TT=226.562	UNSAT TT=27.524	UNSAT TT=15.49	UNSAT TT=20.877
csp/pseudoGLB/ normalized-t2001-13queen13-1111218024.xml	UNSAT TT=8.782	UNSAT TT=312.814	UNSAT TT=620.855	UNSAT TT=8.782	UNSAT TT=34.238	UNSAT TT=228.435	UNSAT TT=25.875	UNSAT TT=20.636	UNSAT TT=27.828
csp/pseudoGLB/ normalized-t2001-13queen13-1111218308.xml	UNSAT TT=14.188	UNSAT TT=1420.27	UNSAT TT=1450.78	UNSAT TT=14.188	UNSAT TT=76.437	UNSAT TT=229.259	UNSAT TT=43.713	UNSAT TT=51.68	UNSAT TT=53.967
csp/pseudoGLB/ normalized-t2001-13queen13-1111218757.xml	UNSAT TT=7.895	UNSAT TT=240.955	UNSAT TT=811.569	UNSAT TT=7.895	UNSAT TT=28.678	UNSAT TT=144.149	UNSAT TT=21.971	UNSAT TT=19.642	UNSAT TT=26.434
csp/pseudoGLB/ normalized-t2001-13queen13-1111219799.xml	UNSAT TT=7.68	UNSAT TT=396.645	UNSAT TT=427.112	UNSAT TT=7.68	UNSAT TT=27.2	UNSAT TT=164.719	UNSAT TT=25.552	UNSAT TT=15.583	UNSAT TT=20.281
csp/pseudoGLB/ normalized-t2001-13queen13-1111223416.xml	SAT TT=1.214	SAT TT=941.914	SAT TT=1034.32	SAT TT=2.608	SAT TT=4.409	SAT TT=2.302	SAT TT=1.214	SAT TT=3.279	SAT TT=13.013
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-t2001-13queen13-1111223596.xml	SAT TT=0.681	SAT TT=465.399	SAT TT=466.537	SAT TT=0.681	SAT TT=3.112	SAT TT=38.778	SAT TT=30.461	SAT TT=28.192	SAT TT=11.089
csp/pseudoGLB/ normalized-t2001-13queen13-1111224451.xml	UNSAT TT=5.596	UNSAT TT=1013.77	? (TO) TT=1800.07	UNSAT TT=5.596	UNSAT TT=29.26	UNSAT TT=144.886	UNSAT TT=43.103	UNSAT TT=12.201	UNSAT TT=17.784
csp/pseudoGLB/ normalized-t2001-13queen13-1111224929.xml	UNSAT TT=5.435	UNSAT TT=920.98	UNSAT TT=287.321	UNSAT TT=5.435	UNSAT TT=24.785	UNSAT TT=108.032	UNSAT TT=35.109	UNSAT TT=15.762	UNSAT TT=20.336
csp/pseudoGLB/ normalized-t2001-13queen13-1111225605.xml	UNSAT TT=8.179	UNSAT TT=804.466	UNSAT TT=550.066	UNSAT TT=8.179	UNSAT TT=18.526	UNSAT TT=108.597	UNSAT TT=23.364	UNSAT TT=17.499	UNSAT TT=19.882
csp/pseudoGLB/ normalized-t2001-13queen13-1111226284.xml	SAT TT=0.801	SAT TT=85.795	? (TO) TT=1800.09	SAT TT=7.74	SAT TT=1.817	SAT TT=30.296	SAT TT=0.801	SAT TT=1.571	SAT TT=7.759
csp/pseudoGLB/ normalized-t2001-13queen13-1111226368.xml	UNSAT TT=4.781	UNSAT TT=423.175	UNSAT TT=231.808	UNSAT TT=4.781	UNSAT TT=13.967	UNSAT TT=109.602	UNSAT TT=26.764	UNSAT TT=11.328	UNSAT TT=12.878
csp/pseudoGLB/ normalized-t2001-13queen13-1111226680.xml	UNSAT TT=7.086	? (TO) TT=1800.14	UNSAT TT=797.617	UNSAT TT=7.086	UNSAT TT=31.063	UNSAT TT=262.167	UNSAT TT=20.624	UNSAT TT=17.526	UNSAT TT=21.319
csp/pseudoGLB/ normalized-t2001-13queen13-1111227418.xml	SAT TT=3.726	SAT TT=412.822	? (TO) TT=1800.16	SAT TT=3.726	SAT TT=6.785	SAT TT=60.711	SAT TT=22.407	SAT TT=8.879	SAT TT=19.401
csp/pseudoGLB/ normalized-t2001-13queen13-1111228076.xml	UNSAT TT=7.186	UNSAT TT=1607.36	UNSAT TT=396.291	UNSAT TT=7.186	UNSAT TT=61.002	UNSAT TT=280.349	UNSAT TT=27.926	UNSAT TT=28.396	UNSAT TT=38.294
csp/pseudoGLB/ normalized-t3002-11tsp11-1900545610.xml	UNSAT TT=41.37	? (TO) TT=1800.1	? (TO) TT=1800.15	UNSAT TT=41.37	? (TO) TT=1800.09	? (TO) TT=1800.05	? (TO) TT=1800.01	UNSAT TT=112.101	UNSAT TT=331.15
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-t3002-11tsp11-1900546076.xml	UNSAT TT=9.615	? (TO) TT=1800.09	? (TO) TT=1800.07	UNSAT TT=19.999	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.1	UNSAT TT=9.615	UNSAT TT=9.819
csp/pseudoGLB/ normalized-t3002-11tsp11-1900547353.xml	SAT TT=0.643	? (TO) TT=1800.13	? (TO) TT=1800.1	SAT TT=0.643	SAT TT=40.138	SAT TT=13.266	SAT TT=5.128	SAT TT=2.945	SAT TT=4.021
csp/pseudoGLB/ normalized-t3002-11tsp11-1900547389.xml	UNSAT TT=20.139	? (TO) TT=1800.12	? (TO) TT=1800.11	UNSAT TT=20.139	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.07	UNSAT TT=88.178	UNSAT TT=301.907
csp/pseudoGLB/ normalized-t3002-11tsp11-1900547416.xml	SAT TT=0.899	? (TO) TT=1800.09	? (TO) TT=1800.08	SAT TT=0.899	SAT TT=20.007	SAT TT=208.986	SAT TT=7.158	SAT TT=2.614	SAT TT=5.411
csp/pseudoGLB/ normalized-t3002-11tsp11-1900552311.xml	SAT TT=0.571	? (TO) TT=1800.05	? (TO) TT=1800.07	SAT TT=0.571	SAT TT=11.353	SAT TT=59.731	SAT TT=0.607	SAT TT=3.078	SAT TT=3.402
csp/pseudoGLB/ normalized-t3002-11tsp11-1900553647.xml	UNSAT TT=25.737	? (TO) TT=1800.09	? (TO) TT=1800.1	UNSAT TT=25.737	? (TO) TT=1800.13	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=114.17	UNSAT TT=310.407
csp/pseudoGLB/ normalized-t3002-11tsp11-1900557413.xml	SAT TT=0.562	? (TO) TT=1800.14	? (TO) TT=1800.11	SAT TT=0.562	SAT TT=9.92	SAT TT=305.327	SAT TT=15.998	SAT TT=7.749	SAT TT=8.393
csp/pseudoGLB/ normalized-t3002-11tsp11-1900557568.xml	SAT TT=0.793	? (TO) TT=1800.12	? (TO) TT=1800.15	SAT TT=1.213	SAT TT=9.835	SAT TT=2.395	SAT TT=0.793	SAT TT=2.502	SAT TT=2.872
csp/pseudoGLB/ normalized-t3002-11tsp11-1900561384.xml	SAT TT=0.692	? (TO) TT=1800.18	? (TO) TT=1800.13	SAT TT=0.692	SAT TT=13.104	SAT TT=33.272	SAT TT=5.452	SAT TT=2.537	SAT TT=3.079
csp/pseudoGLB/ normalized-t3002-11tsp11-1900561389.xml	SAT TT=0.984	? (TO) TT=1800.13	? (TO) TT=1800.1	SAT TT=0.984	SAT TT=231.098	SAT TT=138.425	SAT TT=14.918	SAT TT=3.329	SAT TT=5.519
Bench name	Best results	Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
csp/pseudoGLB/ normalized-t3002-11tsp11-1900562192.xml	UNSAT TT=15.334	? (TO) TT=1800.14	? (TO) TT=1800.1	UNSAT TT=15.334	? (TO) TT=1800.09	? (TO) TT=1800.12	UNSAT TT=1270.02	UNSAT TT=50.667	UNSAT TT=144.353
csp/pseudoGLB/ normalized-t3002-11tsp11-1900568186.xml	UNSAT TT=22.067	? (TO) TT=1800.06	? (TO) TT=1800.12	UNSAT TT=22.067	? (TO) TT=1800.07	? (TO) TT=1800.14	? (TO) TT=1800.08	UNSAT TT=25.521	UNSAT TT=31.803
csp/pseudoGLB/ normalized-t3002-11tsp11-1900568706.xml	UNSAT TT=15.494	? (TO) TT=1800.16	? (TO) TT=1800.12	UNSAT TT=23.037	? (TO) TT=1800.1	? (TO) TT=1800.15	UNSAT TT=977.476	UNSAT TT=15.494	UNSAT TT=15.908
csp/pseudoGLB/ normalized-t3002-11tsp11-1900575442.xml	UNSAT TT=32.2	? (TO) TT=1800.08	? (TO) TT=1800.12	UNSAT TT=53.103	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) TT=1800.03	UNSAT TT=32.2	UNSAT TT=149.299
csp/pseudoGLB/ normalized-t3002-11tsp11-1900576927.xml	UNSAT TT=42.795	? (TO) TT=1800.12	? (TO) TT=1800.11	UNSAT TT=42.795	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) TT=1800.03	UNSAT TT=206.987	UNSAT TT=785.407
csp/pseudoGLB/ normalized-t3002-11tsp11-1900582376.xml	SAT TT=1.852	? (TO) TT=1800.1	? (TO) TT=1800.14	SAT TT=1.852	SAT TT=22.397	SAT TT=80.562	SAT TT=5.669	SAT TT=2.74	SAT TT=5.096
csp/pseudoGLB/ normalized-t3002-11tsp11-1900583053.xml	UNSAT TT=53.321	? (TO) TT=1800.1	? (TO) TT=1800.14	UNSAT TT=53.321	? (TO) TT=1800.13	? (TO) TT=1800.17	? (TO) TT=1800.03	UNSAT TT=159.468	UNSAT TT=867.543
csp/pseudoGLB/ normalized-t3002-11tsp11-1900584219.xml	UNSAT TT=12.054	? (TO) TT=1800.15	? (TO) TT=1800.19	UNSAT TT=31.827	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.01	UNSAT TT=102.258	UNSAT TT=12.054
csp/pseudoGLB/ normalized-t3002-11tsp11-1900669309.xml	SAT TT=1.708	? (TO) TT=1800.08	? (TO) TT=1800.09	SAT TT=1.708	SAT TT=129.266	SAT TT=400.895	SAT TT=46.561	SAT TT=7.442	SAT TT=5.339
csp/pseudoGLB/ normalized-t3002-11tsp11-1900670167.xml	UNSAT TT=26.999	? (TO) TT=1800.11	? (TO) TT=1800.13	UNSAT TT=26.999	? (TO) TT=1800.16	? (TO) TT=1800.05	? (TO) TT=1800.08	UNSAT TT=34.74	UNSAT TT=141.321

Statistics		Abscon 112v4 AC (complete)	Abscon 112v4 ESAC (complete)	bpsolver 09 (complete)	Choco2.1.1 2009-06-10 (complete)	Choco2.1.1b 2009-07-16 (complete)	Mistral 1.545 (complete)	Sugar v1.14.6+minisat (complete)	Sugar v1.14.6+picosat (complete)
Number of times the solver is able to give the best known answer		37	31	70	53	52	59	68	75
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		2	1	46	1	0	14	10	11