CSP 2006 Competition (second round): ranking of solvers

This page presents the ranking of the different solvers. Solvers are ranked on the number of solved instances. Ties are broken by considering the minimum total solution time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "binary constraints in extension" (2-ARY-EXT)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	VALCSP	3.0	1093	59496.9	54.43
2	buggy_2_5_s	2007-01-08	1093	59632.56	54.56
3	buggy_2_5	2007-01-08	1092	58658.14	53.72
4	Abscon	109 AC	1082	57755.56	53.38
5	Abscon	109 ESAC	1081	54184.15	50.12
6	Mistral	2006-12-04	1060	57473.78	54.22
7	Tramontane	2006-12-04	1047	53907.19	51.49
8	CSP4J - MAC	2007-01-16	1025	62580.9	61.05
9	galac	1	940	84144.15	89.52
10	BPrologCSPSolver70a	2006-12-13	938	171619.04	182.96
11	galacJ	beta 1	928	88183.23	95.03
12	sat4jCSP	1.7 RC BF3	917	112685.03	122.88
13	SAT4JCSP-CACHED	1.7 RC BF3	915	109227.69	119.37
14	CSPtoSAT+minisat	0.3	877	82401.02	93.96
15	sugar	0.40	806	54719.92	67.89

Category "binary constraints in intension" (2-ARY-INT)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	buggy_2_5_s	2007-01-08	627	74687.08	119.12
2	buggy_2_5	2007-01-08	626	34572.13	55.23
3	Abscon	109 ESAC	575	61532.88	107.01
4	Abscon	109 AC	561	20169.62	35.95
5	Tramontane	2006-12-04	457	23726.5	51.92
6	CSP4J - Combo	2006-12-19	438	88510.72	202.08
7	CSP4J - MAC	2006-12-19	432	90051.92	208.45
8	Mistral	2006-12-04	423	23648.22	55.91
9	CSP4J - MAC	2007-01-16	405	48439.42	119.6
10	sugar	0.40	276	15977.41	57.89
11	sat4jCSP	1.7 RC BF3	258	18300.66	70.93
12	galac	1	245	41654.44	170.02
13	BPrologCSPSolver70a	2006-12-13	226	64400.8	284.96
14	galacJ	beta 1	220	16881.98	76.74
15	SAT4JCSP-CACHED	1.7 RC BF3	217	8445.94	38.92
16	CSPtoSAT+minisat	0.3	55	13612.87	247.51

Category "global constraints" (GLOBAL)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	BPrologCSPSolver70a	2006-12-13	127	39.9	0.31
2	Abscon	109 AC	127	784.8	6.18
3	Abscon	109 ESAC	126	145.99	1.16
4	CSP4J - MAC	2006-12-19	126	692.14	5.49
5	CSP4J - Combo	2006-12-19	126	1041.41	8.27
6	Mistral	2006-12-04	125	33.38	0.27
7	Tramontane	2006-12-04	125	45.71	0.37
8	CSP4J - MAC	2007-01-16	125	412.16	3.3
9	sugar	0.40	118	884.51	7.5
10	sat4jCSP	1.7 RC BF3	115	2346.01	20.4
11	CSPtoSAT+minisat	0.3	114	645.37	5.66
12	SAT4JCSP-CACHED	1.7 RC BF3	114	1198.54	10.51
13	galac	1	110	492.21	4.47

Category "n-ary constraints in extension" (N-ARY-EXT)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	Abscon	109 AC	277	24132.71	87.12
2	Abscon	109 ESAC	276	25923.82	93.93
3	Mistral	2006-12-04	269	36625.34	136.15
4	Tramontane	2006-12-04	260	41901.79	161.16
5	CSP4J - MAC	2007-01-16	239	29234	122.32
6	CSP4J - MAC	2006-12-19	234	30795.56	131.6
7	CSP4J - Combo	2006-12-19	219	33861.56	154.62
8	galac	1	191	18017.97	94.33
9	SAT4JCSP-CACHED	1.7 RC BF3	187	15004.32	80.24
10	sat4jCSP	1.7 RC BF3	187	15174.91	81.15
11	CSPtoSAT+minisat	0.3	176	5208.18	29.59
12	galacJ	beta 1	175	8659.69	49.48
13	sugar	0.40	161	627.05	3.89
14	BPrologCSPSolver70a	2006-12-13	150	22206.17	148.04

Category "n-ary constraints in intension" (N-ARY-INT)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	BPrologCSPSolver70a	2006-12-13	579	185136.57	319.75
2	Abscon	109 ESAC	509	43430.74	85.33
3	Abscon	109 AC	490	42068.73	85.85
4	sugar	0.40	431	14414	33.44
5	CSPtoSAT+minisat	0.3	395	39764.89	100.67
6	CSP4J - MAC	2006-12-19	370	49866.12	134.77
7	CSP4J - Combo	2006-12-19	364	64358.81	176.81
8	galac	1	352	37116.12	105.44
9	galacJ	beta 1	331	38483.58	116.26
10	Tramontane	2006-12-04	313	29473.79	94.17
11	Mistral	2006-12-04	304	34541.39	113.62
12	sat4jCSP	1.7 RC BF3	228	15734.88	69.01