SAT 2007 Competition Phase 1: ranking of solvers

This page presents the ranking of the different solvers. Solvers are ranked on the number of solved instances. Ties are broken by considering the minimum total solution time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "crafted instances" (CRAFTED)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	minisat	SAT 2007	71	13105	184.58
2	minisat	SAT 2007 (with assertions)	71	13567.6	191.09
3	SATzilla	FULL	69	11673.72	169.18
4	SATzilla	CRAFTED	69	12281.44	177.99
5	minimarch	2007-04-26 (fixed)	67	16433.82	245.28
6	MiraXT	v3	57	13929.06	244.37
7	MiraXT	v2	55	13445.05	244.46
8	MiraXT	v1	55	14146.51	257.21
9	MXC	2007-02-08	53	16084.61	303.48
10	CMUSAT	2007-02-08	52	9361.42	180.03
11	picosat	535	46	12383	269.2
12	SAT7	2007-02-08	46	14162.3	307.88
13	SATzilla	RANDOM	44	8865.79	201.5
14	Barcelogic Fixed	2007-04-13	40	8704.21	217.61
15	TTS	4.0	37	3347.89	90.48
16	CMUSAT BASE	2007-02-08	37	9957.74	269.13
17	Rsat	2007-02-08	37	12315.78	332.86
18	March KS	2007-02-08	33	6519.65	197.57
19	Spear FH	1.0	33	12790.81	387.6
20	Spear FHS	1.0	32	11814.4	369.2
21	SAT4J	SAT 2007	32	12631.54	394.74
22	Spear	2007-02-12	31	11312.68	364.93
23	TiniSatELite	2007-02-08	27	8026.56	297.28
24	tinisat	2007-02-08	25	5322.47	212.9
25	DEWSATZ 1A	2007-02-08	20	5100.84	255.04
26	DEWSATZ	2007-04-26 (fixed)	15	3961.61	264.11
27	KCNFS	SMP	15	5153.27	343.55
28	KCNFS	2006	14	3157.38	225.53
29	KCNFS	2004	14	3274.5	233.89
30	ornithorynque	0.1 alpha	9	3296.36	366.26
31	Hybrid1	2007-02-08	8	229.51	28.69
32	FH	2007-02-08	7	261.54	37.36
33	adaptg2wsat	2007-02-08	6	199.81	33.3
34	adaptg2wsat+	2007-02-08	6	478.89	79.82
35	ranov	2007-02-08	6	2773.86	462.31
36	adaptg2wsat0	2007-02-08	4	259.56	64.89
37	adaptg2wsatp	2007-02-08	4	323.17	80.79
38	UnitMarch	2007-02-08	3	421.62	140.54
39	adaptnovelty	2007-02-08	3	1768.33	589.44
40	Mmisat	2007-02-08	2	757.88	378.94
41	saps	2007-02-08	2	1590.7	795.35
42	gnovelty+	2007-02-08	1	61.94	61.94
43	sapsrt	2007-02-08	1	147.42	147.42

Category "industrial instances" (INDUST)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	Rsat	2007-02-08	106	18043.45	170.22
2	TiniSatELite	2007-02-08	103	22193.27	215.47
3	picosat	535	103	30610.29	297.19
4	Barcelogic Fixed	2007-04-13	99	21811.01	220.31
5	Spear FHS	1.0	99	22610.38	228.39
6	minisat	SAT 2007	97	20931.03	215.78
7	minisat	SAT 2007 (with assertions)	96	20540.63	213.96
8	Spear FH	1.0	93	20041.31	215.5
9	MiraXT	v1	91	18849.09	207.13
10	tinisat	2007-02-08	90	21826.71	242.52
11	CMUSAT	2007-02-08	89	17871.18	200.8
12	MiraXT	v3	89	20671.2	232.26
13	minimarch	2007-04-26 (fixed)	88	18993.44	215.83
14	SATzilla	FULL	88	29939.41	340.22
15	MXC	2007-02-08	84	18692.39	222.53
16	MiraXT	v2	83	17916.25	215.86
17	Spear	2007-02-12	82	15355.45	187.26
18	SATzilla	CRAFTED	81	26772.7	330.53
19	CMUSAT BASE	2007-02-08	80	22286.33	278.58
20	SAT7	2007-02-08	69	21188.66	307.08
21	SATzilla	RANDOM	66	23303.59	353.08
22	SAT4J JVM PARAM CHANGED	SAT 2007	66	23870.23	361.67
23	SAT4J	SAT 2007	57	19670.66	345.1
24	ornithorynque	0.1 alpha	33	5553.22	168.28
25	DEWSATZ 1A	2007-02-08	22	6010.53	273.21
26	DEWSATZ	2007-04-26 (fixed)	19	5044.84	265.52
27	KCNFS	2006	16	2701.24	168.83
28	March KS	2007-02-08	12	2977.72	248.14
29	KCNFS	SMP	10	1798.24	179.82
30	adaptg2wsat+	2007-02-08	9	1458.57	162.06
31	adaptg2wsat	2007-02-08	8	1288.02	161
32	FH	2007-02-08	8	1810.91	226.36
33	adaptg2wsat0	2007-02-08	7	681.15	97.31
34	adaptg2wsatp	2007-02-08	7	1145.09	163.58
35	Hybrid1	2007-02-08	7	1380.6	197.23
36	ranov	2007-02-08	6	2025.9	337.65
37	UnitMarch	2007-02-08	4	1734.5	433.63
38	TTS	4.0	3	13.4	4.47
39	adaptnovelty	2007-02-08	3	1143.1	381.03
40	KCNFS	2004	2	6.57	3.29
41	gnovelty+	2007-02-08	2	575.18	287.59
42	saps	2007-02-08	2	728.4	364.2
43	sapsrt	2007-02-08	2	1291.78	645.89
44	Mmisat	2007-02-08	0	0	-

Category "random instances" (RANDOM)

Rank	Solver	Version	Number of solved instances	Cumulated CPU time on solved instances	Average CPU time per solved instance
1	adaptg2wsat0	2007-02-08	231	25875.21	112.01
2	adaptg2wsat+	2007-02-08	227	22724.01	100.11
3	Hybrid1	2007-02-08	226	24345.15	107.72
4	adaptg2wsat	2007-02-08	222	20570.71	92.66
5	adaptg2wsatp	2007-02-08	222	26237.69	118.19
6	FH	2007-02-08	220	22428.66	101.95
7	adaptnovelty	2007-02-08	218	27987.63	128.38
8	ranov	2007-02-08	216	27044.14	125.2
9	SATzilla	RANDOM	215	31979.77	148.74
10	March KS	2007-02-08	208	36762.87	176.74
11	SATzilla	FULL	205	23696.12	115.59
12	gnovelty+	2007-02-08	203	21409.29	105.46
13	KCNFS	2004	191	40265.13	210.81
14	SATzilla	CRAFTED	189	24417.95	129.2
15	KCNFS	2006	189	38423.92	203.3
16	saps	2007-02-08	169	6197.6	36.67
17	sapsrt	2007-02-08	167	7135.46	42.73
18	KCNFS	SMP	165	35051.84	212.44
19	DEWSATZ	2007-04-26 (fixed)	142	38365.15	270.18
20	minimarch	2007-04-26 (fixed)	125	36271.09	290.17
21	MXC	2007-02-08	105	28842.42	274.69
22	minisat	SAT 2007	101	27872.79	275.97
23	minisat	SAT 2007 (with assertions)	100	27622.24	276.22
24	SAT7	2007-02-08	95	28423.19	299.19
25	DEWSATZ 1A	2007-02-08	84	24117.97	287.12
26	SAT4J	SAT 2007	78	19580.72	251.03
27	MiraXT	v3	78	25731.14	329.89
28	MiraXT	v1	74	18904.06	255.46
29	MiraXT	v2	68	24496.65	360.24
30	picosat	535	67	19041.69	284.2
31	UnitMarch	2007-02-08	62	17324.63	279.43
32	CMUSAT BASE	2007-02-08	48	9710.38	202.3
33	CMUSAT	2007-02-08	46	9128.25	198.44
34	Rsat	2007-02-08	43	7117.74	165.53
35	Spear FHS	1.0	40	12725.5	318.14
36	Spear FH	1.0	39	12585.65	322.71
37	Spear	2007-02-12	38	7746.95	203.87
38	TiniSatELite	2007-02-08	32	9104.94	284.53
39	tinisat	2007-02-08	30	5302.66	176.76
40	Barcelogic Fixed	2007-04-13	29	6332.62	218.37
41	ornithorynque	0.1 alpha	11	1459.23	132.66
42	Mmisat	2007-02-08	1	426.37	426.37
43	TTS	4.0	0	0	-