SAT 2011 Competition: main competition (phase 1): ranking of solvers
max CPU time for sequential solvers=1200 s
max CPU time for parallel solvers=4800 s
max WC time for any solver=1210 s

This page presents the ranking of the different solvers. Solvers are ranked on the number of solved instances. Ties are broken by considering the minimum total solution time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

The Virtual Best Solver (VBS) is a theoretical construction which returns the best answer provided by one of the submitted solver. It corresponds to the 'Best result' column in the bench per bench view of the results. It can be considered as a meta solver which uses a perfect oracle to guess instantaneously which of the submitted solver will be the most efficient to solve the instance. Another way to look at it is to consider this VBS as a solver which would run all other solvers in parallel, bringing together all the solvers strenghs. This VBS is essentially the same notion as State Of The Art (SOTA) solver defined in "Evaluating general purpose automated theorem proving systems", Artificial Intelligence, Volume 131, Issues 1-2, September 2001, Pages 39-54 Geoff Sutcliffe, Christian Suttner. However, SOTA is a more general notion: the VBS can be seen as the least powerful SOTA solver.

Category "applications instances" (APPLICATION): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			242	114 SAT, 128 UNSAT	81%	100%	72175.98	27.17	298.25	-	-	-	-
1	Plingeling	587f (fixed)	192	96 SAT, 96 UNSAT	64%	79%	109844.85	144.94	572.11	27875.01	36.84	145.18	3.94
2	CryptoMiniSat	Strange-Night-2-mt (fixed)	192	87 SAT, 105 UNSAT	64%	79%	130357.47	228.51	678.95	33328.80	58.54	173.59	3.91
3	ppfolio	par	184	86 SAT, 98 UNSAT	61%	76%	122874.65	161.42	667.80	31371.91	40.45	170.50	3.92
4	claspmt	2.0-R4095-patched (fixed)	181	89 SAT, 92 UNSAT	60%	75%	119900.89	98.73	662.44	30521.15	27.64	168.63	3.93
5	CryptoMiniSat	Strange-Night-1-mt (fixed)	180	78 SAT, 102 UNSAT	60%	74%	111807.13	184.72	621.15	29001.30	46.96	161.12	3.86
6	borg-sat	11.03.02-par-industrial (fixed)	179	85 SAT, 94 UNSAT	60%	74%	120611.14	258.82	673.81	42756.99	112.24	238.87	2.82
7	glucose	2	174	82 SAT, 92 UNSAT	58%	72%	32565.62	32.79	187.16	32569.63	32.80	187.18	1.00
8	CryptoMiniSat	Strange-Night2-st (fixed)	172	77 SAT, 95 UNSAT	57%	71%	32319.46	69.85	187.90	32334.92	69.86	187.99	1.00
9	Lingeling	587f (fixed)	171	79 SAT, 92 UNSAT	57%	71%	28641.98	51.61	167.50	28655.39	51.62	167.58	1.00
10	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	171	85 SAT, 86 UNSAT	57%	71%	44330.35	149.09	259.24	44623.16	151.06	260.95	0.99
11	QuteRSat	2011-05-12 (fixed)	170	77 SAT, 93 UNSAT	57%	70%	28607.58	62.12	168.28	28728.50	62.15	168.99	1.00
12	glueminisat	2.2.5	169	73 SAT, 96 UNSAT	56%	70%	26617.79	35.38	157.50	26622.23	35.39	157.53	1.00
13	RestartSAT	A75	169	77 SAT, 92 UNSAT	56%	70%	34049.31	43.10	201.48	34050.19	43.10	201.48	1.00
14	SAT09 reference solver: ManySAT 1.1	aimd 1/2009-03-20	168	78 SAT, 90 UNSAT	56%	69%	110670.04	89.43	658.75	28608.14	24.31	170.29	3.87
15	MPhaseSAT64	2011-05-14 (fixed)	168	84 SAT, 84 UNSAT	56%	69%	32437.60	58.40	193.08	32439.10	58.44	193.09	1.00
16	RestartSAT	B95	168	80 SAT, 88 UNSAT	56%	69%	32997.62	46.86	196.41	33003.75	46.87	196.45	1.00
17	minisat hack: cir_minisat	2011-05-13 (simp)	164	82 SAT, 82 UNSAT	55%	68%	25957.95	37.80	158.28	25961.86	37.81	158.30	1.00
18	SAT09 reference solver:precosat	236	164	78 SAT, 86 UNSAT	55%	68%	26023.19	46.83	158.68	26033.46	46.84	158.74	1.00
19	CryptoMiniSat	Strange-Night-1-st (fixed)	163	72 SAT, 91 UNSAT	54%	67%	27929.31	55.40	171.35	27931.19	55.41	171.36	1.00
20	minisat hack: minisat_psm	2011-04-01	162	85 SAT, 77 UNSAT	54%	67%	25508.30	29.77	157.46	25515.11	30.05	157.50	1.00
21	MPhaseSAT_M	2011-02-16	162	80 SAT, 82 UNSAT	54%	67%	28091.77	56.69	173.41	28094.93	56.69	173.43	1.00
22	minisat hack: contrasat	2011-03-02	160	88 SAT, 72 UNSAT	53%	66%	25136.64	42.31	157.10	25153.63	42.36	157.21	1.00
23	borg-sat	11.03.02-seq-industrial (fixed, 1200 s timeout)	160	75 SAT, 85 UNSAT	53%	66%	40180.79	127.07	251.13	40303.27	127.22	251.90	1.00
24	minisat hack: EBMiniSAT	2011-03-02	159	82 SAT, 77 UNSAT	53%	66%	23044.27	31.71	144.93	23045.46	31.72	144.94	1.00
25	minisat hack reference: minisat	2.2.0	159	86 SAT, 73 UNSAT	53%	66%	24238.92	31.42	152.45	24240.40	31.42	152.46	1.00
26	MPhaseSAT	2011-02-15	159	77 SAT, 82 UNSAT	53%	66%	25026.13	50.76	157.40	25028.41	50.76	157.41	1.00
27	borg-sat	11.03.02-seq-industrial (fixed)	159	76 SAT, 83 UNSAT	53%	66%	40720.36	112.59	256.10	40852.49	112.58	256.93	1.00
28	minisat hack: MiniSAT	2.2.0-agile-26	158	84 SAT, 74 UNSAT	53%	65%	22715.89	29.00	143.77	22719.11	29.01	143.79	1.00
29	minisat hack: cir_minisat	2011-03-02	158	84 SAT, 74 UNSAT	53%	65%	31208.56	45.78	197.52	31216.11	45.79	197.57	1.00
30	EBGlucose	1.0	157	76 SAT, 81 UNSAT	52%	65%	22619.68	37.47	144.07	22678.49	37.88	144.45	1.00
31	rcl	2011-04-01	156	83 SAT, 73 UNSAT	52%	64%	27802.74	41.33	178.22	27948.78	43.43	179.16	0.99
32	minisat hack: LR_GL_NOSHR	2011-03-02	155	84 SAT, 71 UNSAT	52%	64%	21261.88	29.83	137.17	21266.57	29.83	137.20	1.00
33	minisat hack: LR_GL_SHR	2011-03-02	155	67 SAT, 88 UNSAT	52%	64%	25222.93	27.96	162.73	25225.00	27.96	162.74	1.00
34	SAT09 reference solver:glucose	1.0	155	74 SAT, 81 UNSAT	52%	64%	25901.39	42.32	167.11	25927.82	42.33	167.28	1.00
35	minisat hack: LR_NOGL_NOSHR	2011-03-02	154	83 SAT, 71 UNSAT	51%	64%	20474.47	32.41	132.95	20472.12	32.42	132.94	1.00
36	kw	2011-02-28	154	73 SAT, 81 UNSAT	51%	64%	28361.17	44.10	184.16	28375.05	44.10	184.25	1.00
37	ppfolio	seq	154	71 SAT, 83 UNSAT	51%	64%	31801.16	79.90	206.50	31823.81	79.90	206.65	1.00
38	SApperloT2010	2011-05-15 (fixed)	153	74 SAT, 79 UNSAT	51%	63%	23011.00	30.33	150.40	23021.31	30.35	150.47	1.00
39	SArTagnan	2011-05-15 (fixed)	153	74 SAT, 79 UNSAT	51%	63%	84717.32	150.40	553.71	24713.80	51.31	161.53	3.43
40	Satisfiability Solver Selector	2011-03-18	153	77 SAT, 76 UNSAT	51%	63%	38155.46	120.21	249.38	38300.94	120.35	250.33	1.00
41	clasp	2.0-R4092-indust	152	71 SAT, 81 UNSAT	51%	63%	28365.33	52.24	186.61	28395.66	52.25	186.81	1.00
42	priss	Auto	152	74 SAT, 78 UNSAT	51%	63%	122763.71	214.31	807.66	31119.70	55.35	204.73	3.94
43	SApperloT	2011-03-23	151	75 SAT, 76 UNSAT	50%	62%	30017.72	49.06	198.79	30020.94	49.07	198.81	1.00
44	priss	1.0	147	70 SAT, 77 UNSAT	49%	61%	106420.21	195.13	723.95	27104.38	49.89	184.38	3.93
45	priss	UP	144	71 SAT, 73 UNSAT	48%	60%	42433.92	79.79	294.68	25869.61	54.85	179.65	1.64
46	riss	1.0	144	69 SAT, 75 UNSAT	48%	60%	27332.15	48.80	189.81	27334.80	48.80	189.82	1.00
47	riss	Auto	144	69 SAT, 75 UNSAT	48%	60%	27684.65	51.34	192.25	27685.01	51.34	192.26	1.00
48	PicoSAT	941	138	72 SAT, 66 UNSAT	46%	57%	21461.72	34.65	155.52	21473.68	34.76	155.61	1.00
49	riss	Auto (1200 s timeout)	138	65 SAT, 73 UNSAT	46%	57%	22891.64	47.40	165.88	22887.98	47.40	165.85	1.00
50	CryptoMiniSat	TechDemo-Alpha-mt	138	67 SAT, 71 UNSAT	46%	57%	112438.42	268.21	814.77	29019.18	67.75	210.28	3.87
51	riss	Exp	137	68 SAT, 69 UNSAT	46%	57%	26524.82	52.65	193.61	26525.06	52.65	193.61	1.00
52	sathys	2011-04-01	137	69 SAT, 68 UNSAT	46%	57%	26512.24	45.15	193.52	26561.85	45.17	193.88	1.00
53	SAT09 reference solver:LySAT	i/2009-03-20	129	58 SAT, 71 UNSAT	43%	53%	21125.95	33.68	163.77	21147.12	33.68	163.93	1.00
54	jMiniSat	2011	129	68 SAT, 61 UNSAT	43%	53%	30177.27	62.95	233.93	29961.78	61.53	232.26	1.01
55	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	118	63 SAT, 55 UNSAT	39%	49%	31912.83	92.27	270.45	31783.59	91.02	269.35	1.00
56	Sol	2011-04-04	117	57 SAT, 60 UNSAT	39%	48%	25099.51	74.15	214.53	24966.48	71.16	213.39	1.01
57	UmSat	2011-05-12 (fixed)	90	47 SAT, 43 UNSAT	30%	37%	21627.24	107.86	240.30	21629.30	108.21	240.33	1.00
58	STRUCTure	0.1	31	17 SAT, 14 UNSAT	10%	13%	29161.34	120.72	940.69	7596.97	35.54	245.06	3.84
59	STRUCTure	0.1 (hbr=2)	28	14 SAT, 14 UNSAT	9%	12%	16613.11	61.35	593.33	4598.46	27.38	164.23	3.61
60	Black Hole SAT	2011-03-08	27	22 SAT, 5 UNSAT	9%	11%	1959.00	1.20	72.56	1963.20	1.20	72.71	1.00
61	march_rw	2011-03-02	23	15 SAT, 8 UNSAT	8%	10%	3398.18	2.58	147.75	3398.28	2.58	147.75	1.00
62	shortsattime+	2011-03-02	22	15 SAT, 7 UNSAT	7%	9%	4257.57	5.39	193.53	4257.66	5.39	193.53	1.00
63	sattime+	2011-03-02	19	12 SAT, 7 UNSAT	6%	8%	3215.42	4.12	169.23	3215.46	4.12	169.23	1.00
64	sattime	2011-03-02	16	16 SAT	5%	7%	2046.54	5.11	127.91	2046.54	5.11	127.91	1.00
65	sattime2011	2011-03-02	16	16 SAT	5%	7%	2599.59	5.51	162.47	2599.70	5.51	162.48	1.00
66	shortsattime2011	2011-03-02	15	15 SAT	5%	6%	1115.58	5.39	74.37	1117.99	5.39	74.53	1.00
67	adaptg2wsat2011	2011-03-02	15	15 SAT	5%	6%	1350.26	4.87	90.02	1350.27	4.87	90.02	1.00

Category "applications instances" (APPLICATION): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			114	38%	100%	17998.14	20.77	157.88	-	-	-	-
1	Plingeling	587f (fixed)	96	32%	84%	47517.08	106.73	494.97	12148.58	30.10	126.55	3.91
2	claspmt	2.0-R4095-patched (fixed)	89	30%	78%	52668.27	66.94	591.78	13519.74	20.53	151.91	3.90
3	minisat hack: contrasat	2011-03-02	88	29%	77%	14197.14	39.54	161.33	14206.67	41.40	161.44	1.00
4	CryptoMiniSat	Strange-Night-2-mt (fixed)	87	29%	76%	44944.82	186.53	516.61	11373.89	47.13	130.73	3.95
5	minisat hack reference: minisat	2.2.0	86	29%	75%	12595.40	31.76	146.46	12596.60	31.76	146.47	1.00
6	ppfolio	par	86	29%	75%	53261.23	120.76	619.32	13348.70	30.31	155.22	3.99
7	minisat hack: minisat_psm	2011-04-01	85	28%	75%	14632.08	20.38	172.14	14636.69	20.43	172.20	1.00
8	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	85	28%	75%	21407.86	146.98	251.86	21547.10	151.06	253.50	0.99
9	borg-sat	11.03.02-par-industrial (fixed)	85	28%	75%	56318.88	242.21	662.58	22599.57	113.30	265.88	2.49
10	minisat hack: MiniSAT	2.2.0-agile-26	84	28%	74%	10113.15	27.77	120.39	10116.20	27.81	120.43	1.00
11	minisat hack: LR_GL_NOSHR	2011-03-02	84	28%	74%	11720.66	33.21	139.53	11722.68	33.22	139.56	1.00
12	MPhaseSAT64	2011-05-14 (fixed)	84	28%	74%	13543.31	54.93	161.23	13544.45	54.99	161.24	1.00
13	minisat hack: cir_minisat	2011-03-02	84	28%	74%	13980.12	37.04	166.43	13987.51	37.04	166.52	1.00
14	minisat hack: LR_NOGL_NOSHR	2011-03-02	83	28%	73%	10479.44	29.22	126.26	10481.10	29.24	126.28	1.00
15	rcl	2011-04-01	83	28%	73%	13210.39	40.07	159.16	13316.35	41.74	160.44	0.99
16	minisat hack: cir_minisat	2011-05-13 (simp)	82	27%	72%	10611.60	30.29	129.41	10615.23	30.31	129.45	1.00
17	minisat hack: EBMiniSAT	2011-03-02	82	27%	72%	11233.48	30.26	136.99	11234.49	30.27	137.01	1.00
18	glucose	2	82	27%	72%	13597.50	25.36	165.82	13601.39	26.07	165.87	1.00
19	MPhaseSAT_M	2011-02-16	80	27%	70%	12250.52	38.89	153.13	12253.52	38.89	153.17	1.00
20	RestartSAT	B95	80	27%	70%	13533.69	41.25	169.17	13534.56	41.26	169.18	1.00
21	Lingeling	587f (fixed)	79	26%	69%	10090.91	50.60	127.73	10098.17	50.61	127.82	1.00
22	CryptoMiniSat	Strange-Night-1-mt (fixed)	78	26%	68%	39940.55	119.46	512.06	10399.17	30.47	133.32	3.84
23	SAT09 reference solver:precosat	236	78	26%	68%	11775.83	41.89	150.97	11780.12	41.91	151.03	1.00
24	SAT09 reference solver: ManySAT 1.1	aimd 1/2009-03-20	78	26%	68%	49375.97	67.85	633.03	12632.01	21.46	161.95	3.91
25	MPhaseSAT	2011-02-15	77	26%	68%	9764.63	25.84	126.81	9765.94	25.84	126.83	1.00
26	CryptoMiniSat	Strange-Night2-st (fixed)	77	26%	68%	12757.17	64.09	165.68	12762.31	64.11	165.74	1.00
27	QuteRSat	2011-05-12 (fixed)	77	26%	68%	12798.29	69.63	166.21	12800.05	69.64	166.23	1.00
28	RestartSAT	A75	77	26%	68%	13033.56	42.28	169.27	13034.45	42.28	169.28	1.00
29	Satisfiability Solver Selector	2011-03-18	77	26%	68%	21823.40	119.84	283.42	21920.08	119.99	284.68	1.00
30	EBGlucose	1.0	76	25%	67%	10209.81	25.33	134.34	10242.64	25.33	134.77	1.00
31	borg-sat	11.03.02-seq-industrial (fixed)	76	25%	67%	19851.76	109.75	261.21	19929.74	110.29	262.23	1.00
32	SApperloT	2011-03-23	75	25%	66%	13122.21	28.18	174.96	13125.00	28.23	175.00	1.00
33	borg-sat	11.03.02-seq-industrial (fixed, 1200 s timeout)	75	25%	66%	18665.46	125.05	248.87	18700.51	125.15	249.34	1.00
34	SApperloT2010	2011-05-15 (fixed)	74	25%	65%	8996.30	26.85	121.57	8997.78	26.85	121.59	1.00
35	SAT09 reference solver:glucose	1.0	74	25%	65%	9964.89	35.30	134.66	9965.88	35.31	134.67	1.00
36	SArTagnan	2011-05-15 (fixed)	74	25%	65%	36552.56	81.03	493.95	10596.05	20.53	143.19	3.45
37	priss	Auto	74	25%	65%	47824.58	173.73	646.28	12087.54	52.30	163.35	3.96
38	glueminisat	2.2.5	73	24%	64%	8854.46	33.52	121.29	8855.29	33.53	121.31	1.00
39	kw	2011-02-28	73	24%	64%	15462.33	49.24	211.81	15463.05	49.26	211.82	1.00
40	PicoSAT	941	72	24%	63%	7745.40	23.09	107.58	7757.48	23.18	107.74	1.00
41	CryptoMiniSat	Strange-Night-1-st (fixed)	72	24%	63%	9750.46	42.55	135.42	9751.94	42.57	135.44	1.00
42	clasp	2.0-R4092-indust	71	24%	62%	8746.67	17.61	123.19	8779.73	19.52	123.66	1.00
43	ppfolio	seq	71	24%	62%	10252.73	37.41	144.40	10273.45	38.76	144.70	1.00
44	priss	UP	71	24%	62%	18261.60	66.30	257.21	11363.70	40.06	160.05	1.61
45	priss	1.0	70	23%	61%	39512.75	89.31	564.47	10203.32	27.39	145.76	3.87
46	riss	1.0	69	23%	61%	11223.82	33.42	162.66	11224.72	33.42	162.68	1.00
47	riss	Auto	69	23%	61%	11484.80	34.76	166.45	11485.11	34.77	166.45	1.00
48	sathys	2011-04-01	69	23%	61%	12365.33	43.25	179.21	12413.12	43.25	179.90	1.00
49	jMiniSat	2011	68	23%	60%	11496.04	34.65	169.06	11368.22	33.77	167.18	1.01
50	riss	Exp	68	23%	60%	12650.34	47.35	186.03	12650.66	47.36	186.04	1.00
51	minisat hack: LR_GL_SHR	2011-03-02	67	22%	59%	7856.21	22.63	117.26	7857.70	22.79	117.28	1.00
52	CryptoMiniSat	TechDemo-Alpha-mt	67	22%	59%	45691.11	136.62	681.96	11844.27	44.10	176.78	3.86
53	riss	Auto (1200 s timeout)	65	22%	57%	8853.35	22.52	136.21	8853.97	22.52	136.21	1.00
54	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	63	21%	55%	14149.34	48.32	224.59	14083.96	47.64	223.55	1.00
55	SAT09 reference solver:LySAT	i/2009-03-20	58	19%	51%	8905.40	21.00	153.54	8907.47	21.00	153.58	1.00
56	Sol	2011-04-04	57	19%	50%	9404.97	41.42	165.00	9352.98	39.09	164.09	1.01
57	UmSat	2011-05-12 (fixed)	47	16%	41%	10269.61	49.90	218.50	10271.52	49.90	218.54	1.00
58	Black Hole SAT	2011-03-08	22	7%	19%	1473.84	1.54	66.99	1478.01	1.57	67.18	1.00
59	STRUCTure	0.1	17	6%	15%	13083.90	120.72	769.64	3451.76	30.92	203.04	3.79
60	sattime	2011-03-02	16	5%	14%	2046.54	5.11	127.91	2046.54	5.11	127.91	1.00
61	sattime2011	2011-03-02	16	5%	14%	2599.59	5.51	162.47	2599.70	5.51	162.48	1.00
62	shortsattime2011	2011-03-02	15	5%	13%	1115.58	5.39	74.37	1117.99	5.39	74.53	1.00
63	adaptg2wsat2011	2011-03-02	15	5%	13%	1350.26	4.87	90.02	1350.27	4.87	90.02	1.00
64	march_rw	2011-03-02	15	5%	13%	1685.28	1.00	112.35	1685.34	1.00	112.36	1.00
65	shortsattime+	2011-03-02	15	5%	13%	2536.47	10.31	169.10	2536.44	10.31	169.10	1.00
66	STRUCTure	0.1 (hbr=2)	14	5%	12%	7082.42	53.56	505.89	2027.62	16.55	144.83	3.49
67	sattime+	2011-03-02	12	4%	11%	1459.97	4.33	121.66	1459.99	4.33	121.67	1.00

Category "applications instances" (APPLICATION): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			128	43%	100%	54177.84	41.92	423.26	-	-	-	-
1	CryptoMiniSat	Strange-Night-2-mt (fixed)	105	35%	82%	85412.65	256.67	813.45	21954.91	66.13	209.09	3.89
2	CryptoMiniSat	Strange-Night-1-mt (fixed)	102	34%	80%	71866.58	231.15	704.57	18602.14	58.05	182.37	3.86
3	ppfolio	par	98	33%	77%	69613.42	180.24	710.34	18023.20	45.14	183.91	3.86
4	Plingeling	587f (fixed)	96	32%	75%	62327.77	165.20	649.25	15726.43	41.84	163.82	3.96
5	glueminisat	2.2.5	96	32%	75%	17763.33	37.35	185.03	17766.94	37.36	185.07	1.00
6	CryptoMiniSat	Strange-Night2-st (fixed)	95	32%	74%	19562.30	73.84	205.92	19572.61	73.84	206.03	1.00
7	borg-sat	11.03.02-par-industrial (fixed)	94	31%	73%	64292.26	259.82	683.96	20157.42	106.50	214.44	3.19
8	QuteRSat	2011-05-12 (fixed)	93	31%	73%	15809.29	56.38	169.99	15928.45	56.40	171.27	0.99
9	claspmt	2.0-R4095-patched (fixed)	92	31%	72%	67232.63	168.97	730.79	17001.41	44.47	184.80	3.95
10	Lingeling	587f (fixed)	92	31%	72%	18551.07	55.33	201.64	18557.22	55.34	201.71	1.00
11	glucose	2	92	31%	72%	18968.12	58.89	206.18	18968.23	58.90	206.18	1.00
12	RestartSAT	A75	92	31%	72%	21015.74	46.21	228.43	21015.75	46.22	228.43	1.00
13	CryptoMiniSat	Strange-Night-1-st (fixed)	91	30%	71%	18178.85	58.04	199.77	18179.25	58.06	199.77	1.00
14	SAT09 reference solver: ManySAT 1.1	aimd 1/2009-03-20	90	30%	70%	61294.07	105.79	681.05	15976.13	27.03	177.51	3.84
15	minisat hack: LR_GL_SHR	2011-03-02	88	29%	69%	17366.71	42.78	197.35	17367.31	42.88	197.36	1.00
16	RestartSAT	B95	88	29%	69%	19463.93	49.93	221.18	19469.18	49.93	221.24	1.00
17	SAT09 reference solver:precosat	236	86	29%	67%	14247.36	56.68	165.67	14253.34	56.69	165.74	1.00
18	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	86	29%	67%	22922.49	149.22	266.54	23076.06	151.77	268.33	0.99
19	borg-sat	11.03.02-seq-industrial (fixed, 1200 s timeout)	85	28%	66%	21515.33	132.35	253.12	21602.76	134.26	254.15	1.00
20	MPhaseSAT64	2011-05-14 (fixed)	84	28%	66%	18894.29	60.50	224.93	18894.65	60.58	224.94	1.00
21	borg-sat	11.03.02-seq-industrial (fixed)	83	28%	65%	20868.59	122.00	251.43	20922.75	122.10	252.08	1.00
22	ppfolio	seq	83	28%	65%	21548.43	110.96	259.62	21550.36	110.96	259.64	1.00
23	MPhaseSAT	2011-02-15	82	27%	64%	15261.50	67.05	186.12	15262.46	67.17	186.13	1.00
24	minisat hack: cir_minisat	2011-05-13 (simp)	82	27%	64%	15346.35	42.78	187.15	15346.62	42.79	187.15	1.00
25	MPhaseSAT_M	2011-02-16	82	27%	64%	15841.25	67.95	193.19	15841.41	67.95	193.19	1.00
26	EBGlucose	1.0	81	27%	63%	12409.87	40.22	153.21	12435.86	40.23	153.53	1.00
27	kw	2011-02-28	81	27%	63%	12898.85	42.86	159.25	12912.00	42.86	159.41	1.00
28	SAT09 reference solver:glucose	1.0	81	27%	63%	15936.50	44.17	196.75	15961.94	44.18	197.06	1.00
29	clasp	2.0-R4092-indust	81	27%	63%	19618.65	76.15	242.21	19615.92	76.15	242.17	1.00
30	SApperloT2010	2011-05-15 (fixed)	79	26%	62%	14014.71	39.27	177.40	14023.54	39.27	177.51	1.00
31	SArTagnan	2011-05-15 (fixed)	79	26%	62%	48164.77	233.73	609.68	14117.75	69.02	178.71	3.41
32	priss	Auto	78	26%	61%	74939.13	233.12	960.76	19032.17	60.97	244.00	3.94
33	minisat hack: minisat_psm	2011-04-01	77	26%	60%	10876.22	33.92	141.25	10878.42	33.92	141.28	1.00
34	minisat hack: EBMiniSAT	2011-03-02	77	26%	60%	11810.79	39.77	153.39	11810.97	39.77	153.39	1.00
35	priss	1.0	77	26%	60%	66907.46	223.33	868.93	16901.06	56.07	219.49	3.96
36	Satisfiability Solver Selector	2011-03-18	76	25%	59%	16332.05	121.41	214.90	16380.86	121.52	215.54	1.00
37	SApperloT	2011-03-23	76	25%	59%	16895.51	83.07	222.31	16895.93	83.07	222.31	1.00
38	riss	1.0	75	25%	59%	16108.33	58.27	214.78	16110.08	60.03	214.80	1.00
39	riss	Auto	75	25%	59%	16199.85	57.12	216.00	16199.90	57.12	216.00	1.00
40	minisat hack: MiniSAT	2.2.0-agile-26	74	25%	58%	12602.74	29.00	170.31	12602.91	29.01	170.31	1.00
41	minisat hack: cir_minisat	2011-03-02	74	25%	58%	17228.44	95.60	232.82	17228.60	95.61	232.82	1.00
42	minisat hack reference: minisat	2.2.0	73	24%	57%	11643.52	27.88	159.50	11643.81	27.88	159.50	1.00
43	riss	Auto (1200 s timeout)	73	24%	57%	14038.29	56.23	192.31	14034.01	56.23	192.25	1.00
44	priss	UP	73	24%	57%	24172.32	94.17	331.13	14505.91	56.23	198.71	1.67
45	rcl	2011-04-01	73	24%	57%	14592.36	46.94	199.90	14632.43	46.94	200.44	1.00
46	minisat hack: contrasat	2011-03-02	72	24%	56%	10939.50	42.51	151.94	10946.96	42.55	152.04	1.00
47	minisat hack: LR_GL_NOSHR	2011-03-02	71	24%	55%	9541.22	25.19	134.38	9543.90	25.90	134.42	1.00
48	minisat hack: LR_NOGL_NOSHR	2011-03-02	71	24%	55%	9995.03	38.40	140.78	9991.02	38.41	140.72	1.00
49	SAT09 reference solver:LySAT	i/2009-03-20	71	24%	55%	12220.56	41.76	172.12	12239.64	41.76	172.39	1.00
50	CryptoMiniSat	TechDemo-Alpha-mt	71	24%	55%	66747.31	370.42	940.10	17174.91	114.77	241.90	3.89
51	riss	Exp	69	23%	54%	13874.48	61.02	201.08	13874.40	61.02	201.08	1.00
52	sathys	2011-04-01	68	23%	53%	14146.91	49.03	208.04	14148.73	49.04	208.07	1.00
53	PicoSAT	941	66	22%	52%	13716.32	85.93	207.82	13716.19	85.93	207.82	1.00
54	jMiniSat	2011	61	20%	48%	18681.23	110.27	306.25	18593.55	109.40	304.81	1.00
55	Sol	2011-04-04	60	20%	47%	15694.54	103.49	261.58	15613.50	103.15	260.23	1.01
56	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	55	18%	43%	17763.50	129.41	322.97	17699.63	128.42	321.81	1.00
57	UmSat	2011-05-12 (fixed)	43	14%	34%	11357.62	139.41	264.13	11357.77	139.41	264.13	1.00
58	STRUCTure	0.1 (hbr=2)	14	5%	11%	9530.70	82.16	680.76	2570.84	36.02	183.63	3.71
59	STRUCTure	0.1	14	5%	11%	16077.44	83.74	1148.39	4145.21	36.15	296.09	3.88
60	march_rw	2011-03-02	8	3%	6%	1712.90	21.91	214.11	1712.95	21.91	214.12	1.00
61	shortsattime+	2011-03-02	7	2%	5%	1721.10	4.13	245.87	1721.22	4.13	245.89	1.00
62	sattime+	2011-03-02	7	2%	5%	1755.45	4.12	250.78	1755.47	4.12	250.78	1.00
63	Black Hole SAT	2011-03-08	5	2%	4%	485.16	0.03	97.03	485.19	0.03	97.04	1.00
64	sattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
65	sattime	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
66	shortsattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
67	adaptg2wsat2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "crafted instances" (CRAFTED): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			207	143 SAT, 64 UNSAT	69%	100%	43913.53	1.70	212.14	-	-	-	-
1	ppfolio	par	151	113 SAT, 38 UNSAT	50%	73%	63874.52	15.12	423.01	16911.03	3.82	111.99	3.78
2	ppfolio	seq	138	106 SAT, 32 UNSAT	46%	67%	17991.42	5.72	130.37	17998.11	5.73	130.42	1.00
3	Satisfiability Solver Selector	2011-03-18	136	107 SAT, 29 UNSAT	45%	66%	24195.73	30.19	177.91	24343.85	30.60	179.00	0.99
4	claspmt	2.0-R4095-patched (fixed)	133	90 SAT, 43 UNSAT	44%	64%	94520.12	124.26	710.68	23743.50	31.37	178.52	3.98
5	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	131	100 SAT, 31 UNSAT	44%	63%	23183.51	13.77	176.97	23312.05	16.35	177.95	0.99
6	CryptoMiniSat	Strange-Night-1-mt (fixed)	113	76 SAT, 37 UNSAT	38%	55%	74145.90	66.97	656.16	18751.73	16.80	165.94	3.95
7	MPhaseSAT	2011-02-15	107	81 SAT, 26 UNSAT	36%	52%	14248.05	5.74	133.16	14248.58	5.74	133.16	1.00
8	MPhaseSAT64	2011-05-14 (fixed)	107	80 SAT, 27 UNSAT	36%	52%	18057.38	5.42	168.76	18057.80	5.42	168.76	1.00
9	CryptoMiniSat	Strange-Night-2-mt (fixed)	105	69 SAT, 36 UNSAT	35%	51%	56604.33	32.59	539.09	14206.55	8.26	135.30	3.98
10	sattime2011	2011-03-02	104	104 SAT	35%	50%	6150.28	1.56	59.14	6151.22	1.56	59.15	1.00
11	shortsattime+	2011-03-02	104	104 SAT	35%	50%	11191.41	3.48	107.61	11191.35	3.48	107.61	1.00
12	MPhaseSAT_M	2011-02-16	104	79 SAT, 25 UNSAT	35%	50%	13199.41	3.00	126.92	13199.24	3.00	126.92	1.00
13	shortsattime2011	2011-03-02	103	103 SAT	34%	50%	6537.74	0.91	63.47	6538.19	0.91	63.48	1.00
14	sattime	2011-03-02	103	103 SAT	34%	50%	8489.33	0.60	82.42	8489.35	0.60	82.42	1.00
15	Sol	2011-04-04	103	73 SAT, 30 UNSAT	34%	50%	11258.24	3.58	109.30	11174.26	3.56	108.49	1.01
16	Plingeling	587f (fixed)	103	73 SAT, 30 UNSAT	34%	50%	62216.91	16.14	604.05	15638.01	4.17	151.83	3.98
17	sattime+	2011-03-02	102	102 SAT	34%	49%	9242.73	1.90	90.61	9243.00	1.90	90.62	1.00
18	clasp	2.0-R4092-crafted	102	67 SAT, 35 UNSAT	34%	49%	18146.98	33.86	177.91	18147.45	33.86	177.92	1.00
19	adaptg2wsat2011	2011-03-02	99	99 SAT	33%	48%	7748.97	0.67	78.27	7749.27	0.68	78.28	1.00
20	RestartSAT	B95	94	66 SAT, 28 UNSAT	31%	45%	16670.27	16.78	177.34	16670.07	16.78	177.34	1.00
21	SApperloT2010	2011-05-15 (fixed)	93	67 SAT, 26 UNSAT	31%	45%	15410.65	8.73	165.71	15410.84	8.73	165.71	1.00
22	QuteRSat	2011-05-12 (fixed)	93	61 SAT, 32 UNSAT	31%	45%	15580.41	14.22	167.53	15592.95	14.22	167.67	1.00
23	CryptoMiniSat	Strange-Night2-st (fixed)	93	63 SAT, 30 UNSAT	31%	45%	16181.96	16.28	174.00	16182.10	16.28	174.00	1.00
24	PicoSAT	941	92	65 SAT, 27 UNSAT	31%	44%	18030.14	12.63	195.98	18030.05	12.63	195.98	1.00
25	SAT09 reference solver: clasp	1.2.0-SAT09-32	91	58 SAT, 33 UNSAT	30%	44%	15729.84	30.60	172.86	15729.79	30.60	172.85	1.00
26	glucose	2	90	60 SAT, 30 UNSAT	30%	43%	16321.44	23.28	181.35	16321.46	23.28	181.35	1.00
27	Lingeling	587f (fixed)	89	62 SAT, 27 UNSAT	30%	43%	11681.10	5.64	131.25	11681.22	5.64	131.25	1.00
28	CryptoMiniSat	Strange-Night-1-st (fixed)	89	57 SAT, 32 UNSAT	30%	43%	17914.86	20.97	201.29	17914.95	20.97	201.29	1.00
29	sathys	2011-04-01	88	68 SAT, 20 UNSAT	29%	43%	12078.28	1.08	137.25	12078.43	1.08	137.25	1.00
30	RestartSAT	A75	88	62 SAT, 26 UNSAT	29%	43%	15664.64	7.71	178.01	15664.47	7.71	178.01	1.00
31	SAT07 reference solver: minisat	SAT 2007	87	57 SAT, 30 UNSAT	29%	42%	12077.12	6.52	138.82	12076.96	6.53	138.82	1.00
32	rcl	2011-04-01	86	61 SAT, 25 UNSAT	29%	42%	14280.09	5.07	166.05	14281.47	5.07	166.06	1.00
33	SArTagnan	2011-05-15 (fixed)	85	60 SAT, 25 UNSAT	28%	41%	37827.49	7.64	445.03	11956.27	2.54	140.66	3.16
34	SApperloT	2011-03-23	85	58 SAT, 27 UNSAT	28%	41%	12483.09	25.03	146.86	12483.00	25.03	146.86	1.00
35	jMiniSat	2011	84	61 SAT, 23 UNSAT	28%	41%	17543.21	15.44	208.85	17507.50	15.04	208.42	1.00
36	priss	1.0	82	62 SAT, 20 UNSAT	27%	40%	45279.02	12.16	552.18	11339.61	3.07	138.29	3.99
37	priss	Auto	81	61 SAT, 20 UNSAT	27%	39%	40098.15	10.99	495.04	10042.44	2.77	123.98	3.99
38	SAT09 reference solver: SATzilla2009_C	2009-03-22	81	50 SAT, 31 UNSAT	27%	39%	12052.55	5.01	148.80	12044.70	5.06	148.70	1.00
39	march_rw	2011-03-02	80	55 SAT, 25 UNSAT	27%	39%	11117.06	0.62	138.96	11116.91	0.62	138.96	1.00
40	priss	UP	80	58 SAT, 22 UNSAT	27%	39%	19915.08	10.27	248.94	11248.46	7.12	140.61	1.77
41	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	77	53 SAT, 24 UNSAT	26%	37%	9551.80	17.65	124.05	9500.91	16.81	123.39	1.01
42	riss	1.0	76	55 SAT, 21 UNSAT	25%	37%	11220.54	3.23	147.64	11220.43	3.23	147.64	1.00
43	riss	Auto	76	55 SAT, 21 UNSAT	25%	37%	11245.46	3.19	147.97	11245.89	3.19	147.97	1.00
44	riss	Auto (1200 s timeout)	76	54 SAT, 22 UNSAT	25%	37%	11537.35	3.07	151.81	11537.62	3.09	151.81	1.00
45	riss	Exp	74	54 SAT, 20 UNSAT	25%	36%	11913.11	3.49	160.99	11913.54	3.49	160.99	1.00
46	CryptoMiniSat	TechDemo-Alpha-mt	71	46 SAT, 25 UNSAT	24%	34%	29873.23	25.87	420.75	8300.39	7.14	116.91	3.60
47	UmSat	2011-05-12 (fixed)	68	49 SAT, 19 UNSAT	23%	33%	9046.48	2.42	133.04	9046.55	2.42	133.04	1.00
48	STRUCTure	0.1 (hbr=2)	61	44 SAT, 17 UNSAT	20%	29%	42977.60	12.26	704.55	10847.90	3.35	177.83	3.96
49	STRUCTure	0.1	60	43 SAT, 17 UNSAT	20%	29%	34629.70	9.19	577.16	8747.94	2.79	145.80	3.96
50	ttsth	5-4	46	26 SAT, 20 UNSAT	15%	22%	29075.16	30.32	632.07	7397.66	11.30	160.82	3.93
51	tts	5-4	37	20 SAT, 17 UNSAT	12%	18%	4581.72	7.70	123.83	4581.82	7.70	123.83	1.00
52	Black Hole SAT	2011-03-08	36	25 SAT, 11 UNSAT	12%	17%	3583.16	0.17	99.53	3583.17	0.17	99.53	1.00

Category "crafted instances" (CRAFTED): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			143	48%	100%	5283.63	0.73	36.95	-	-	-	-
1	ppfolio	par	113	38%	79%	37643.49	8.33	333.13	9732.28	2.11	86.13	3.87
2	Satisfiability Solver Selector	2011-03-18	107	36%	75%	16430.43	30.63	153.56	16573.80	30.81	154.90	0.99
3	ppfolio	seq	106	35%	74%	10189.02	3.73	96.12	10194.89	3.76	96.18	1.00
4	sattime2011	2011-03-02	104	35%	73%	6150.28	1.56	59.14	6151.22	1.56	59.15	1.00
5	shortsattime+	2011-03-02	104	35%	73%	11191.41	3.48	107.61	11191.35	3.48	107.61	1.00
6	shortsattime2011	2011-03-02	103	34%	72%	6537.74	0.91	63.47	6538.19	0.91	63.48	1.00
7	sattime	2011-03-02	103	34%	72%	8489.33	0.60	82.42	8489.35	0.60	82.42	1.00
8	sattime+	2011-03-02	102	34%	71%	9242.73	1.90	90.61	9243.00	1.90	90.62	1.00
9	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	100	33%	70%	13354.24	12.87	133.54	13460.97	14.00	134.61	0.99
10	adaptg2wsat2011	2011-03-02	99	33%	69%	7748.97	0.67	78.27	7749.27	0.68	78.28	1.00
11	claspmt	2.0-R4095-patched (fixed)	90	30%	63%	56986.94	65.08	633.19	14330.98	16.40	159.23	3.98
12	MPhaseSAT	2011-02-15	81	27%	57%	9128.99	6.64	112.70	9129.53	6.64	112.71	1.00
13	MPhaseSAT64	2011-05-14 (fixed)	80	27%	56%	11679.86	4.75	146.00	11680.33	4.75	146.00	1.00
14	MPhaseSAT_M	2011-02-16	79	26%	55%	8939.49	2.77	113.16	8939.37	2.77	113.16	1.00
15	CryptoMiniSat	Strange-Night-1-mt (fixed)	76	25%	53%	38976.71	28.48	512.85	9924.04	7.49	130.58	3.93
16	Sol	2011-04-04	73	24%	51%	6388.07	2.83	87.51	6345.08	2.80	86.92	1.01
17	Plingeling	587f (fixed)	73	24%	51%	34261.07	12.10	469.33	8609.15	3.09	117.93	3.98
18	CryptoMiniSat	Strange-Night-2-mt (fixed)	69	23%	48%	28357.37	15.79	410.98	7124.37	4.69	103.25	3.98
19	sathys	2011-04-01	68	23%	48%	7805.24	0.42	114.78	7805.36	0.44	114.78	1.00
20	SApperloT2010	2011-05-15 (fixed)	67	22%	47%	9368.34	3.73	139.83	9368.61	3.73	139.83	1.00
21	clasp	2.0-R4092-crafted	67	22%	47%	10213.42	16.32	152.44	10214.06	16.33	152.45	1.00
22	RestartSAT	B95	66	22%	46%	8613.43	11.04	130.51	8613.37	11.05	130.51	1.00
23	PicoSAT	941	65	22%	45%	10587.91	6.89	162.89	10587.85	6.90	162.89	1.00
24	CryptoMiniSat	Strange-Night2-st (fixed)	63	21%	44%	10783.51	20.48	171.17	10783.60	20.49	171.17	1.00
25	Lingeling	587f (fixed)	62	21%	43%	5797.21	2.71	93.50	5797.29	2.71	93.50	1.00
26	priss	1.0	62	21%	43%	33920.86	13.66	547.11	8495.52	3.45	137.02	3.99
27	RestartSAT	A75	62	21%	43%	8851.21	2.69	142.76	8851.16	2.70	142.76	1.00
28	rcl	2011-04-01	61	20%	43%	7208.65	2.57	118.17	7210.06	2.57	118.20	1.00
29	priss	Auto	61	20%	43%	29364.77	13.82	481.39	7354.65	3.48	120.57	3.99
30	QuteRSat	2011-05-12 (fixed)	61	20%	43%	8897.04	13.33	145.85	8909.64	13.33	146.06	1.00
31	jMiniSat	2011	61	20%	43%	11716.47	2.35	192.07	11693.65	1.89	191.70	1.00
32	SArTagnan	2011-05-15 (fixed)	60	20%	42%	25345.97	1.01	422.43	7959.38	0.34	132.66	3.18
33	glucose	2	60	20%	42%	10509.43	21.30	175.16	10509.43	21.30	175.16	1.00
34	priss	UP	58	19%	41%	13361.99	10.65	230.38	7673.49	7.19	132.30	1.74
35	SApperloT	2011-03-23	58	19%	41%	8546.38	20.77	147.35	8546.33	20.77	147.35	1.00
36	SAT09 reference solver: clasp	1.2.0-SAT09-32	58	19%	41%	8896.80	10.92	153.39	8896.75	10.92	153.39	1.00
37	SAT07 reference solver: minisat	SAT 2007	57	19%	40%	7500.75	2.22	131.59	7500.65	2.22	131.59	1.00
38	CryptoMiniSat	Strange-Night-1-st (fixed)	57	19%	40%	10124.37	20.50	177.62	10124.45	20.50	177.62	1.00
39	march_rw	2011-03-02	55	18%	38%	7349.38	0.41	133.63	7349.25	0.41	133.62	1.00
40	riss	Auto	55	18%	38%	7492.64	2.93	136.23	7493.13	2.94	136.24	1.00
41	riss	1.0	55	18%	38%	7537.77	2.79	137.05	7537.70	2.79	137.05	1.00
42	riss	Auto (1200 s timeout)	54	18%	38%	6436.95	2.62	119.20	6437.20	2.62	119.21	1.00
43	riss	Exp	54	18%	38%	9127.51	3.05	169.03	9127.37	3.05	169.03	1.00
44	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	53	18%	37%	5828.50	11.30	109.97	5801.02	10.43	109.45	1.00
45	SAT09 reference solver: SATzilla2009_C	2009-03-22	50	17%	35%	4757.99	1.06	95.16	4755.06	1.06	95.10	1.00
46	UmSat	2011-05-12 (fixed)	49	16%	34%	6337.98	0.18	129.35	6338.05	0.18	129.35	1.00
47	CryptoMiniSat	TechDemo-Alpha-mt	46	15%	32%	14643.03	3.95	318.33	4472.96	1.03	97.24	3.27
48	STRUCTure	0.1 (hbr=2)	44	15%	31%	32887.34	8.48	747.44	8292.56	2.55	188.47	3.97
49	STRUCTure	0.1	43	14%	30%	24138.46	7.68	561.36	6092.39	2.30	141.68	3.96
50	ttsth	5-4	26	9%	18%	9056.03	25.45	348.31	2310.78	11.30	88.88	3.92
51	Black Hole SAT	2011-03-08	25	8%	17%	2752.49	0.03	110.10	2752.48	0.03	110.10	1.00
52	tts	5-4	20	7%	14%	1703.75	8.62	85.19	1703.75	8.62	85.19	1.00

Category "crafted instances" (CRAFTED): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			64	21%	100%	38629.90	108.23	603.59	-	-	-	-
1	claspmt	2.0-R4095-patched (fixed)	43	14%	67%	37533.17	283.86	872.86	9412.52	71.18	218.90	3.99
2	ppfolio	par	38	13%	59%	26231.03	107.78	690.29	7178.76	27.06	188.91	3.65
3	CryptoMiniSat	Strange-Night-1-mt (fixed)	37	12%	58%	35169.19	286.98	950.52	8827.69	72.04	238.59	3.98
4	CryptoMiniSat	Strange-Night-2-mt (fixed)	36	12%	56%	28246.96	208.74	784.64	7082.17	52.35	196.73	3.99
5	clasp	2.0-R4092-crafted	35	12%	55%	7933.56	64.98	226.67	7933.39	64.98	226.67	1.00
6	SAT09 reference solver: clasp	1.2.0-SAT09-32	33	11%	52%	6833.03	88.30	207.06	6833.03	88.30	207.06	1.00
7	QuteRSat	2011-05-12 (fixed)	32	11%	50%	6683.36	33.72	208.86	6683.32	33.74	208.85	1.00
8	CryptoMiniSat	Strange-Night-1-st (fixed)	32	11%	50%	7790.49	55.46	243.45	7790.50	55.46	243.45	1.00
9	ppfolio	seq	32	11%	50%	7802.40	64.16	243.82	7803.22	64.16	243.85	1.00
10	SAT09 reference solver: SATzilla2009_C	2009-03-22	31	10%	48%	7294.56	19.36	235.31	7289.64	19.37	235.15	1.00
11	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	31	10%	48%	9829.27	24.64	317.07	9851.08	24.74	317.78	1.00
12	SAT07 reference solver: minisat	SAT 2007	30	10%	47%	4576.37	63.76	152.55	4576.31	63.76	152.54	1.00
13	Sol	2011-04-04	30	10%	47%	4870.17	14.06	162.34	4829.17	13.97	160.97	1.01
14	CryptoMiniSat	Strange-Night2-st (fixed)	30	10%	47%	5398.45	13.06	179.95	5398.50	13.06	179.95	1.00
15	glucose	2	30	10%	47%	5812.02	71.66	193.73	5812.03	71.67	193.73	1.00
16	Plingeling	587f (fixed)	30	10%	47%	27955.84	167.71	931.86	7028.86	42.26	234.30	3.98
17	Satisfiability Solver Selector	2011-03-18	29	10%	45%	7765.30	24.84	267.77	7770.04	24.98	267.93	1.00
18	RestartSAT	B95	28	9%	44%	8056.84	187.34	287.74	8056.69	187.33	287.74	1.00
19	SApperloT	2011-03-23	27	9%	42%	3936.70	112.08	145.80	3936.66	112.09	145.80	1.00
20	Lingeling	587f (fixed)	27	9%	42%	5883.89	27.47	217.92	5883.92	27.47	217.92	1.00
21	MPhaseSAT64	2011-05-14 (fixed)	27	9%	42%	6377.52	6.65	236.20	6377.48	6.65	236.20	1.00
22	PicoSAT	941	27	9%	42%	7442.23	187.28	275.64	7442.20	187.27	275.64	1.00
23	MPhaseSAT	2011-02-15	26	9%	41%	5119.05	4.97	196.89	5119.06	4.98	196.89	1.00
24	SApperloT2010	2011-05-15 (fixed)	26	9%	41%	6042.31	139.46	232.40	6042.23	139.46	232.39	1.00
25	RestartSAT	A75	26	9%	41%	6813.43	28.67	262.06	6813.31	28.68	262.05	1.00
26	march_rw	2011-03-02	25	8%	39%	3767.68	4.97	150.71	3767.66	4.97	150.71	1.00
27	CryptoMiniSat	TechDemo-Alpha-mt	25	8%	39%	15230.20	107.34	609.21	3827.43	27.15	153.10	3.98
28	SArTagnan	2011-05-15 (fixed)	25	8%	39%	12481.52	49.17	499.26	3996.89	12.35	159.88	3.12
29	MPhaseSAT_M	2011-02-16	25	8%	39%	4259.92	3.24	170.40	4259.87	3.24	170.39	1.00
30	rcl	2011-04-01	25	8%	39%	7071.44	165.91	282.86	7071.41	165.91	282.86	1.00
31	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	24	8%	38%	3723.30	34.57	155.14	3699.89	33.24	154.16	1.01
32	jMiniSat	2011	23	8%	36%	5826.74	78.11	253.34	5813.85	77.66	252.78	1.00
33	priss	UP	22	7%	34%	6553.10	10.27	297.87	3574.97	7.12	162.50	1.83
34	riss	Auto (1200 s timeout)	22	7%	34%	5100.40	6.44	231.84	5100.42	6.44	231.84	1.00
35	riss	1.0	21	7%	33%	3682.77	3.57	175.37	3682.72	3.57	175.37	1.00
36	riss	Auto	21	7%	33%	3752.82	3.59	178.71	3752.77	3.59	178.70	1.00
37	priss	Auto	20	7%	31%	10733.39	10.42	536.67	2687.80	2.62	134.39	3.99
38	riss	Exp	20	7%	31%	2785.61	10.12	139.28	2786.17	10.31	139.31	1.00
39	priss	1.0	20	7%	31%	11358.16	9.97	567.91	2844.09	2.51	142.20	3.99
40	sathys	2011-04-01	20	7%	31%	4273.04	12.75	213.65	4273.07	12.75	213.65	1.00
41	ttsth	5-4	20	7%	31%	20019.13	157.86	1000.96	5086.88	40.85	254.34	3.94
42	UmSat	2011-05-12 (fixed)	19	6%	30%	2708.50	13.33	142.55	2708.50	13.33	142.55	1.00
43	STRUCTure	0.1 (hbr=2)	17	6%	27%	10090.26	94.50	593.54	2555.33	25.83	150.31	3.95
44	STRUCTure	0.1	17	6%	27%	10491.24	92.49	617.13	2655.55	24.38	156.21	3.95
45	tts	5-4	17	6%	27%	2877.96	7.63	169.29	2878.07	7.63	169.30	1.00
46	Black Hole SAT	2011-03-08	11	4%	17%	830.67	12.68	75.52	830.69	12.68	75.52	1.00
47	sattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
48	sattime	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
49	shortsattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
50	adaptg2wsat2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
51	shortsattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
52	sattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "random instances" (RANDOM): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 600
Virtual Best Solver (VBS)			466	358 SAT, 108 UNSAT	78%	100%	43024.93	1.68	92.33	-	-	-	-
1	borg-sat	11.03.02-par-random (fixed)	408	302 SAT, 106 UNSAT	68%	88%	111071.89	9.00	272.24	30067.64	3.26	73.70	3.69
2	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	399	293 SAT, 106 UNSAT	67%	86%	48566.02	20.13	121.72	48654.12	20.44	121.94	1.00
3	borg-sat	11.03.02-seq-random (fixed)	398	292 SAT, 106 UNSAT	66%	85%	52173.90	14.08	131.09	52370.05	14.43	131.58	1.00
4	Satisfiability Solver Selector	2011-03-18	379	283 SAT, 96 UNSAT	63%	81%	81047.07	37.35	213.84	81640.96	38.84	215.41	0.99
5	ppfolio	par	375	273 SAT, 102 UNSAT	63%	80%	181124.36	11.75	483.00	46724.06	3.17	124.60	3.88
6	SAT09 reference solver: SATzilla2009_R	2009-03-22	351	268 SAT, 83 UNSAT	59%	75%	36827.82	3.36	104.92	36828.62	3.36	104.92	1.00
7	sparrow2011	sparrow2011_ubcsat1.2_2011-03-02	342	342 SAT	57%	73%	29318.01	2.09	85.73	29318.37	2.10	85.73	1.00
8	MPhaseSAT_M	2011-02-16	341	242 SAT, 99 UNSAT	57%	73%	34895.57	6.50	102.33	34896.22	6.51	102.33	1.00
9	ppfolio	seq	322	262 SAT, 60 UNSAT	54%	69%	39143.95	4.23	121.57	39152.56	4.19	121.59	1.00
10	EagleUP	1.565.350	313	313 SAT	52%	67%	19498.17	0.74	62.29	19498.49	0.74	62.30	1.00
11	MPhaseSAT64	2011-05-14 (fixed)	312	243 SAT, 69 UNSAT	52%	67%	37258.37	2.27	119.42	37258.40	2.28	119.42	1.00
12	MPhaseSAT	2011-02-15	308	241 SAT, 67 UNSAT	51%	66%	30441.77	2.09	98.84	30442.48	2.10	98.84	1.00
13	csls	pnorm-8cores	307	307 SAT	51%	66%	85817.16	1.46	279.53	21483.86	0.37	69.98	3.99
14	sattime2011	2011-03-02	307	307 SAT	51%	66%	40428.29	1.87	131.69	40432.46	1.96	131.70	1.00
15	shortsattime2011	2011-03-02	304	304 SAT	51%	65%	39438.09	1.84	129.73	39437.83	1.73	129.73	1.00
16	csls	prob-4cores	303	303 SAT	51%	65%	78858.12	0.89	260.26	19742.54	0.23	65.16	3.99
17	csls	prob-8cores	301	301 SAT	50%	65%	70451.06	0.90	234.06	17637.07	0.23	58.59	3.99
18	adaptg2wsat2011	2011-03-02	299	299 SAT	50%	64%	40184.75	1.59	134.40	40184.47	1.60	134.40	1.00
19	SAT09 reference solver: TNM	2009-03-22	292	292 SAT	49%	63%	29562.50	1.27	101.24	29562.68	1.27	101.24	1.00
20	SAT09 reference solver: gnovelty+2	2009-03-22	275	275 SAT	46%	59%	23349.21	0.57	84.91	23349.09	0.58	84.91	1.00
21	MultiFlip	2011-02-11	272	272 SAT	45%	58%	22181.31	0.83	81.55	22181.00	0.83	81.55	1.00
22	sattime	2011-03-02	250	250 SAT	42%	54%	16641.40	0.65	66.57	16641.45	0.65	66.57	1.00
23	sattime+	2011-03-02	216	216 SAT	36%	46%	16294.70	1.75	75.44	16286.41	1.75	75.40	1.00
24	march_rw	2011-03-02	212	106 SAT, 106 UNSAT	35%	45%	27576.88	23.68	130.08	27576.97	23.69	130.08	1.00
25	shortsattime+	2011-03-02	211	211 SAT	35%	45%	14703.56	1.61	69.69	14706.10	1.61	69.70	1.00
26	SAT09 reference solver: march_hi	hi	211	105 SAT, 106 UNSAT	35%	45%	29171.96	22.76	138.26	29171.86	22.76	138.26	1.00
27	sathys	2011-04-01	181	159 SAT, 22 UNSAT	30%	39%	21875.26	22.71	120.86	21876.84	22.71	120.87	1.00
28	STRUCTure	0.1 (hbr=2)	107	47 SAT, 60 UNSAT	18%	23%	78320.48	354.29	731.97	19767.07	89.69	184.74	3.96
29	STRUCTure	0.1	103	43 SAT, 60 UNSAT	17%	22%	79533.32	380.15	772.17	20071.03	96.56	194.86	3.96
30	CryptoMiniSat	Strange-Night-2-mt (fixed)	89	40 SAT, 49 UNSAT	15%	19%	82734.07	239.02	929.60	20744.53	59.91	233.08	3.99
31	CryptoMiniSat	Strange-Night-1-mt (fixed)	86	37 SAT, 49 UNSAT	14%	18%	81195.29	243.71	944.13	20367.95	61.08	236.84	3.99
32	RestartSAT	B95	82	34 SAT, 48 UNSAT	14%	18%	20973.75	84.39	255.78	20973.43	84.39	255.77	1.00
33	CryptoMiniSat	Strange-Night2-st (fixed)	80	32 SAT, 48 UNSAT	13%	17%	23145.14	132.29	289.31	23145.12	132.29	289.31	1.00
34	QuteRSat	2011-05-12 (fixed)	79	30 SAT, 49 UNSAT	13%	17%	21395.40	172.17	270.83	21400.91	172.17	270.90	1.00
35	jMiniSat	2011	79	37 SAT, 42 UNSAT	13%	17%	22353.44	101.10	282.95	22316.56	100.81	282.49	1.00
36	RestartSAT	A75	76	38 SAT, 38 UNSAT	13%	16%	21608.24	102.90	284.32	21608.76	102.90	284.33	1.00
37	CryptoMiniSat	Strange-Night-1-st (fixed)	74	26 SAT, 48 UNSAT	12%	16%	19478.78	94.79	263.23	19478.74	94.78	263.23	1.00
38	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	65	33 SAT, 32 UNSAT	11%	14%	18743.17	117.48	288.36	18683.92	117.15	287.44	1.00
39	CryptoMiniSat	TechDemo-Alpha-mt	59	28 SAT, 31 UNSAT	10%	13%	62626.37	332.47	1061.46	15696.77	83.70	266.05	3.99
40	Sol	2011-04-04	52	22 SAT, 30 UNSAT	9%	11%	11375.45	46.15	218.76	11325.16	45.86	217.79	1.00
41	Black Hole SAT	2011-03-08	44	18 SAT, 26 UNSAT	7%	9%	19503.25	295.15	443.26	19502.98	295.15	443.25	1.00
42	SAT_Intersec_01	2011-03-01	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "random instances" (RANDOM): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 600
Virtual Best Solver (VBS)			358	60%	100%	26493.64	0.45	74.00	-	-	-	-
1	sparrow2011	sparrow2011_ubcsat1.2_2011-03-02	342	57%	96%	29318.01	2.09	85.73	29318.37	2.10	85.73	1.00
2	EagleUP	1.565.350	313	52%	87%	19498.17	0.74	62.29	19498.49	0.74	62.30	1.00
3	csls	pnorm-8cores	307	51%	86%	85817.16	1.46	279.53	21483.86	0.37	69.98	3.99
4	sattime2011	2011-03-02	307	51%	86%	40428.29	1.87	131.69	40432.46	1.96	131.70	1.00
5	shortsattime2011	2011-03-02	304	51%	85%	39438.09	1.84	129.73	39437.83	1.73	129.73	1.00
6	csls	prob-4cores	303	51%	85%	78858.12	0.89	260.26	19742.54	0.23	65.16	3.99
7	borg-sat	11.03.02-par-random (fixed)	302	50%	84%	48862.35	4.73	161.80	13012.83	2.10	43.09	3.75
8	csls	prob-8cores	301	50%	84%	70451.06	0.90	234.06	17637.07	0.23	58.59	3.99
9	adaptg2wsat2011	2011-03-02	299	50%	84%	40184.75	1.59	134.40	40184.47	1.60	134.40	1.00
10	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	293	49%	82%	29619.17	18.60	101.09	29733.95	18.83	101.48	1.00
11	borg-sat	11.03.02-seq-random (fixed)	292	49%	82%	15865.63	2.03	54.33	16063.13	3.06	55.01	0.99
12	SAT09 reference solver: TNM	2009-03-22	292	49%	82%	29562.50	1.27	101.24	29562.68	1.27	101.24	1.00
13	Satisfiability Solver Selector	2011-03-18	283	47%	79%	53116.01	35.35	187.69	53608.37	35.95	189.43	0.99
14	SAT09 reference solver: gnovelty+2	2009-03-22	275	46%	77%	23349.21	0.57	84.91	23349.09	0.58	84.91	1.00
15	ppfolio	par	273	46%	76%	99905.56	6.13	365.95	25188.02	1.63	92.26	3.97
16	MultiFlip	2011-02-11	272	45%	76%	22181.31	0.83	81.55	22181.00	0.83	81.55	1.00
17	SAT09 reference solver: SATzilla2009_R	2009-03-22	268	45%	75%	23400.86	0.68	87.32	23401.42	0.68	87.32	1.00
18	ppfolio	seq	262	44%	73%	22877.90	2.12	87.32	22887.86	2.12	87.36	1.00
19	sattime	2011-03-02	250	42%	70%	16641.40	0.65	66.57	16641.45	0.65	66.57	1.00
20	MPhaseSAT64	2011-05-14 (fixed)	243	41%	68%	14339.56	1.18	59.01	14339.81	1.18	59.01	1.00
21	MPhaseSAT_M	2011-02-16	242	40%	68%	15962.15	1.32	65.96	15962.42	1.32	65.96	1.00
22	MPhaseSAT	2011-02-15	241	40%	67%	14085.63	1.26	58.45	14086.43	1.27	58.45	1.00
23	sattime+	2011-03-02	216	36%	60%	16294.70	1.75	75.44	16286.41	1.75	75.40	1.00
24	shortsattime+	2011-03-02	211	35%	59%	14703.56	1.61	69.69	14706.10	1.61	69.70	1.00
25	sathys	2011-04-01	159	27%	44%	19287.48	22.56	121.30	19288.78	22.56	121.31	1.00
26	march_rw	2011-03-02	106	18%	30%	13430.83	18.56	126.71	13430.76	18.56	126.71	1.00
27	SAT09 reference solver: march_hi	hi	105	18%	29%	14326.32	17.51	136.44	14326.27	17.51	136.44	1.00
28	STRUCTure	0.1 (hbr=2)	47	8%	13%	40817.21	441.14	868.45	10276.33	111.24	218.65	3.97
29	STRUCTure	0.1	43	7%	12%	38694.82	411.02	899.88	9741.70	103.64	226.55	3.97
30	CryptoMiniSat	Strange-Night-2-mt (fixed)	40	7%	11%	38361.31	408.48	959.03	9619.51	102.41	240.49	3.99
31	RestartSAT	A75	38	6%	11%	9559.74	159.27	251.57	9559.82	159.27	251.57	1.00
32	CryptoMiniSat	Strange-Night-1-mt (fixed)	37	6%	10%	36136.84	349.36	976.67	9070.34	88.60	245.14	3.98
33	jMiniSat	2011	37	6%	10%	11180.17	221.88	302.17	11164.54	221.44	301.74	1.00
34	RestartSAT	B95	34	6%	9%	8008.92	118.41	235.56	8008.80	118.41	235.55	1.00
35	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	33	6%	9%	15245.65	475.78	461.99	15226.75	475.52	461.42	1.00
36	CryptoMiniSat	Strange-Night2-st (fixed)	32	5%	9%	9929.16	202.50	310.29	9929.11	202.49	310.28	1.00
37	QuteRSat	2011-05-12 (fixed)	30	5%	8%	7975.11	221.96	265.84	7975.17	221.98	265.84	1.00
38	CryptoMiniSat	TechDemo-Alpha-mt	28	5%	8%	47789.19	1276.98	1706.76	11976.37	319.95	427.73	3.99
39	CryptoMiniSat	Strange-Night-1-st (fixed)	26	4%	7%	6869.85	142.64	264.23	6869.93	142.69	264.23	1.00
40	Sol	2011-04-04	22	4%	6%	9060.70	222.73	411.85	9021.80	221.24	410.08	1.00
41	Black Hole SAT	2011-03-08	18	3%	5%	11463.61	695.93	636.87	11463.49	695.92	636.86	1.00
42	SAT_Intersec_01	2011-03-01	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "random instances" (RANDOM): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 600
Virtual Best Solver (VBS)			108	18%	100%	16531.29	29.14	153.07	-	-	-	-
1	march_rw	2011-03-02	106	18%	98%	14146.04	26.57	133.45	14146.21	26.57	133.45	1.00
2	SAT09 reference solver: march_hi	hi	106	18%	98%	14845.64	30.27	140.05	14845.59	30.27	140.05	1.00
3	borg-sat	11.03.02-par-random (fixed)	106	18%	98%	62209.53	87.98	586.88	17054.81	29.65	160.89	3.65
4	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	106	18%	98%	18946.85	71.86	178.74	18920.17	74.11	178.49	1.00
5	borg-sat	11.03.02-seq-random (fixed)	106	18%	98%	36308.27	224.23	342.53	36306.93	224.02	342.52	1.00
6	ppfolio	par	102	17%	94%	81218.80	190.89	796.26	21536.04	47.84	211.14	3.77
7	MPhaseSAT_M	2011-02-16	99	17%	92%	18933.42	64.90	191.25	18933.81	64.90	191.25	1.00
8	Satisfiability Solver Selector	2011-03-18	96	16%	89%	27931.06	170.13	290.95	28032.59	170.40	292.01	1.00
9	SAT09 reference solver: SATzilla2009_R	2009-03-22	83	14%	77%	13426.96	105.66	161.77	13427.20	105.66	161.77	1.00
10	MPhaseSAT64	2011-05-14 (fixed)	69	12%	64%	22918.80	119.21	332.16	22918.59	119.21	332.15	1.00
11	MPhaseSAT	2011-02-15	67	11%	62%	16356.14	121.23	244.12	16356.05	121.23	244.12	1.00
12	STRUCTure	0.1 (hbr=2)	60	10%	56%	37503.27	331.83	625.05	9490.74	84.75	158.18	3.95
13	STRUCTure	0.1	60	10%	56%	40838.50	356.13	680.64	10329.34	90.86	172.16	3.95
14	ppfolio	seq	60	10%	56%	16266.05	154.95	271.10	16264.70	155.02	271.08	1.00
15	CryptoMiniSat	Strange-Night-2-mt (fixed)	49	8%	45%	44372.76	102.62	905.57	11125.02	25.72	227.04	3.99
16	CryptoMiniSat	Strange-Night-1-mt (fixed)	49	8%	45%	45058.45	138.52	919.56	11297.60	34.73	230.56	3.99
17	QuteRSat	2011-05-12 (fixed)	49	8%	45%	13420.30	40.47	273.88	13425.74	40.47	273.99	1.00
18	CryptoMiniSat	Strange-Night-1-st (fixed)	48	8%	44%	12608.92	36.14	262.69	12608.82	36.14	262.68	1.00
19	RestartSAT	B95	48	8%	44%	12964.83	38.05	270.10	12964.63	38.05	270.10	1.00
20	CryptoMiniSat	Strange-Night2-st (fixed)	48	8%	44%	13215.98	38.39	275.33	13216.01	38.39	275.33	1.00
21	jMiniSat	2011	42	7%	39%	11173.27	36.65	266.03	11152.02	36.17	265.52	1.00
22	RestartSAT	A75	38	6%	35%	12048.50	47.67	317.07	12048.94	47.68	317.08	1.00
23	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	32	5%	30%	3497.52	9.47	109.30	3457.17	7.73	108.04	1.01
24	CryptoMiniSat	TechDemo-Alpha-mt	31	5%	29%	14837.17	45.66	478.62	3720.40	11.45	120.01	3.99
25	Sol	2011-04-04	30	5%	28%	2314.75	7.08	77.16	2303.37	7.01	76.78	1.00
26	Black Hole SAT	2011-03-08	26	4%	24%	8039.65	159.69	309.22	8039.49	159.69	309.21	1.00
27	sathys	2011-04-01	22	4%	20%	2587.78	46.17	117.63	2588.05	46.29	117.64	1.00
28	EagleUP	1.565.350	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
29	MultiFlip	2011-02-11	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
30	sattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
31	sattime	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
32	shortsattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
33	csls	prob-4cores	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
34	csls	prob-8cores	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
35	csls	pnorm-8cores	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
36	sparrow2011	sparrow2011_ubcsat1.2_2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
37	adaptg2wsat2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
38	shortsattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
39	sattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
40	SAT_Intersec_01	2011-03-01	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
41	SAT09 reference solver: TNM	2009-03-22	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
42	SAT09 reference solver: gnovelty+2	2009-03-22	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

SAT 2011 Competition: main competition (phase 1): ranking of solvers max CPU time for sequential solvers=1200 s max CPU time for parallel solvers=4800 s max WC time for any solver=1210 s