SAT 2011 Competition: main competition (phase 1): ranking of solvers
max CPU time for any solver=1200 s
max WC time for any solver=1210 s

This page presents the ranking of the different solvers. Solvers are ranked on the number of solved instances. Ties are broken by considering the minimum total solution time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

The Virtual Best Solver (VBS) is a theoretical construction which returns the best answer provided by one of the submitted solver. It corresponds to the 'Best result' column in the bench per bench view of the results. It can be considered as a meta solver which uses a perfect oracle to guess instantaneously which of the submitted solver will be the most efficient to solve the instance. Another way to look at it is to consider this VBS as a solver which would run all other solvers in parallel, bringing together all the solvers strenghs. This VBS is essentially the same notion as State Of The Art (SOTA) solver defined in "Evaluating general purpose automated theorem proving systems", Artificial Intelligence, Volume 131, Issues 1-2, September 2001, Pages 39-54 Geoff Sutcliffe, Christian Suttner. However, SOTA is a more general notion: the VBS can be seen as the least powerful SOTA solver.

Category "applications instances" (APPLICATION): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			242	114 SAT, 128 UNSAT	81%	100%	72175.98	27.17	298.25	-	-	-	-
1	glucose	2	174	82 SAT, 92 UNSAT	58%	72%	32565.62	32.79	187.16	32569.63	32.80	187.18	1.00
2	CryptoMiniSat	Strange-Night2-st (fixed)	172	77 SAT, 95 UNSAT	57%	71%	32319.46	69.85	187.90	32334.92	69.86	187.99	1.00
3	Lingeling	587f (fixed)	171	79 SAT, 92 UNSAT	57%	71%	28641.98	51.61	167.50	28655.39	51.62	167.58	1.00
4	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	171	85 SAT, 86 UNSAT	57%	71%	44330.35	149.09	259.24	44623.16	151.06	260.95	0.99
5	QuteRSat	2011-05-12 (fixed)	170	77 SAT, 93 UNSAT	57%	70%	28607.58	62.12	168.28	28728.50	62.15	168.99	1.00
6	glueminisat	2.2.5	169	73 SAT, 96 UNSAT	56%	70%	26617.79	35.38	157.50	26622.23	35.39	157.53	1.00
7	RestartSAT	A75	169	77 SAT, 92 UNSAT	56%	70%	34049.31	43.10	201.48	34050.19	43.10	201.48	1.00
8	MPhaseSAT64	2011-05-14 (fixed)	168	84 SAT, 84 UNSAT	56%	69%	32437.60	58.40	193.08	32439.10	58.44	193.09	1.00
9	RestartSAT	B95	168	80 SAT, 88 UNSAT	56%	69%	32997.62	46.86	196.41	33003.75	46.87	196.45	1.00
10	minisat hack: cir_minisat	2011-05-13 (simp)	164	82 SAT, 82 UNSAT	55%	68%	25957.95	37.80	158.28	25961.86	37.81	158.30	1.00
11	SAT09 reference solver:precosat	236	164	78 SAT, 86 UNSAT	55%	68%	26023.19	46.83	158.68	26033.46	46.84	158.74	1.00
12	CryptoMiniSat	Strange-Night-1-st (fixed)	163	72 SAT, 91 UNSAT	54%	67%	27929.31	55.40	171.35	27931.19	55.41	171.36	1.00
13	minisat hack: minisat_psm	2011-04-01	162	85 SAT, 77 UNSAT	54%	67%	25508.30	29.77	157.46	25515.11	30.05	157.50	1.00
14	MPhaseSAT_M	2011-02-16	162	80 SAT, 82 UNSAT	54%	67%	28091.77	56.69	173.41	28094.93	56.69	173.43	1.00
15	Plingeling	587f (fixed)	162	84 SAT, 78 UNSAT	54%	67%	35135.36	96.48	216.88	9095.84	27.35	56.15	3.86
16	minisat hack: contrasat	2011-03-02	160	88 SAT, 72 UNSAT	53%	66%	25136.64	42.31	157.10	25153.63	42.36	157.21	1.00
17	borg-sat	11.03.02-seq-industrial (fixed, 1200 s timeout)	160	75 SAT, 85 UNSAT	53%	66%	40180.79	127.07	251.13	40303.27	127.22	251.90	1.00
18	minisat hack: EBMiniSAT	2011-03-02	159	82 SAT, 77 UNSAT	53%	66%	23044.27	31.71	144.93	23045.46	31.72	144.94	1.00
19	minisat hack reference: minisat	2.2.0	159	86 SAT, 73 UNSAT	53%	66%	24238.92	31.42	152.45	24240.40	31.42	152.46	1.00
20	MPhaseSAT	2011-02-15	159	77 SAT, 82 UNSAT	53%	66%	25026.13	50.76	157.40	25028.41	50.76	157.41	1.00
21	borg-sat	11.03.02-seq-industrial (fixed)	159	76 SAT, 83 UNSAT	53%	66%	40720.36	112.59	256.10	40852.49	112.58	256.93	1.00
22	minisat hack: MiniSAT	2.2.0-agile-26	158	84 SAT, 74 UNSAT	53%	65%	22715.89	29.00	143.77	22719.11	29.01	143.79	1.00
23	minisat hack: cir_minisat	2011-03-02	158	84 SAT, 74 UNSAT	53%	65%	31208.56	45.78	197.52	31216.11	45.79	197.57	1.00
24	EBGlucose	1.0	157	76 SAT, 81 UNSAT	52%	65%	22619.68	37.47	144.07	22678.49	37.88	144.45	1.00
25	rcl	2011-04-01	156	83 SAT, 73 UNSAT	52%	64%	27802.74	41.33	178.22	27948.78	43.43	179.16	0.99
26	minisat hack: LR_GL_NOSHR	2011-03-02	155	84 SAT, 71 UNSAT	52%	64%	21261.88	29.83	137.17	21266.57	29.83	137.20	1.00
27	minisat hack: LR_GL_SHR	2011-03-02	155	67 SAT, 88 UNSAT	52%	64%	25222.93	27.96	162.73	25225.00	27.96	162.74	1.00
28	SAT09 reference solver:glucose	1.0	155	74 SAT, 81 UNSAT	52%	64%	25901.39	42.32	167.11	25927.82	42.33	167.28	1.00
29	minisat hack: LR_NOGL_NOSHR	2011-03-02	154	83 SAT, 71 UNSAT	51%	64%	20474.47	32.41	132.95	20472.12	32.42	132.94	1.00
30	kw	2011-02-28	154	73 SAT, 81 UNSAT	51%	64%	28361.17	44.10	184.16	28375.05	44.10	184.25	1.00
31	ppfolio	seq	154	71 SAT, 83 UNSAT	51%	64%	31801.16	79.90	206.50	31823.81	79.90	206.65	1.00
32	SApperloT2010	2011-05-15 (fixed)	153	74 SAT, 79 UNSAT	51%	63%	23011.00	30.33	150.40	23021.31	30.35	150.47	1.00
33	Satisfiability Solver Selector	2011-03-18	153	77 SAT, 76 UNSAT	51%	63%	38155.46	120.21	249.38	38300.94	120.35	250.33	1.00
34	clasp	2.0-R4092-indust	152	71 SAT, 81 UNSAT	51%	63%	28365.33	52.24	186.61	28395.66	52.25	186.81	1.00
35	SApperloT	2011-03-23	151	75 SAT, 76 UNSAT	50%	62%	30017.72	49.06	198.79	30020.94	49.07	198.81	1.00
36	claspmt	2.0-R4095-patched (fixed)	148	75 SAT, 73 UNSAT	49%	61%	28191.99	55.73	190.49	7468.95	17.30	50.47	3.77
37	ppfolio	par	147	72 SAT, 75 UNSAT	49%	61%	28962.34	87.91	197.02	7265.06	22.01	49.42	3.99
38	CryptoMiniSat	Strange-Night-2-mt (fixed)	147	74 SAT, 73 UNSAT	49%	61%	36018.98	123.87	245.03	9157.30	31.26	62.29	3.93
39	riss	1.0	144	69 SAT, 75 UNSAT	48%	60%	27332.15	48.80	189.81	27334.80	48.80	189.82	1.00
40	riss	Auto	144	69 SAT, 75 UNSAT	48%	60%	27684.65	51.34	192.25	27685.01	51.34	192.26	1.00
41	borg-sat	11.03.02-par-industrial (fixed)	144	71 SAT, 73 UNSAT	48%	60%	40928.89	172.72	284.23	18946.20	74.71	131.57	2.16
42	CryptoMiniSat	Strange-Night-1-mt (fixed)	142	63 SAT, 79 UNSAT	47%	59%	33177.11	102.55	233.64	8701.21	26.56	61.28	3.81
43	PicoSAT	941	138	72 SAT, 66 UNSAT	46%	57%	21461.72	34.65	155.52	21473.68	34.76	155.61	1.00
44	riss	Auto (1200 s timeout)	138	65 SAT, 73 UNSAT	46%	57%	22891.64	47.40	165.88	22887.98	47.40	165.85	1.00
45	priss	UP	138	68 SAT, 70 UNSAT	46%	57%	32606.21	70.93	236.28	20285.33	45.81	147.00	1.61
46	sathys	2011-04-01	137	69 SAT, 68 UNSAT	46%	57%	26512.24	45.15	193.52	26561.85	45.17	193.88	1.00
47	riss	Exp	137	68 SAT, 69 UNSAT	46%	57%	26524.82	52.65	193.61	26525.06	52.65	193.61	1.00
48	SAT09 reference solver: ManySAT 1.1	aimd 1/2009-03-20	135	64 SAT, 71 UNSAT	45%	56%	23966.59	38.31	177.53	6760.40	14.09	50.08	3.55
49	SArTagnan	2011-05-15 (fixed)	132	64 SAT, 68 UNSAT	44%	55%	27430.17	89.24	207.80	8481.48	25.70	64.25	3.23
50	SAT09 reference solver:LySAT	i/2009-03-20	129	58 SAT, 71 UNSAT	43%	53%	21125.95	33.68	163.77	21147.12	33.68	163.93	1.00
51	jMiniSat	2011	129	68 SAT, 61 UNSAT	43%	53%	30177.27	62.95	233.93	29961.78	61.53	232.26	1.01
52	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	118	63 SAT, 55 UNSAT	39%	49%	31912.83	92.27	270.45	31783.59	91.02	269.35	1.00
53	Sol	2011-04-04	117	57 SAT, 60 UNSAT	39%	48%	25099.51	74.15	214.53	24966.48	71.16	213.39	1.01
54	priss	1.0	117	57 SAT, 60 UNSAT	39%	48%	25250.94	72.57	215.82	6718.90	18.55	57.43	3.76
55	priss	Auto	116	59 SAT, 57 UNSAT	39%	48%	25160.78	86.04	216.90	6522.89	23.26	56.23	3.86
56	CryptoMiniSat	TechDemo-Alpha-mt	104	53 SAT, 51 UNSAT	35%	43%	25492.62	113.93	245.12	7034.88	32.63	67.64	3.62
57	UmSat	2011-05-12 (fixed)	90	47 SAT, 43 UNSAT	30%	37%	21627.24	107.86	240.30	21629.30	108.21	240.33	1.00
58	Black Hole SAT	2011-03-08	27	22 SAT, 5 UNSAT	9%	11%	1959.00	1.20	72.56	1963.20	1.20	72.71	1.00
59	march_rw	2011-03-02	23	15 SAT, 8 UNSAT	8%	10%	3398.18	2.58	147.75	3398.28	2.58	147.75	1.00
60	shortsattime+	2011-03-02	22	15 SAT, 7 UNSAT	7%	9%	4257.57	5.39	193.53	4257.66	5.39	193.53	1.00
61	STRUCTure	0.1	21	13 SAT, 8 UNSAT	7%	9%	1617.48	19.08	77.02	532.04	9.06	25.34	3.04
62	STRUCTure	0.1 (hbr=2)	20	11 SAT, 9 UNSAT	7%	8%	1844.96	15.37	92.25	643.98	6.45	32.20	2.86
63	sattime+	2011-03-02	19	12 SAT, 7 UNSAT	6%	8%	3215.42	4.12	169.23	3215.46	4.12	169.23	1.00
64	sattime	2011-03-02	16	16 SAT	5%	7%	2046.54	5.11	127.91	2046.54	5.11	127.91	1.00
65	sattime2011	2011-03-02	16	16 SAT	5%	7%	2599.59	5.51	162.47	2599.70	5.51	162.48	1.00
66	shortsattime2011	2011-03-02	15	15 SAT	5%	6%	1115.58	5.39	74.37	1117.99	5.39	74.53	1.00
67	adaptg2wsat2011	2011-03-02	15	15 SAT	5%	6%	1350.26	4.87	90.02	1350.27	4.87	90.02	1.00

Category "applications instances" (APPLICATION): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			114	38%	100%	17998.14	20.77	157.88	-	-	-	-
1	minisat hack: contrasat	2011-03-02	88	29%	77%	14197.14	39.54	161.33	14206.67	41.40	161.44	1.00
2	minisat hack reference: minisat	2.2.0	86	29%	75%	12595.40	31.76	146.46	12596.60	31.76	146.47	1.00
3	minisat hack: minisat_psm	2011-04-01	85	28%	75%	14632.08	20.38	172.14	14636.69	20.43	172.20	1.00
4	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	85	28%	75%	21407.86	146.98	251.86	21547.10	151.06	253.50	0.99
5	minisat hack: MiniSAT	2.2.0-agile-26	84	28%	74%	10113.15	27.77	120.39	10116.20	27.81	120.43	1.00
6	minisat hack: LR_GL_NOSHR	2011-03-02	84	28%	74%	11720.66	33.21	139.53	11722.68	33.22	139.56	1.00
7	MPhaseSAT64	2011-05-14 (fixed)	84	28%	74%	13543.31	54.93	161.23	13544.45	54.99	161.24	1.00
8	minisat hack: cir_minisat	2011-03-02	84	28%	74%	13980.12	37.04	166.43	13987.51	37.04	166.52	1.00
9	Plingeling	587f (fixed)	84	28%	74%	17718.52	67.30	210.93	4673.11	22.04	55.63	3.79
10	minisat hack: LR_NOGL_NOSHR	2011-03-02	83	28%	73%	10479.44	29.22	126.26	10481.10	29.24	126.28	1.00
11	rcl	2011-04-01	83	28%	73%	13210.39	40.07	159.16	13316.35	41.74	160.44	0.99
12	minisat hack: cir_minisat	2011-05-13 (simp)	82	27%	72%	10611.60	30.29	129.41	10615.23	30.31	129.45	1.00
13	minisat hack: EBMiniSAT	2011-03-02	82	27%	72%	11233.48	30.26	136.99	11234.49	30.27	137.01	1.00
14	glucose	2	82	27%	72%	13597.50	25.36	165.82	13601.39	26.07	165.87	1.00
15	MPhaseSAT_M	2011-02-16	80	27%	70%	12250.52	38.89	153.13	12253.52	38.89	153.17	1.00
16	RestartSAT	B95	80	27%	70%	13533.69	41.25	169.17	13534.56	41.26	169.18	1.00
17	Lingeling	587f (fixed)	79	26%	69%	10090.91	50.60	127.73	10098.17	50.61	127.82	1.00
18	SAT09 reference solver:precosat	236	78	26%	68%	11775.83	41.89	150.97	11780.12	41.91	151.03	1.00
19	MPhaseSAT	2011-02-15	77	26%	68%	9764.63	25.84	126.81	9765.94	25.84	126.83	1.00
20	CryptoMiniSat	Strange-Night2-st (fixed)	77	26%	68%	12757.17	64.09	165.68	12762.31	64.11	165.74	1.00
21	QuteRSat	2011-05-12 (fixed)	77	26%	68%	12798.29	69.63	166.21	12800.05	69.64	166.23	1.00
22	RestartSAT	A75	77	26%	68%	13033.56	42.28	169.27	13034.45	42.28	169.28	1.00
23	Satisfiability Solver Selector	2011-03-18	77	26%	68%	21823.40	119.84	283.42	21920.08	119.99	284.68	1.00
24	EBGlucose	1.0	76	25%	67%	10209.81	25.33	134.34	10242.64	25.33	134.77	1.00
25	borg-sat	11.03.02-seq-industrial (fixed)	76	25%	67%	19851.76	109.75	261.21	19929.74	110.29	262.23	1.00
26	claspmt	2.0-R4095-patched (fixed)	75	25%	66%	11991.15	22.02	159.88	3302.58	8.99	44.03	3.63
27	SApperloT	2011-03-23	75	25%	66%	13122.21	28.18	174.96	13125.00	28.23	175.00	1.00
28	borg-sat	11.03.02-seq-industrial (fixed, 1200 s timeout)	75	25%	66%	18665.46	125.05	248.87	18700.51	125.15	249.34	1.00
29	SApperloT2010	2011-05-15 (fixed)	74	25%	65%	8996.30	26.85	121.57	8997.78	26.85	121.59	1.00
30	SAT09 reference solver:glucose	1.0	74	25%	65%	9964.89	35.30	134.66	9965.88	35.31	134.67	1.00
31	CryptoMiniSat	Strange-Night-2-mt (fixed)	74	25%	65%	20507.02	104.41	277.12	5186.94	28.21	70.09	3.95
32	glueminisat	2.2.5	73	24%	64%	8854.46	33.52	121.29	8855.29	33.53	121.31	1.00
33	kw	2011-02-28	73	24%	64%	15462.33	49.24	211.81	15463.05	49.26	211.82	1.00
34	PicoSAT	941	72	24%	63%	7745.40	23.09	107.58	7757.48	23.18	107.74	1.00
35	CryptoMiniSat	Strange-Night-1-st (fixed)	72	24%	63%	9750.46	42.55	135.42	9751.94	42.57	135.44	1.00
36	ppfolio	par	72	24%	63%	12363.55	63.49	171.72	3107.29	16.05	43.16	3.98
37	clasp	2.0-R4092-indust	71	24%	62%	8746.67	17.61	123.19	8779.73	19.52	123.66	1.00
38	ppfolio	seq	71	24%	62%	10252.73	37.41	144.40	10273.45	38.76	144.70	1.00
39	borg-sat	11.03.02-par-industrial (fixed)	71	24%	62%	22180.03	183.40	312.39	11407.47	95.38	160.67	1.94
40	riss	1.0	69	23%	61%	11223.82	33.42	162.66	11224.72	33.42	162.68	1.00
41	riss	Auto	69	23%	61%	11484.80	34.76	166.45	11485.11	34.77	166.45	1.00
42	sathys	2011-04-01	69	23%	61%	12365.33	43.25	179.21	12413.12	43.25	179.90	1.00
43	jMiniSat	2011	68	23%	60%	11496.04	34.65	169.06	11368.22	33.77	167.18	1.01
44	riss	Exp	68	23%	60%	12650.34	47.35	186.03	12650.66	47.36	186.04	1.00
45	priss	UP	68	23%	60%	13598.09	46.96	199.97	8652.42	36.49	127.24	1.57
46	minisat hack: LR_GL_SHR	2011-03-02	67	22%	59%	7856.21	22.63	117.26	7857.70	22.79	117.28	1.00
47	riss	Auto (1200 s timeout)	65	22%	57%	8853.35	22.52	136.21	8853.97	22.52	136.21	1.00
48	SAT09 reference solver: ManySAT 1.1	aimd 1/2009-03-20	64	21%	56%	9809.14	23.03	153.27	2657.50	11.55	41.52	3.69
49	SArTagnan	2011-05-15 (fixed)	64	21%	56%	10656.91	44.46	166.51	3368.14	13.67	52.63	3.16
50	CryptoMiniSat	Strange-Night-1-mt (fixed)	63	21%	55%	12400.97	78.38	196.84	3248.42	21.10	51.56	3.82
51	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	63	21%	55%	14149.34	48.32	224.59	14083.96	47.64	223.55	1.00
52	priss	Auto	59	20%	52%	12276.79	64.38	208.08	3175.62	16.18	53.82	3.87
53	SAT09 reference solver:LySAT	i/2009-03-20	58	19%	51%	8905.40	21.00	153.54	8907.47	21.00	153.58	1.00
54	Sol	2011-04-04	57	19%	50%	9404.97	41.42	165.00	9352.98	39.09	164.09	1.01
55	priss	1.0	57	19%	50%	10679.58	44.13	187.36	2948.28	13.26	51.72	3.62
56	CryptoMiniSat	TechDemo-Alpha-mt	53	18%	46%	11088.25	86.72	209.21	3087.13	24.28	58.25	3.59
57	UmSat	2011-05-12 (fixed)	47	16%	41%	10269.61	49.90	218.50	10271.52	49.90	218.54	1.00
58	Black Hole SAT	2011-03-08	22	7%	19%	1473.84	1.54	66.99	1478.01	1.57	67.18	1.00
59	sattime	2011-03-02	16	5%	14%	2046.54	5.11	127.91	2046.54	5.11	127.91	1.00
60	sattime2011	2011-03-02	16	5%	14%	2599.59	5.51	162.47	2599.70	5.51	162.48	1.00
61	shortsattime2011	2011-03-02	15	5%	13%	1115.58	5.39	74.37	1117.99	5.39	74.53	1.00
62	adaptg2wsat2011	2011-03-02	15	5%	13%	1350.26	4.87	90.02	1350.27	4.87	90.02	1.00
63	march_rw	2011-03-02	15	5%	13%	1685.28	1.00	112.35	1685.34	1.00	112.36	1.00
64	shortsattime+	2011-03-02	15	5%	13%	2536.47	10.31	169.10	2536.44	10.31	169.10	1.00
65	STRUCTure	0.1	13	4%	11%	1380.70	11.30	106.21	421.54	3.27	32.43	3.28
66	sattime+	2011-03-02	12	4%	11%	1459.97	4.33	121.66	1459.99	4.33	121.67	1.00
67	STRUCTure	0.1 (hbr=2)	11	4%	10%	1023.21	8.39	93.02	356.12	2.53	32.37	2.87

Category "applications instances" (APPLICATION): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			128	43%	100%	54177.84	41.92	423.26	-	-	-	-
1	glueminisat	2.2.5	96	32%	75%	17763.33	37.35	185.03	17766.94	37.36	185.07	1.00
2	CryptoMiniSat	Strange-Night2-st (fixed)	95	32%	74%	19562.30	73.84	205.92	19572.61	73.84	206.03	1.00
3	QuteRSat	2011-05-12 (fixed)	93	31%	73%	15809.29	56.38	169.99	15928.45	56.40	171.27	0.99
4	Lingeling	587f (fixed)	92	31%	72%	18551.07	55.33	201.64	18557.22	55.34	201.71	1.00
5	glucose	2	92	31%	72%	18968.12	58.89	206.18	18968.23	58.90	206.18	1.00
6	RestartSAT	A75	92	31%	72%	21015.74	46.21	228.43	21015.75	46.22	228.43	1.00
7	CryptoMiniSat	Strange-Night-1-st (fixed)	91	30%	71%	18178.85	58.04	199.77	18179.25	58.06	199.77	1.00
8	minisat hack: LR_GL_SHR	2011-03-02	88	29%	69%	17366.71	42.78	197.35	17367.31	42.88	197.36	1.00
9	RestartSAT	B95	88	29%	69%	19463.93	49.93	221.18	19469.18	49.93	221.24	1.00
10	SAT09 reference solver:precosat	236	86	29%	67%	14247.36	56.68	165.67	14253.34	56.69	165.74	1.00
11	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	86	29%	67%	22922.49	149.22	266.54	23076.06	151.77	268.33	0.99
12	borg-sat	11.03.02-seq-industrial (fixed, 1200 s timeout)	85	28%	66%	21515.33	132.35	253.12	21602.76	134.26	254.15	1.00
13	MPhaseSAT64	2011-05-14 (fixed)	84	28%	66%	18894.29	60.50	224.93	18894.65	60.58	224.94	1.00
14	borg-sat	11.03.02-seq-industrial (fixed)	83	28%	65%	20868.59	122.00	251.43	20922.75	122.10	252.08	1.00
15	ppfolio	seq	83	28%	65%	21548.43	110.96	259.62	21550.36	110.96	259.64	1.00
16	MPhaseSAT	2011-02-15	82	27%	64%	15261.50	67.05	186.12	15262.46	67.17	186.13	1.00
17	minisat hack: cir_minisat	2011-05-13 (simp)	82	27%	64%	15346.35	42.78	187.15	15346.62	42.79	187.15	1.00
18	MPhaseSAT_M	2011-02-16	82	27%	64%	15841.25	67.95	193.19	15841.41	67.95	193.19	1.00
19	EBGlucose	1.0	81	27%	63%	12409.87	40.22	153.21	12435.86	40.23	153.53	1.00
20	kw	2011-02-28	81	27%	63%	12898.85	42.86	159.25	12912.00	42.86	159.41	1.00
21	SAT09 reference solver:glucose	1.0	81	27%	63%	15936.50	44.17	196.75	15961.94	44.18	197.06	1.00
22	clasp	2.0-R4092-indust	81	27%	63%	19618.65	76.15	242.21	19615.92	76.15	242.17	1.00
23	SApperloT2010	2011-05-15 (fixed)	79	26%	62%	14014.71	39.27	177.40	14023.54	39.27	177.51	1.00
24	CryptoMiniSat	Strange-Night-1-mt (fixed)	79	26%	62%	20776.14	170.43	262.99	5452.79	42.81	69.02	3.81
25	Plingeling	587f (fixed)	78	26%	61%	17416.84	121.48	223.29	4422.73	31.05	56.70	3.94
26	minisat hack: minisat_psm	2011-04-01	77	26%	60%	10876.22	33.92	141.25	10878.42	33.92	141.28	1.00
27	minisat hack: EBMiniSAT	2011-03-02	77	26%	60%	11810.79	39.77	153.39	11810.97	39.77	153.39	1.00
28	Satisfiability Solver Selector	2011-03-18	76	25%	59%	16332.05	121.41	214.90	16380.86	121.52	215.54	1.00
29	SApperloT	2011-03-23	76	25%	59%	16895.51	83.07	222.31	16895.93	83.07	222.31	1.00
30	riss	1.0	75	25%	59%	16108.33	58.27	214.78	16110.08	60.03	214.80	1.00
31	riss	Auto	75	25%	59%	16199.85	57.12	216.00	16199.90	57.12	216.00	1.00
32	ppfolio	par	75	25%	59%	16598.79	112.80	221.32	4157.77	28.24	55.44	3.99
33	minisat hack: MiniSAT	2.2.0-agile-26	74	25%	58%	12602.74	29.00	170.31	12602.91	29.01	170.31	1.00
34	minisat hack: cir_minisat	2011-03-02	74	25%	58%	17228.44	95.60	232.82	17228.60	95.61	232.82	1.00
35	minisat hack reference: minisat	2.2.0	73	24%	57%	11643.52	27.88	159.50	11643.81	27.88	159.50	1.00
36	riss	Auto (1200 s timeout)	73	24%	57%	14038.29	56.23	192.31	14034.01	56.23	192.25	1.00
37	rcl	2011-04-01	73	24%	57%	14592.36	46.94	199.90	14632.43	46.94	200.44	1.00
38	CryptoMiniSat	Strange-Night-2-mt (fixed)	73	24%	57%	15511.96	149.71	212.49	3970.36	37.60	54.39	3.91
39	claspmt	2.0-R4095-patched (fixed)	73	24%	57%	16200.85	79.67	221.93	4166.36	20.60	57.07	3.89
40	borg-sat	11.03.02-par-industrial (fixed)	73	24%	57%	18748.86	165.73	256.83	7538.73	66.81	103.27	2.49
41	minisat hack: contrasat	2011-03-02	72	24%	56%	10939.50	42.51	151.94	10946.96	42.55	152.04	1.00
42	minisat hack: LR_GL_NOSHR	2011-03-02	71	24%	55%	9541.22	25.19	134.38	9543.90	25.90	134.42	1.00
43	minisat hack: LR_NOGL_NOSHR	2011-03-02	71	24%	55%	9995.03	38.40	140.78	9991.02	38.41	140.72	1.00
44	SAT09 reference solver:LySAT	i/2009-03-20	71	24%	55%	12220.56	41.76	172.12	12239.64	41.76	172.39	1.00
45	SAT09 reference solver: ManySAT 1.1	aimd 1/2009-03-20	71	24%	55%	14157.45	65.22	199.40	4102.90	18.68	57.79	3.45
46	priss	UP	70	23%	55%	19008.12	79.79	271.54	11632.91	54.85	166.18	1.63
47	riss	Exp	69	23%	54%	13874.48	61.02	201.08	13874.40	61.02	201.08	1.00
48	sathys	2011-04-01	68	23%	53%	14146.91	49.03	208.04	14148.73	49.04	208.07	1.00
49	SArTagnan	2011-05-15 (fixed)	68	23%	53%	16773.27	138.87	246.67	5113.34	46.57	75.20	3.28
50	PicoSAT	941	66	22%	52%	13716.32	85.93	207.82	13716.19	85.93	207.82	1.00
51	jMiniSat	2011	61	20%	48%	18681.23	110.27	306.25	18593.55	109.40	304.81	1.00
52	priss	1.0	60	20%	47%	14571.36	132.42	242.86	3770.63	33.72	62.84	3.86
53	Sol	2011-04-04	60	20%	47%	15694.54	103.49	261.58	15613.50	103.15	260.23	1.01
54	priss	Auto	57	19%	45%	12883.99	127.32	226.03	3347.27	32.59	58.72	3.85
55	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	55	18%	43%	17763.50	129.41	322.97	17699.63	128.42	321.81	1.00
56	CryptoMiniSat	TechDemo-Alpha-mt	51	17%	40%	14404.37	199.63	282.44	3947.75	53.92	77.41	3.65
57	UmSat	2011-05-12 (fixed)	43	14%	34%	11357.62	139.41	264.13	11357.77	139.41	264.13	1.00
58	STRUCTure	0.1 (hbr=2)	9	3%	7%	821.76	21.56	91.31	287.86	12.01	31.98	2.85
59	STRUCTure	0.1	8	3%	6%	236.78	19.98	29.60	110.49	9.99	13.81	2.14
60	march_rw	2011-03-02	8	3%	6%	1712.90	21.91	214.11	1712.95	21.91	214.12	1.00
61	shortsattime+	2011-03-02	7	2%	5%	1721.10	4.13	245.87	1721.22	4.13	245.89	1.00
62	sattime+	2011-03-02	7	2%	5%	1755.45	4.12	250.78	1755.47	4.12	250.78	1.00
63	Black Hole SAT	2011-03-08	5	2%	4%	485.16	0.03	97.03	485.19	0.03	97.04	1.00
64	sattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
65	sattime	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
66	shortsattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
67	adaptg2wsat2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "crafted instances" (CRAFTED): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			207	143 SAT, 64 UNSAT	69%	100%	43913.53	1.70	212.14	-	-	-	-
1	ppfolio	seq	138	106 SAT, 32 UNSAT	46%	67%	17991.42	5.72	130.37	17998.11	5.73	130.42	1.00
2	Satisfiability Solver Selector	2011-03-18	136	107 SAT, 29 UNSAT	45%	66%	24195.73	30.19	177.91	24343.85	30.60	179.00	0.99
3	ppfolio	par	134	103 SAT, 31 UNSAT	45%	65%	20781.50	5.41	155.09	5347.00	1.55	39.90	3.89
4	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	131	100 SAT, 31 UNSAT	44%	63%	23183.51	13.77	176.97	23312.05	16.35	177.95	0.99
5	MPhaseSAT	2011-02-15	107	81 SAT, 26 UNSAT	36%	52%	14248.05	5.74	133.16	14248.58	5.74	133.16	1.00
6	MPhaseSAT64	2011-05-14 (fixed)	107	80 SAT, 27 UNSAT	36%	52%	18057.38	5.42	168.76	18057.80	5.42	168.76	1.00
7	claspmt	2.0-R4095-patched (fixed)	105	73 SAT, 32 UNSAT	35%	51%	20383.63	35.78	194.13	5142.13	9.24	48.97	3.96
8	sattime2011	2011-03-02	104	104 SAT	35%	50%	6150.28	1.56	59.14	6151.22	1.56	59.15	1.00
9	shortsattime+	2011-03-02	104	104 SAT	35%	50%	11191.41	3.48	107.61	11191.35	3.48	107.61	1.00
10	MPhaseSAT_M	2011-02-16	104	79 SAT, 25 UNSAT	35%	50%	13199.41	3.00	126.92	13199.24	3.00	126.92	1.00
11	shortsattime2011	2011-03-02	103	103 SAT	34%	50%	6537.74	0.91	63.47	6538.19	0.91	63.48	1.00
12	sattime	2011-03-02	103	103 SAT	34%	50%	8489.33	0.60	82.42	8489.35	0.60	82.42	1.00
13	Sol	2011-04-04	103	73 SAT, 30 UNSAT	34%	50%	11258.24	3.58	109.30	11174.26	3.56	108.49	1.01
14	sattime+	2011-03-02	102	102 SAT	34%	49%	9242.73	1.90	90.61	9243.00	1.90	90.62	1.00
15	clasp	2.0-R4092-crafted	102	67 SAT, 35 UNSAT	34%	49%	18146.98	33.86	177.91	18147.45	33.86	177.92	1.00
16	adaptg2wsat2011	2011-03-02	99	99 SAT	33%	48%	7748.97	0.67	78.27	7749.27	0.68	78.28	1.00
17	RestartSAT	B95	94	66 SAT, 28 UNSAT	31%	45%	16670.27	16.78	177.34	16670.07	16.78	177.34	1.00
18	SApperloT2010	2011-05-15 (fixed)	93	67 SAT, 26 UNSAT	31%	45%	15410.65	8.73	165.71	15410.84	8.73	165.71	1.00
19	QuteRSat	2011-05-12 (fixed)	93	61 SAT, 32 UNSAT	31%	45%	15580.41	14.22	167.53	15592.95	14.22	167.67	1.00
20	CryptoMiniSat	Strange-Night2-st (fixed)	93	63 SAT, 30 UNSAT	31%	45%	16181.96	16.28	174.00	16182.10	16.28	174.00	1.00
21	PicoSAT	941	92	65 SAT, 27 UNSAT	31%	44%	18030.14	12.63	195.98	18030.05	12.63	195.98	1.00
22	CryptoMiniSat	Strange-Night-1-mt (fixed)	92	64 SAT, 28 UNSAT	31%	44%	19114.99	15.30	207.77	4829.82	4.50	52.50	3.96
23	SAT09 reference solver: clasp	1.2.0-SAT09-32	91	58 SAT, 33 UNSAT	30%	44%	15729.84	30.60	172.86	15729.79	30.60	172.85	1.00
24	glucose	2	90	60 SAT, 30 UNSAT	30%	43%	16321.44	23.28	181.35	16321.46	23.28	181.35	1.00
25	Lingeling	587f (fixed)	89	62 SAT, 27 UNSAT	30%	43%	11681.10	5.64	131.25	11681.22	5.64	131.25	1.00
26	CryptoMiniSat	Strange-Night-1-st (fixed)	89	57 SAT, 32 UNSAT	30%	43%	17914.86	20.97	201.29	17914.95	20.97	201.29	1.00
27	sathys	2011-04-01	88	68 SAT, 20 UNSAT	29%	43%	12078.28	1.08	137.25	12078.43	1.08	137.25	1.00
28	Plingeling	587f (fixed)	88	65 SAT, 23 UNSAT	29%	43%	13972.85	6.66	158.78	3520.41	1.85	40.00	3.97
29	CryptoMiniSat	Strange-Night-2-mt (fixed)	88	61 SAT, 27 UNSAT	29%	43%	14052.72	7.86	159.69	3538.49	1.98	40.21	3.97
30	RestartSAT	A75	88	62 SAT, 26 UNSAT	29%	43%	15664.64	7.71	178.01	15664.47	7.71	178.01	1.00
31	SAT07 reference solver: minisat	SAT 2007	87	57 SAT, 30 UNSAT	29%	42%	12077.12	6.52	138.82	12076.96	6.53	138.82	1.00
32	rcl	2011-04-01	86	61 SAT, 25 UNSAT	29%	42%	14280.09	5.07	166.05	14281.47	5.07	166.06	1.00
33	SApperloT	2011-03-23	85	58 SAT, 27 UNSAT	28%	41%	12483.09	25.03	146.86	12483.00	25.03	146.86	1.00
34	jMiniSat	2011	84	61 SAT, 23 UNSAT	28%	41%	17543.21	15.44	208.85	17507.50	15.04	208.42	1.00
35	SAT09 reference solver: SATzilla2009_C	2009-03-22	81	50 SAT, 31 UNSAT	27%	39%	12052.55	5.01	148.80	12044.70	5.06	148.70	1.00
36	march_rw	2011-03-02	80	55 SAT, 25 UNSAT	27%	39%	11117.06	0.62	138.96	11116.91	0.62	138.96	1.00
37	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	77	53 SAT, 24 UNSAT	26%	37%	9551.80	17.65	124.05	9500.91	16.81	123.39	1.01
38	riss	1.0	76	55 SAT, 21 UNSAT	25%	37%	11220.54	3.23	147.64	11220.43	3.23	147.64	1.00
39	riss	Auto	76	55 SAT, 21 UNSAT	25%	37%	11245.46	3.19	147.97	11245.89	3.19	147.97	1.00
40	riss	Auto (1200 s timeout)	76	54 SAT, 22 UNSAT	25%	37%	11537.35	3.07	151.81	11537.62	3.09	151.81	1.00
41	priss	UP	75	55 SAT, 20 UNSAT	25%	36%	12334.68	6.23	164.46	6999.06	4.31	93.32	1.76
42	riss	Exp	74	54 SAT, 20 UNSAT	25%	36%	11913.11	3.49	160.99	11913.54	3.49	160.99	1.00
43	SArTagnan	2011-05-15 (fixed)	73	51 SAT, 22 UNSAT	24%	35%	9101.49	2.77	124.68	2698.56	0.77	36.97	3.37
44	priss	Auto	70	53 SAT, 17 UNSAT	23%	34%	10583.26	9.09	151.19	2652.76	2.29	37.90	3.99
45	UmSat	2011-05-12 (fixed)	68	49 SAT, 19 UNSAT	23%	33%	9046.48	2.42	133.04	9046.55	2.42	133.04	1.00
46	priss	1.0	65	49 SAT, 16 UNSAT	22%	31%	6083.82	8.21	93.60	1526.15	2.19	23.48	3.99
47	CryptoMiniSat	TechDemo-Alpha-mt	61	41 SAT, 20 UNSAT	20%	29%	7514.96	3.91	123.20	1905.93	1.02	31.24	3.94
48	STRUCTure	0.1 (hbr=2)	48	34 SAT, 14 UNSAT	16%	23%	3533.16	7.32	73.61	919.47	2.34	19.16	3.84
49	STRUCTure	0.1	48	34 SAT, 14 UNSAT	16%	23%	4120.35	6.41	85.84	1067.98	2.21	22.25	3.86
50	ttsth	5-4	39	24 SAT, 15 UNSAT	13%	19%	7917.07	13.20	203.00	2033.75	7.79	52.15	3.89
51	tts	5-4	37	20 SAT, 17 UNSAT	12%	18%	4581.72	7.70	123.83	4581.82	7.70	123.83	1.00
52	Black Hole SAT	2011-03-08	36	25 SAT, 11 UNSAT	12%	17%	3583.16	0.17	99.53	3583.17	0.17	99.53	1.00

Category "crafted instances" (CRAFTED): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			143	48%	100%	5283.63	0.73	36.95	-	-	-	-
1	Satisfiability Solver Selector	2011-03-18	107	36%	75%	16430.43	30.63	153.56	16573.80	30.81	154.90	0.99
2	ppfolio	seq	106	35%	74%	10189.02	3.73	96.12	10194.89	3.76	96.18	1.00
3	sattime2011	2011-03-02	104	35%	73%	6150.28	1.56	59.14	6151.22	1.56	59.15	1.00
4	shortsattime+	2011-03-02	104	35%	73%	11191.41	3.48	107.61	11191.35	3.48	107.61	1.00
5	shortsattime2011	2011-03-02	103	34%	72%	6537.74	0.91	63.47	6538.19	0.91	63.48	1.00
6	sattime	2011-03-02	103	34%	72%	8489.33	0.60	82.42	8489.35	0.60	82.42	1.00
7	ppfolio	par	103	34%	72%	13141.57	4.76	127.59	3360.49	1.52	32.63	3.91
8	sattime+	2011-03-02	102	34%	71%	9242.73	1.90	90.61	9243.00	1.90	90.62	1.00
9	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	100	33%	70%	13354.24	12.87	133.54	13460.97	14.00	134.61	0.99
10	adaptg2wsat2011	2011-03-02	99	33%	69%	7748.97	0.67	78.27	7749.27	0.68	78.28	1.00
11	MPhaseSAT	2011-02-15	81	27%	57%	9128.99	6.64	112.70	9129.53	6.64	112.71	1.00
12	MPhaseSAT64	2011-05-14 (fixed)	80	27%	56%	11679.86	4.75	146.00	11680.33	4.75	146.00	1.00
13	MPhaseSAT_M	2011-02-16	79	26%	55%	8939.49	2.77	113.16	8939.37	2.77	113.16	1.00
14	Sol	2011-04-04	73	24%	51%	6388.07	2.83	87.51	6345.08	2.80	86.92	1.01
15	claspmt	2.0-R4095-patched (fixed)	73	24%	51%	13209.48	19.39	180.95	3340.39	5.37	45.76	3.95
16	sathys	2011-04-01	68	23%	48%	7805.24	0.42	114.78	7805.36	0.44	114.78	1.00
17	SApperloT2010	2011-05-15 (fixed)	67	22%	47%	9368.34	3.73	139.83	9368.61	3.73	139.83	1.00
18	clasp	2.0-R4092-crafted	67	22%	47%	10213.42	16.32	152.44	10214.06	16.33	152.45	1.00
19	RestartSAT	B95	66	22%	46%	8613.43	11.04	130.51	8613.37	11.05	130.51	1.00
20	Plingeling	587f (fixed)	65	22%	45%	8944.62	4.65	137.61	2253.67	1.24	34.67	3.97
21	PicoSAT	941	65	22%	45%	10587.91	6.89	162.89	10587.85	6.90	162.89	1.00
22	CryptoMiniSat	Strange-Night-1-mt (fixed)	64	21%	45%	11166.75	5.15	174.48	2829.02	1.30	44.20	3.95
23	CryptoMiniSat	Strange-Night2-st (fixed)	63	21%	44%	10783.51	20.48	171.17	10783.60	20.49	171.17	1.00
24	Lingeling	587f (fixed)	62	21%	43%	5797.21	2.71	93.50	5797.29	2.71	93.50	1.00
25	RestartSAT	A75	62	21%	43%	8851.21	2.69	142.76	8851.16	2.70	142.76	1.00
26	rcl	2011-04-01	61	20%	43%	7208.65	2.57	118.17	7210.06	2.57	118.20	1.00
27	CryptoMiniSat	Strange-Night-2-mt (fixed)	61	20%	43%	7352.34	2.93	120.53	1857.91	0.74	30.46	3.96
28	QuteRSat	2011-05-12 (fixed)	61	20%	43%	8897.04	13.33	145.85	8909.64	13.33	146.06	1.00
29	jMiniSat	2011	61	20%	43%	11716.47	2.35	192.07	11693.65	1.89	191.70	1.00
30	glucose	2	60	20%	42%	10509.43	21.30	175.16	10509.43	21.30	175.16	1.00
31	SApperloT	2011-03-23	58	19%	41%	8546.38	20.77	147.35	8546.33	20.77	147.35	1.00
32	SAT09 reference solver: clasp	1.2.0-SAT09-32	58	19%	41%	8896.80	10.92	153.39	8896.75	10.92	153.39	1.00
33	SAT07 reference solver: minisat	SAT 2007	57	19%	40%	7500.75	2.22	131.59	7500.65	2.22	131.59	1.00
34	CryptoMiniSat	Strange-Night-1-st (fixed)	57	19%	40%	10124.37	20.50	177.62	10124.45	20.50	177.62	1.00
35	march_rw	2011-03-02	55	18%	38%	7349.38	0.41	133.63	7349.25	0.41	133.62	1.00
36	riss	Auto	55	18%	38%	7492.64	2.93	136.23	7493.13	2.94	136.24	1.00
37	riss	1.0	55	18%	38%	7537.77	2.79	137.05	7537.70	2.79	137.05	1.00
38	priss	UP	55	18%	38%	8986.32	6.23	163.39	5186.33	4.31	94.30	1.73
39	riss	Auto (1200 s timeout)	54	18%	38%	6436.95	2.62	119.20	6437.20	2.62	119.21	1.00
40	riss	Exp	54	18%	38%	9127.51	3.05	169.03	9127.37	3.05	169.03	1.00
41	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	53	18%	37%	5828.50	11.30	109.97	5801.02	10.43	109.45	1.00
42	priss	Auto	53	18%	37%	8253.46	7.44	155.73	2068.96	1.95	39.04	3.99
43	SArTagnan	2011-05-15 (fixed)	51	17%	36%	4075.41	0.21	79.91	1126.47	0.10	22.09	3.62
44	SAT09 reference solver: SATzilla2009_C	2009-03-22	50	17%	35%	4757.99	1.06	95.16	4755.06	1.06	95.10	1.00
45	priss	1.0	49	16%	34%	4804.34	7.12	98.05	1205.34	1.88	24.60	3.99
46	UmSat	2011-05-12 (fixed)	49	16%	34%	6337.98	0.18	129.35	6338.05	0.18	129.35	1.00
47	CryptoMiniSat	TechDemo-Alpha-mt	41	14%	29%	4804.98	0.97	117.19	1214.86	0.28	29.63	3.96
48	STRUCTure	0.1 (hbr=2)	34	11%	24%	1825.03	5.98	53.68	476.32	1.91	14.01	3.83
49	STRUCTure	0.1	34	11%	24%	2368.19	5.23	69.65	614.17	1.63	18.06	3.86
50	Black Hole SAT	2011-03-08	25	8%	17%	2752.49	0.03	110.10	2752.48	0.03	110.10	1.00
51	ttsth	5-4	24	8%	17%	4640.48	14.23	193.35	1200.02	9.19	50.00	3.87
52	tts	5-4	20	7%	14%	1703.75	8.62	85.19	1703.75	8.62	85.19	1.00

Category "crafted instances" (CRAFTED): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 300
Virtual Best Solver (VBS)			64	21%	100%	38629.90	108.23	603.59	-	-	-	-
1	clasp	2.0-R4092-crafted	35	12%	55%	7933.56	64.98	226.67	7933.39	64.98	226.67	1.00
2	SAT09 reference solver: clasp	1.2.0-SAT09-32	33	11%	52%	6833.03	88.30	207.06	6833.03	88.30	207.06	1.00
3	QuteRSat	2011-05-12 (fixed)	32	11%	50%	6683.36	33.72	208.86	6683.32	33.74	208.85	1.00
4	claspmt	2.0-R4095-patched (fixed)	32	11%	50%	7174.14	108.14	224.19	1801.74	27.22	56.30	3.98
5	CryptoMiniSat	Strange-Night-1-st (fixed)	32	11%	50%	7790.49	55.46	243.45	7790.50	55.46	243.45	1.00
6	ppfolio	seq	32	11%	50%	7802.40	64.16	243.82	7803.22	64.16	243.85	1.00
7	SAT09 reference solver: SATzilla2009_C	2009-03-22	31	10%	48%	7294.56	19.36	235.31	7289.64	19.37	235.15	1.00
8	ppfolio	par	31	10%	48%	7639.93	18.69	246.45	1986.51	4.68	64.08	3.85
9	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	31	10%	48%	9829.27	24.64	317.07	9851.08	24.74	317.78	1.00
10	SAT07 reference solver: minisat	SAT 2007	30	10%	47%	4576.37	63.76	152.55	4576.31	63.76	152.54	1.00
11	Sol	2011-04-04	30	10%	47%	4870.17	14.06	162.34	4829.17	13.97	160.97	1.01
12	CryptoMiniSat	Strange-Night2-st (fixed)	30	10%	47%	5398.45	13.06	179.95	5398.50	13.06	179.95	1.00
13	glucose	2	30	10%	47%	5812.02	71.66	193.73	5812.03	71.67	193.73	1.00
14	Satisfiability Solver Selector	2011-03-18	29	10%	45%	7765.30	24.84	267.77	7770.04	24.98	267.93	1.00
15	CryptoMiniSat	Strange-Night-1-mt (fixed)	28	9%	44%	7948.24	33.44	283.87	2000.80	8.39	71.46	3.97
16	RestartSAT	B95	28	9%	44%	8056.84	187.34	287.74	8056.69	187.33	287.74	1.00
17	SApperloT	2011-03-23	27	9%	42%	3936.70	112.08	145.80	3936.66	112.09	145.80	1.00
18	Lingeling	587f (fixed)	27	9%	42%	5883.89	27.47	217.92	5883.92	27.47	217.92	1.00
19	MPhaseSAT64	2011-05-14 (fixed)	27	9%	42%	6377.52	6.65	236.20	6377.48	6.65	236.20	1.00
20	CryptoMiniSat	Strange-Night-2-mt (fixed)	27	9%	42%	6700.38	29.02	248.16	1680.58	7.28	62.24	3.99
21	PicoSAT	941	27	9%	42%	7442.23	187.28	275.64	7442.20	187.27	275.64	1.00
22	MPhaseSAT	2011-02-15	26	9%	41%	5119.05	4.97	196.89	5119.06	4.98	196.89	1.00
23	SApperloT2010	2011-05-15 (fixed)	26	9%	41%	6042.31	139.46	232.40	6042.23	139.46	232.39	1.00
24	RestartSAT	A75	26	9%	41%	6813.43	28.67	262.06	6813.31	28.68	262.05	1.00
25	march_rw	2011-03-02	25	8%	39%	3767.68	4.97	150.71	3767.66	4.97	150.71	1.00
26	MPhaseSAT_M	2011-02-16	25	8%	39%	4259.92	3.24	170.40	4259.87	3.24	170.39	1.00
27	rcl	2011-04-01	25	8%	39%	7071.44	165.91	282.86	7071.41	165.91	282.86	1.00
28	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	24	8%	38%	3723.30	34.57	155.14	3699.89	33.24	154.16	1.01
29	Plingeling	587f (fixed)	23	8%	36%	5028.23	19.03	218.62	1266.75	4.85	55.08	3.97
30	jMiniSat	2011	23	8%	36%	5826.74	78.11	253.34	5813.85	77.66	252.78	1.00
31	SArTagnan	2011-05-15 (fixed)	22	7%	34%	5026.08	25.05	228.46	1572.09	7.64	71.46	3.20
32	riss	Auto (1200 s timeout)	22	7%	34%	5100.40	6.44	231.84	5100.42	6.44	231.84	1.00
33	riss	1.0	21	7%	33%	3682.77	3.57	175.37	3682.72	3.57	175.37	1.00
34	riss	Auto	21	7%	33%	3752.82	3.59	178.71	3752.77	3.59	178.70	1.00
35	CryptoMiniSat	TechDemo-Alpha-mt	20	7%	31%	2709.98	26.50	135.50	691.07	6.76	34.55	3.92
36	riss	Exp	20	7%	31%	2785.61	10.12	139.28	2786.17	10.31	139.31	1.00
37	priss	UP	20	7%	31%	3348.37	6.12	167.42	1812.72	4.53	90.64	1.85
38	sathys	2011-04-01	20	7%	31%	4273.04	12.75	213.65	4273.07	12.75	213.65	1.00
39	UmSat	2011-05-12 (fixed)	19	6%	30%	2708.50	13.33	142.55	2708.50	13.33	142.55	1.00
40	priss	Auto	17	6%	27%	2329.81	9.14	137.05	583.80	2.30	34.34	3.99
41	tts	5-4	17	6%	27%	2877.96	7.63	169.29	2878.07	7.63	169.30	1.00
42	priss	1.0	16	5%	25%	1279.48	8.82	79.97	320.80	2.25	20.05	3.99
43	ttsth	5-4	15	5%	23%	3276.59	1.40	218.44	833.73	0.96	55.58	3.93
44	STRUCTure	0.1 (hbr=2)	14	5%	22%	1708.13	51.91	122.01	443.15	14.00	31.65	3.85
45	STRUCTure	0.1	14	5%	22%	1752.16	48.77	125.15	453.81	13.51	32.41	3.86
46	Black Hole SAT	2011-03-08	11	4%	17%	830.67	12.68	75.52	830.69	12.68	75.52	1.00
47	sattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
48	sattime	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
49	shortsattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
50	adaptg2wsat2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
51	shortsattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
52	sattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "random instances" (RANDOM): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 600
Virtual Best Solver (VBS)			466	358 SAT, 108 UNSAT	78%	100%	43024.93	1.68	92.33	-	-	-	-
1	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	399	293 SAT, 106 UNSAT	67%	86%	48566.02	20.13	121.72	48654.12	20.44	121.94	1.00
2	borg-sat	11.03.02-seq-random (fixed)	398	292 SAT, 106 UNSAT	66%	85%	52173.90	14.08	131.09	52370.05	14.43	131.58	1.00
3	Satisfiability Solver Selector	2011-03-18	379	283 SAT, 96 UNSAT	63%	81%	81047.07	37.35	213.84	81640.96	38.84	215.41	0.99
4	borg-sat	11.03.02-par-random (fixed)	373	293 SAT, 80 UNSAT	62%	80%	34748.26	6.75	93.16	9656.44	2.49	25.89	3.60
5	SAT09 reference solver: SATzilla2009_R	2009-03-22	351	268 SAT, 83 UNSAT	59%	75%	36827.82	3.36	104.92	36828.62	3.36	104.92	1.00
6	sparrow2011	sparrow2011_ubcsat1.2_2011-03-02	342	342 SAT	57%	73%	29318.01	2.09	85.73	29318.37	2.10	85.73	1.00
7	MPhaseSAT_M	2011-02-16	341	242 SAT, 99 UNSAT	57%	73%	34895.57	6.50	102.33	34896.22	6.51	102.33	1.00
8	ppfolio	seq	322	262 SAT, 60 UNSAT	54%	69%	39143.95	4.23	121.57	39152.56	4.19	121.59	1.00
9	ppfolio	par	317	242 SAT, 75 UNSAT	53%	68%	30648.50	6.50	96.68	7766.70	1.70	24.50	3.95
10	EagleUP	1.565.350	313	313 SAT	52%	67%	19498.17	0.74	62.29	19498.49	0.74	62.30	1.00
11	MPhaseSAT64	2011-05-14 (fixed)	312	243 SAT, 69 UNSAT	52%	67%	37258.37	2.27	119.42	37258.40	2.28	119.42	1.00
12	MPhaseSAT	2011-02-15	308	241 SAT, 67 UNSAT	51%	66%	30441.77	2.09	98.84	30442.48	2.10	98.84	1.00
13	sattime2011	2011-03-02	307	307 SAT	51%	66%	40428.29	1.87	131.69	40432.46	1.96	131.70	1.00
14	shortsattime2011	2011-03-02	304	304 SAT	51%	65%	39438.09	1.84	129.73	39437.83	1.73	129.73	1.00
15	adaptg2wsat2011	2011-03-02	299	299 SAT	50%	64%	40184.75	1.59	134.40	40184.47	1.60	134.40	1.00
16	SAT09 reference solver: TNM	2009-03-22	292	292 SAT	49%	63%	29562.50	1.27	101.24	29562.68	1.27	101.24	1.00
17	csls	pnorm-8cores	282	282 SAT	47%	61%	21716.31	0.72	77.01	5437.64	0.18	19.28	3.99
18	csls	prob-4cores	280	280 SAT	47%	60%	22868.93	0.71	81.67	5726.40	0.18	20.45	3.99
19	csls	prob-8cores	279	279 SAT	47%	60%	20041.78	0.60	71.83	5018.39	0.16	17.99	3.99
20	SAT09 reference solver: gnovelty+2	2009-03-22	275	275 SAT	46%	59%	23349.21	0.57	84.91	23349.09	0.58	84.91	1.00
21	MultiFlip	2011-02-11	272	272 SAT	45%	58%	22181.31	0.83	81.55	22181.00	0.83	81.55	1.00
22	sattime	2011-03-02	250	250 SAT	42%	54%	16641.40	0.65	66.57	16641.45	0.65	66.57	1.00
23	sattime+	2011-03-02	216	216 SAT	36%	46%	16294.70	1.75	75.44	16286.41	1.75	75.40	1.00
24	march_rw	2011-03-02	212	106 SAT, 106 UNSAT	35%	45%	27576.88	23.68	130.08	27576.97	23.69	130.08	1.00
25	shortsattime+	2011-03-02	211	211 SAT	35%	45%	14703.56	1.61	69.69	14706.10	1.61	69.70	1.00
26	SAT09 reference solver: march_hi	hi	211	105 SAT, 106 UNSAT	35%	45%	29171.96	22.76	138.26	29171.86	22.76	138.26	1.00
27	sathys	2011-04-01	181	159 SAT, 22 UNSAT	30%	39%	21875.26	22.71	120.86	21876.84	22.71	120.87	1.00
28	STRUCTure	0.1 (hbr=2)	86	36 SAT, 50 UNSAT	14%	18%	33378.40	232.18	388.12	8451.57	59.05	98.27	3.95
29	RestartSAT	B95	82	34 SAT, 48 UNSAT	14%	18%	20973.75	84.39	255.78	20973.43	84.39	255.77	1.00
30	CryptoMiniSat	Strange-Night2-st (fixed)	80	32 SAT, 48 UNSAT	13%	17%	23145.14	132.29	289.31	23145.12	132.29	289.31	1.00
31	QuteRSat	2011-05-12 (fixed)	79	30 SAT, 49 UNSAT	13%	17%	21395.40	172.17	270.83	21400.91	172.17	270.90	1.00
32	jMiniSat	2011	79	37 SAT, 42 UNSAT	13%	17%	22353.44	101.10	282.95	22316.56	100.81	282.49	1.00
33	RestartSAT	A75	76	38 SAT, 38 UNSAT	13%	16%	21608.24	102.90	284.32	21608.76	102.90	284.33	1.00
34	CryptoMiniSat	Strange-Night-1-st (fixed)	74	26 SAT, 48 UNSAT	12%	16%	19478.78	94.79	263.23	19478.74	94.78	263.23	1.00
35	STRUCTure	0.1	73	33 SAT, 40 UNSAT	12%	16%	19267.83	195.40	263.94	4893.04	50.20	67.03	3.94
36	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	65	33 SAT, 32 UNSAT	11%	14%	18743.17	117.48	288.36	18683.92	117.15	287.44	1.00
37	CryptoMiniSat	Strange-Night-2-mt (fixed)	59	28 SAT, 31 UNSAT	10%	13%	11656.55	67.12	197.57	2924.38	16.91	49.57	3.99
38	CryptoMiniSat	Strange-Night-1-mt (fixed)	55	25 SAT, 30 UNSAT	9%	12%	9191.60	56.27	167.12	2307.11	14.33	41.95	3.98
39	Sol	2011-04-04	52	22 SAT, 30 UNSAT	9%	11%	11375.45	46.15	218.76	11325.16	45.86	217.79	1.00
40	Black Hole SAT	2011-03-08	44	18 SAT, 26 UNSAT	7%	9%	19503.25	295.15	443.26	19502.98	295.15	443.25	1.00
41	CryptoMiniSat	TechDemo-Alpha-mt	42	14 SAT, 28 UNSAT	7%	9%	10864.81	82.91	258.69	2729.20	21.21	64.98	3.98
42	SAT_Intersec_01	2011-03-01	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "random instances" (RANDOM): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 600
Virtual Best Solver (VBS)			358	60%	100%	26493.64	0.45	74.00	-	-	-	-
1	sparrow2011	sparrow2011_ubcsat1.2_2011-03-02	342	57%	96%	29318.01	2.09	85.73	29318.37	2.10	85.73	1.00
2	EagleUP	1.565.350	313	52%	87%	19498.17	0.74	62.29	19498.49	0.74	62.30	1.00
3	sattime2011	2011-03-02	307	51%	86%	40428.29	1.87	131.69	40432.46	1.96	131.70	1.00
4	shortsattime2011	2011-03-02	304	51%	85%	39438.09	1.84	129.73	39437.83	1.73	129.73	1.00
5	adaptg2wsat2011	2011-03-02	299	50%	84%	40184.75	1.59	134.40	40184.47	1.60	134.40	1.00
6	borg-sat	11.03.02-par-random (fixed)	293	49%	82%	23892.04	3.70	81.54	6593.65	1.73	22.50	3.62
7	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	293	49%	82%	29619.17	18.60	101.09	29733.95	18.83	101.48	1.00
8	borg-sat	11.03.02-seq-random (fixed)	292	49%	82%	15865.63	2.03	54.33	16063.13	3.06	55.01	0.99
9	SAT09 reference solver: TNM	2009-03-22	292	49%	82%	29562.50	1.27	101.24	29562.68	1.27	101.24	1.00
10	Satisfiability Solver Selector	2011-03-18	283	47%	79%	53116.01	35.35	187.69	53608.37	35.95	189.43	0.99
11	csls	pnorm-8cores	282	47%	79%	21716.31	0.72	77.01	5437.64	0.18	19.28	3.99
12	csls	prob-4cores	280	47%	78%	22868.93	0.71	81.67	5726.40	0.18	20.45	3.99
13	csls	prob-8cores	279	47%	78%	20041.78	0.60	71.83	5018.39	0.16	17.99	3.99
14	SAT09 reference solver: gnovelty+2	2009-03-22	275	46%	77%	23349.21	0.57	84.91	23349.09	0.58	84.91	1.00
15	MultiFlip	2011-02-11	272	45%	76%	22181.31	0.83	81.55	22181.00	0.83	81.55	1.00
16	SAT09 reference solver: SATzilla2009_R	2009-03-22	268	45%	75%	23400.86	0.68	87.32	23401.42	0.68	87.32	1.00
17	ppfolio	seq	262	44%	73%	22877.90	2.12	87.32	22887.86	2.12	87.36	1.00
18	sattime	2011-03-02	250	42%	70%	16641.40	0.65	66.57	16641.45	0.65	66.57	1.00
19	MPhaseSAT64	2011-05-14 (fixed)	243	41%	68%	14339.56	1.18	59.01	14339.81	1.18	59.01	1.00
20	MPhaseSAT_M	2011-02-16	242	40%	68%	15962.15	1.32	65.96	15962.42	1.32	65.96	1.00
21	ppfolio	par	242	40%	68%	18731.75	3.17	77.40	4756.27	0.99	19.65	3.94
22	MPhaseSAT	2011-02-15	241	40%	67%	14085.63	1.26	58.45	14086.43	1.27	58.45	1.00
23	sattime+	2011-03-02	216	36%	60%	16294.70	1.75	75.44	16286.41	1.75	75.40	1.00
24	shortsattime+	2011-03-02	211	35%	59%	14703.56	1.61	69.69	14706.10	1.61	69.70	1.00
25	sathys	2011-04-01	159	27%	44%	19287.48	22.56	121.30	19288.78	22.56	121.31	1.00
26	march_rw	2011-03-02	106	18%	30%	13430.83	18.56	126.71	13430.76	18.56	126.71	1.00
27	SAT09 reference solver: march_hi	hi	105	18%	29%	14326.32	17.51	136.44	14326.27	17.51	136.44	1.00
28	RestartSAT	A75	38	6%	11%	9559.74	159.27	251.57	9559.82	159.27	251.57	1.00
29	jMiniSat	2011	37	6%	10%	11180.17	221.88	302.17	11164.54	221.44	301.74	1.00
30	STRUCTure	0.1 (hbr=2)	36	6%	10%	15388.09	265.98	427.45	3879.91	67.19	107.78	3.97
31	RestartSAT	B95	34	6%	9%	8008.92	118.41	235.56	8008.80	118.41	235.55	1.00
32	STRUCTure	0.1	33	6%	9%	12358.10	292.11	374.49	3116.57	73.79	94.44	3.97
33	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	33	6%	9%	15245.65	475.78	461.99	15226.75	475.52	461.42	1.00
34	CryptoMiniSat	Strange-Night2-st (fixed)	32	5%	9%	9929.16	202.50	310.29	9929.11	202.49	310.28	1.00
35	QuteRSat	2011-05-12 (fixed)	30	5%	8%	7975.11	221.96	265.84	7975.17	221.98	265.84	1.00
36	CryptoMiniSat	Strange-Night-2-mt (fixed)	28	5%	8%	8513.15	186.94	304.04	2135.76	46.89	76.28	3.99
37	CryptoMiniSat	Strange-Night-1-st (fixed)	26	4%	7%	6869.85	142.64	264.23	6869.93	142.69	264.23	1.00
38	CryptoMiniSat	Strange-Night-1-mt (fixed)	25	4%	7%	7099.02	179.17	283.96	1782.23	45.63	71.29	3.98
39	Sol	2011-04-04	22	4%	6%	9060.70	222.73	411.85	9021.80	221.24	410.08	1.00
40	Black Hole SAT	2011-03-08	18	3%	5%	11463.61	695.93	636.87	11463.49	695.92	636.86	1.00
41	CryptoMiniSat	TechDemo-Alpha-mt	14	2%	4%	7625.49	464.63	544.68	1912.45	116.57	136.60	3.99
42	SAT_Intersec_01	2011-03-01	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "random instances" (RANDOM): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 600
Virtual Best Solver (VBS)			108	18%	100%	16531.29	29.14	153.07	-	-	-	-
1	march_rw	2011-03-02	106	18%	98%	14146.04	26.57	133.45	14146.21	26.57	133.45	1.00
2	SAT09 reference solver: march_hi	hi	106	18%	98%	14845.64	30.27	140.05	14845.59	30.27	140.05	1.00
3	Satisfiability Solver Selector	2011-03-18 (1200 s timeout)	106	18%	98%	18946.85	71.86	178.74	18920.17	74.11	178.49	1.00
4	borg-sat	11.03.02-seq-random (fixed)	106	18%	98%	36308.27	224.23	342.53	36306.93	224.02	342.52	1.00
5	MPhaseSAT_M	2011-02-16	99	17%	92%	18933.42	64.90	191.25	18933.81	64.90	191.25	1.00
6	Satisfiability Solver Selector	2011-03-18	96	16%	89%	27931.06	170.13	290.95	28032.59	170.40	292.01	1.00
7	SAT09 reference solver: SATzilla2009_R	2009-03-22	83	14%	77%	13426.96	105.66	161.77	13427.20	105.66	161.77	1.00
8	borg-sat	11.03.02-par-random (fixed)	80	13%	74%	10856.21	34.37	135.70	3062.79	9.61	38.28	3.54
9	ppfolio	par	75	13%	69%	11916.75	52.29	158.89	3010.43	13.97	40.14	3.96
10	MPhaseSAT64	2011-05-14 (fixed)	69	12%	64%	22918.80	119.21	332.16	22918.59	119.21	332.15	1.00
11	MPhaseSAT	2011-02-15	67	11%	62%	16356.14	121.23	244.12	16356.05	121.23	244.12	1.00
12	ppfolio	seq	60	10%	56%	16266.05	154.95	271.10	16264.70	155.02	271.08	1.00
13	STRUCTure	0.1 (hbr=2)	50	8%	46%	17990.31	205.68	359.81	4571.65	52.74	91.43	3.94
14	QuteRSat	2011-05-12 (fixed)	49	8%	45%	13420.30	40.47	273.88	13425.74	40.47	273.99	1.00
15	CryptoMiniSat	Strange-Night-1-st (fixed)	48	8%	44%	12608.92	36.14	262.69	12608.82	36.14	262.68	1.00
16	RestartSAT	B95	48	8%	44%	12964.83	38.05	270.10	12964.63	38.05	270.10	1.00
17	CryptoMiniSat	Strange-Night2-st (fixed)	48	8%	44%	13215.98	38.39	275.33	13216.01	38.39	275.33	1.00
18	jMiniSat	2011	42	7%	39%	11173.27	36.65	266.03	11152.02	36.17	265.52	1.00
19	STRUCTure	0.1	40	7%	37%	6909.73	73.46	172.74	1776.47	19.42	44.41	3.89
20	RestartSAT	A75	38	6%	35%	12048.50	47.67	317.07	12048.94	47.68	317.08	1.00
21	SAT4J core	2.3.0 SNAPSHOT 2011-02-06	32	5%	30%	3497.52	9.47	109.30	3457.17	7.73	108.04	1.01
22	CryptoMiniSat	Strange-Night-2-mt (fixed)	31	5%	29%	3143.40	16.99	101.40	788.62	4.26	25.44	3.99
23	CryptoMiniSat	Strange-Night-1-mt (fixed)	30	5%	28%	2092.58	16.08	69.75	524.88	4.05	17.50	3.99
24	Sol	2011-04-04	30	5%	28%	2314.75	7.08	77.16	2303.37	7.01	76.78	1.00
25	CryptoMiniSat	TechDemo-Alpha-mt	28	5%	26%	3239.31	39.83	115.69	816.75	10.19	29.17	3.97
26	Black Hole SAT	2011-03-08	26	4%	24%	8039.65	159.69	309.22	8039.49	159.69	309.21	1.00
27	sathys	2011-04-01	22	4%	20%	2587.78	46.17	117.63	2588.05	46.29	117.64	1.00
28	EagleUP	1.565.350	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
29	MultiFlip	2011-02-11	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
30	sattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
31	sattime	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
32	shortsattime2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
33	csls	prob-4cores	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
34	csls	prob-8cores	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
35	csls	pnorm-8cores	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
36	sparrow2011	sparrow2011_ubcsat1.2_2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
37	adaptg2wsat2011	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
38	shortsattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
39	sattime+	2011-03-02	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
40	SAT_Intersec_01	2011-03-01	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
41	SAT09 reference solver: TNM	2009-03-22	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-
42	SAT09 reference solver: gnovelty+2	2009-03-22	0	0%	0%	0.00	0.00	-	0.00	0.00	-	-

SAT 2011 Competition: main competition (phase 1): ranking of solvers max CPU time for any solver=1200 s max WC time for any solver=1210 s