SAT 2011 Competition: main competition (phase 1): solvers page (tabular format, summary)

Some explanations

Category "applications instances" (APPLICATION)

Solver Name#benchsSolvedUNSATSATUNKNOWNBad Answers
adaptg2wsat2011
2011-03-02
(incomplete)		 30015
(5%)		 0
(0%)		 15
(5%)		 285
(95%)		 0
(0%)
Black Hole SAT
2011-03-08
(complete)		 30027
(9%)		 5
(1.67%)		 22
(7.33%)		 273
(91%)		 0
(0%)
borg-sat
11.03.02-industrial [DEPRECATED]
(complete)		 300175
(58.33%)		 89
(29.67%)		 86
(28.67%)		 125
(41.67%)		 0
(0%)
borg-sat
11.03.02-par-industrial (fixed)
(complete)		 300179
(59.67%)		 94
(31.33%)		 85
(28.33%)		 120
(40%)		 1
(0.33%)
borg-sat
11.03.02-seq-industrial (fixed)
(complete)		 300160
(53.33%)		 84
(28%)		 76
(25.33%)		 140
(46.67%)		 0
(0%)
borg-sat
11.03.02-seq-industrial (fixed, 1200 s timeout)
(complete)		 300160
(53.33%)		 85
(28.33%)		 75
(25%)		 140
(46.67%)		 0
(0%)
clasp
2.0-R4092-indust
(complete)		 300154
(51.33%)		 81
(27%)		 73
(24.33%)		 146
(48.67%)		 0
(0%)
claspmt
2.0-R4095 [DEPRECATED]
(complete)		 300178
(59.33%)		 89
(29.67%)		 89
(29.67%)		 116
(38.67%)		 6
(2%)
claspmt
2.0-R4095-patched (fixed)
(complete)		 300181
(60.33%)		 92
(30.67%)		 89
(29.67%)		 119
(39.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)		 300181
(60.33%)		 104
(34.67%)		 77
(25.67%)		 119
(39.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)		 300181
(60.33%)		 103
(34.33%)		 78
(26%)		 119
(39.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)		 300163
(54.33%)		 91
(30.33%)		 72
(24%)		 137
(45.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)		 300163
(54.33%)		 91
(30.33%)		 72
(24%)		 137
(45.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)		 300191
(63.67%)		 103
(34.33%)		 88
(29.33%)		 109
(36.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)		 300192
(64%)		 105
(35%)		 87
(29%)		 108
(36%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)		 300173
(57.67%)		 96
(32%)		 77
(25.67%)		 127
(42.33%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)		 300174
(58%)		 96
(32%)		 78
(26%)		 126
(42%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-mt
(complete)		 300139
(46.33%)		 72
(24%)		 67
(22.33%)		 161
(53.67%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-st
(complete)		 300136
(45.33%)		 73
(24.33%)		 63
(21%)		 162
(54%)		 2
(0.67%)
EBGlucose
1.0
(incomplete)		 300158
(52.67%)		 82
(27.33%)		 76
(25.33%)		 142
(47.33%)		 0
(0%)
glucose
2
(complete)		 300177
(59%)		 94
(31.33%)		 83
(27.67%)		 123
(41%)		 0
(0%)
glueminisat
2.2.5
(complete)		 300171
(57%)		 98
(32.67%)		 73
(24.33%)		 129
(43%)		 0
(0%)
jMiniSat
2011
(complete)		 300132
(44%)		 62
(20.67%)		 70
(23.33%)		 168
(56%)		 0
(0%)
kw
2011-02-28
(complete)		 300156
(52%)		 83
(27.67%)		 73
(24.33%)		 144
(48%)		 0
(0%)
Lingeling
587 [DEPRECATED]
(complete)		 300172
(57.33%)		 93
(31%)		 79
(26.33%)		 128
(42.67%)		 0
(0%)
Lingeling
587f (fixed)
(complete)		 300172
(57.33%)		 93
(31%)		 79
(26.33%)		 128
(42.67%)		 0
(0%)
march_rw
2011-03-02
(complete)		 30023
(7.67%)		 8
(2.67%)		 15
(5%)		 277
(92.33%)		 0
(0%)
minisat hack: cir_minisat
2011-03-02
(complete)		 300159
(53%)		 74
(24.67%)		 85
(28.33%)		 141
(47%)		 0
(0%)
minisat hack: cir_minisat
2011-05-13 (simp)
(complete)		 300164
(54.67%)		 82
(27.33%)		 82
(27.33%)		 136
(45.33%)		 0
(0%)
minisat hack: contrasat
2011-03-02
(complete)		 300161
(53.67%)		 72
(24%)		 89
(29.67%)		 139
(46.33%)		 0
(0%)
minisat hack: EBMiniSAT
2011-03-02
(incomplete)		 300162
(54%)		 78
(26%)		 84
(28%)		 138
(46%)		 0
(0%)
minisat hack: LR_GL_NOSHR
2011-03-02
(complete)		 300155
(51.67%)		 71
(23.67%)		 84
(28%)		 145
(48.33%)		 0
(0%)
minisat hack: LR_GL_SHR
2011-03-02
(complete)		 300158
(52.67%)		 90
(30%)		 68
(22.67%)		 142
(47.33%)		 0
(0%)
minisat hack: LR_NOGL_NOSHR
2011-03-02
(complete)		 300154
(51.33%)		 71
(23.67%)		 83
(27.67%)		 146
(48.67%)		 0
(0%)
minisat hack: MiniSAT
2.2.0-agile-26
(complete)		 300160
(53.33%)		 74
(24.67%)		 86
(28.67%)		 140
(46.67%)		 0
(0%)
minisat hack: minisat_psm
2011-04-01
(complete)		 300163
(54.33%)		 77
(25.67%)		 86
(28.67%)		 137
(45.67%)		 0
(0%)
minisat hack reference: minisat
2.2.0
(complete)		 300161
(53.67%)		 74
(24.67%)		 87
(29%)		 139
(46.33%)		 0
(0%)
MPhaseSAT
2011-02-15
(complete)		 300162
(54%)		 82
(27.33%)		 80
(26.67%)		 138
(46%)		 0
(0%)
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)		 300161
(53.67%)		 79
(26.33%)		 82
(27.33%)		 139
(46.33%)		 0
(0%)
MPhaseSAT64
2011-05-14 (fixed)
(complete)		 300170
(56.67%)		 86
(28.67%)		 84
(28%)		 130
(43.33%)		 0
(0%)
MPhaseSAT_M
2011-02-16
(complete)		 300166
(55.33%)		 82
(27.33%)		 84
(28%)		 134
(44.67%)		 0
(0%)
PicoSAT
941
(complete)		 300141
(47%)		 67
(22.33%)		 74
(24.67%)		 159
(53%)		 0
(0%)
Plingeling
587 [DEPRECATED]
(complete)		 300190
(63.33%)		 95
(31.67%)		 95
(31.67%)		 110
(36.67%)		 0
(0%)
Plingeling
587f (fixed)
(complete)		 300192
(64%)		 96
(32%)		 96
(32%)		 108
(36%)		 0
(0%)
ppfolio
par
(complete)		 300184
(61.33%)		 98
(32.67%)		 86
(28.67%)		 116
(38.67%)		 0
(0%)
ppfolio
seq
(complete)		 300155
(51.67%)		 84
(28%)		 71
(23.67%)		 145
(48.33%)		 0
(0%)
priss
1.0
(complete)		 300147
(49%)		 77
(25.67%)		 70
(23.33%)		 153
(51%)		 0
(0%)
priss
Auto
(complete)		 300152
(50.67%)		 78
(26%)		 74
(24.67%)		 148
(49.33%)		 0
(0%)
priss
UP
(complete)		 300144
(48%)		 73
(24.33%)		 71
(23.67%)		 156
(52%)		 0
(0%)
QuteRSat
2011-03-01 [DEPRECATED]
(complete)		 300169
(56.33%)		 97
(32.33%)		 72
(24%)		 130
(43.33%)		 1
(0.33%)
QuteRSat
2011-05-12 (fixed)
(complete)		 300174
(58%)		 96
(32%)		 78
(26%)		 126
(42%)		 0
(0%)
rcl
2011-04-01
(complete)		 300157
(52.33%)		 74
(24.67%)		 83
(27.67%)		 143
(47.67%)		 0
(0%)
RestartSAT
A75
(complete)		 300171
(57%)		 93
(31%)		 78
(26%)		 129
(43%)		 0
(0%)
RestartSAT
B95
(complete)		 300172
(57.33%)		 89
(29.67%)		 83
(27.67%)		 128
(42.67%)		 0
(0%)
riss
1.0
(complete)		 300145
(48.33%)		 75
(25%)		 70
(23.33%)		 155
(51.67%)		 0
(0%)
riss
Auto
(complete)		 300145
(48.33%)		 75
(25%)		 70
(23.33%)		 155
(51.67%)		 0
(0%)
riss
Auto (1200 s timeout)
(complete)		 300138
(46%)		 73
(24.33%)		 65
(21.67%)		 162
(54%)		 0
(0%)
riss
Exp
(complete)		 300138
(46%)		 69
(23%)		 69
(23%)		 162
(54%)		 0
(0%)
SApperloT
2011-03-23
(complete)		 300152
(50.67%)		 77
(25.67%)		 75
(25%)		 148
(49.33%)		 0
(0%)
SApperloT2010
2011-03-04 [DEPRECATED]
(complete)		 300144
(48%)		 81
(27%)		 63
(21%)		 143
(47.67%)		 13
(4.33%)
SApperloT2010
2011-05-15 (fixed)
(complete)		 300157
(52.33%)		 82
(27.33%)		 75
(25%)		 143
(47.67%)		 0
(0%)
SArTagnan
2011-03-24 [DEPRECATED]
(complete)		 300135
(45%)		 80
(26.67%)		 55
(18.33%)		 143
(47.67%)		 22
(7.33%)
SArTagnan
2011-05-15 (fixed)
(complete)		 300154
(51.33%)		 79
(26.33%)		 75
(25%)		 146
(48.67%)		 0
(0%)
SAT09 reference solver:glucose
1.0
(complete)		 300159
(53%)		 84
(28%)		 75
(25%)		 141
(47%)		 0
(0%)
SAT09 reference solver:LySAT
i/2009-03-20
(complete)		 300131
(43.67%)		 71
(23.67%)		 60
(20%)		 169
(56.33%)		 0
(0%)
SAT09 reference solver: ManySAT 1.1
aimd 1/2009-03-20
(complete)		 300169
(56.33%)		 91
(30.33%)		 78
(26%)		 131
(43.67%)		 0
(0%)
SAT09 reference solver:precosat
236
(complete)		 300167
(55.67%)		 88
(29.33%)		 79
(26.33%)		 133
(44.33%)		 0
(0%)
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)		 300119
(39.67%)		 56
(18.67%)		 63
(21%)		 181
(60.33%)		 0
(0%)
sathys
2011-04-01
(complete)		 300139
(46.33%)		 70
(23.33%)		 69
(23%)		 161
(53.67%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18
(complete)		 300158
(52.67%)		 79
(26.33%)		 79
(26.33%)		 142
(47.33%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)		 300171
(57%)		 86
(28.67%)		 85
(28.33%)		 129
(43%)		 0
(0%)
sattime
2011-03-02
(incomplete)		 30016
(5.33%)		 0
(0%)		 16
(5.33%)		 284
(94.67%)		 0
(0%)
sattime+
2011-03-02
(incomplete)		 30019
(6.33%)		 7
(2.33%)		 12
(4%)		 281
(93.67%)		 0
(0%)
sattime2011
2011-03-02
(incomplete)		 30016
(5.33%)		 0
(0%)		 16
(5.33%)		 284
(94.67%)		 0
(0%)
shortsattime+
2011-03-02
(incomplete)		 30022
(7.33%)		 7
(2.33%)		 15
(5%)		 278
(92.67%)		 0
(0%)
shortsattime2011
2011-03-02
(incomplete)		 30015
(5%)		 0
(0%)		 15
(5%)		 285
(95%)		 0
(0%)
Sol
2011-04-04
(complete)		 300119
(39.67%)		 61
(20.33%)		 58
(19.33%)		 181
(60.33%)		 0
(0%)
STRUCTure
0.1
(complete)		 30031
(10.33%)		 14
(4.67%)		 17
(5.67%)		 269
(89.67%)		 0
(0%)
STRUCTure
0.1 (hbr=2)
(complete)		 30028
(9.33%)		 14
(4.67%)		 14
(4.67%)		 272
(90.67%)		 0
(0%)
synit_sat
2011-03-03
(complete)		 300163
(54.33%)		 96
(32%)		 67
(22.33%)		 136
(45.33%)		 1
(0.33%)
UmSat
2011-02-25 [DEPRECATED]
(complete)		 30041
(13.67%)		 23
(7.67%)		 18
(6%)		 258
(86%)		 1
(0.33%)
UmSat
2011-05-12 (fixed)
(complete)		 30091
(30.33%)		 44
(14.67%)		 47
(15.67%)		 209
(69.67%)		 0
(0%)

Solver NameProgress
adaptg2wsat2011
2011-03-02
(incomplete)
done
300
SAT
15		?
285
Black Hole SAT
2011-03-08
(complete)
done
300
UNSAT
5		SAT
22		?
273
borg-sat
11.03.02-industrial [DEPRECATED]
(complete)
done
300
UNSAT
89		SAT
86		?
125
borg-sat
11.03.02-par-industrial (fixed)
(complete)
done
300
UNSAT
94		SAT
85		?
120		ERR
1
borg-sat
11.03.02-seq-industrial (fixed)
(complete)
done
300
UNSAT
84		SAT
76		?
140
borg-sat
11.03.02-seq-industrial (fixed, 1200 s timeout)
(complete)
done
300
UNSAT
85		SAT
75		?
140
clasp
2.0-R4092-indust
(complete)
done
300
UNSAT
81		SAT
73		?
146
claspmt
2.0-R4095 [DEPRECATED]
(complete)
done
300
UNSAT
89		SAT
89		?
116		ERR
6
claspmt
2.0-R4095-patched (fixed)
(complete)
done
300
UNSAT
92		SAT
89		?
119
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)
done
300
UNSAT
104		SAT
77		?
119
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)
done
300
UNSAT
103		SAT
78		?
119
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)
done
300
UNSAT
91		SAT
72		?
137
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)
done
300
UNSAT
91		SAT
72		?
137
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)
done
300
UNSAT
103		SAT
88		?
109
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)
done
300
UNSAT
105		SAT
87		?
108
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)
done
300
UNSAT
96		SAT
77		?
127
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)
done
300
UNSAT
96		SAT
78		?
126
CryptoMiniSat
TechDemo-Alpha-mt
(complete)
done
300
UNSAT
72		SAT
67		?
161
CryptoMiniSat
TechDemo-Alpha-st
(complete)
done
300
UNSAT
73		SAT
63		?
162		ERR
2
EBGlucose
1.0
(incomplete)
done
300
UNSAT
82		SAT
76		?
142
glucose
2
(complete)
done
300
UNSAT
94		SAT
83		?
123
glueminisat
2.2.5
(complete)
done
300
UNSAT
98		SAT
73		?
129
jMiniSat
2011
(complete)
done
300
UNSAT
62		SAT
70		?
168
kw
2011-02-28
(complete)
done
300
UNSAT
83		SAT
73		?
144
Lingeling
587 [DEPRECATED]
(complete)
done
300
UNSAT
93		SAT
79		?
128
Lingeling
587f (fixed)
(complete)
done
300
UNSAT
93		SAT
79		?
128
march_rw
2011-03-02
(complete)
done
300
UNSAT
8		SAT
15		?
277
minisat hack: cir_minisat
2011-03-02
(complete)
done
300
UNSAT
74		SAT
85		?
141
minisat hack: cir_minisat
2011-05-13 (simp)
(complete)
done
300
UNSAT
82		SAT
82		?
136
minisat hack: contrasat
2011-03-02
(complete)
done
300
UNSAT
72		SAT
89		?
139
minisat hack: EBMiniSAT
2011-03-02
(incomplete)
done
300
UNSAT
78		SAT
84		?
138
minisat hack: LR_GL_NOSHR
2011-03-02
(complete)
done
300
UNSAT
71		SAT
84		?
145
minisat hack: LR_GL_SHR
2011-03-02
(complete)
done
300
UNSAT
90		SAT
68		?
142
minisat hack: LR_NOGL_NOSHR
2011-03-02
(complete)
done
300
UNSAT
71		SAT
83		?
146
minisat hack: MiniSAT
2.2.0-agile-26
(complete)
done
300
UNSAT
74		SAT
86		?
140
minisat hack: minisat_psm
2011-04-01
(complete)
done
300
UNSAT
77		SAT
86		?
137
minisat hack reference: minisat
2.2.0
(complete)
done
300
UNSAT
74		SAT
87		?
139
MPhaseSAT
2011-02-15
(complete)
done
300
UNSAT
82		SAT
80		?
138
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)
done
300
UNSAT
79		SAT
82		?
139
MPhaseSAT64
2011-05-14 (fixed)
(complete)
done
300
UNSAT
86		SAT
84		?
130
MPhaseSAT_M
2011-02-16
(complete)
done
300
UNSAT
82		SAT
84		?
134
PicoSAT
941
(complete)
done
300
UNSAT
67		SAT
74		?
159
Plingeling
587 [DEPRECATED]
(complete)
done
300
UNSAT
95		SAT
95		?
110
Plingeling
587f (fixed)
(complete)
done
300
UNSAT
96		SAT
96		?
108
ppfolio
par
(complete)
done
300
UNSAT
98		SAT
86		?
116
ppfolio
seq
(complete)
done
300
UNSAT
84		SAT
71		?
145
priss
1.0
(complete)
done
300
UNSAT
77		SAT
70		?
153
priss
Auto
(complete)
done
300
UNSAT
78		SAT
74		?
148
priss
UP
(complete)
done
300
UNSAT
73		SAT
71		?
156
QuteRSat
2011-03-01 [DEPRECATED]
(complete)
done
300
UNSAT
97		SAT
72		?
130		ERR
1
QuteRSat
2011-05-12 (fixed)
(complete)
done
300
UNSAT
96		SAT
78		?
126
rcl
2011-04-01
(complete)
done
300
UNSAT
74		SAT
83		?
143
RestartSAT
A75
(complete)
done
300
UNSAT
93		SAT
78		?
129
RestartSAT
B95
(complete)
done
300
UNSAT
89		SAT
83		?
128
riss
1.0
(complete)
done
300
UNSAT
75		SAT
70		?
155
riss
Auto
(complete)
done
300
UNSAT
75		SAT
70		?
155
riss
Auto (1200 s timeout)
(complete)
done
300
UNSAT
73		SAT
65		?
162
riss
Exp
(complete)
done
300
UNSAT
69		SAT
69		?
162
SApperloT
2011-03-23
(complete)
done
300
UNSAT
77		SAT
75		?
148
SApperloT2010
2011-03-04 [DEPRECATED]
(complete)
done
300
UNSAT
81		SAT
63		?
143		ERR
13
SApperloT2010
2011-05-15 (fixed)
(complete)
done
300
UNSAT
82		SAT
75		?
143
SArTagnan
2011-03-24 [DEPRECATED]
(complete)
done
300
UNSAT
80		SAT
55		?
143		ERR
22
SArTagnan
2011-05-15 (fixed)
(complete)
done
300
UNSAT
79		SAT
75		?
146
SAT09 reference solver:glucose
1.0
(complete)
done
300
UNSAT
84		SAT
75		?
141
SAT09 reference solver:LySAT
i/2009-03-20
(complete)
done
300
UNSAT
71		SAT
60		?
169
SAT09 reference solver: ManySAT 1.1
aimd 1/2009-03-20
(complete)
done
300
UNSAT
91		SAT
78		?
131
SAT09 reference solver:precosat
236
(complete)
done
300
UNSAT
88		SAT
79		?
133
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)
done
300
UNSAT
56		SAT
63		?
181
sathys
2011-04-01
(complete)
done
300
UNSAT
70		SAT
69		?
161
Satisfiability Solver Selector
2011-03-18
(complete)
done
300
UNSAT
79		SAT
79		?
142
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)
done
300
UNSAT
86		SAT
85		?
129
sattime
2011-03-02
(incomplete)
done
300
SAT
16		?
284
sattime+
2011-03-02
(incomplete)
done
300
UNSAT
7		SAT
12		?
281
sattime2011
2011-03-02
(incomplete)
done
300
SAT
16		?
284
shortsattime+
2011-03-02
(incomplete)
done
300
UNSAT
7		SAT
15		?
278
shortsattime2011
2011-03-02
(incomplete)
done
300
SAT
15		?
285
Sol
2011-04-04
(complete)
done
300
UNSAT
61		SAT
58		?
181
STRUCTure
0.1
(complete)
done
300
UNSAT
14		SAT
17		?
269
STRUCTure
0.1 (hbr=2)
(complete)
done
300
UNSAT
14		SAT
14		?
272
synit_sat
2011-03-03
(complete)
done
300
UNSAT
96		SAT
67		?
136		ERR
1
UmSat
2011-02-25 [DEPRECATED]
(complete)
done
300
UNSAT
23		SAT
18		?
258		ERR
1
UmSat
2011-05-12 (fixed)
(complete)
done
300
UNSAT
44		SAT
47		?
209

Category "crafted instances" (CRAFTED)

Solver Name#benchsSolvedUNSATSATUNKNOWNBad Answers
adaptg2wsat2011
2011-03-02
(incomplete)		 30099
(33%)		 0
(0%)		 99
(33%)		 201
(67%)		 0
(0%)
Black Hole SAT
2011-03-08
(complete)		 30036
(12%)		 11
(3.67%)		 25
(8.33%)		 264
(88%)		 0
(0%)
clasp
2.0-R4092-crafted
(complete)		 300105
(35%)		 37
(12.33%)		 68
(22.67%)		 195
(65%)		 0
(0%)
claspmt
2.0-R4095 [DEPRECATED]
(complete)		 300129
(43%)		 41
(13.67%)		 88
(29.33%)		 170
(56.67%)		 1
(0.33%)
claspmt
2.0-R4095-patched (fixed)
(complete)		 300133
(44.33%)		 43
(14.33%)		 90
(30%)		 167
(55.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)		 300111
(37%)		 38
(12.67%)		 73
(24.33%)		 189
(63%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)		 300113
(37.67%)		 37
(12.33%)		 76
(25.33%)		 187
(62.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)		 30089
(29.67%)		 32
(10.67%)		 57
(19%)		 211
(70.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)		 30089
(29.67%)		 32
(10.67%)		 57
(19%)		 211
(70.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)		 300107
(35.67%)		 35
(11.67%)		 72
(24%)		 193
(64.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)		 300105
(35%)		 36
(12%)		 69
(23%)		 195
(65%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)		 30095
(31.67%)		 32
(10.67%)		 63
(21%)		 205
(68.33%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)		 30096
(32%)		 32
(10.67%)		 64
(21.33%)		 204
(68%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-mt
(complete)		 30071
(23.67%)		 25
(8.33%)		 46
(15.33%)		 229
(76.33%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-st
(complete)		 30062
(20.67%)		 18
(6%)		 44
(14.67%)		 238
(79.33%)		 0
(0%)
glucose
2
(complete)		 30091
(30.33%)		 31
(10.33%)		 60
(20%)		 209
(69.67%)		 0
(0%)
jMiniSat
2011
(complete)		 30084
(28%)		 23
(7.67%)		 61
(20.33%)		 216
(72%)		 0
(0%)
Lingeling
587 [DEPRECATED]
(complete)		 30089
(29.67%)		 27
(9%)		 62
(20.67%)		 211
(70.33%)		 0
(0%)
Lingeling
587f (fixed)
(complete)		 30089
(29.67%)		 27
(9%)		 62
(20.67%)		 211
(70.33%)		 0
(0%)
march_rw
2011-03-02
(complete)		 30083
(27.67%)		 26
(8.67%)		 57
(19%)		 217
(72.33%)		 0
(0%)
MPhaseSAT
2011-02-15
(complete)		 300107
(35.67%)		 26
(8.67%)		 81
(27%)		 193
(64.33%)		 0
(0%)
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)		 30099
(33%)		 26
(8.67%)		 73
(24.33%)		 201
(67%)		 0
(0%)
MPhaseSAT64
2011-05-14 (fixed)
(complete)		 300107
(35.67%)		 27
(9%)		 80
(26.67%)		 193
(64.33%)		 0
(0%)
MPhaseSAT_M
2011-02-16
(complete)		 300104
(34.67%)		 25
(8.33%)		 79
(26.33%)		 196
(65.33%)		 0
(0%)
PicoSAT
941
(complete)		 30092
(30.67%)		 27
(9%)		 65
(21.67%)		 208
(69.33%)		 0
(0%)
Plingeling
587 [DEPRECATED]
(complete)		 300103
(34.33%)		 29
(9.67%)		 74
(24.67%)		 197
(65.67%)		 0
(0%)
Plingeling
587f (fixed)
(complete)		 300103
(34.33%)		 30
(10%)		 73
(24.33%)		 197
(65.67%)		 0
(0%)
ppfolio
par
(complete)		 300151
(50.33%)		 38
(12.67%)		 113
(37.67%)		 149
(49.67%)		 0
(0%)
ppfolio
seq
(complete)		 300139
(46.33%)		 32
(10.67%)		 107
(35.67%)		 161
(53.67%)		 0
(0%)
priss
1.0
(complete)		 30082
(27.33%)		 20
(6.67%)		 62
(20.67%)		 218
(72.67%)		 0
(0%)
priss
Auto
(complete)		 30081
(27%)		 20
(6.67%)		 61
(20.33%)		 219
(73%)		 0
(0%)
priss
UP
(complete)		 30082
(27.33%)		 22
(7.33%)		 60
(20%)		 218
(72.67%)		 0
(0%)
QuteRSat
2011-03-01 [DEPRECATED]
(complete)		 30090
(30%)		 32
(10.67%)		 58
(19.33%)		 210
(70%)		 0
(0%)
QuteRSat
2011-05-12 (fixed)
(complete)		 30094
(31.33%)		 33
(11%)		 61
(20.33%)		 206
(68.67%)		 0
(0%)
rcl
2011-04-01
(complete)		 30086
(28.67%)		 25
(8.33%)		 61
(20.33%)		 214
(71.33%)		 0
(0%)
RestartSAT
A75
(complete)		 30089
(29.67%)		 26
(8.67%)		 63
(21%)		 211
(70.33%)		 0
(0%)
RestartSAT
B95
(complete)		 30094
(31.33%)		 28
(9.33%)		 66
(22%)		 206
(68.67%)		 0
(0%)
riss
1.0
(complete)		 30077
(25.67%)		 22
(7.33%)		 55
(18.33%)		 223
(74.33%)		 0
(0%)
riss
Auto
(complete)		 30077
(25.67%)		 22
(7.33%)		 55
(18.33%)		 223
(74.33%)		 0
(0%)
riss
Auto (1200 s timeout)
(complete)		 30076
(25.33%)		 22
(7.33%)		 54
(18%)		 224
(74.67%)		 0
(0%)
riss
Exp
(complete)		 30074
(24.67%)		 20
(6.67%)		 54
(18%)		 226
(75.33%)		 0
(0%)
SApperloT
2011-03-23
(complete)		 30088
(29.33%)		 27
(9%)		 61
(20.33%)		 212
(70.67%)		 0
(0%)
SApperloT2010
2011-03-04 [DEPRECATED]
(complete)		 30064
(21.33%)		 27
(9%)		 37
(12.33%)		 210
(70%)		 26
(8.67%)
SApperloT2010
2011-05-15 (fixed)
(complete)		 30093
(31%)		 26
(8.67%)		 67
(22.33%)		 207
(69%)		 0
(0%)
SArTagnan
2011-03-24 [DEPRECATED]
(complete)		 30067
(22.33%)		 27
(9%)		 40
(13.33%)		 214
(71.33%)		 19
(6.33%)
SArTagnan
2011-05-15 (fixed)
(complete)		 30085
(28.33%)		 25
(8.33%)		 60
(20%)		 215
(71.67%)		 0
(0%)
SAT07 reference solver: minisat
SAT 2007
(complete)		 30092
(30.67%)		 32
(10.67%)		 60
(20%)		 208
(69.33%)		 0
(0%)
SAT09 reference solver: clasp
1.2.0-SAT09-32
(complete)		 30093
(31%)		 34
(11.33%)		 59
(19.67%)		 207
(69%)		 0
(0%)
SAT09 reference solver: SATzilla2009_C
2009-03-22
(complete)		 30081
(27%)		 31
(10.33%)		 50
(16.67%)		 219
(73%)		 0
(0%)
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)		 30078
(26%)		 25
(8.33%)		 53
(17.67%)		 222
(74%)		 0
(0%)
sathys
2011-04-01
(complete)		 30088
(29.33%)		 20
(6.67%)		 68
(22.67%)		 212
(70.67%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18
(complete)		 300137
(45.67%)		 29
(9.67%)		 108
(36%)		 163
(54.33%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)		 300131
(43.67%)		 31
(10.33%)		 100
(33.33%)		 169
(56.33%)		 0
(0%)
sattime
2011-03-02
(incomplete)		 300104
(34.67%)		 0
(0%)		 104
(34.67%)		 196
(65.33%)		 0
(0%)
sattime+
2011-03-02
(incomplete)		 300102
(34%)		 0
(0%)		 102
(34%)		 198
(66%)		 0
(0%)
sattime2011
2011-03-02
(incomplete)		 300104
(34.67%)		 0
(0%)		 104
(34.67%)		 196
(65.33%)		 0
(0%)
shortsattime+
2011-03-02
(incomplete)		 300105
(35%)		 0
(0%)		 105
(35%)		 195
(65%)		 0
(0%)
shortsattime2011
2011-03-02
(incomplete)		 300103
(34.33%)		 0
(0%)		 103
(34.33%)		 197
(65.67%)		 0
(0%)
Sol
2011-04-04
(complete)		 300103
(34.33%)		 30
(10%)		 73
(24.33%)		 197
(65.67%)		 0
(0%)
STRUCTure
0.1
(complete)		 30060
(20%)		 17
(5.67%)		 43
(14.33%)		 240
(80%)		 0
(0%)
STRUCTure
0.1 (hbr=2)
(complete)		 30061
(20.33%)		 17
(5.67%)		 44
(14.67%)		 239
(79.67%)		 0
(0%)
synit_sat
2011-03-03
(complete)		 30094
(31.33%)		 32
(10.67%)		 62
(20.67%)		 206
(68.67%)		 0
(0%)
tts
5-4
(complete)		 30037
(12.33%)		 17
(5.67%)		 20
(6.67%)		 263
(87.67%)		 0
(0%)
ttsth
5-4
(complete)		 30046
(15.33%)		 20
(6.67%)		 26
(8.67%)		 254
(84.67%)		 0
(0%)
UmSat
2011-02-25 [DEPRECATED]
(complete)		 30062
(20.67%)		 17
(5.67%)		 45
(15%)		 238
(79.33%)		 0
(0%)
UmSat
2011-05-12 (fixed)
(complete)		 30068
(22.67%)		 19
(6.33%)		 49
(16.33%)		 232
(77.33%)		 0
(0%)

Solver NameProgress
adaptg2wsat2011
2011-03-02
(incomplete)
done
300
SAT
99		?
201
Black Hole SAT
2011-03-08
(complete)
done
300
UNSAT
11		SAT
25		?
264
clasp
2.0-R4092-crafted
(complete)
done
300
UNSAT
37		SAT
68		?
195
claspmt
2.0-R4095 [DEPRECATED]
(complete)
done
300
UNSAT
41		SAT
88		?
170		ERR
1
claspmt
2.0-R4095-patched (fixed)
(complete)
done
300
UNSAT
43		SAT
90		?
167
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)
done
300
UNSAT
38		SAT
73		?
189
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)
done
300
UNSAT
37		SAT
76		?
187
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)
done
300
UNSAT
32		SAT
57		?
211
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)
done
300
UNSAT
32		SAT
57		?
211
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)
done
300
UNSAT
35		SAT
72		?
193
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)
done
300
UNSAT
36		SAT
69		?
195
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)
done
300
UNSAT
32		SAT
63		?
205
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)
done
300
UNSAT
32		SAT
64		?
204
CryptoMiniSat
TechDemo-Alpha-mt
(complete)
done
300
UNSAT
25		SAT
46		?
229
CryptoMiniSat
TechDemo-Alpha-st
(complete)
done
300
UNSAT
18		SAT
44		?
238
glucose
2
(complete)
done
300
UNSAT
31		SAT
60		?
209
jMiniSat
2011
(complete)
done
300
UNSAT
23		SAT
61		?
216
Lingeling
587 [DEPRECATED]
(complete)
done
300
UNSAT
27		SAT
62		?
211
Lingeling
587f (fixed)
(complete)
done
300
UNSAT
27		SAT
62		?
211
march_rw
2011-03-02
(complete)
done
300
UNSAT
26		SAT
57		?
217
MPhaseSAT
2011-02-15
(complete)
done
300
UNSAT
26		SAT
81		?
193
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)
done
300
UNSAT
26		SAT
73		?
201
MPhaseSAT64
2011-05-14 (fixed)
(complete)
done
300
UNSAT
27		SAT
80		?
193
MPhaseSAT_M
2011-02-16
(complete)
done
300
UNSAT
25		SAT
79		?
196
PicoSAT
941
(complete)
done
300
UNSAT
27		SAT
65		?
208
Plingeling
587 [DEPRECATED]
(complete)
done
300
UNSAT
29		SAT
74		?
197
Plingeling
587f (fixed)
(complete)
done
300
UNSAT
30		SAT
73		?
197
ppfolio
par
(complete)
done
300
UNSAT
38		SAT
113		?
149
ppfolio
seq
(complete)
done
300
UNSAT
32		SAT
107		?
161
priss
1.0
(complete)
done
300
UNSAT
20		SAT
62		?
218
priss
Auto
(complete)
done
300
UNSAT
20		SAT
61		?
219
priss
UP
(complete)
done
300
UNSAT
22		SAT
60		?
218
QuteRSat
2011-03-01 [DEPRECATED]
(complete)
done
300
UNSAT
32		SAT
58		?
210
QuteRSat
2011-05-12 (fixed)
(complete)
done
300
UNSAT
33		SAT
61		?
206
rcl
2011-04-01
(complete)
done
300
UNSAT
25		SAT
61		?
214
RestartSAT
A75
(complete)
done
300
UNSAT
26		SAT
63		?
211
RestartSAT
B95
(complete)
done
300
UNSAT
28		SAT
66		?
206
riss
1.0
(complete)
done
300
UNSAT
22		SAT
55		?
223
riss
Auto
(complete)
done
300
UNSAT
22		SAT
55		?
223
riss
Auto (1200 s timeout)
(complete)
done
300
UNSAT
22		SAT
54		?
224
riss
Exp
(complete)
done
300
UNSAT
20		SAT
54		?
226
SApperloT
2011-03-23
(complete)
done
300
UNSAT
27		SAT
61		?
212
SApperloT2010
2011-03-04 [DEPRECATED]
(complete)
done
300
UNSAT
27		SAT
37		?
210		ERR
26
SApperloT2010
2011-05-15 (fixed)
(complete)
done
300
UNSAT
26		SAT
67		?
207
SArTagnan
2011-03-24 [DEPRECATED]
(complete)
done
300
UNSAT
27		SAT
40		?
214		ERR
19
SArTagnan
2011-05-15 (fixed)
(complete)
done
300
UNSAT
25		SAT
60		?
215
SAT07 reference solver: minisat
SAT 2007
(complete)
done
300
UNSAT
32		SAT
60		?
208
SAT09 reference solver: clasp
1.2.0-SAT09-32
(complete)
done
300
UNSAT
34		SAT
59		?
207
SAT09 reference solver: SATzilla2009_C
2009-03-22
(complete)
done
300
UNSAT
31		SAT
50		?
219
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)
done
300
UNSAT
25		SAT
53		?
222
sathys
2011-04-01
(complete)
done
300
UNSAT
20		SAT
68		?
212
Satisfiability Solver Selector
2011-03-18
(complete)
done
300
UNSAT
29		SAT
108		?
163
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)
done
300
UNSAT
31		SAT
100		?
169
sattime
2011-03-02
(incomplete)
done
300
SAT
104		?
196
sattime+
2011-03-02
(incomplete)
done
300
SAT
102		?
198
sattime2011
2011-03-02
(incomplete)
done
300
SAT
104		?
196
shortsattime+
2011-03-02
(incomplete)
done
300
SAT
105		?
195
shortsattime2011
2011-03-02
(incomplete)
done
300
SAT
103		?
197
Sol
2011-04-04
(complete)
done
300
UNSAT
30		SAT
73		?
197
STRUCTure
0.1
(complete)
done
300
UNSAT
17		SAT
43		?
240
STRUCTure
0.1 (hbr=2)
(complete)
done
300
UNSAT
17		SAT
44		?
239
synit_sat
2011-03-03
(complete)
done
300
UNSAT
32		SAT
62		?
206
tts
5-4
(complete)
done
300
UNSAT
17		SAT
20		?
263
ttsth
5-4
(complete)
done
300
UNSAT
20		SAT
26		?
254
UmSat
2011-02-25 [DEPRECATED]
(complete)
done
300
UNSAT
17		SAT
45		?
238
UmSat
2011-05-12 (fixed)
(complete)
done
300
UNSAT
19		SAT
49		?
232

Category "random instances" (RANDOM)

Solver Name#benchsSolvedUNSATSATUNKNOWNBad Answers
adaptg2wsat2011
2011-03-02
(incomplete)		 600300
(50%)		 0
(0%)		 300
(50%)		 300
(50%)		 0
(0%)
Black Hole SAT
2011-03-08
(complete)		 60047
(7.83%)		 29
(4.83%)		 18
(3%)		 553
(92.17%)		 0
(0%)
borg-sat
11.03.02-par-random (fixed)
(complete)		 600408
(68%)		 106
(17.67%)		 302
(50.33%)		 192
(32%)		 0
(0%)
borg-sat
11.03.02-random [DEPRECATED]
(complete)		 600403
(67.17%)		 106
(17.67%)		 297
(49.5%)		 196
(32.67%)		 1
(0.17%)
borg-sat
11.03.02-seq-random (fixed)
(complete)		 600398
(66.33%)		 106
(17.67%)		 292
(48.67%)		 202
(33.67%)		 0
(0%)
borg-sat
11.03.02-seq-random (fixed, 1200 s timeout)
(complete)		 600396
(66%)		 102
(17%)		 294
(49%)		 203
(33.83%)		 1
(0.17%)
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)		 60086
(14.33%)		 49
(8.17%)		 37
(6.17%)		 514
(85.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)		 60086
(14.33%)		 49
(8.17%)		 37
(6.17%)		 514
(85.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)		 60076
(12.67%)		 48
(8%)		 28
(4.67%)		 524
(87.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)		 60075
(12.5%)		 48
(8%)		 27
(4.5%)		 525
(87.5%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)		 60089
(14.83%)		 49
(8.17%)		 40
(6.67%)		 511
(85.17%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)		 60089
(14.83%)		 49
(8.17%)		 40
(6.67%)		 511
(85.17%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)		 60080
(13.33%)		 48
(8%)		 32
(5.33%)		 520
(86.67%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)		 60081
(13.5%)		 48
(8%)		 33
(5.5%)		 519
(86.5%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-mt
(complete)		 60059
(9.83%)		 31
(5.17%)		 28
(4.67%)		 541
(90.17%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-st
(complete)		 60055
(9.17%)		 31
(5.17%)		 24
(4%)		 545
(90.83%)		 0
(0%)
csls
pnorm-8cores
(incomplete)		 600307
(51.17%)		 0
(0%)		 307
(51.17%)		 293
(48.83%)		 0
(0%)
csls
prob-4cores
(incomplete)		 600303
(50.5%)		 0
(0%)		 303
(50.5%)		 297
(49.5%)		 0
(0%)
csls
prob-8cores
(incomplete)		 600301
(50.17%)		 0
(0%)		 301
(50.17%)		 299
(49.83%)		 0
(0%)
EagleUP
1.565.350
(incomplete)		 600315
(52.5%)		 0
(0%)		 315
(52.5%)		 285
(47.5%)		 0
(0%)
jMiniSat
2011
(complete)		 60081
(13.5%)		 43
(7.17%)		 38
(6.33%)		 519
(86.5%)		 0
(0%)
march_rw
2011-03-02
(complete)		 600217
(36.17%)		 108
(18%)		 109
(18.17%)		 383
(63.83%)		 0
(0%)
MPhaseSAT
2011-02-15
(complete)		 600310
(51.67%)		 68
(11.33%)		 242
(40.33%)		 290
(48.33%)		 0
(0%)
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)		 600306
(51%)		 64
(10.67%)		 242
(40.33%)		 294
(49%)		 0
(0%)
MPhaseSAT64
2011-05-14 (fixed)
(complete)		 600312
(52%)		 69
(11.5%)		 243
(40.5%)		 288
(48%)		 0
(0%)
MPhaseSAT_M
2011-02-16
(complete)		 600342
(57%)		 100
(16.67%)		 242
(40.33%)		 258
(43%)		 0
(0%)
MultiFlip
2011-02-11
(incomplete)		 600274
(45.67%)		 0
(0%)		 274
(45.67%)		 326
(54.33%)		 0
(0%)
ppfolio
par
(complete)		 600375
(62.5%)		 102
(17%)		 273
(45.5%)		 225
(37.5%)		 0
(0%)
ppfolio
seq
(complete)		 600323
(53.83%)		 60
(10%)		 263
(43.83%)		 277
(46.17%)		 0
(0%)
QuteRSat
2011-03-01 [DEPRECATED]
(complete)		 60079
(13.17%)		 48
(8%)		 31
(5.17%)		 521
(86.83%)		 0
(0%)
QuteRSat
2011-05-12 (fixed)
(complete)		 60083
(13.83%)		 49
(8.17%)		 34
(5.67%)		 517
(86.17%)		 0
(0%)
RestartSAT
A75
(complete)		 60077
(12.83%)		 39
(6.5%)		 38
(6.33%)		 523
(87.17%)		 0
(0%)
RestartSAT
B95
(complete)		 60083
(13.83%)		 49
(8.17%)		 34
(5.67%)		 517
(86.17%)		 0
(0%)
SAT09 reference solver: gnovelty+2
2009-03-22
(incomplete)		 600276
(46%)		 0
(0%)		 276
(46%)		 324
(54%)		 0
(0%)
SAT09 reference solver: march_hi
hi
(complete)		 600212
(35.33%)		 106
(17.67%)		 106
(17.67%)		 388
(64.67%)		 0
(0%)
SAT09 reference solver: SATzilla2009_R
2009-03-22
(complete)		 600352
(58.67%)		 83
(13.83%)		 269
(44.83%)		 248
(41.33%)		 0
(0%)
SAT09 reference solver: TNM
2009-03-22
(incomplete)		 600294
(49%)		 0
(0%)		 294
(49%)		 306
(51%)		 0
(0%)
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)		 60066
(11%)		 32
(5.33%)		 34
(5.67%)		 534
(89%)		 0
(0%)
sathys
2011-04-01
(complete)		 600183
(30.5%)		 23
(3.83%)		 160
(26.67%)		 417
(69.5%)		 0
(0%)
SAT_Intersec_01
2011-03-01
(complete)		 6000
(0%)		 0
(0%)		 0
(0%)		 600
(100%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18
(complete)		 600380
(63.33%)		 96
(16%)		 284
(47.33%)		 220
(36.67%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)		 600399
(66.5%)		 106
(17.67%)		 293
(48.83%)		 201
(33.5%)		 0
(0%)
sattime
2011-03-02
(incomplete)		 600251
(41.83%)		 0
(0%)		 251
(41.83%)		 349
(58.17%)		 0
(0%)
sattime+
2011-03-02
(incomplete)		 600216
(36%)		 0
(0%)		 216
(36%)		 384
(64%)		 0
(0%)
sattime2011
2011-03-02
(incomplete)		 600309
(51.5%)		 0
(0%)		 309
(51.5%)		 291
(48.5%)		 0
(0%)
shortsattime+
2011-03-02
(incomplete)		 600211
(35.17%)		 0
(0%)		 211
(35.17%)		 389
(64.83%)		 0
(0%)
shortsattime2011
2011-03-02
(incomplete)		 600304
(50.67%)		 0
(0%)		 304
(50.67%)		 296
(49.33%)		 0
(0%)
Sol
2011-04-04
(complete)		 60053
(8.83%)		 30
(5%)		 23
(3.83%)		 547
(91.17%)		 0
(0%)
sparrow2011
sparrow2011_ubcsat1.2_2011-03-02
(incomplete)		 600343
(57.17%)		 0
(0%)		 343
(57.17%)		 257
(42.83%)		 0
(0%)
STRUCTure
0.1
(complete)		 600103
(17.17%)		 60
(10%)		 43
(7.17%)		 497
(82.83%)		 0
(0%)
STRUCTure
0.1 (hbr=2)
(complete)		 600107
(17.83%)		 60
(10%)		 47
(7.83%)		 493
(82.17%)		 0
(0%)
synit_sat
2011-03-03
(complete)		 60076
(12.67%)		 48
(8%)		 28
(4.67%)		 524
(87.33%)		 0
(0%)
UmSat
2011-02-25 [DEPRECATED]
(complete)		 60062
(10.33%)		 36
(6%)		 26
(4.33%)		 538
(89.67%)		 0
(0%)

Solver NameProgress
adaptg2wsat2011
2011-03-02
(incomplete)
done
600
SAT
300		?
300
Black Hole SAT
2011-03-08
(complete)
done
600
UNSAT
29		SAT
18		?
553
borg-sat
11.03.02-par-random (fixed)
(complete)
done
600
UNSAT
106		SAT
302		?
192
borg-sat
11.03.02-random [DEPRECATED]
(complete)
done
600
UNSAT
106		SAT
297		?
196		ERR
1
borg-sat
11.03.02-seq-random (fixed)
(complete)
done
600
UNSAT
106		SAT
292		?
202
borg-sat
11.03.02-seq-random (fixed, 1200 s timeout)
(complete)
done
600
UNSAT
102		SAT
294		?
203		ERR
1
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)
done
600
UNSAT
49		SAT
37		?
514
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)
done
600
UNSAT
49		SAT
37		?
514
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)
done
600
UNSAT
48		SAT
28		?
524
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)
done
600
UNSAT
48		SAT
27		?
525
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)
done
600
UNSAT
49		SAT
40		?
511
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)
done
600
UNSAT
49		SAT
40		?
511
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)
done
600
UNSAT
48		SAT
32		?
520
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)
done
600
UNSAT
48		SAT
33		?
519
CryptoMiniSat
TechDemo-Alpha-mt
(complete)
done
600
UNSAT
31		SAT
28		?
541
CryptoMiniSat
TechDemo-Alpha-st
(complete)
done
600
UNSAT
31		SAT
24		?
545
csls
pnorm-8cores
(incomplete)
done
600
SAT
307		?
293
csls
prob-4cores
(incomplete)
done
600
SAT
303		?
297
csls
prob-8cores
(incomplete)
done
600
SAT
301		?
299
EagleUP
1.565.350
(incomplete)
done
600
SAT
315		?
285
jMiniSat
2011
(complete)
done
600
UNSAT
43		SAT
38		?
519
march_rw
2011-03-02
(complete)
done
600
UNSAT
108		SAT
109		?
383
MPhaseSAT
2011-02-15
(complete)
done
600
UNSAT
68		SAT
242		?
290
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)
done
600
UNSAT
64		SAT
242		?
294
MPhaseSAT64
2011-05-14 (fixed)
(complete)
done
600
UNSAT
69		SAT
243		?
288
MPhaseSAT_M
2011-02-16
(complete)
done
600
UNSAT
100		SAT
242		?
258
MultiFlip
2011-02-11
(incomplete)
done
600
SAT
274		?
326
ppfolio
par
(complete)
done
600
UNSAT
102		SAT
273		?
225
ppfolio
seq
(complete)
done
600
UNSAT
60		SAT
263		?
277
QuteRSat
2011-03-01 [DEPRECATED]
(complete)
done
600
UNSAT
48		SAT
31		?
521
QuteRSat
2011-05-12 (fixed)
(complete)
done
600
UNSAT
49		SAT
34		?
517
RestartSAT
A75
(complete)
done
600
UNSAT
39		SAT
38		?
523
RestartSAT
B95
(complete)
done
600
UNSAT
49		SAT
34		?
517
SAT09 reference solver: gnovelty+2
2009-03-22
(incomplete)
done
600
SAT
276		?
324
SAT09 reference solver: march_hi
hi
(complete)
done
600
UNSAT
106		SAT
106		?
388
SAT09 reference solver: SATzilla2009_R
2009-03-22
(complete)
done
600
UNSAT
83		SAT
269		?
248
SAT09 reference solver: TNM
2009-03-22
(incomplete)
done
600
SAT
294		?
306
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)
done
600
UNSAT
32		SAT
34		?
534
sathys
2011-04-01
(complete)
done
600
UNSAT
23		SAT
160		?
417
SAT_Intersec_01
2011-03-01
(complete)
done
600
?
600
Satisfiability Solver Selector
2011-03-18
(complete)
done
600
UNSAT
96		SAT
284		?
220
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)
done
600
UNSAT
106		SAT
293		?
201
sattime
2011-03-02
(incomplete)
done
600
SAT
251		?
349
sattime+
2011-03-02
(incomplete)
done
600
SAT
216		?
384
sattime2011
2011-03-02
(incomplete)
done
600
SAT
309		?
291
shortsattime+
2011-03-02
(incomplete)
done
600
SAT
211		?
389
shortsattime2011
2011-03-02
(incomplete)
done
600
SAT
304		?
296
Sol
2011-04-04
(complete)
done
600
UNSAT
30		SAT
23		?
547
sparrow2011
sparrow2011_ubcsat1.2_2011-03-02
(incomplete)
done
600
SAT
343		?
257
STRUCTure
0.1
(complete)
done
600
UNSAT
60		SAT
43		?
497
STRUCTure
0.1 (hbr=2)
(complete)
done
600
UNSAT
60		SAT
47		?
493
synit_sat
2011-03-03
(complete)
done
600
UNSAT
48		SAT
28		?
524
UmSat
2011-02-25 [DEPRECATED]
(complete)
done
600
UNSAT
36		SAT
26		?
538

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATSATUNKNOWNBad Answers
adaptg2wsat2011
2011-03-02
(incomplete)		 1200414
(34.5%)		 0
(0%)		 414
(34.5%)		 786
(65.5%)		 0
(0%)
Black Hole SAT
2011-03-08
(complete)		 1200110
(9.17%)		 45
(3.75%)		 65
(5.42%)		 1090
(90.83%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)		 1200378
(31.5%)		 191
(15.92%)		 187
(15.58%)		 822
(68.5%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)		 1200380
(31.67%)		 189
(15.75%)		 191
(15.92%)		 820
(68.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)		 1200328
(27.33%)		 171
(14.25%)		 157
(13.08%)		 872
(72.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)		 1200327
(27.25%)		 171
(14.25%)		 156
(13%)		 873
(72.75%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)		 1200387
(32.25%)		 187
(15.58%)		 200
(16.67%)		 813
(67.75%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)		 1200386
(32.17%)		 190
(15.83%)		 196
(16.33%)		 814
(67.83%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)		 1200348
(29%)		 176
(14.67%)		 172
(14.33%)		 852
(71%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)		 1200351
(29.25%)		 176
(14.67%)		 175
(14.58%)		 849
(70.75%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-mt
(complete)		 1200269
(22.42%)		 128
(10.67%)		 141
(11.75%)		 931
(77.58%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-st
(complete)		 1200253
(21.08%)		 122
(10.17%)		 131
(10.92%)		 945
(78.75%)		 2
(0.17%)
jMiniSat
2011
(complete)		 1200297
(24.75%)		 128
(10.67%)		 169
(14.08%)		 903
(75.25%)		 0
(0%)
march_rw
2011-03-02
(complete)		 1200323
(26.92%)		 142
(11.83%)		 181
(15.08%)		 877
(73.08%)		 0
(0%)
MPhaseSAT
2011-02-15
(complete)		 1200579
(48.25%)		 176
(14.67%)		 403
(33.58%)		 621
(51.75%)		 0
(0%)
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)		 1200566
(47.17%)		 169
(14.08%)		 397
(33.08%)		 634
(52.83%)		 0
(0%)
MPhaseSAT64
2011-05-14 (fixed)
(complete)		 1200589
(49.08%)		 182
(15.17%)		 407
(33.92%)		 611
(50.92%)		 0
(0%)
MPhaseSAT_M
2011-02-16
(complete)		 1200612
(51%)		 207
(17.25%)		 405
(33.75%)		 588
(49%)		 0
(0%)
ppfolio
par
(complete)		 1200710
(59.17%)		 238
(19.83%)		 472
(39.33%)		 490
(40.83%)		 0
(0%)
ppfolio
seq
(complete)		 1200617
(51.42%)		 176
(14.67%)		 441
(36.75%)		 583
(48.58%)		 0
(0%)
QuteRSat
2011-03-01 [DEPRECATED]
(complete)		 1200338
(28.17%)		 177
(14.75%)		 161
(13.42%)		 861
(71.75%)		 1
(0.08%)
QuteRSat
2011-05-12 (fixed)
(complete)		 1200351
(29.25%)		 178
(14.83%)		 173
(14.42%)		 849
(70.75%)		 0
(0%)
RestartSAT
A75
(complete)		 1200337
(28.08%)		 158
(13.17%)		 179
(14.92%)		 863
(71.92%)		 0
(0%)
RestartSAT
B95
(complete)		 1200349
(29.08%)		 166
(13.83%)		 183
(15.25%)		 851
(70.92%)		 0
(0%)
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)		 1200263
(21.92%)		 113
(9.42%)		 150
(12.5%)		 937
(78.08%)		 0
(0%)
sathys
2011-04-01
(complete)		 1200410
(34.17%)		 113
(9.42%)		 297
(24.75%)		 790
(65.83%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18
(complete)		 1200675
(56.25%)		 204
(17%)		 471
(39.25%)		 525
(43.75%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)		 1200701
(58.42%)		 223
(18.58%)		 478
(39.83%)		 499
(41.58%)		 0
(0%)
sattime
2011-03-02
(incomplete)		 1200371
(30.92%)		 0
(0%)		 371
(30.92%)		 829
(69.08%)		 0
(0%)
sattime+
2011-03-02
(incomplete)		 1200337
(28.08%)		 7
(0.58%)		 330
(27.5%)		 863
(71.92%)		 0
(0%)
sattime2011
2011-03-02
(incomplete)		 1200429
(35.75%)		 0
(0%)		 429
(35.75%)		 771
(64.25%)		 0
(0%)
shortsattime+
2011-03-02
(incomplete)		 1200338
(28.17%)		 7
(0.58%)		 331
(27.58%)		 862
(71.83%)		 0
(0%)
shortsattime2011
2011-03-02
(incomplete)		 1200422
(35.17%)		 0
(0%)		 422
(35.17%)		 778
(64.83%)		 0
(0%)
Sol
2011-04-04
(complete)		 1200275
(22.92%)		 121
(10.08%)		 154
(12.83%)		 925
(77.08%)		 0
(0%)
STRUCTure
0.1
(complete)		 1200194
(16.17%)		 91
(7.58%)		 103
(8.58%)		 1006
(83.83%)		 0
(0%)
STRUCTure
0.1 (hbr=2)
(complete)		 1200196
(16.33%)		 91
(7.58%)		 105
(8.75%)		 1004
(83.67%)		 0
(0%)
synit_sat
2011-03-03
(complete)		 1200333
(27.75%)		 176
(14.67%)		 157
(13.08%)		 866
(72.17%)		 1
(0.08%)
UmSat
2011-02-25 [DEPRECATED]
(complete)		 1200165
(13.75%)		 76
(6.33%)		 89
(7.42%)		 1034
(86.17%)		 1
(0.08%)

Solver NameProgress
adaptg2wsat2011
2011-03-02
(incomplete)
done
1200
SAT
414		?
786
Black Hole SAT
2011-03-08
(complete)
done
1200
UNSAT
45		SAT
65		?
1090
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)
done
1200
UNSAT
191		SAT
187		?
822
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)
done
1200
UNSAT
189		SAT
191		?
820
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)
done
1200
UNSAT
171		SAT
157		?
872
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)
done
1200
UNSAT
171		SAT
156		?
873
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)
done
1200
UNSAT
187		SAT
200		?
813
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)
done
1200
UNSAT
190		SAT
196		?
814
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)
done
1200
UNSAT
176		SAT
172		?
852
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)
done
1200
UNSAT
176		SAT
175		?
849
CryptoMiniSat
TechDemo-Alpha-mt
(complete)
done
1200
UNSAT
128		SAT
141		?
931
CryptoMiniSat
TechDemo-Alpha-st
(complete)
done
1200
UNSAT
122		SAT
131		?
945		ERR
2
jMiniSat
2011
(complete)
done
1200
UNSAT
128		SAT
169		?
903
march_rw
2011-03-02
(complete)
done
1200
UNSAT
142		SAT
181		?
877
MPhaseSAT
2011-02-15
(complete)
done
1200
UNSAT
176		SAT
403		?
621
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)
done
1200
UNSAT
169		SAT
397		?
634
MPhaseSAT64
2011-05-14 (fixed)
(complete)
done
1200
UNSAT
182		SAT
407		?
611
MPhaseSAT_M
2011-02-16
(complete)
done
1200
UNSAT
207		SAT
405		?
588
ppfolio
par
(complete)
done
1200
UNSAT
238		SAT
472		?
490
ppfolio
seq
(complete)
done
1200
UNSAT
176		SAT
441		?
583
QuteRSat
2011-03-01 [DEPRECATED]
(complete)
done
1200
UNSAT
177		SAT
161		?
861		ERR
1
QuteRSat
2011-05-12 (fixed)
(complete)
done
1200
UNSAT
178		SAT
173		?
849
RestartSAT
A75
(complete)
done
1200
UNSAT
158		SAT
179		?
863
RestartSAT
B95
(complete)
done
1200
UNSAT
166		SAT
183		?
851
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)
done
1200
UNSAT
113		SAT
150		?
937
sathys
2011-04-01
(complete)
done
1200
UNSAT
113		SAT
297		?
790
Satisfiability Solver Selector
2011-03-18
(complete)
done
1200
UNSAT
204		SAT
471		?
525
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)
done
1200
UNSAT
223		SAT
478		?
499
sattime
2011-03-02
(incomplete)
done
1200
SAT
371		?
829
sattime+
2011-03-02
(incomplete)
done
1200
UNSAT
7		SAT
330		?
863
sattime2011
2011-03-02
(incomplete)
done
1200
SAT
429		?
771
shortsattime+
2011-03-02
(incomplete)
done
1200
UNSAT
7		SAT
331		?
862
shortsattime2011
2011-03-02
(incomplete)
done
1200
SAT
422		?
778
Sol
2011-04-04
(complete)
done
1200
UNSAT
121		SAT
154		?
925
STRUCTure
0.1
(complete)
done
1200
UNSAT
91		SAT
103		?
1006
STRUCTure
0.1 (hbr=2)
(complete)
done
1200
UNSAT
91		SAT
105		?
1004
synit_sat
2011-03-03
(complete)
done
1200
UNSAT
176		SAT
157		?
866		ERR
1
UmSat
2011-02-25 [DEPRECATED]
(complete)
done
1200
UNSAT
76		SAT
89		?
1034		ERR
1

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATSATUNKNOWNBad Answers
adaptg2wsat2011
2011-03-02
(incomplete)		 1200414
(34.5%)		 0
(0%)		 414
(34.5%)		 786
(65.5%)		 0
(0%)
Black Hole SAT
2011-03-08
(complete)		 1200110
(9.17%)		 45
(3.75%)		 65
(5.42%)		 1090
(90.83%)		 0
(0%)
borg-sat
11.03.02-industrial [DEPRECATED]
(complete)		 300175
(58.33%)		 89
(29.67%)		 86
(28.67%)		 125
(41.67%)		 0
(0%)
borg-sat
11.03.02-par-industrial (fixed)
(complete)		 300179
(59.67%)		 94
(31.33%)		 85
(28.33%)		 120
(40%)		 1
(0.33%)
borg-sat
11.03.02-par-random (fixed)
(complete)		 600408
(68%)		 106
(17.67%)		 302
(50.33%)		 192
(32%)		 0
(0%)
borg-sat
11.03.02-random [DEPRECATED]
(complete)		 600403
(67.17%)		 106
(17.67%)		 297
(49.5%)		 196
(32.67%)		 1
(0.17%)
borg-sat
11.03.02-seq-industrial (fixed)
(complete)		 300160
(53.33%)		 84
(28%)		 76
(25.33%)		 140
(46.67%)		 0
(0%)
borg-sat
11.03.02-seq-industrial (fixed, 1200 s timeout)
(complete)		 300160
(53.33%)		 85
(28.33%)		 75
(25%)		 140
(46.67%)		 0
(0%)
borg-sat
11.03.02-seq-random (fixed)
(complete)		 600398
(66.33%)		 106
(17.67%)		 292
(48.67%)		 202
(33.67%)		 0
(0%)
borg-sat
11.03.02-seq-random (fixed, 1200 s timeout)
(complete)		 600396
(66%)		 102
(17%)		 294
(49%)		 203
(33.83%)		 1
(0.17%)
clasp
2.0-R4092-crafted
(complete)		 300105
(35%)		 37
(12.33%)		 68
(22.67%)		 195
(65%)		 0
(0%)
clasp
2.0-R4092-indust
(complete)		 300154
(51.33%)		 81
(27%)		 73
(24.33%)		 146
(48.67%)		 0
(0%)
claspmt
2.0-R4095 [DEPRECATED]
(complete)		 600307
(51.17%)		 130
(21.67%)		 177
(29.5%)		 286
(47.67%)		 7
(1.17%)
claspmt
2.0-R4095-patched (fixed)
(complete)		 600314
(52.33%)		 135
(22.5%)		 179
(29.83%)		 286
(47.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)		 1200378
(31.5%)		 191
(15.92%)		 187
(15.58%)		 822
(68.5%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)		 1200380
(31.67%)		 189
(15.75%)		 191
(15.92%)		 820
(68.33%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)		 1200328
(27.33%)		 171
(14.25%)		 157
(13.08%)		 872
(72.67%)		 0
(0%)
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)		 1200327
(27.25%)		 171
(14.25%)		 156
(13%)		 873
(72.75%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)		 1200387
(32.25%)		 187
(15.58%)		 200
(16.67%)		 813
(67.75%)		 0
(0%)
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)		 1200386
(32.17%)		 190
(15.83%)		 196
(16.33%)		 814
(67.83%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)		 1200348
(29%)		 176
(14.67%)		 172
(14.33%)		 852
(71%)		 0
(0%)
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)		 1200351
(29.25%)		 176
(14.67%)		 175
(14.58%)		 849
(70.75%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-mt
(complete)		 1200269
(22.42%)		 128
(10.67%)		 141
(11.75%)		 931
(77.58%)		 0
(0%)
CryptoMiniSat
TechDemo-Alpha-st
(complete)		 1200253
(21.08%)		 122
(10.17%)		 131
(10.92%)		 945
(78.75%)		 2
(0.17%)
csls
pnorm-8cores
(incomplete)		 600307
(51.17%)		 0
(0%)		 307
(51.17%)		 293
(48.83%)		 0
(0%)
csls
prob-4cores
(incomplete)		 600303
(50.5%)		 0
(0%)		 303
(50.5%)		 297
(49.5%)		 0
(0%)
csls
prob-8cores
(incomplete)		 600301
(50.17%)		 0
(0%)		 301
(50.17%)		 299
(49.83%)		 0
(0%)
EagleUP
1.565.350
(incomplete)		 600315
(52.5%)		 0
(0%)		 315
(52.5%)		 285
(47.5%)		 0
(0%)
EBGlucose
1.0
(incomplete)		 300158
(52.67%)		 82
(27.33%)		 76
(25.33%)		 142
(47.33%)		 0
(0%)
glucose
2
(complete)		 600268
(44.67%)		 125
(20.83%)		 143
(23.83%)		 332
(55.33%)		 0
(0%)
glueminisat
2.2.5
(complete)		 300171
(57%)		 98
(32.67%)		 73
(24.33%)		 129
(43%)		 0
(0%)
jMiniSat
2011
(complete)		 1200297
(24.75%)		 128
(10.67%)		 169
(14.08%)		 903
(75.25%)		 0
(0%)
kw
2011-02-28
(complete)		 300156
(52%)		 83
(27.67%)		 73
(24.33%)		 144
(48%)		 0
(0%)
Lingeling
587 [DEPRECATED]
(complete)		 600261
(43.5%)		 120
(20%)		 141
(23.5%)		 339
(56.5%)		 0
(0%)
Lingeling
587f (fixed)
(complete)		 600261
(43.5%)		 120
(20%)		 141
(23.5%)		 339
(56.5%)		 0
(0%)
march_rw
2011-03-02
(complete)		 1200323
(26.92%)		 142
(11.83%)		 181
(15.08%)		 877
(73.08%)		 0
(0%)
minisat hack: cir_minisat
2011-03-02
(complete)		 300159
(53%)		 74
(24.67%)		 85
(28.33%)		 141
(47%)		 0
(0%)
minisat hack: cir_minisat
2011-05-13 (simp)
(complete)		 300164
(54.67%)		 82
(27.33%)		 82
(27.33%)		 136
(45.33%)		 0
(0%)
minisat hack: contrasat
2011-03-02
(complete)		 300161
(53.67%)		 72
(24%)		 89
(29.67%)		 139
(46.33%)		 0
(0%)
minisat hack: EBMiniSAT
2011-03-02
(incomplete)		 300162
(54%)		 78
(26%)		 84
(28%)		 138
(46%)		 0
(0%)
minisat hack: LR_GL_NOSHR
2011-03-02
(complete)		 300155
(51.67%)		 71
(23.67%)		 84
(28%)		 145
(48.33%)		 0
(0%)
minisat hack: LR_GL_SHR
2011-03-02
(complete)		 300158
(52.67%)		 90
(30%)		 68
(22.67%)		 142
(47.33%)		 0
(0%)
minisat hack: LR_NOGL_NOSHR
2011-03-02
(complete)		 300154
(51.33%)		 71
(23.67%)		 83
(27.67%)		 146
(48.67%)		 0
(0%)
minisat hack: MiniSAT
2.2.0-agile-26
(complete)		 300160
(53.33%)		 74
(24.67%)		 86
(28.67%)		 140
(46.67%)		 0
(0%)
minisat hack: minisat_psm
2011-04-01
(complete)		 300163
(54.33%)		 77
(25.67%)		 86
(28.67%)		 137
(45.67%)		 0
(0%)
minisat hack reference: minisat
2.2.0
(complete)		 300161
(53.67%)		 74
(24.67%)		 87
(29%)		 139
(46.33%)		 0
(0%)
MPhaseSAT
2011-02-15
(complete)		 1200579
(48.25%)		 176
(14.67%)		 403
(33.58%)		 621
(51.75%)		 0
(0%)
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)		 1200566
(47.17%)		 169
(14.08%)		 397
(33.08%)		 634
(52.83%)		 0
(0%)
MPhaseSAT64
2011-05-14 (fixed)
(complete)		 1200589
(49.08%)		 182
(15.17%)		 407
(33.92%)		 611
(50.92%)		 0
(0%)
MPhaseSAT_M
2011-02-16
(complete)		 1200612
(51%)		 207
(17.25%)		 405
(33.75%)		 588
(49%)		 0
(0%)
MultiFlip
2011-02-11
(incomplete)		 600274
(45.67%)		 0
(0%)		 274
(45.67%)		 326
(54.33%)		 0
(0%)
PicoSAT
941
(complete)		 600233
(38.83%)		 94
(15.67%)		 139
(23.17%)		 367
(61.17%)		 0
(0%)
Plingeling
587 [DEPRECATED]
(complete)		 600293
(48.83%)		 124
(20.67%)		 169
(28.17%)		 307
(51.17%)		 0
(0%)
Plingeling
587f (fixed)
(complete)		 600295
(49.17%)		 126
(21%)		 169
(28.17%)		 305
(50.83%)		 0
(0%)
ppfolio
par
(complete)		 1200710
(59.17%)		 238
(19.83%)		 472
(39.33%)		 490
(40.83%)		 0
(0%)
ppfolio
seq
(complete)		 1200617
(51.42%)		 176
(14.67%)		 441
(36.75%)		 583
(48.58%)		 0
(0%)
priss
1.0
(complete)		 600229
(38.17%)		 97
(16.17%)		 132
(22%)		 371
(61.83%)		 0
(0%)
priss
Auto
(complete)		 600233
(38.83%)		 98
(16.33%)		 135
(22.5%)		 367
(61.17%)		 0
(0%)
priss
UP
(complete)		 600226
(37.67%)		 95
(15.83%)		 131
(21.83%)		 374
(62.33%)		 0
(0%)
QuteRSat
2011-03-01 [DEPRECATED]
(complete)		 1200338
(28.17%)		 177
(14.75%)		 161
(13.42%)		 861
(71.75%)		 1
(0.08%)
QuteRSat
2011-05-12 (fixed)
(complete)		 1200351
(29.25%)		 178
(14.83%)		 173
(14.42%)		 849
(70.75%)		 0
(0%)
rcl
2011-04-01
(complete)		 600243
(40.5%)		 99
(16.5%)		 144
(24%)		 357
(59.5%)		 0
(0%)
RestartSAT
A75
(complete)		 1200337
(28.08%)		 158
(13.17%)		 179
(14.92%)		 863
(71.92%)		 0
(0%)
RestartSAT
B95
(complete)		 1200349
(29.08%)		 166
(13.83%)		 183
(15.25%)		 851
(70.92%)		 0
(0%)
riss
1.0
(complete)		 600222
(37%)		 97
(16.17%)		 125
(20.83%)		 378
(63%)		 0
(0%)
riss
Auto
(complete)		 600222
(37%)		 97
(16.17%)		 125
(20.83%)		 378
(63%)		 0
(0%)
riss
Auto (1200 s timeout)
(complete)		 600214
(35.67%)		 95
(15.83%)		 119
(19.83%)		 386
(64.33%)		 0
(0%)
riss
Exp
(complete)		 600212
(35.33%)		 89
(14.83%)		 123
(20.5%)		 388
(64.67%)		 0
(0%)
SApperloT
2011-03-23
(complete)		 600240
(40%)		 104
(17.33%)		 136
(22.67%)		 360
(60%)		 0
(0%)
SApperloT2010
2011-03-04 [DEPRECATED]
(complete)		 600208
(34.67%)		 108
(18%)		 100
(16.67%)		 353
(58.83%)		 39
(6.5%)
SApperloT2010
2011-05-15 (fixed)
(complete)		 600250
(41.67%)		 108
(18%)		 142
(23.67%)		 350
(58.33%)		 0
(0%)
SArTagnan
2011-03-24 [DEPRECATED]
(complete)		 600202
(33.67%)		 107
(17.83%)		 95
(15.83%)		 357
(59.5%)		 41
(6.83%)
SArTagnan
2011-05-15 (fixed)
(complete)		 600239
(39.83%)		 104
(17.33%)		 135
(22.5%)		 361
(60.17%)		 0
(0%)
SAT07 reference solver: minisat
SAT 2007
(complete)		 30092
(30.67%)		 32
(10.67%)		 60
(20%)		 208
(69.33%)		 0
(0%)
SAT09 reference solver: clasp
1.2.0-SAT09-32
(complete)		 30093
(31%)		 34
(11.33%)		 59
(19.67%)		 207
(69%)		 0
(0%)
SAT09 reference solver:glucose
1.0
(complete)		 300159
(53%)		 84
(28%)		 75
(25%)		 141
(47%)		 0
(0%)
SAT09 reference solver: gnovelty+2
2009-03-22
(incomplete)		 600276
(46%)		 0
(0%)		 276
(46%)		 324
(54%)		 0
(0%)
SAT09 reference solver:LySAT
i/2009-03-20
(complete)		 300131
(43.67%)		 71
(23.67%)		 60
(20%)		 169
(56.33%)		 0
(0%)
SAT09 reference solver: ManySAT 1.1
aimd 1/2009-03-20
(complete)		 300169
(56.33%)		 91
(30.33%)		 78
(26%)		 131
(43.67%)		 0
(0%)
SAT09 reference solver: march_hi
hi
(complete)		 600212
(35.33%)		 106
(17.67%)		 106
(17.67%)		 388
(64.67%)		 0
(0%)
SAT09 reference solver:precosat
236
(complete)		 300167
(55.67%)		 88
(29.33%)		 79
(26.33%)		 133
(44.33%)		 0
(0%)
SAT09 reference solver: SATzilla2009_C
2009-03-22
(complete)		 30081
(27%)		 31
(10.33%)		 50
(16.67%)		 219
(73%)		 0
(0%)
SAT09 reference solver: SATzilla2009_R
2009-03-22
(complete)		 600352
(58.67%)		 83
(13.83%)		 269
(44.83%)		 248
(41.33%)		 0
(0%)
SAT09 reference solver: TNM
2009-03-22
(incomplete)		 600294
(49%)		 0
(0%)		 294
(49%)		 306
(51%)		 0
(0%)
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)		 1200263
(21.92%)		 113
(9.42%)		 150
(12.5%)		 937
(78.08%)		 0
(0%)
sathys
2011-04-01
(complete)		 1200410
(34.17%)		 113
(9.42%)		 297
(24.75%)		 790
(65.83%)		 0
(0%)
SAT_Intersec_01
2011-03-01
(complete)		 6000
(0%)		 0
(0%)		 0
(0%)		 600
(100%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18
(complete)		 1200675
(56.25%)		 204
(17%)		 471
(39.25%)		 525
(43.75%)		 0
(0%)
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)		 1200701
(58.42%)		 223
(18.58%)		 478
(39.83%)		 499
(41.58%)		 0
(0%)
sattime
2011-03-02
(incomplete)		 1200371
(30.92%)		 0
(0%)		 371
(30.92%)		 829
(69.08%)		 0
(0%)
sattime+
2011-03-02
(incomplete)		 1200337
(28.08%)		 7
(0.58%)		 330
(27.5%)		 863
(71.92%)		 0
(0%)
sattime2011
2011-03-02
(incomplete)		 1200429
(35.75%)		 0
(0%)		 429
(35.75%)		 771
(64.25%)		 0
(0%)
shortsattime+
2011-03-02
(incomplete)		 1200338
(28.17%)		 7
(0.58%)		 331
(27.58%)		 862
(71.83%)		 0
(0%)
shortsattime2011
2011-03-02
(incomplete)		 1200422
(35.17%)		 0
(0%)		 422
(35.17%)		 778
(64.83%)		 0
(0%)
Sol
2011-04-04
(complete)		 1200275
(22.92%)		 121
(10.08%)		 154
(12.83%)		 925
(77.08%)		 0
(0%)
sparrow2011
sparrow2011_ubcsat1.2_2011-03-02
(incomplete)		 600343
(57.17%)		 0
(0%)		 343
(57.17%)		 257
(42.83%)		 0
(0%)
STRUCTure
0.1
(complete)		 1200194
(16.17%)		 91
(7.58%)		 103
(8.58%)		 1006
(83.83%)		 0
(0%)
STRUCTure
0.1 (hbr=2)
(complete)		 1200196
(16.33%)		 91
(7.58%)		 105
(8.75%)		 1004
(83.67%)		 0
(0%)
synit_sat
2011-03-03
(complete)		 1200333
(27.75%)		 176
(14.67%)		 157
(13.08%)		 866
(72.17%)		 1
(0.08%)
tts
5-4
(complete)		 30037
(12.33%)		 17
(5.67%)		 20
(6.67%)		 263
(87.67%)		 0
(0%)
ttsth
5-4
(complete)		 30046
(15.33%)		 20
(6.67%)		 26
(8.67%)		 254
(84.67%)		 0
(0%)
UmSat
2011-02-25 [DEPRECATED]
(complete)		 1200165
(13.75%)		 76
(6.33%)		 89
(7.42%)		 1034
(86.17%)		 1
(0.08%)
UmSat
2011-05-12 (fixed)
(complete)		 600159
(26.5%)		 63
(10.5%)		 96
(16%)		 441
(73.5%)		 0
(0%)

Solver NameProgress
adaptg2wsat2011
2011-03-02
(incomplete)
done
1200
SAT
414		?
786
Black Hole SAT
2011-03-08
(complete)
done
1200
UNSAT
45		SAT
65		?
1090
borg-sat
11.03.02-industrial [DEPRECATED]
(complete)
done
300
UNSAT
89		SAT
86		?
125
borg-sat
11.03.02-par-industrial (fixed)
(complete)
done
300
UNSAT
94		SAT
85		?
120		ERR
1
borg-sat
11.03.02-par-random (fixed)
(complete)
done
600
UNSAT
106		SAT
302		?
192
borg-sat
11.03.02-random [DEPRECATED]
(complete)
done
600
UNSAT
106		SAT
297		?
196		ERR
1
borg-sat
11.03.02-seq-industrial (fixed)
(complete)
done
300
UNSAT
84		SAT
76		?
140
borg-sat
11.03.02-seq-industrial (fixed, 1200 s timeout)
(complete)
done
300
UNSAT
85		SAT
75		?
140
borg-sat
11.03.02-seq-random (fixed)
(complete)
done
600
UNSAT
106		SAT
292		?
202
borg-sat
11.03.02-seq-random (fixed, 1200 s timeout)
(complete)
done
600
UNSAT
102		SAT
294		?
203		ERR
1
clasp
2.0-R4092-crafted
(complete)
done
300
UNSAT
37		SAT
68		?
195
clasp
2.0-R4092-indust
(complete)
done
300
UNSAT
81		SAT
73		?
146
claspmt
2.0-R4095 [DEPRECATED]
(complete)
done
600
UNSAT
130		SAT
177		?
286		ERR
7
claspmt
2.0-R4095-patched (fixed)
(complete)
done
600
UNSAT
135		SAT
179		?
286
CryptoMiniSat
Strange-Night-1-mt [DEPRECATED]
(complete)
done
1200
UNSAT
191		SAT
187		?
822
CryptoMiniSat
Strange-Night-1-mt (fixed)
(complete)
done
1200
UNSAT
189		SAT
191		?
820
CryptoMiniSat
Strange-Night-1-st [DEPRECATED]
(complete)
done
1200
UNSAT
171		SAT
157		?
872
CryptoMiniSat
Strange-Night-1-st (fixed)
(complete)
done
1200
UNSAT
171		SAT
156		?
873
CryptoMiniSat
Strange-Night-2-mt [DEPRECATED]
(complete)
done
1200
UNSAT
187		SAT
200		?
813
CryptoMiniSat
Strange-Night-2-mt (fixed)
(complete)
done
1200
UNSAT
190		SAT
196		?
814
CryptoMiniSat
Strange-Night2-st [DEPRECATED]
(complete)
done
1200
UNSAT
176		SAT
172		?
852
CryptoMiniSat
Strange-Night2-st (fixed)
(complete)
done
1200
UNSAT
176		SAT
175		?
849
CryptoMiniSat
TechDemo-Alpha-mt
(complete)
done
1200
UNSAT
128		SAT
141		?
931
CryptoMiniSat
TechDemo-Alpha-st
(complete)
done
1200
UNSAT
122		SAT
131		?
945		ERR
2
csls
pnorm-8cores
(incomplete)
done
600
SAT
307		?
293
csls
prob-4cores
(incomplete)
done
600
SAT
303		?
297
csls
prob-8cores
(incomplete)
done
600
SAT
301		?
299
EagleUP
1.565.350
(incomplete)
done
600
SAT
315		?
285
EBGlucose
1.0
(incomplete)
done
300
UNSAT
82		SAT
76		?
142
glucose
2
(complete)
done
600
UNSAT
125		SAT
143		?
332
glueminisat
2.2.5
(complete)
done
300
UNSAT
98		SAT
73		?
129
jMiniSat
2011
(complete)
done
1200
UNSAT
128		SAT
169		?
903
kw
2011-02-28
(complete)
done
300
UNSAT
83		SAT
73		?
144
Lingeling
587 [DEPRECATED]
(complete)
done
600
UNSAT
120		SAT
141		?
339
Lingeling
587f (fixed)
(complete)
done
600
UNSAT
120		SAT
141		?
339
march_rw
2011-03-02
(complete)
done
1200
UNSAT
142		SAT
181		?
877
minisat hack: cir_minisat
2011-03-02
(complete)
done
300
UNSAT
74		SAT
85		?
141
minisat hack: cir_minisat
2011-05-13 (simp)
(complete)
done
300
UNSAT
82		SAT
82		?
136
minisat hack: contrasat
2011-03-02
(complete)
done
300
UNSAT
72		SAT
89		?
139
minisat hack: EBMiniSAT
2011-03-02
(incomplete)
done
300
UNSAT
78		SAT
84		?
138
minisat hack: LR_GL_NOSHR
2011-03-02
(complete)
done
300
UNSAT
71		SAT
84		?
145
minisat hack: LR_GL_SHR
2011-03-02
(complete)
done
300
UNSAT
90		SAT
68		?
142
minisat hack: LR_NOGL_NOSHR
2011-03-02
(complete)
done
300
UNSAT
71		SAT
83		?
146
minisat hack: MiniSAT
2.2.0-agile-26
(complete)
done
300
UNSAT
74		SAT
86		?
140
minisat hack: minisat_psm
2011-04-01
(complete)
done
300
UNSAT
77		SAT
86		?
137
minisat hack reference: minisat
2.2.0
(complete)
done
300
UNSAT
74		SAT
87		?
139
MPhaseSAT
2011-02-15
(complete)
done
1200
UNSAT
176		SAT
403		?
621
MPhaseSAT64
2011-03-03 [DEPRECATED]
(complete)
done
1200
UNSAT
169		SAT
397		?
634
MPhaseSAT64
2011-05-14 (fixed)
(complete)
done
1200
UNSAT
182		SAT
407		?
611
MPhaseSAT_M
2011-02-16
(complete)
done
1200
UNSAT
207		SAT
405		?
588
MultiFlip
2011-02-11
(incomplete)
done
600
SAT
274		?
326
PicoSAT
941
(complete)
done
600
UNSAT
94		SAT
139		?
367
Plingeling
587 [DEPRECATED]
(complete)
done
600
UNSAT
124		SAT
169		?
307
Plingeling
587f (fixed)
(complete)
done
600
UNSAT
126		SAT
169		?
305
ppfolio
par
(complete)
done
1200
UNSAT
238		SAT
472		?
490
ppfolio
seq
(complete)
done
1200
UNSAT
176		SAT
441		?
583
priss
1.0
(complete)
done
600
UNSAT
97		SAT
132		?
371
priss
Auto
(complete)
done
600
UNSAT
98		SAT
135		?
367
priss
UP
(complete)
done
600
UNSAT
95		SAT
131		?
374
QuteRSat
2011-03-01 [DEPRECATED]
(complete)
done
1200
UNSAT
177		SAT
161		?
861		ERR
1
QuteRSat
2011-05-12 (fixed)
(complete)
done
1200
UNSAT
178		SAT
173		?
849
rcl
2011-04-01
(complete)
done
600
UNSAT
99		SAT
144		?
357
RestartSAT
A75
(complete)
done
1200
UNSAT
158		SAT
179		?
863
RestartSAT
B95
(complete)
done
1200
UNSAT
166		SAT
183		?
851
riss
1.0
(complete)
done
600
UNSAT
97		SAT
125		?
378
riss
Auto
(complete)
done
600
UNSAT
97		SAT
125		?
378
riss
Auto (1200 s timeout)
(complete)
done
600
UNSAT
95		SAT
119		?
386
riss
Exp
(complete)
done
600
UNSAT
89		SAT
123		?
388
SApperloT
2011-03-23
(complete)
done
600
UNSAT
104		SAT
136		?
360
SApperloT2010
2011-03-04 [DEPRECATED]
(complete)
done
600
UNSAT
108		SAT
100		?
353		ERR
39
SApperloT2010
2011-05-15 (fixed)
(complete)
done
600
UNSAT
108		SAT
142		?
350
SArTagnan
2011-03-24 [DEPRECATED]
(complete)
done
600
UNSAT
107		SAT
95		?
357		ERR
41
SArTagnan
2011-05-15 (fixed)
(complete)
done
600
UNSAT
104		SAT
135		?
361
SAT07 reference solver: minisat
SAT 2007
(complete)
done
300
UNSAT
32		SAT
60		?
208
SAT09 reference solver: clasp
1.2.0-SAT09-32
(complete)
done
300
UNSAT
34		SAT
59		?
207
SAT09 reference solver:glucose
1.0
(complete)
done
300
UNSAT
84		SAT
75		?
141
SAT09 reference solver: gnovelty+2
2009-03-22
(incomplete)
done
600
SAT
276		?
324
SAT09 reference solver:LySAT
i/2009-03-20
(complete)
done
300
UNSAT
71		SAT
60		?
169
SAT09 reference solver: ManySAT 1.1
aimd 1/2009-03-20
(complete)
done
300
UNSAT
91		SAT
78		?
131
SAT09 reference solver: march_hi
hi
(complete)
done
600
UNSAT
106		SAT
106		?
388
SAT09 reference solver:precosat
236
(complete)
done
300
UNSAT
88		SAT
79		?
133
SAT09 reference solver: SATzilla2009_C
2009-03-22
(complete)
done
300
UNSAT
31		SAT
50		?
219
SAT09 reference solver: SATzilla2009_R
2009-03-22
(complete)
done
600
UNSAT
83		SAT
269		?
248
SAT09 reference solver: TNM
2009-03-22
(incomplete)
done
600
SAT
294		?
306
SAT4J core
2.3.0 SNAPSHOT 2011-02-06
(complete)
done
1200
UNSAT
113		SAT
150		?
937
sathys
2011-04-01
(complete)
done
1200
UNSAT
113		SAT
297		?
790
SAT_Intersec_01
2011-03-01
(complete)
done
600
?
600
Satisfiability Solver Selector
2011-03-18
(complete)
done
1200
UNSAT
204		SAT
471		?
525
Satisfiability Solver Selector
2011-03-18 (1200 s timeout)
(complete)
done
1200
UNSAT
223		SAT
478		?
499
sattime
2011-03-02
(incomplete)
done
1200
SAT
371		?
829
sattime+
2011-03-02
(incomplete)
done
1200
UNSAT
7		SAT
330		?
863
sattime2011
2011-03-02
(incomplete)
done
1200
SAT
429		?
771
shortsattime+
2011-03-02
(incomplete)
done
1200
UNSAT
7		SAT
331		?
862
shortsattime2011
2011-03-02
(incomplete)
done
1200
SAT
422		?
778
Sol
2011-04-04
(complete)
done
1200
UNSAT
121		SAT
154		?
925
sparrow2011
sparrow2011_ubcsat1.2_2011-03-02
(incomplete)
done
600
SAT
343		?
257
STRUCTure
0.1
(complete)
done
1200
UNSAT
91		SAT
103		?
1006
STRUCTure
0.1 (hbr=2)
(complete)
done
1200
UNSAT
91		SAT
105		?
1004
synit_sat
2011-03-03
(complete)
done
1200
UNSAT
176		SAT
157		?
866		ERR
1
tts
5-4
(complete)
done
300
UNSAT
17		SAT
20		?
263
ttsth
5-4
(complete)
done
300
UNSAT
20		SAT
26		?
254
UmSat
2011-02-25 [DEPRECATED]
(complete)
done
1200
UNSAT
76		SAT
89		?
1034		ERR
1
UmSat
2011-05-12 (fixed)
(complete)
done
600
UNSAT
63		SAT
96		?
441