SAT 2007 Competition Phase 2: solvers page (tabular format, summary)

Category "crafted instances" (CRAFTED)

Solver Name#benchsSolvedUNSATSATUNKNOWNBad AnswersMisc.
CMUSAT
2007-02-08		 20166
(32.84%)		 45
(22.39%)		 21
(10.45%)		 135
(67.16%)		 0
(0%)		 0
March KS
2007-02-08		 20147
(23.38%)		 29
(14.43%)		 18
(8.96%)		 154
(76.62%)		 0
(0%)		 0
minisat
SAT 2007		 20198
(48.76%)		 72
(35.82%)		 26
(12.94%)		 103
(51.24%)		 0
(0%)		 0
MiraXT
v3		 20178
(38.81%)		 54
(26.87%)		 24
(11.94%)		 123
(61.19%)		 0
(0%)		 0
MXC
2007-02-08		 20177
(38.31%)		 57
(28.36%)		 20
(9.95%)		 124
(61.69%)		 0
(0%)		 0
picosat
535		 20160
(29.85%)		 38
(18.91%)		 22
(10.95%)		 141
(70.15%)		 0
(0%)		 0
Rsat
2007-02-08		 20155
(27.36%)		 40
(19.9%)		 15
(7.46%)		 146
(72.64%)		 0
(0%)		 0
SATzilla
CRAFTED		 20194
(46.77%)		 67
(33.33%)		 27
(13.43%)		 107
(53.23%)		 0
(0%)		 0
TTS
4.0		 20142
(20.9%)		 39
(19.4%)		 3
(1.49%)		 159
(79.1%)		 0
(0%)		 0

Solver NameProgress
CMUSAT
2007-02-08
done
201
UNSAT
45		SAT
21		?
135
March KS
2007-02-08
done
201
UNSAT
29		SAT
18		?
154
minisat
SAT 2007
done
201
UNSAT
72		SAT
26		?
103
MiraXT
v3
done
201
UNSAT
54		SAT
24		?
123
MXC
2007-02-08
done
201
UNSAT
57		SAT
20		?
124
picosat
535
done
201
UNSAT
38		SAT
22		?
141
Rsat
2007-02-08
done
201
UNSAT
40		SAT
15		?
146
SATzilla
CRAFTED
done
201
UNSAT
67		SAT
27		?
107
TTS
4.0
done
201
UNSAT
39		SAT
3		?
159

Category "industrial instances" (INDUST)

Solver Name#benchsSolvedUNSATSATUNKNOWNBad AnswersMisc.
CMUSAT
2007-02-08		 234123
(52.56%)		 77
(32.91%)		 46
(19.66%)		 111
(47.44%)		 0
(0%)		 0
minisat
SAT 2007		 234132
(56.41%)		 79
(33.76%)		 53
(22.65%)		 102
(43.59%)		 0
(0%)		 0
MiraXT
v3		 234126
(53.85%)		 74
(31.62%)		 52
(22.22%)		 108
(46.15%)		 0
(0%)		 0
MXC
2007-02-08		 234116
(49.57%)		 69
(29.49%)		 47
(20.09%)		 118
(50.43%)		 0
(0%)		 0
picosat
535		 234139
(59.4%)		 67
(28.63%)		 72
(30.77%)		 95
(40.6%)		 0
(0%)		 0
Rsat
2007-02-08		 234139
(59.4%)		 76
(32.48%)		 63
(26.92%)		 95
(40.6%)		 0
(0%)		 0
SAT7
2007-02-08		 234114
(48.72%)		 63
(26.92%)		 51
(21.79%)		 120
(51.28%)		 0
(0%)		 0
SATzilla
CRAFTED		 234114
(48.72%)		 67
(28.63%)		 47
(20.09%)		 117
(50%)		 3
(1.28%)		 0
Siege
V4		 234122
(52.14%)		 72
(30.77%)		 50
(21.37%)		 112
(47.86%)		 0
(0%)		 0
TiniSatELite
2007-02-08		 234130
(55.56%)		 75
(32.05%)		 55
(23.5%)		 104
(44.44%)		 0
(0%)		 0

Solver NameProgress
CMUSAT
2007-02-08
done
234
UNSAT
77		SAT
46		?
111
minisat
SAT 2007
done
234
UNSAT
79		SAT
53		?
102
MiraXT
v3
done
234
UNSAT
74		SAT
52		?
108
MXC
2007-02-08
done
234
UNSAT
69		SAT
47		?
118
picosat
535
done
234
UNSAT
67		SAT
72		?
95
Rsat
2007-02-08
done
234
UNSAT
76		SAT
63		?
95
SAT7
2007-02-08
done
234
UNSAT
63		SAT
51		?
120
SATzilla
CRAFTED
done
234
UNSAT
67		SAT
47		?
117		ERR
3
Siege
V4
done
234
UNSAT
72		SAT
50		?
112
TiniSatELite
2007-02-08
done
234
UNSAT
75		SAT
55		?
104

Category "random instances" (RANDOM)

Solver Name#benchsSolvedUNSATSATUNKNOWNBad AnswersMisc.
adaptg2wsat+
2007-02-08		 511252
(49.32%)		 0
(0%)		 252
(49.32%)		 259
(50.68%)		 0
(0%)		 0
adaptg2wsat0
2007-02-08		 511248
(48.53%)		 0
(0%)		 248
(48.53%)		 263
(51.47%)		 0
(0%)		 0
adaptnovelty
2007-02-08		 511240
(46.97%)		 0
(0%)		 240
(46.97%)		 271
(53.03%)		 0
(0%)		 0
DEWSATZ 1A
2007-02-08		 511121
(23.68%)		 48
(9.39%)		 73
(14.29%)		 390
(76.32%)		 0
(0%)		 0
gnovelty+
2007-02-08		 511242
(47.36%)		 0
(0%)		 242
(47.36%)		 269
(52.64%)		 0
(0%)		 0
KCNFS
2004		 511237
(46.38%)		 107
(20.94%)		 130
(25.44%)		 274
(53.62%)		 0
(0%)		 0
March KS
2007-02-08		 511257
(50.29%)		 111
(21.72%)		 146
(28.57%)		 254
(49.71%)		 0
(0%)		 0
minisat
SAT 2007		 511140
(27.4%)		 57
(11.15%)		 83
(16.24%)		 371
(72.6%)		 0
(0%)		 0
MiraXT
v3		 511106
(20.74%)		 37
(7.24%)		 69
(13.5%)		 405
(79.26%)		 0
(0%)		 0
MXC
2007-02-08		 511135
(26.42%)		 53
(10.37%)		 82
(16.05%)		 376
(73.58%)		 0
(0%)		 0
ranov
2007-02-08		 511242
(47.36%)		 0
(0%)		 242
(47.36%)		 269
(52.64%)		 0
(0%)		 0
sapsrt
2007-02-08		 511188
(36.79%)		 0
(0%)		 188
(36.79%)		 323
(63.21%)		 0
(0%)		 0
SAT7
2007-02-08		 511122
(23.87%)		 48
(9.39%)		 74
(14.48%)		 389
(76.13%)		 0
(0%)		 0
SATzilla
RANDOM		 511248
(48.53%)		 101
(19.77%)		 147
(28.77%)		 263
(51.47%)		 0
(0%)		 0

Solver NameProgress
adaptg2wsat+
2007-02-08
done
511
SAT
252		?
259
adaptg2wsat0
2007-02-08
done
511
SAT
248		?
263
adaptnovelty
2007-02-08
done
511
SAT
240		?
271
DEWSATZ 1A
2007-02-08
done
511
UNSAT
48		SAT
73		?
390
gnovelty+
2007-02-08
done
511
SAT
242		?
269
KCNFS
2004
done
511
UNSAT
107		SAT
130		?
274
March KS
2007-02-08
done
511
UNSAT
111		SAT
146		?
254
minisat
SAT 2007
done
511
UNSAT
57		SAT
83		?
371
MiraXT
v3
done
511
UNSAT
37		SAT
69		?
405
MXC
2007-02-08
done
511
UNSAT
53		SAT
82		?
376
ranov
2007-02-08
done
511
SAT
242		?
269
sapsrt
2007-02-08
done
511
SAT
188		?
323
SAT7
2007-02-08
done
511
UNSAT
48		SAT
74		?
389
SATzilla
RANDOM
done
511
UNSAT
101		SAT
147		?
263

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATSATUNKNOWNBad AnswersMisc.
minisat
SAT 2007		 946370
(39.11%)		 208
(21.99%)		 162
(17.12%)		 576
(60.89%)		 0
(0%)		 0
MiraXT
v3		 946310
(32.77%)		 165
(17.44%)		 145
(15.33%)		 636
(67.23%)		 0
(0%)		 0
MXC
2007-02-08		 946328
(34.67%)		 179
(18.92%)		 149
(15.75%)		 618
(65.33%)		 0
(0%)		 0

Solver NameProgress
minisat
SAT 2007
done
946
UNSAT
208		SAT
162		?
576
MiraXT
v3
done
946
UNSAT
165		SAT
145		?
636
MXC
2007-02-08
done
946
UNSAT
179		SAT
149		?
618

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATSATUNKNOWNBad AnswersMisc.
adaptg2wsat+
2007-02-08		 511252
(49.32%)		 0
(0%)		 252
(49.32%)		 259
(50.68%)		 0
(0%)		 0
adaptg2wsat0
2007-02-08		 511248
(48.53%)		 0
(0%)		 248
(48.53%)		 263
(51.47%)		 0
(0%)		 0
adaptnovelty
2007-02-08		 511240
(46.97%)		 0
(0%)		 240
(46.97%)		 271
(53.03%)		 0
(0%)		 0
CMUSAT
2007-02-08		 435189
(43.45%)		 122
(28.05%)		 67
(15.4%)		 246
(56.55%)		 0
(0%)		 0
DEWSATZ 1A
2007-02-08		 511121
(23.68%)		 48
(9.39%)		 73
(14.29%)		 390
(76.32%)		 0
(0%)		 0
gnovelty+
2007-02-08		 511242
(47.36%)		 0
(0%)		 242
(47.36%)		 269
(52.64%)		 0
(0%)		 0
KCNFS
2004		 511237
(46.38%)		 107
(20.94%)		 130
(25.44%)		 274
(53.62%)		 0
(0%)		 0
March KS
2007-02-08		 712304
(42.7%)		 140
(19.66%)		 164
(23.03%)		 408
(57.3%)		 0
(0%)		 0
minisat
SAT 2007		 946370
(39.11%)		 208
(21.99%)		 162
(17.12%)		 576
(60.89%)		 0
(0%)		 0
MiraXT
v3		 946310
(32.77%)		 165
(17.44%)		 145
(15.33%)		 636
(67.23%)		 0
(0%)		 0
MXC
2007-02-08		 946328
(34.67%)		 179
(18.92%)		 149
(15.75%)		 618
(65.33%)		 0
(0%)		 0
picosat
535		 435199
(45.75%)		 105
(24.14%)		 94
(21.61%)		 236
(54.25%)		 0
(0%)		 0
ranov
2007-02-08		 511242
(47.36%)		 0
(0%)		 242
(47.36%)		 269
(52.64%)		 0
(0%)		 0
Rsat
2007-02-08		 435194
(44.6%)		 116
(26.67%)		 78
(17.93%)		 241
(55.4%)		 0
(0%)		 0
sapsrt
2007-02-08		 511188
(36.79%)		 0
(0%)		 188
(36.79%)		 323
(63.21%)		 0
(0%)		 0
SAT7
2007-02-08		 745236
(31.68%)		 111
(14.9%)		 125
(16.78%)		 509
(68.32%)		 0
(0%)		 0
SATzilla
CRAFTED		 435208
(47.82%)		 134
(30.8%)		 74
(17.01%)		 224
(51.49%)		 3
(0.69%)		 0
SATzilla
RANDOM		 511248
(48.53%)		 101
(19.77%)		 147
(28.77%)		 263
(51.47%)		 0
(0%)		 0
Siege
V4		 234122
(52.14%)		 72
(30.77%)		 50
(21.37%)		 112
(47.86%)		 0
(0%)		 0
TiniSatELite
2007-02-08		 234130
(55.56%)		 75
(32.05%)		 55
(23.5%)		 104
(44.44%)		 0
(0%)		 0
TTS
4.0		 20142
(20.9%)		 39
(19.4%)		 3
(1.49%)		 159
(79.1%)		 0
(0%)		 0

Solver NameProgress
adaptg2wsat+
2007-02-08
done
511
SAT
252		?
259
adaptg2wsat0
2007-02-08
done
511
SAT
248		?
263
adaptnovelty
2007-02-08
done
511
SAT
240		?
271
CMUSAT
2007-02-08
done
435
UNSAT
122		SAT
67		?
246
DEWSATZ 1A
2007-02-08
done
511
UNSAT
48		SAT
73		?
390
gnovelty+
2007-02-08
done
511
SAT
242		?
269
KCNFS
2004
done
511
UNSAT
107		SAT
130		?
274
March KS
2007-02-08
done
712
UNSAT
140		SAT
164		?
408
minisat
SAT 2007
done
946
UNSAT
208		SAT
162		?
576
MiraXT
v3
done
946
UNSAT
165		SAT
145		?
636
MXC
2007-02-08
done
946
UNSAT
179		SAT
149		?
618
picosat
535
done
435
UNSAT
105		SAT
94		?
236
ranov
2007-02-08
done
511
SAT
242		?
269
Rsat
2007-02-08
done
435
UNSAT
116		SAT
78		?
241
sapsrt
2007-02-08
done
511
SAT
188		?
323
SAT7
2007-02-08
done
745
UNSAT
111		SAT
125		?
509
SATzilla
CRAFTED
done
435
UNSAT
134		SAT
74		?
224		ERR
3
SATzilla
RANDOM
done
511
UNSAT
101		SAT
147		?
263
Siege
V4
done
234
UNSAT
72		SAT
50		?
112
TiniSatELite
2007-02-08
done
234
UNSAT
75		SAT
55		?
104
TTS
4.0
done
201
UNSAT
39		SAT
3		?
159

Some explanations