PB'07 evaluation: solvers page (tabular format, detailed results)

Category "optimisation, big integers, linear constraints" (OPT-BIGINT-LIN)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
bsolo
3.0.16		 38854
(13.92%)		 29
(7.47%)		 25
(6.44%)		 0
(0%)		 18
(4.64%)		 47
(12.11%)		 22
(5.67%)		 141
(36.34%)		 53
(13.66%)		 27
(6.96%)		 0
(0%)		 0
(0%)		 21
(5.41%)		 0
(0%)		 4
(1.03%)		 1
(0.26%)		 0
(0%)		 0
bsolo
3.0.17		 38854
(13.92%)		 29
(7.47%)		 25
(6.44%)		 0
(0%)		 0
(0%)		 69
(17.78%)		 23
(5.93%)		 135
(34.79%)		 59
(15.21%)		 27
(6.96%)		 0
(0%)		 0
(0%)		 21
(5.41%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 38874
(19.07%)		 38
(9.79%)		 36
(9.28%)		 0
(0%)		 0
(0%)		 67
(17.27%)		 0
(0%)		 0
(0%)		 166
(42.78%)		 20
(5.15%)		 0
(0%)		 0
(0%)		 61
(15.72%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 38848
(12.37%)		 38
(9.79%)		 10
(2.58%)		 0
(0%)		 0
(0%)		 52
(13.4%)		 3
(0.77%)		 0
(0%)		 225
(57.99%)		 60
(15.46%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 38898
(25.26%)		 38
(9.79%)		 60
(15.46%)		 0
(0%)		 0
(0%)		 92
(23.71%)		 0
(0%)		 2
(0.52%)		 194
(50%)		 1
(0.26%)		 1
(0.26%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 38893
(23.97%)		 36
(9.28%)		 57
(14.69%)		 0
(0%)		 0
(0%)		 116
(29.9%)		 0
(0%)		 2
(0.52%)		 175
(45.1%)		 0
(0%)		 2
(0.52%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 388118
(30.41%)		 40
(10.31%)		 78
(20.1%)		 0
(0%)		 1
(0.26%)		 111
(28.61%)		 0
(0%)		 2
(0.52%)		 152
(39.18%)		 1
(0.26%)		 2
(0.52%)		 0
(0%)		 0
(0%)		 1
(0.26%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
bsolo
3.0.16
done
388
UNSAT.
29		OPT
25		SAT (subopt.)
18		SAT (TO)
47		SAT (MO)
22		?
141		? (TO)
53		? (MO)
27		? (SIG)
21		ERR CERT
4		ERR OPT
1
bsolo
3.0.17
done
388
UNSAT.
29		OPT
25		SAT (TO)
69		SAT (MO)
23		?
135		? (TO)
59		? (MO)
27		? (SIG)
21
minisat+
1.14
done
388
UNSAT.
38		OPT
36		SAT (TO)
67		? (TO)
166		? (MO)
20		? (SIG)
61
oree
0.1.2 alpha
done
388
UNSAT.
38		OPT
10		SAT (TO)
52		SAT (MO)
3		? (TO)
225		? (MO)
60
sat4jPseudoCP
2007-03-23
done
388
UNSAT.
38		OPT
60		SAT (TO)
92		?
2		? (TO)
194		? (MO)
1		? (EC)
1
sat4jPseudoCPClause
2007-03-23
done
388
UNSAT.
36		OPT
57		SAT (TO)
116		?
2		? (TO)
175		? (EC)
2
SAT4JPseudoResolution
2007-03-23
done
388
UNSAT.
40		OPT
78		SAT (subopt.)
1		SAT (TO)
111		?
2		? (TO)
152		? (MO)
1		? (EC)
2		NO CERT.
1

Category "optimisation, small integers, linear constraints" (OPT-SMALLINT-LIN)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
absconPseudo
102		 80799
(12.27%)		 8
(0.99%)		 91
(11.28%)		 0
(0%)		 0
(0%)		 245
(30.36%)		 0
(0%)		 53
(6.57%)		 407
(50.43%)		 3
(0.37%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.16		 807263
(32.59%)		 33
(4.09%)		 230
(28.5%)		 0
(0%)		 141
(17.47%)		 138
(17.1%)		 40
(4.96%)		 52
(6.44%)		 105
(13.01%)		 30
(3.72%)		 0
(0%)		 0
(0%)		 25
(3.1%)		 0
(0%)		 12
(1.49%)		 1
(0.12%)		 0
(0%)		 0
bsolo
3.0.17		 807270
(33.46%)		 33
(4.09%)		 237
(29.37%)		 0
(0%)		 0
(0%)		 285
(35.32%)		 40
(4.96%)		 0
(0%)		 157
(19.45%)		 30
(3.72%)		 0
(0%)		 0
(0%)		 25
(3.1%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
glpPB
0.2		 807252
(31.23%)		 40
(4.96%)		 212
(26.27%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 423
(52.42%)		 0
(0%)		 130
(16.11%)		 2
(0.25%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 807243
(30.11%)		 35
(4.34%)		 208
(25.77%)		 0
(0%)		 0
(0%)		 297
(36.8%)		 5
(0.62%)		 0
(0%)		 243
(30.11%)		 4
(0.5%)		 0
(0%)		 0
(0%)		 15
(1.86%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 80792
(11.4%)		 15
(1.86%)		 77
(9.54%)		 0
(0%)		 0
(0%)		 299
(37.05%)		 22
(2.73%)		 0
(0%)		 374
(46.34%)		 19
(2.35%)		 1
(0.12%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-03-23		 807154
(19.08%)		 8
(0.99%)		 146
(18.09%)		 0
(0%)		 0
(0%)		 130
(16.11%)		 77
(9.54%)		 0
(0%)		 175
(21.69%)		 271
(33.58%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-04-10		 807181
(22.43%)		 23
(2.85%)		 158
(19.58%)		 0
(0%)		 0
(0%)		 309
(38.29%)		 0
(0%)		 0
(0%)		 300
(37.17%)		 17
(2.11%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4
2007-03-23		 807164
(20.32%)		 23
(2.85%)		 141
(17.47%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 226
(28%)		 0
(0%)		 0
(0%)		 1
(0.12%)		 0
(0%)		 416
(51.55%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4_v2
2007-03-23		 807164
(20.32%)		 23
(2.85%)		 141
(17.47%)		 0
(0%)		 0
(0%)		 416
(51.55%)		 0
(0%)		 0
(0%)		 226
(28%)		 0
(0%)		 0
(0%)		 1
(0.12%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Pueblo
1.4		 807211
(26.15%)		 31
(3.84%)		 180
(22.3%)		 0
(0%)		 389
(48.2%)		 0
(0%)		 0
(0%)		 0
(0%)		 192
(23.79%)		 13
(1.61%)		 0
(0%)		 0
(0%)		 2
(0.25%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 807194
(24.04%)		 47
(5.82%)		 147
(18.22%)		 0
(0%)		 2
(0.25%)		 395
(48.95%)		 0
(0%)		 0
(0%)		 216
(26.77%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 807191
(23.67%)		 46
(5.7%)		 145
(17.97%)		 0
(0%)		 1
(0.12%)		 397
(49.19%)		 0
(0%)		 0
(0%)		 218
(27.01%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 807191
(23.67%)		 35
(4.34%)		 156
(19.33%)		 0
(0%)		 6
(0.74%)		 370
(45.85%)		 0
(0%)		 0
(0%)		 225
(27.88%)		 4
(0.5%)		 9
(1.12%)		 0
(0%)		 0
(0%)		 2
(0.25%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-rnp
2007-03-21		 8071
(0.12%)		 0
(0%)		 1
(0.12%)		 0
(0%)		 0
(0%)		 411
(50.93%)		 0
(0%)		 0
(0%)		 392
(48.57%)		 3
(0.37%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-skc
2007-03-21		 8071
(0.12%)		 0
(0%)		 1
(0.12%)		 0
(0%)		 3
(0.37%)		 444
(55.02%)		 0
(0%)		 0
(0%)		 356
(44.11%)		 3
(0.37%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
absconPseudo
102
done
807
UNSAT.
8		OPT
91		SAT (TO)
245		?
53		? (TO)
407		? (MO)
3
bsolo
3.0.16
done
807
UNSAT.
33		OPT
230		SAT (subopt.)
141		SAT (TO)
138		SAT (MO)
40		?
52		? (TO)
105		? (MO)
30		? (SIG)
25		ERR CERT
12		ERR OPT
1
bsolo
3.0.17
done
807
UNSAT.
33		OPT
237		SAT (TO)
285		SAT (MO)
40		? (TO)
157		? (MO)
30		? (SIG)
25
glpPB
0.2
done
807
UNSAT.
40		OPT
212		? (TO)
423		? (EC)
130		? (?)
2
minisat+
1.14
done
807
UNSAT.
35		OPT
208		SAT (TO)
297		SAT (MO)
5		? (TO)
243		? (MO)
4		? (SIG)
15
oree
0.1.2 alpha
done
807
UNSAT.
15		OPT
77		SAT (TO)
299		SAT (MO)
22		? (TO)
374		? (MO)
19		? (EC)
1
PB-clasp
2007-03-23
done
807
UNSAT.
8		OPT
146		SAT (TO)
130		SAT (MO)
77		? (TO)
175		? (MO)
271
PB-clasp
2007-04-10
done
807
UNSAT.
23		OPT
158		SAT (TO)
309		? (TO)
300		? (MO)
17
PBS4
2007-03-23
done
807
UNSAT.
23		OPT
141		? (TO)
226		? (?)
1		NO CERT.
416
PBS4_v2
2007-03-23
done
807
UNSAT.
23		OPT
141		SAT (TO)
416		? (TO)
226		? (?)
1
Pueblo
1.4
done
807
UNSAT.
31		OPT
180		SAT (subopt.)
389		? (TO)
192		? (MO)
13		? (SIG)
2
sat4jPseudoCP
2007-03-23
done
807
UNSAT.
47		OPT
147		SAT (subopt.)
2		SAT (TO)
395		? (TO)
216
sat4jPseudoCPClause
2007-03-23
done
807
UNSAT.
46		OPT
145		SAT (subopt.)
1		SAT (TO)
397		? (TO)
218
SAT4JPseudoResolution
2007-03-23
done
807
UNSAT.
35		OPT
156		SAT (subopt.)
6		SAT (TO)
370		? (TO)
225		? (MO)
4		? (EC)
9		NO CERT.
2
wildcat-rnp
2007-03-21
done
807
OPT
1		SAT (TO)
411		? (TO)
392		? (MO)
3
wildcat-skc
2007-03-21
done
807
OPT
1		SAT (subopt.)
3		SAT (TO)
444		? (TO)
356		? (MO)
3

Category "optimisation, small integers, non linear constraints" (OPT-SMALLINT-NLC)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
absconPseudo
102		 40588
(21.73%)		 0
(0%)		 88
(21.73%)		 0
(0%)		 0
(0%)		 87
(21.48%)		 0
(0%)		 0
(0%)		 230
(56.79%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.16		 405239
(59.01%)		 0
(0%)		 239
(59.01%)		 0
(0%)		 52
(12.84%)		 84
(20.74%)		 14
(3.46%)		 9
(2.22%)		 6
(1.48%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.25%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.17		 405239
(59.01%)		 0
(0%)		 239
(59.01%)		 0
(0%)		 0
(0%)		 139
(34.32%)		 12
(2.96%)		 0
(0%)		 15
(3.7%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
glpPB
0.2		 40567
(16.54%)		 0
(0%)		 67
(16.54%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 7
(1.73%)		 322
(79.51%)		 0
(0%)		 9
(2.22%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 405275
(67.9%)		 0
(0%)		 275
(67.9%)		 0
(0%)		 0
(0%)		 120
(29.63%)		 0
(0%)		 0
(0%)		 10
(2.47%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 40599
(24.44%)		 0
(0%)		 99
(24.44%)		 0
(0%)		 0
(0%)		 101
(24.94%)		 0
(0%)		 0
(0%)		 200
(49.38%)		 5
(1.23%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-03-23		 4050
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 97
(23.95%)		 219
(54.07%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 60
(14.81%)		 0
(0%)		 29
(7.16%)		 0
PB-clasp
2007-04-10		 405177
(43.7%)		 0
(0%)		 177
(43.7%)		 0
(0%)		 0
(0%)		 35
(8.64%)		 0
(0%)		 0
(0%)		 133
(32.84%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 60
(14.81%)		 0
(0%)		 0
(0%)		 0
PBS4
2007-03-23		 405171
(42.22%)		 0
(0%)		 171
(42.22%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 18
(4.44%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 216
(53.33%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4_v2
2007-03-23		 405171
(42.22%)		 0
(0%)		 171
(42.22%)		 0
(0%)		 0
(0%)		 216
(53.33%)		 0
(0%)		 0
(0%)		 18
(4.44%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Pueblo
1.4		 405248
(61.23%)		 0
(0%)		 248
(61.23%)		 0
(0%)		 157
(38.77%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 40578
(19.26%)		 0
(0%)		 78
(19.26%)		 0
(0%)		 0
(0%)		 124
(30.62%)		 0
(0%)		 120
(29.63%)		 83
(20.49%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 40576
(18.77%)		 0
(0%)		 76
(18.77%)		 0
(0%)		 0
(0%)		 126
(31.11%)		 0
(0%)		 120
(29.63%)		 83
(20.49%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 405216
(53.33%)		 0
(0%)		 216
(53.33%)		 0
(0%)		 0
(0%)		 68
(16.79%)		 0
(0%)		 120
(29.63%)		 1
(0.25%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-rnp
2007-03-21		 4050
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 178
(43.95%)		 0
(0%)		 0
(0%)		 166
(40.99%)		 0
(0%)		 0
(0%)		 60
(14.81%)		 0
(0%)		 1
(0.25%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-skc
2007-03-21		 4050
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 153
(37.78%)		 0
(0%)		 0
(0%)		 190
(46.91%)		 0
(0%)		 0
(0%)		 60
(14.81%)		 0
(0%)		 2
(0.49%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
absconPseudo
102
done
405
OPT
88		SAT (TO)
87		? (TO)
230
bsolo
3.0.16
done
405
OPT
239		SAT (subopt.)
52		SAT (TO)
84		SAT (MO)
14		?
9		? (TO)
6		ERR CERT
1
bsolo
3.0.17
done
405
OPT
239		SAT (TO)
139		SAT (MO)
12		? (TO)
15
glpPB
0.2
done
405
OPT
67		?
7		? (TO)
322		? (EC)
9
minisat+
1.14
done
405
OPT
275		SAT (TO)
120		? (TO)
10
oree
0.1.2 alpha
done
405
OPT
99		SAT (TO)
101		? (TO)
200		? (MO)
5
PB-clasp
2007-03-23
done
405
? (TO)
97		? (MO)
219		ERR CERT
60		ERR UNSAT
29
PB-clasp
2007-04-10
done
405
OPT
177		SAT (TO)
35		? (TO)
133		ERR CERT
60
PBS4
2007-03-23
done
405
OPT
171		? (TO)
18		NO CERT.
216
PBS4_v2
2007-03-23
done
405
OPT
171		SAT (TO)
216		? (TO)
18
Pueblo
1.4
done
405
OPT
248		SAT (subopt.)
157
sat4jPseudoCP
2007-03-23
done
405
OPT
78		SAT (TO)
124		?
120		? (TO)
83
sat4jPseudoCPClause
2007-03-23
done
405
OPT
76		SAT (TO)
126		?
120		? (TO)
83
SAT4JPseudoResolution
2007-03-23
done
405
OPT
216		SAT (TO)
68		?
120		? (TO)
1
wildcat-rnp
2007-03-21
done
405
SAT (TO)
178		? (TO)
166		? (?)
60		NO CERT.
1
wildcat-skc
2007-03-21
done
405
SAT (TO)
153		? (TO)
190		? (?)
60		NO CERT.
2

Category "instances containing only clauses" (PURE-SAT)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
absconPseudo
102		 1669
(5.42%)		 9
(5.42%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 46
(27.71%)		 109
(65.66%)		 2
(1.2%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.16		 16660
(36.14%)		 45
(27.11%)		 0
(0%)		 15
(9.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 38
(22.89%)		 45
(27.11%)		 21
(12.65%)		 0
(0%)		 0
(0%)		 2
(1.2%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.17		 16663
(37.95%)		 45
(27.11%)		 0
(0%)		 18
(10.84%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 80
(48.19%)		 21
(12.65%)		 0
(0%)		 0
(0%)		 2
(1.2%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
glpPB
0.2		 1661
(0.6%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 153
(92.17%)		 10
(6.02%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 166115
(69.28%)		 75
(45.18%)		 0
(0%)		 40
(24.1%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 33
(19.88%)		 17
(10.24%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 16613
(7.83%)		 7
(4.22%)		 0
(0%)		 6
(3.61%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 142
(85.54%)		 9
(5.42%)		 2
(1.2%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-03-23		 166102
(61.45%)		 72
(43.37%)		 0
(0%)		 30
(18.07%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 52
(31.33%)		 11
(6.63%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-04-10		 166125
(75.3%)		 83
(50%)		 0
(0%)		 42
(25.3%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 25
(15.06%)		 15
(9.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4
2007-03-23		 16635
(21.08%)		 21
(12.65%)		 0
(0%)		 14
(8.43%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 51
(30.72%)		 0
(0%)		 0
(0%)		 3
(1.81%)		 70
(42.17%)		 7
(4.22%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4_v2
2007-03-23		 16635
(21.08%)		 21
(12.65%)		 0
(0%)		 14
(8.43%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 51
(30.72%)		 0
(0%)		 0
(0%)		 3
(1.81%)		 70
(42.17%)		 7
(4.22%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Pueblo
1.4		 16685
(51.2%)		 62
(37.35%)		 0
(0%)		 23
(13.86%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 71
(42.77%)		 9
(5.42%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 16647
(28.31%)		 30
(18.07%)		 0
(0%)		 17
(10.24%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 118
(71.08%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 16647
(28.31%)		 30
(18.07%)		 0
(0%)		 17
(10.24%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 118
(71.08%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 16698
(59.04%)		 57
(34.34%)		 0
(0%)		 41
(24.7%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 66
(39.76%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-rnp
2007-03-21		 1661
(0.6%)		 0
(0%)		 0
(0%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 11
(6.63%)		 143
(86.14%)		 10
(6.02%)		 0
(0%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-skc
2007-03-21		 1664
(2.41%)		 0
(0%)		 0
(0%)		 4
(2.41%)		 0
(0%)		 0
(0%)		 0
(0%)		 39
(23.49%)		 112
(67.47%)		 10
(6.02%)		 0
(0%)		 1
(0.6%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
absconPseudo
102
done
166
UNSAT.
9		?
46		? (TO)
109		? (MO)
2
bsolo
3.0.16
done
166
UNSAT.
45		SAT
15		?
38		? (TO)
45		? (MO)
21		? (SIG)
2
bsolo
3.0.17
done
166
UNSAT.
45		SAT
18		? (TO)
80		? (MO)
21		? (SIG)
2
glpPB
0.2
done
166
UNSAT.
1		?
1		? (TO)
153		? (MO)
10		? (EC)
1
minisat+
1.14
done
166
UNSAT.
75		SAT
40		? (TO)
33		? (MO)
17		NO CERT.
1
oree
0.1.2 alpha
done
166
UNSAT.
7		SAT
6		? (TO)
142		? (MO)
9		? (EC)
2
PB-clasp
2007-03-23
done
166
UNSAT.
72		SAT
30		? (TO)
52		? (MO)
11		NO CERT.
1
PB-clasp
2007-04-10
done
166
UNSAT.
83		SAT
42		? (TO)
25		? (MO)
15		NO CERT.
1
PBS4
2007-03-23
done
166
UNSAT.
21		SAT
14		? (TO)
51		? (?)
3		? (SIG)
70		NO CERT.
7
PBS4_v2
2007-03-23
done
166
UNSAT.
21		SAT
14		? (TO)
51		? (?)
3		? (SIG)
70		NO CERT.
7
Pueblo
1.4
done
166
UNSAT.
62		SAT
23		?
1		? (TO)
71		? (MO)
9
sat4jPseudoCP
2007-03-23
done
166
UNSAT.
30		SAT
17		?
1		? (TO)
118
sat4jPseudoCPClause
2007-03-23
done
166
UNSAT.
30		SAT
17		?
1		? (TO)
118
SAT4JPseudoResolution
2007-03-23
done
166
UNSAT.
57		SAT
41		?
1		? (TO)
66		? (MO)
1
wildcat-rnp
2007-03-21
done
166
SAT
1		?
11		? (TO)
143		? (MO)
10		? (?)
1
wildcat-skc
2007-03-21
done
166
SAT
4		?
39		? (TO)
112		? (MO)
10		? (?)
1

Category "no optimisation, big integers, linear constraints" (SATUNSAT-BIGINT-LIN)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
bsolo
3.0.16		 147
(50%)		 0
(0%)		 0
(0%)		 7
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 7
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.17		 147
(50%)		 0
(0%)		 0
(0%)		 7
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 7
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 1411
(78.57%)		 0
(0%)		 0
(0%)		 11
(78.57%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 2
(14.29%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(7.14%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 141
(7.14%)		 0
(0%)		 0
(0%)		 1
(7.14%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 13
(92.86%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 143
(21.43%)		 0
(0%)		 0
(0%)		 3
(21.43%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 11
(78.57%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 142
(14.29%)		 0
(0%)		 0
(0%)		 2
(14.29%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 12
(85.71%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 144
(28.57%)		 0
(0%)		 0
(0%)		 4
(28.57%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 10
(71.43%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
bsolo
3.0.16
done
14
SAT
7		?
7
bsolo
3.0.17
done
14
SAT
7		? (TO)
7
minisat+
1.14
done
14
SAT
11		? (TO)
2		? (SIG)
1
oree
0.1.2 alpha
done
14
SAT
1		? (TO)
13
sat4jPseudoCP
2007-03-23
done
14
SAT
3		? (TO)
11
sat4jPseudoCPClause
2007-03-23
done
14
SAT
2		? (TO)
12
SAT4JPseudoResolution
2007-03-23
done
14
SAT
4		? (TO)
10

Category "no optimisation, small integers, linear constraints" (SATUNSAT-SMALLINT-LIN)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
absconPseudo
102		 371160
(43.13%)		 79
(21.29%)		 0
(0%)		 81
(21.83%)		 0
(0%)		 0
(0%)		 0
(0%)		 9
(2.43%)		 201
(54.18%)		 1
(0.27%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.16		 371271
(73.05%)		 137
(36.93%)		 0
(0%)		 134
(36.12%)		 0
(0%)		 0
(0%)		 0
(0%)		 48
(12.94%)		 35
(9.43%)		 17
(4.58%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.17		 371271
(73.05%)		 137
(36.93%)		 0
(0%)		 134
(36.12%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 83
(22.37%)		 17
(4.58%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
glpPB
0.2		 371200
(53.91%)		 121
(32.61%)		 0
(0%)		 79
(21.29%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 168
(45.28%)		 0
(0%)		 3
(0.81%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 371305
(82.21%)		 170
(45.82%)		 0
(0%)		 135
(36.39%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 66
(17.79%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 37198
(26.42%)		 36
(9.7%)		 0
(0%)		 62
(16.71%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 273
(73.58%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-03-23		 371283
(76.28%)		 171
(46.09%)		 0
(0%)		 112
(30.19%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 68
(18.33%)		 20
(5.39%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-04-10		 371312
(84.1%)		 175
(47.17%)		 0
(0%)		 137
(36.93%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 59
(15.9%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4
2007-03-23		 371327
(88.14%)		 197
(53.1%)		 0
(0%)		 130
(35.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 44
(11.86%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4_v2
2007-03-23		 371327
(88.14%)		 197
(53.1%)		 0
(0%)		 130
(35.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 44
(11.86%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Pueblo
1.4		 371341
(91.91%)		 202
(54.45%)		 0
(0%)		 139
(37.47%)		 0
(0%)		 0
(0%)		 0
(0%)		 2
(0.54%)		 27
(7.28%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.27%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 371141
(38.01%)		 88
(23.72%)		 0
(0%)		 53
(14.29%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 230
(61.99%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 371151
(40.7%)		 95
(25.61%)		 0
(0%)		 56
(15.09%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 220
(59.3%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 371273
(73.58%)		 167
(45.01%)		 0
(0%)		 106
(28.57%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 98
(26.42%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-rnp
2007-03-21		 371142
(38.27%)		 0
(0%)		 0
(0%)		 142
(38.27%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.27%)		 228
(61.46%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-skc
2007-03-21		 371148
(39.89%)		 0
(0%)		 0
(0%)		 148
(39.89%)		 0
(0%)		 0
(0%)		 0
(0%)		 37
(9.97%)		 186
(50.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
absconPseudo
102
done
371
UNSAT.
79		SAT
81		?
9		? (TO)
201		? (MO)
1
bsolo
3.0.16
done
371
UNSAT.
137		SAT
134		?
48		? (TO)
35		? (MO)
17
bsolo
3.0.17
done
371
UNSAT.
137		SAT
134		? (TO)
83		? (MO)
17
glpPB
0.2
done
371
UNSAT.
121		SAT
79		? (TO)
168		? (EC)
3
minisat+
1.14
done
371
UNSAT.
170		SAT
135		? (TO)
66
oree
0.1.2 alpha
done
371
UNSAT.
36		SAT
62		? (TO)
273
PB-clasp
2007-03-23
done
371
UNSAT.
171		SAT
112		? (TO)
68		? (MO)
20
PB-clasp
2007-04-10
done
371
UNSAT.
175		SAT
137		? (TO)
59
PBS4
2007-03-23
done
371
UNSAT.
197		SAT
130		? (TO)
44
PBS4_v2
2007-03-23
done
371
UNSAT.
197		SAT
130		? (TO)
44
Pueblo
1.4
done
371
UNSAT.
202		SAT
139		?
2		? (TO)
27		? (SIG)
1
sat4jPseudoCP
2007-03-23
done
371
UNSAT.
88		SAT
53		? (TO)
230
sat4jPseudoCPClause
2007-03-23
done
371
UNSAT.
95		SAT
56		? (TO)
220
SAT4JPseudoResolution
2007-03-23
done
371
UNSAT.
167		SAT
106		? (TO)
98
wildcat-rnp
2007-03-21
done
371
SAT
142		?
1		? (TO)
228
wildcat-skc
2007-03-21
done
371
SAT
148		?
37		? (TO)
186

Category "no optimisation, small integers, non linear constraints" (SATUNSAT-SMALLINT-NLC)

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
absconPseudo
102		 10050
(50%)		 10
(10%)		 0
(0%)		 40
(40%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 50
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.16		 10027
(27%)		 10
(10%)		 0
(0%)		 17
(17%)		 0
(0%)		 0
(0%)		 0
(0%)		 9
(9%)		 64
(64%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.17		 10028
(28%)		 10
(10%)		 0
(0%)		 18
(18%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 71
(71%)		 1
(1%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
glpPB
0.2		 10065
(65%)		 20
(20%)		 0
(0%)		 45
(45%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 35
(35%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 10065
(65%)		 15
(15%)		 0
(0%)		 50
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 35
(35%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 10055
(55%)		 10
(10%)		 0
(0%)		 45
(45%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 45
(45%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-03-23		 10062
(62%)		 15
(15%)		 0
(0%)		 47
(47%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 38
(38%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-04-10		 10064
(64%)		 15
(15%)		 0
(0%)		 49
(49%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 36
(36%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4
2007-03-23		 10065
(65%)		 15
(15%)		 0
(0%)		 50
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 35
(35%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4_v2
2007-03-23		 10065
(65%)		 15
(15%)		 0
(0%)		 50
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 35
(35%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Pueblo
1.4		 10064
(64%)		 14
(14%)		 0
(0%)		 50
(50%)		 0
(0%)		 0
(0%)		 0
(0%)		 3
(3%)		 32
(32%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(1%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 1000
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 100
(100%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 1000
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 100
(100%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 1000
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 100
(100%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-rnp
2007-03-21		 10060
(60%)		 0
(0%)		 0
(0%)		 60
(60%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 40
(40%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-skc
2007-03-21		 10060
(60%)		 0
(0%)		 0
(0%)		 60
(60%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 40
(40%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
absconPseudo
102
done
100
UNSAT.
10		SAT
40		? (TO)
50
bsolo
3.0.16
done
100
UNSAT.
10		SAT
17		?
9		? (TO)
64
bsolo
3.0.17
done
100
UNSAT.
10		SAT
18		? (TO)
71		? (MO)
1
glpPB
0.2
done
100
UNSAT.
20		SAT
45		? (TO)
35
minisat+
1.14
done
100
UNSAT.
15		SAT
50		? (TO)
35
oree
0.1.2 alpha
done
100
UNSAT.
10		SAT
45		? (TO)
45
PB-clasp
2007-03-23
done
100
UNSAT.
15		SAT
47		? (TO)
38
PB-clasp
2007-04-10
done
100
UNSAT.
15		SAT
49		? (TO)
36
PBS4
2007-03-23
done
100
UNSAT.
15		SAT
50		? (TO)
35
PBS4_v2
2007-03-23
done
100
UNSAT.
15		SAT
50		? (TO)
35
Pueblo
1.4
done
100
UNSAT.
14		SAT
50		?
3		? (TO)
32		? (SIG)
1
sat4jPseudoCP
2007-03-23
done
100
?
100
sat4jPseudoCPClause
2007-03-23
done
100
?
100
SAT4JPseudoResolution
2007-03-23
done
100
?
100
wildcat-rnp
2007-03-21
done
100
SAT
60		? (TO)
40
wildcat-skc
2007-03-21
done
100
SAT
60		? (TO)
40

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
bsolo
3.0.16		 2251921
(40.92%)		 254
(11.28%)		 494
(21.95%)		 173
(7.69%)		 211
(9.37%)		 269
(11.95%)		 76
(3.38%)		 304
(13.51%)		 308
(13.68%)		 95
(4.22%)		 0
(0%)		 0
(0%)		 48
(2.13%)		 0
(0%)		 17
(0.76%)		 2
(0.09%)		 0
(0%)		 0
bsolo
3.0.17		 2251932
(41.4%)		 254
(11.28%)		 501
(22.26%)		 177
(7.86%)		 0
(0%)		 493
(21.9%)		 75
(3.33%)		 135
(6%)		 472
(20.97%)		 96
(4.26%)		 0
(0%)		 0
(0%)		 48
(2.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 22511088
(48.33%)		 333
(14.79%)		 519
(23.06%)		 236
(10.48%)		 0
(0%)		 484
(21.5%)		 5
(0.22%)		 0
(0%)		 555
(24.66%)		 41
(1.82%)		 0
(0%)		 0
(0%)		 77
(3.42%)		 1
(0.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 2251406
(18.04%)		 106
(4.71%)		 186
(8.26%)		 114
(5.06%)		 0
(0%)		 452
(20.08%)		 25
(1.11%)		 0
(0%)		 1272
(56.51%)		 93
(4.13%)		 3
(0.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 2251561
(24.92%)		 203
(9.02%)		 285
(12.66%)		 73
(3.24%)		 2
(0.09%)		 611
(27.14%)		 0
(0%)		 223
(9.91%)		 852
(37.85%)		 1
(0.04%)		 1
(0.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 2251560
(24.88%)		 207
(9.2%)		 278
(12.35%)		 75
(3.33%)		 1
(0.04%)		 639
(28.39%)		 0
(0%)		 223
(9.91%)		 826
(36.69%)		 0
(0%)		 2
(0.09%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 2251900
(39.98%)		 299
(13.28%)		 450
(19.99%)		 151
(6.71%)		 7
(0.31%)		 549
(24.39%)		 0
(0%)		 223
(9.91%)		 552
(24.52%)		 6
(0.27%)		 11
(0.49%)		 0
(0%)		 0
(0%)		 3
(0.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
bsolo
3.0.16
done
2251
UNSAT.
254		OPT
494		SAT
173		SAT (subopt.)
211		SAT (TO)
269		SAT (MO)
76		?
304		? (TO)
308		? (MO)
95		? (SIG)
48		ERR CERT
17		ERR OPT
2
bsolo
3.0.17
done
2251
UNSAT.
254		OPT
501		SAT
177		SAT (TO)
493		SAT (MO)
75		?
135		? (TO)
472		? (MO)
96		? (SIG)
48
minisat+
1.14
done
2251
UNSAT.
333		OPT
519		SAT
236		SAT (TO)
484		SAT (MO)
5		? (TO)
555		? (MO)
41		? (SIG)
77		NO CERT.
1
oree
0.1.2 alpha
done
2251
UNSAT.
106		OPT
186		SAT
114		SAT (TO)
452		SAT (MO)
25		? (TO)
1272		? (MO)
93		? (EC)
3
sat4jPseudoCP
2007-03-23
done
2251
UNSAT.
203		OPT
285		SAT
73		SAT (subopt.)
2		SAT (TO)
611		?
223		? (TO)
852		? (MO)
1		? (EC)
1
sat4jPseudoCPClause
2007-03-23
done
2251
UNSAT.
207		OPT
278		SAT
75		SAT (subopt.)
1		SAT (TO)
639		?
223		? (TO)
826		? (EC)
2
SAT4JPseudoResolution
2007-03-23
done
2251
UNSAT.
299		OPT
450		SAT
151		SAT (subopt.)
7		SAT (TO)
549		?
223		? (TO)
552		? (MO)
6		? (EC)
11		NO CERT.
3

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSAT.OPTSATSAT (subopt.)SAT (timeout)SAT (out of mem.)UNKNOWNUNKNOWN (timeout)UNKNOWN (out of mem.)UNKNOWN (exit code)UNKNOWN (problem)Sig. CaughtNO CERT.WRONG CERT.WRONG OPT.WRONG UNSAT.Misc.
absconPseudo
102		 1849406
(21.96%)		 106
(5.73%)		 179
(9.68%)		 121
(6.54%)		 0
(0%)		 332
(17.96%)		 0
(0%)		 108
(5.84%)		 997
(53.92%)		 6
(0.32%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
bsolo
3.0.16		 2251921
(40.92%)		 254
(11.28%)		 494
(21.95%)		 173
(7.69%)		 211
(9.37%)		 269
(11.95%)		 76
(3.38%)		 304
(13.51%)		 308
(13.68%)		 95
(4.22%)		 0
(0%)		 0
(0%)		 48
(2.13%)		 0
(0%)		 17
(0.76%)		 2
(0.09%)		 0
(0%)		 0
bsolo
3.0.17		 2251932
(41.4%)		 254
(11.28%)		 501
(22.26%)		 177
(7.86%)		 0
(0%)		 493
(21.9%)		 75
(3.33%)		 135
(6%)		 472
(20.97%)		 96
(4.26%)		 0
(0%)		 0
(0%)		 48
(2.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
glpPB
0.2		 1849585
(31.64%)		 182
(9.84%)		 279
(15.09%)		 124
(6.71%)		 0
(0%)		 0
(0%)		 0
(0%)		 8
(0.43%)		 1101
(59.55%)		 10
(0.54%)		 143
(7.73%)		 2
(0.11%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
minisat+
1.14		 22511088
(48.33%)		 333
(14.79%)		 519
(23.06%)		 236
(10.48%)		 0
(0%)		 484
(21.5%)		 5
(0.22%)		 0
(0%)		 555
(24.66%)		 41
(1.82%)		 0
(0%)		 0
(0%)		 77
(3.42%)		 1
(0.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
oree
0.1.2 alpha		 2251406
(18.04%)		 106
(4.71%)		 186
(8.26%)		 114
(5.06%)		 0
(0%)		 452
(20.08%)		 25
(1.11%)		 0
(0%)		 1272
(56.51%)		 93
(4.13%)		 3
(0.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PB-clasp
2007-03-23		 1849601
(32.5%)		 266
(14.39%)		 146
(7.9%)		 189
(10.22%)		 0
(0%)		 130
(7.03%)		 77
(4.16%)		 0
(0%)		 430
(23.26%)		 521
(28.18%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.05%)		 60
(3.24%)		 0
(0%)		 29
(1.57%)		 0
PB-clasp
2007-04-10		 1849859
(46.46%)		 296
(16.01%)		 335
(18.12%)		 228
(12.33%)		 0
(0%)		 344
(18.6%)		 0
(0%)		 0
(0%)		 553
(29.91%)		 32
(1.73%)		 0
(0%)		 0
(0%)		 0
(0%)		 1
(0.05%)		 60
(3.24%)		 0
(0%)		 0
(0%)		 0
PBS4
2007-03-23		 1849762
(41.21%)		 256
(13.85%)		 312
(16.87%)		 194
(10.49%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 374
(20.23%)		 0
(0%)		 0
(0%)		 4
(0.22%)		 70
(3.79%)		 639
(34.56%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
PBS4_v2
2007-03-23		 1849762
(41.21%)		 256
(13.85%)		 312
(16.87%)		 194
(10.49%)		 0
(0%)		 632
(34.18%)		 0
(0%)		 0
(0%)		 374
(20.23%)		 0
(0%)		 0
(0%)		 4
(0.22%)		 70
(3.79%)		 7
(0.38%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
Pueblo
1.4		 1849949
(51.33%)		 309
(16.71%)		 428
(23.15%)		 212
(11.47%)		 546
(29.53%)		 0
(0%)		 0
(0%)		 6
(0.32%)		 322
(17.41%)		 22
(1.19%)		 0
(0%)		 0
(0%)		 4
(0.22%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCP
2007-03-23		 2251561
(24.92%)		 203
(9.02%)		 285
(12.66%)		 73
(3.24%)		 2
(0.09%)		 611
(27.14%)		 0
(0%)		 223
(9.91%)		 852
(37.85%)		 1
(0.04%)		 1
(0.04%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
sat4jPseudoCPClause
2007-03-23		 2251560
(24.88%)		 207
(9.2%)		 278
(12.35%)		 75
(3.33%)		 1
(0.04%)		 639
(28.39%)		 0
(0%)		 223
(9.91%)		 826
(36.69%)		 0
(0%)		 2
(0.09%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
SAT4JPseudoResolution
2007-03-23		 2251900
(39.98%)		 299
(13.28%)		 450
(19.99%)		 151
(6.71%)		 7
(0.31%)		 549
(24.39%)		 0
(0%)		 223
(9.91%)		 552
(24.52%)		 6
(0.27%)		 11
(0.49%)		 0
(0%)		 0
(0%)		 3
(0.13%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-rnp
2007-03-21		 1849204
(11.03%)		 0
(0%)		 1
(0.05%)		 203
(10.98%)		 0
(0%)		 589
(31.86%)		 0
(0%)		 12
(0.65%)		 969
(52.41%)		 13
(0.7%)		 0
(0%)		 61
(3.3%)		 0
(0%)		 1
(0.05%)		 0
(0%)		 0
(0%)		 0
(0%)		 0
wildcat-skc
2007-03-21		 1849213
(11.52%)		 0
(0%)		 1
(0.05%)		 212
(11.47%)		 3
(0.16%)		 597
(32.29%)		 0
(0%)		 76
(4.11%)		 884
(47.81%)		 13
(0.7%)		 0
(0%)		 61
(3.3%)		 0
(0%)		 2
(0.11%)		 0
(0%)		 0
(0%)		 0
(0%)		 0

Solver NameProgress
absconPseudo
102
done
1849
UNSAT.
106		OPT
179		SAT
121		SAT (TO)
332		?
108		? (TO)
997		? (MO)
6
bsolo
3.0.16
done
2251
UNSAT.
254		OPT
494		SAT
173		SAT (subopt.)
211		SAT (TO)
269		SAT (MO)
76		?
304		? (TO)
308		? (MO)
95		? (SIG)
48		ERR CERT
17		ERR OPT
2
bsolo
3.0.17
done
2251
UNSAT.
254		OPT
501		SAT
177		SAT (TO)
493		SAT (MO)
75		?
135		? (TO)
472		? (MO)
96		? (SIG)
48
glpPB
0.2
done
1849
UNSAT.
182		OPT
279		SAT
124		?
8		? (TO)
1101		? (MO)
10		? (EC)
143		? (?)
2
minisat+
1.14
done
2251
UNSAT.
333		OPT
519		SAT
236		SAT (TO)
484		SAT (MO)
5		? (TO)
555		? (MO)
41		? (SIG)
77		NO CERT.
1
oree
0.1.2 alpha
done
2251
UNSAT.
106		OPT
186		SAT
114		SAT (TO)
452		SAT (MO)
25		? (TO)
1272		? (MO)
93		? (EC)
3
PB-clasp
2007-03-23
done
1849
UNSAT.
266		OPT
146		SAT
189		SAT (TO)
130		SAT (MO)
77		? (TO)
430		? (MO)
521		NO CERT.
1		ERR CERT
60		ERR UNSAT
29
PB-clasp
2007-04-10
done
1849
UNSAT.
296		OPT
335		SAT
228		SAT (TO)
344		? (TO)
553		? (MO)
32		NO CERT.
1		ERR CERT
60
PBS4
2007-03-23
done
1849
UNSAT.
256		OPT
312		SAT
194		? (TO)
374		? (?)
4		? (SIG)
70		NO CERT.
639
PBS4_v2
2007-03-23
done
1849
UNSAT.
256		OPT
312		SAT
194		SAT (TO)
632		? (TO)
374		? (?)
4		? (SIG)
70		NO CERT.
7
Pueblo
1.4
done
1849
UNSAT.
309		OPT
428		SAT
212		SAT (subopt.)
546		?
6		? (TO)
322		? (MO)
22		? (SIG)
4
sat4jPseudoCP
2007-03-23
done
2251
UNSAT.
203		OPT
285		SAT
73		SAT (subopt.)
2		SAT (TO)
611		?
223		? (TO)
852		? (MO)
1		? (EC)
1
sat4jPseudoCPClause
2007-03-23
done
2251
UNSAT.
207		OPT
278		SAT
75		SAT (subopt.)
1		SAT (TO)
639		?
223		? (TO)
826		? (EC)
2
SAT4JPseudoResolution
2007-03-23
done
2251
UNSAT.
299		OPT
450		SAT
151		SAT (subopt.)
7		SAT (TO)
549		?
223		? (TO)
552		? (MO)
6		? (EC)
11		NO CERT.
3
wildcat-rnp
2007-03-21
done
1849
OPT
1		SAT
203		SAT (TO)
589		?
12		? (TO)
969		? (MO)
13		? (?)
61		NO CERT.
1
wildcat-skc
2007-03-21
done
1849
OPT
1		SAT
212		SAT (subopt.)
3		SAT (TO)
597		?
76		? (TO)
884		? (MO)
13		? (?)
61		NO CERT.
2

Some explanations