SAT 2007 Competition AIG special track: solvers page (tabular format, detailed results)

Category "all instances" (ALL)

Solver Name	#benchs	Solved	UNSAT.	SAT.	UNKNOWN (timeout)	UNKNOWN (out of mem.)	UNKNOWN (problem)	WRONG CERT.	WRONG UNSAT.
aig-cmusat 2007-05-15	341	233 (68.33%)	185 (54.25%)	48 (14.08%)	98 (28.74%)	3 (0.88%)	6 (1.76%)	0 (0%)	1 (0.29%)
aigsat-sp 2007-05-15	341	196 (57.48%)	160 (46.92%)	36 (10.56%)	103 (30.21%)	33 (9.68%)	8 (2.35%)	1 (0.29%)	0 (0%)
aigsat-wc 2007-05-15	341	204 (59.82%)	164 (48.09%)	40 (11.73%)	101 (29.62%)	28 (8.21%)	7 (2.05%)	1 (0.29%)	0 (0%)
minicirc 2007-05-17	341	208 (61%)	169 (49.56%)	39 (11.44%)	93 (27.27%)	0 (0%)	39 (11.44%)	0 (0%)	1 (0.29%)
picoaigersat 2007-05-15	341	228 (66.86%)	175 (51.32%)	53 (15.54%)	106 (31.09%)	0 (0%)	6 (1.76%)	0 (0%)	1 (0.29%)

Solver Name

Progress

aig-cmusat
2007-05-15

done
341

UNSAT.
185

SAT.
48

? (TO)
98

? (MO)
3

? (?)
6

WRONG UNSAT.
1

aigsat-sp
2007-05-15

done
341

UNSAT.
160

SAT.
36

? (TO)
103

? (MO)
33

? (?)
8

ERR CERT
1

aigsat-wc
2007-05-15

done
341

UNSAT.
164

SAT.
40

? (TO)
101

? (MO)
28

? (?)
7

ERR CERT
1

minicirc
2007-05-17

done
341

UNSAT.
169

SAT.
39

? (TO)
93

? (?)
39

WRONG UNSAT.
1

picoaigersat
2007-05-15

done
341

UNSAT.
175

SAT.
53

? (TO)
106

? (?)
6

WRONG UNSAT.
1

Some explanations

#Benchs: Number of benchs for this solver
Solved: Total number of solved instances (this is the sum of the SAT and UNSAT columns)
UNSAT.: solver proved unsatitisfiability ("s UNSATISFIABLE" was output)
SAT.: solver found a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint
SAT (timeout): solver exceeded the time limit but was able to find a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint
UNKNOWN: solver couldn't decide ("s UNKNOWN" was output)
UNKNOWN (timeout): solver exceeded the time limit and gave no answer. These runs are considered to give result UNKNOWN
UNKNOWN (out of mem.): solver exceeded the memory limit and gave no answer. These runs are considered to give result UNKNOWN.
UNKNOWN (exit code): solver didn't output a solution line and terminated with an unexpected exit code. These runs are considered to give result UNKNOWN.
UNKNOWN (conv. script): The conversion script failed to convert the instance. These runs are considered to give result UNKNOWN.
UNKNOWN (problem): No solution line was correctly recognized. These runs are considered to give result UNKNOWN.
Sig. Caught: solver was terminated by a signal (SIGSEGV for example) and didn't output a solution line
NO CERT.: solver answered SATISFIABLE but either didn't provide a certificate (the "v " line) or either gave a truncated certificate (which doesn't end in a new line)
WRONG CERT.: solver gave an implicant but it appears that it doesn't satisfy every constraint
WRONG UNSAT.: solver proved unsatisfiability but was wrong ("s UNSATISFIABLE" was output)
Misc: This is a internal check. It counts the runs that don't fall into one of the previous category. Should be 0 if categories are exhaustive and mutually exclusive. Will be negative if categories are not mutually exclusive and positive if categories are not exhaustive.