2017 XCSP3 competition: fast COP track (sequential and parallel solvers): solvers page (tabular format, detailed results)

Some explanations

• #Benchs: Number of benchs for this solver
• Solved: Total number of solved instances (this is the sum of the SAT and UNSAT columns)
• UNSAT.: solver proved unsatitisfiability ("s UNSATISFIABLE" was output)
• OPT: solver found an optimum solution ("s OPTIMUM FOUND" was output), the provided implicant satisfies each constraint and no solver gave a better solution
• SAT: solver found a solution ("s SATISFIABLE" was output) of a SAT/UNSAT intances and the provided implicant satisfies each constraint
• SAT (subopt.): solver found a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint but this may not be the optimal solution
• SAT (timeout): solver exceeded the time limit but was able to find a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint
• SAT (out of mem.): solver exceeded the memory limit but was able to find a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint
• UNKNOWN: solver couldn't decide ("s UNKNOWN" was output)
• UNKNOWN (no support): Solver does not handle one of the constraint
• UNKNOWN (timeout): solver exceeded the time limit and gave no answer. These runs are considered to give result UNKNOWN
• UNKNOWN (out of mem.): solver exceeded the memory limit and gave no answer. These runs are considered to give result UNKNOWN.
• UNKNOWN (exit code): solver didn't output a solution line and terminated with an unexpected exit code. These runs are considered to give result UNKNOWN.
• UNKNOWN (conv. script): The conversion script failed to convert the instance. These runs are considered to give result UNKNOWN.
• UNKNOWN (problem): No solution line was correctly recognized. These runs are considered to give result UNKNOWN.
• Sig. Caught: solver was terminated by a signal (SIGSEGV for example) and didn't output a solution line
• NO CERT.: solver answered SATISFIABLE but either didn't provide a certificate (the "v " line) or either gave a truncated certificate (which doesn't end in a new line)
• WRONG CERT.: solver gave an implicant but it appears that it doesn't satisfy every constraint
• WRONG OPT.: solver found an optimum solution ("s OPTIMUM FOUND" was output) but there exists an implicant which gives a better value to the objective function
• WRONG UNSAT.: solver proved unsatisfiability but was wrong ("s UNSATISFIABLE" was output)
• WRONG ANS.: solver gave an inconsistent answer (e.g. OPTIMUM FOUND on a decision problem)

• solver

  a "light red" background denotes a solver which was disqualified because it gave an incorrect answer, and which will not appear in the ranking

• solver

  a "light yellow" background denotes a solver which was replaced by another version, and which will not appear in the ranking

Category "optimization problem" (COP)