2018 XCSP3 competition: mini-solvers track: solvers page (tabular format, detailed results)

Some explanations

• #Benchs: Number of benchs for this solver
• Solved: Total number of solved instances (this is the sum of the SAT and UNSAT columns)
• UNSAT.: solver proved unsatitisfiability ("s UNSATISFIABLE" was output)
• OPT: solver found an optimum solution ("s OPTIMUM FOUND" was output), the provided implicant satisfies each constraint and no solver gave a better solution
• SAT: solver found a solution ("s SATISFIABLE" was output) of a SAT/UNSAT intances and the provided implicant satisfies each constraint
• SAT (subopt.): solver found a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint but this may not be the optimal solution
• SAT (timeout): solver exceeded the time limit but was able to find a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint
• SAT (out of mem.): solver exceeded the memory limit but was able to find a solution ("s SATISFIABLE" was output) and the provided implicant satisfies each constraint
• UNKNOWN: solver couldn't decide ("s UNKNOWN" was output)
• UNKNOWN (no support): Solver does not handle one of the constraint
• UNKNOWN (timeout): solver exceeded the time limit and gave no answer. These runs are considered to give result UNKNOWN
• UNKNOWN (out of mem.): solver exceeded the memory limit and gave no answer. These runs are considered to give result UNKNOWN.
• UNKNOWN (exit code): solver didn't output a solution line and terminated with an unexpected exit code. These runs are considered to give result UNKNOWN.
• UNKNOWN (conv. script): The conversion script failed to convert the instance. These runs are considered to give result UNKNOWN.
• UNKNOWN (problem): No solution line was correctly recognized. These runs are considered to give result UNKNOWN.
• Sig. Caught: solver was terminated by a signal (SIGSEGV for example) and didn't output a solution line
• NO CERT.: solver answered SATISFIABLE but either didn't provide a certificate (the "v " line) or either gave a truncated certificate (which doesn't end in a new line)
• WRONG CERT.: solver gave an implicant but it appears that it doesn't satisfy every constraint
• WRONG OPT.: solver found an optimum solution ("s OPTIMUM FOUND" was output) but there exists an implicant which gives a better value to the objective function
• WRONG UNSAT.: solver proved unsatisfiability but was wrong ("s UNSATISFIABLE" was output)
• WRONG ANS.: solver gave an inconsistent answer (e.g. OPTIMUM FOUND on a decision problem)

• solver

  a "light red" background denotes a solver which was disqualified because it gave an incorrect answer, and which will not appear in the ranking

• solver

  a "light yellow" background denotes a solver which was replaced by another version, and which will not appear in the ranking

Category "optimization problem" (COP)

Category "decision problem" (CSP)

All categories for solvers registered in all categories

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.