2018 XCSP3 competition: mini-solvers track: results by benchmark

Results by benchmark for category decision problem (CSP)

This page displays the results of the different solvers for each benchmark for category decision problem (CSP)

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
cosoco 1.12 (complete)
GG's minicp 2018-04-29 (complete)
miniBTD 2018.07.27_3 (complete)
miniBTD_12 2018.07.27_12 (complete)
MiniCPFever 2018-04-29 (complete)
minimacht 2018.07.27 (complete)
NACRE 1.0.4 (complete)
slowpoke 2018-04-29 (incomplete)
Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)
SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)
The dodo solver 2018-04-29 (complete)

Bench name	Best results on this instance	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
ColouredQueens/ ColouredQueens-05_c18.xml	SAT TT=0.004	SAT TT=0.005	SAT TT=0.506	SAT TT=0.004	SAT TT=0.004	SAT TT=0.513	SAT TT=0.005	SAT TT=0.004	SAT TT=0.53	SAT TT=0.515	SAT TT=0.528	SAT (TO) TT=2520.1
ColouredQueens/ ColouredQueens-06_c18.xml	UNSAT TT=0.666	UNSAT TT=2.024	UNSAT TT=1.64	UNSAT TT=0.666	UNSAT TT=0.951	UNSAT TT=1.408	UNSAT TT=0.878	UNSAT TT=7.156	? (TO) TT=2520.05	UNSAT TT=1.936	? TT=304.191	UNSAT TT=2.268
ColouredQueens/ ColouredQueens-07_c18.xml	SAT TT=0.004	SAT TT=0.065	SAT TT=0.619	SAT TT=0.014	SAT TT=0.005	SAT TT=0.532	SAT TT=0.005	SAT TT=0.004	SAT TT=0.563	SAT TT=0.551	SAT TT=0.562	SAT (TO) TT=2520.09
ColouredQueens/ ColouredQueens-08_c18.xml		? (TO) TT=2520.07	? (TO) TT=2520.05	? TT=2520.01	? (TO) TT=2519.89	? (TO) TT=2520.1	? (TO) TT=2519.87	? (TO) TT=2519.89	? (TO) TT=2520.07	? (TO) TT=2520.02	? (TO) TT=2520.11	? (TO) TT=2520.03
ColouredQueens/ ColouredQueens-09_c18.xml		? (TO) TT=2520.08	? (TO) TT=2520.11	? TT=2520.01	? (TO) TT=2519.91	? (TO) TT=2520.02	? (TO) TT=2519.9	? (TO) TT=2519.87	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2520.02	? (TO) TT=2520.04
ColouredQueens/ ColouredQueens-10_c18.xml		? (TO) TT=2520.01	? (TO) TT=2520.09	? (TO) TT=2519.71	? TT=2520	? (TO) TT=2520.07	? (TO) TT=2519.98	? (TO) TT=2519.81	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.08
ColouredQueens/ ColouredQueens-11_c18.xml		? (TO) TT=2519.83	? (TO) TT=2520.08	? (TO) TT=2519.93	? (TO) TT=2519.78	? (TO) TT=2520.11	? (TO) TT=2519.47	? (TO) TT=2519.88	? (TO) TT=2520.07	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.12
ColouredQueens/ ColouredQueens-12_c18.xml		? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2519.66	? TT=2520	? (TO) TT=2520.1	? TT=2520.01	? (TO) TT=2519.8	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.08	? (TO) TT=2520.04
ColouredQueens/ ColouredQueens-13_c18.xml		? (TO) TT=2519.87	? (TO) TT=2520.1	? (TO) TT=2519.96	? (TO) TT=2519.91	? (TO) TT=2520.02	? (TO) TT=2519.96	? (TO) TT=2519.73	? (TO) TT=2520.1	? (TO) TT=2520.03	? (TO) TT=2520.04	? (TO) TT=2520.08
ColouredQueens/ ColouredQueens-14_c18.xml		? (TO) TT=2520.06	? (TO) TT=2520.06	? TT=2520	? TT=2520	? (TO) TT=2520.03	? (TO) TT=2519.94	? (TO) TT=2519.86	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.04
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
ColouredQueens/ ColouredQueens-15_c18.xml		? (TO) TT=2519.59	? (TO) TT=2520.03	? TT=2520	? (TO) TT=2519.82	? (TO) TT=2520.12	? (TO) TT=2519.86	? (TO) TT=2519.53	? (TO) TT=2520.12	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2520.1
ColouredQueens/ ColouredQueens-16_c18.xml		? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2519.94	? TT=2520.01	? (TO) TT=2520.03	? (TO) TT=2519.89	? (TO) TT=2519.58	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.11
Crossword/ Crossword-m18-ogd2008-vg-04-05_c18.xml	SAT TT=0.044	SAT TT=0.139	SAT TT=1.457	SAT TT=0.045	SAT TT=0.044	SAT TT=1.367	SAT TT=0.044	SAT TT=0.101	SAT TT=1.38	SAT TT=1.438	SAT TT=1.398	SAT (TO) TT=2520.04
Crossword/ Crossword-m18-ogd2008-vg-05-06_c18.xml	SAT TT=0.12	SAT TT=0.355	SAT TT=2.012	SAT TT=0.121	SAT TT=0.12	SAT TT=1.83	SAT TT=0.122	SAT TT=0.303	SAT TT=1.785	SAT TT=1.918	SAT TT=1.916	SAT (TO) TT=2520.12
Crossword/ Crossword-m18-ogd2008-vg-06-07_c18.xml	SAT TT=0.27	SAT TT=1.156	SAT TT=3.785	SAT TT=0.279	SAT TT=0.27	SAT TT=3.262	SAT TT=0.506	SAT TT=3.472	SAT TT=5.098	SAT TT=7.35	SAT TT=4.693	SAT (TO) TT=2520.04
Crossword/ Crossword-m18-ogd2008-vg-07-08_c18.xml	SAT TT=41.907	SAT TT=937.693	SAT TT=1152.42	SAT TT=41.907	SAT TT=721.102	SAT TT=2358.89	SAT TT=597.275	SAT TT=1913.96	? (TO) TT=2520.09	? (TO) TT=2520.03	? (TO) TT=2520.03	SAT (TO) TT=2520.05
Crossword/ Crossword-m18-ogd2008-vg-08-09_c18.xml		? (TO) TT=2519.94	? (TO) TT=2520.05	? (TO) TT=2519.89	? (TO) TT=2519.89	? (TO) TT=2520.11	? (TO) TT=2519.93	? (TO) TT=2519.89	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2520.03
Crossword/ Crossword-m18-ogd2008-vg-09-10_c18.xml		? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2519.77	? (TO) TT=2520.06	? (TO) TT=2519.91	? (TO) TT=2519.92	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.07
Crossword/ Crossword-m18-ogd2008-vg-10-11_c18.xml		? (TO) TT=2519.76	? (TO) TT=2520.06	? (TO) TT=2520	? (TO) TT=2519.64	? (TO) TT=2520.09	? (TO) TT=2519.87	? (TO) TT=2519.86	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2520.05
Crossword/ Crossword-m18-ogd2008-vg-11-12_c18.xml		? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.02	? (TO) TT=2519.98	? (TO) TT=2520.03	? (TO) TT=2520.11	? (TO) TT=2519.94	? (TO) TT=2520.09	? (TO) TT=2520.04	? (TO) TT=2520.12	? (TO) TT=2520.12
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Crossword/ Crossword-m18-ogd2008-vg-12-13_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520.11	? (TO) TT=2520.04	? (TO) TT=2519.92	? (TO) TT=2520.09	? (TO) TT=2519.97	? (TO) TT=2519.83	? (TO) TT=2520.04	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.1
Crossword/ Crossword-m18-ogd2008-vg-13-14_c18.xml		? (TO) TT=2519.97	? (TO) TT=2520.1	? (TO) TT=2519.87	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2519.9	? (TO) TT=2519.93	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2520.08
Crossword/ Crossword-m18-ogd2008-vg-14-15_c18.xml		? (TO) TT=2519.96	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2520.01	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2519.9	? (TO) TT=2520.05	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.1
Crossword/ Crossword-m18-ogd2008-vg-15-16_c18.xml		? (TO) TT=2519.96	? (TO) TT=2520.06	? (TO) TT=2520.03	? (TO) TT=2519.96	? (TO) TT=2520.11	? (TO) TT=2519.9	? (TO) TT=2519.76	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.1
Crossword/ Crossword-m18-ogd2008-vg-16-17_c18.xml	UNSAT TT=369.771	? (TO) TT=2519.93	? (TO) TT=2520.05	UNSAT TT=369.771	UNSAT TT=640.896	? (TO) TT=2520.12	UNSAT TT=762.414	UNSAT TT=2188.27	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2520.12
Dubois/ Dubois-018_c18.xml	UNSAT TT=0.006	UNSAT TT=20.636	UNSAT TT=5.659	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=6.9	UNSAT TT=18.836	UNSAT TT=0.006	? (TO) TT=2520.06	UNSAT TT=6.776	? TT=308.751	UNSAT TT=7.375
Dubois/ Dubois-020_c18.xml	UNSAT TT=0.006	UNSAT TT=82.741	UNSAT TT=18.359	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=23.577	UNSAT TT=53.534	UNSAT TT=0.006	? (TO) TT=2520.02	UNSAT TT=23.816	? TT=315.995	UNSAT TT=23.882
Dubois/ Dubois-021_c18.xml	UNSAT TT=0.006	UNSAT TT=178.399	UNSAT TT=34.811	UNSAT TT=0.006	UNSAT TT=0.007	UNSAT TT=45.457	UNSAT TT=66.078	UNSAT TT=0.006	? (TO) TT=2520.13	UNSAT TT=45.359	? TT=343.655	UNSAT TT=45.922
Dubois/ Dubois-022_c18.xml	UNSAT TT=0.006	UNSAT TT=385.845	UNSAT TT=69.543	UNSAT TT=0.006	UNSAT TT=0.007	UNSAT TT=88.836	UNSAT TT=140.016	UNSAT TT=0.007	? (TO) TT=2520.02	UNSAT TT=88.015	? TT=377.445	UNSAT TT=92.219
Dubois/ Dubois-023_c18.xml	UNSAT TT=0.007	UNSAT TT=642.665	UNSAT TT=137.666	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=177.563	UNSAT TT=325.883	UNSAT TT=0.007	? (TO) TT=2520.04	UNSAT TT=174.404	? TT=380.009	UNSAT TT=189.366
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Dubois/ Dubois-024_c18.xml	UNSAT TT=0.007	UNSAT TT=1137.12	UNSAT TT=277.05	UNSAT TT=0.007	UNSAT TT=0.008	UNSAT TT=367.049	UNSAT TT=812.513	UNSAT TT=0.007	? (TO) TT=2520.05	UNSAT TT=365.755	? TT=393.152	UNSAT TT=384.317
Dubois/ Dubois-025_c18.xml	UNSAT TT=0.007	? (TO) TT=2520.09	UNSAT TT=560.109	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=729.84	UNSAT TT=1004.9	UNSAT TT=0.007	? (TO) TT=2520.03	UNSAT TT=734.976	? TT=800.706	UNSAT TT=771.319
Dubois/ Dubois-030_c18.xml	UNSAT TT=0.008	? (TO) TT=2520.04	? (TO) TT=2520.1	UNSAT TT=0.008	UNSAT TT=0.008	? (TO) TT=2520.04	? TT=2520	UNSAT TT=0.009	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.09
Dubois/ Dubois-040_c18.xml	UNSAT TT=0.01	? (TO) TT=2520	? (TO) TT=2520.1	UNSAT TT=0.01	UNSAT TT=0.01	? (TO) TT=2520.03	? TT=2520.01	UNSAT TT=0.014	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.03
Dubois/ Dubois-050_c18.xml	UNSAT TT=0.012	? (TO) TT=2520.1	? (TO) TT=2520.06	UNSAT TT=0.012	UNSAT TT=0.012	? (TO) TT=2520.04	? (TO) TT=2519.74	UNSAT TT=0.019	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.05
Dubois/ Dubois-075_c18.xml	UNSAT TT=0.018	? (TO) TT=2520.07	? (TO) TT=2520.09	UNSAT TT=0.018	UNSAT TT=0.019	? (TO) TT=2520.04	? (TO) TT=2519.93	UNSAT TT=0.034	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.04
Dubois/ Dubois-100_c18.xml	UNSAT TT=0.025	? (TO) TT=2520.08	? (TO) TT=2520.06	UNSAT TT=0.025	UNSAT TT=0.026	? (TO) TT=2520.06	? (TO) TT=2519.98	UNSAT TT=0.056	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2520.04
Eternity/ Eternity-04-04_c18.xml	SAT TT=0.007	SAT TT=0.013	SAT TT=0.818	SAT TT=0.009	SAT TT=0.007	SAT TT=0.822	SAT TT=0.008	SAT TT=0.007	SAT TT=0.834	SAT TT=0.839	SAT TT=0.832	SAT (TO) TT=2520.03
Eternity/ Eternity-05-05_c18.xml	SAT TT=0.053	SAT TT=0.14	SAT TT=1.014	SAT TT=0.091	SAT TT=0.122	SAT TT=1.072	SAT TT=0.136	SAT TT=0.053	SAT TT=1.021	SAT TT=1.104	SAT TT=1.05	SAT (TO) TT=2520.1
Eternity/ Eternity-06-06_c18.xml	SAT TT=0.729	SAT TT=1.034	SAT TT=2.166	SAT TT=4.375	SAT TT=0.729	SAT TT=2.704	SAT TT=4.434	SAT TT=1.059	SAT TT=5.85	SAT TT=3.051	SAT TT=2.132	SAT (TO) TT=2520.1
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Eternity/ Eternity-07-07_c18.xml	SAT TT=11.694	SAT TT=13.033	SAT TT=64.035	SAT TT=454.314	SAT TT=11.694	SAT TT=55.954	SAT TT=613.513	SAT TT=375.479	SAT TT=171.361	SAT TT=171.254	SAT TT=101.709	? (TO) TT=2520.03
Eternity/ Eternity-08-08_c18.xml		? (TO) TT=2519.98	? (TO) TT=2520.09	? TT=2520.02	? (TO) TT=2519.93	? (TO) TT=2520.12	? (TO) TT=2519.9	? (TO) TT=2519.78	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.11	? (TO) TT=2520.08
Eternity/ Eternity-09-09_c18.xml		? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2519.92	? TT=2520.04	? (TO) TT=2520.1	? TT=2520.04	? (TO) TT=2519.63	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.09
Eternity/ Eternity-10-04_c18.xml	SAT TT=1.069	SAT TT=2.079	SAT TT=4.523	SAT TT=2.387	SAT TT=4.69	SAT TT=1.504	SAT TT=1.92	SAT TT=1.069	SAT TT=6.013	SAT TT=1.773	SAT TT=1.639	SAT (TO) TT=2520.08
Eternity/ Eternity-10-05_c18.xml	SAT TT=4.849	SAT TT=24.843	SAT TT=269.276	SAT TT=1818.88	SAT TT=43.692	SAT TT=78.848	SAT TT=4.849	SAT TT=20.353	SAT TT=13.031	SAT TT=230.278	SAT TT=130.934	SAT (TO) TT=2520.05
Eternity/ Eternity-10-06_c18.xml	SAT TT=1010.43	? (TO) TT=2519.69	? (TO) TT=2520.1	? TT=2520.03	? (TO) TT=2519.74	SAT TT=1010.43	? TT=2520.02	? (TO) TT=2519.66	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.11
Eternity/ Eternity-10-10_c18.xml		? (TO) TT=2519.94	? (TO) TT=2520.05	? TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.1	? TT=2520.06	? (TO) TT=2519.4	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.03
Eternity/ Eternity-11-11_c18.xml		? (TO) TT=2519.8	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2519.98	? (TO) TT=2520.12	? (TO) TT=2519.94	? (TO) TT=2519.69	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.11
Eternity/ Eternity-12-12_c18.xml		? (TO) TT=2519.73	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2519.73	? (TO) TT=2520.11	? (TO) TT=2519.92	? (TO) TT=2519.36	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2520.07
Eternity/ Eternity-13-13_c18.xml		? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2519.91	? (TO) TT=2520.01	? (TO) TT=2520.04	? (TO) TT=2519.99	? (TO) TT=2519.72	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.07
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Eternity/ Eternity-14-14_c18.xml		? (TO) TT=2519.8	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2519.79	? (TO) TT=2520.11	? (TO) TT=2519.99	? (TO) TT=2519.71	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.1
Eternity/ Eternity-15-15_c18.xml		? (TO) TT=2519.92	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2520.05	? (TO) TT=2519.98	? (TO) TT=2519.63	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2520.06
Frb/ frb-30-15-1_c18.xml	SAT TT=0.112	SAT TT=0.159	SAT TT=5.268	SAT TT=0.36	SAT TT=0.885	SAT TT=7.307	SAT TT=0.835	SAT TT=0.112	SAT TT=13.931	SAT TT=8.791	SAT TT=25.849	SAT (TO) TT=2520.11
Frb/ frb-35-17-1_c18.xml	SAT TT=2.36	SAT TT=2.36	SAT TT=25.327	SAT TT=7.118	SAT TT=12.001	SAT TT=44.476	SAT TT=12.778	SAT TT=2.726	SAT TT=175.82	SAT TT=70.713	SAT TT=101.775	SAT (TO) TT=2520.02
Frb/ frb-40-19-1-mgd_c18.xml	SAT TT=5.218	SAT TT=5.218	SAT TT=135.655	SAT TT=26.676	SAT TT=49.885	SAT TT=229.597	SAT TT=69.511	SAT TT=9.367	SAT TT=507.914	SAT TT=421.009	SAT TT=1239.61	? (TO) TT=2520.11
Frb/ frb-45-21-1_c18.xml	SAT TT=1880.58	SAT TT=1880.58	? (TO) TT=2520.12	? (TO) TT=2520.03	? (TO) TT=2519.99	? (TO) TT=2520.06	? (TO) TT=2519.99	? (TO) TT=2519.9	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.06
Frb/ frb-50-23-1-mgd_c18.xml		? (TO) TT=2519.83	? (TO) TT=2520.06	? TT=2520.05	? TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2519.81	? (TO) TT=2519.85	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.03
Frb/ frb-50-23-2-mgd_c18.xml		? (TO) TT=2519.94	? (TO) TT=2520.05	? TT=2520.04	? TT=2520.04	? (TO) TT=2520.1	? TT=2520.05	? (TO) TT=2519.8	? (TO) TT=2520.1	? (TO) TT=2520.02	? (TO) TT=2520.1	? (TO) TT=2520.04
Frb/ frb-50-23-3-mgd_c18.xml		? (TO) TT=2519.96	? (TO) TT=2520.03	? TT=2520.05	? (TO) TT=2519.75	? (TO) TT=2520.11	? (TO) TT=2520.02	? (TO) TT=2519.81	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.04
Frb/ frb-53-24-1-mgd_c18.xml		? (TO) TT=2519.82	? (TO) TT=2520.05	? TT=2520.05	? (TO) TT=2519.91	? (TO) TT=2520.12	? (TO) TT=2519.93	? (TO) TT=2519.81	? (TO) TT=2520.03	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2520.12
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Frb/ frb-53-24-2-mgd_c18.xml		? (TO) TT=2519.84	? (TO) TT=2520.1	? TT=2520.06	? (TO) TT=2519.83	? (TO) TT=2520.08	? (TO) TT=2519.9	? (TO) TT=2519.96	? (TO) TT=2520.03	? (TO) TT=2520.03	? (TO) TT=2520.11	? (TO) TT=2520.03
Frb/ frb-53-24-3-mgd_c18.xml		? (TO) TT=2519.96	? (TO) TT=2520.04	? TT=2520.06	? TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2519.79	? (TO) TT=2519.86	? (TO) TT=2520.11	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2520.02
Frb/ frb-56-25-1-mgd_c18.xml		? (TO) TT=2520.11	? (TO) TT=2520.01	? (TO) TT=2520.04	? (TO) TT=2519.98	? (TO) TT=2520.04	? (TO) TT=2519.95	? (TO) TT=2519.83	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.07
Frb/ frb-56-25-2-mgd_c18.xml		? (TO) TT=2519.85	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2519.82	? (TO) TT=2520.06	? (TO) TT=2519.93	? (TO) TT=2519.93	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2520.09
Frb/ frb-56-25-3-mgd_c18.xml		? (TO) TT=2520.08	? (TO) TT=2520.02	? TT=2520.07	? (TO) TT=2519.91	? (TO) TT=2520.04	? (TO) TT=2519.93	? (TO) TT=2519.82	? (TO) TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.11
Frb/ frb-59-26-1-mgd_c18.xml		? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2520.01	? (TO) TT=2520.03	? (TO) TT=2519.98	? (TO) TT=2519.93	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.03	? (TO) TT=2520.05
Frb/ frb-59-26-2-mgd_c18.xml		? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.06	? TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2519.99	? (TO) TT=2519.83	? (TO) TT=2520.06	? (TO) TT=2520.09	? (TO) TT=2520.04	? (TO) TT=2520.08
Frb/ frb-59-26-3-mgd_c18.xml		? (TO) TT=2519.98	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.04	? TT=2520.07	? (TO) TT=2519.75	? (TO) TT=2520.09	? (TO) TT=2520.02	? (TO) TT=2520.09	? (TO) TT=2520.11
GracefulGraph/ GracefulGraph-K02-P09_c18.xml	SAT TT=0.021	? (TO) TT=2520.08	? (TO) TT=2520.03	SAT TT=1.17	SAT TT=0.021	? (TO) TT=2520.09	SAT TT=0.021	SAT TT=5.262	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.02	SAT (TO) TT=2520.07
GracefulGraph/ GracefulGraph-K03-P03_c18.xml	SAT TT=0.009	SAT TT=0.575	SAT TT=0.765	SAT TT=0.014	SAT TT=0.009	? (TO) TT=2520.04	SAT TT=0.009	SAT TT=0.217	SAT TT=0.792	? (TO) TT=2520.05	? (TO) TT=2520.05	SAT (TO) TT=2520.1
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
GracefulGraph/ GracefulGraph-K04-P02_c18.xml	SAT TT=0.027	SAT TT=2.54	SAT TT=7.569	SAT TT=0.4	SAT TT=0.06	? (TO) TT=2520.06	SAT TT=0.027	SAT TT=0.527	SAT TT=1.452	? (TO) TT=2520.04	? (TO) TT=2520.02	SAT (TO) TT=2520.1
GracefulGraph/ GracefulGraph-K04-P03_c18.xml	SAT TT=0.809	SAT TT=1322.77	? (TO) TT=2520.08	SAT TT=438.893	SAT TT=5.399	? (TO) TT=2520.02	SAT TT=0.809	SAT TT=179.596	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.04	SAT (TO) TT=2520.02
GracefulGraph/ GracefulGraph-K04-P04_c18.xml	SAT TT=54.16	? (TO) TT=2520.08	? (TO) TT=2520.06	SAT TT=2104.75	SAT TT=425.388	? (TO) TT=2520.04	SAT TT=54.16	? (TO) TT=2519.85	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2520.09
GracefulGraph/ GracefulGraph-K05-P02_c18.xml	SAT TT=68.119	? (TO) TT=2519.83	? (TO) TT=2520.05	SAT TT=451.632	SAT TT=86.302	? (TO) TT=2520.07	SAT TT=354.745	? (TO) TT=2519.88	SAT TT=68.119	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.06
GracefulGraph/ GracefulGraph-K05-P03_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520.04	? TT=2520	? TT=2520.01	? (TO) TT=2520.05	? (TO) TT=2519.87	? (TO) TT=2519.84	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.09
GracefulGraph/ GracefulGraph-K06-P02_c18.xml		? (TO) TT=2520.07	? (TO) TT=2520.11	? TT=2520	? (TO) TT=2519.97	? (TO) TT=2520.12	? (TO) TT=2519.96	? (TO) TT=2519.9	? (TO) TT=2520.03	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.04
GracefulGraph/ GracefulGraph-K06-P03_c18.xml		? (TO) TT=2520.02	? (TO) TT=2520.02	? TT=2520.02	? TT=2520.02	? (TO) TT=2520.1	? (TO) TT=2519.96	? (TO) TT=2519.94	? (TO) TT=2520.12	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.02
GracefulGraph/ GracefulGraph-K07-P02_c18.xml		? (TO) TT=2519.9	? (TO) TT=2520.06	? (TO) TT=2519.95	? TT=2520.02	? (TO) TT=2520.05	? (TO) TT=2519.79	? (TO) TT=2519.73	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2520.12	? (TO) TT=2520.06
GracefulGraph/ GracefulGraph-K07-P03_c18.xml		? (TO) TT=2520.07	? (TO) TT=2520.03	? (TO) TT=2519.95	? (TO) TT=2519.73	? (TO) TT=2520.04	? TT=2520.02	? (TO) TT=2519.92	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2520.01
Haystacks/ Haystacks-ext-04_c18.xml	UNSAT TT=0.004	UNSAT TT=0.005	UNSAT TT=0.564	UNSAT TT=0.004	UNSAT TT=0.004	UNSAT TT=0.629	UNSAT TT=0.007	UNSAT TT=0.004	? (TO) TT=2520.03	UNSAT TT=0.6	? TT=304.654	UNSAT TT=0.586
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Haystacks/ Haystacks-ext-05_c18.xml	UNSAT TT=0.005	UNSAT TT=3.648	UNSAT TT=1.526	UNSAT TT=0.006	UNSAT TT=0.005	UNSAT TT=1.698	UNSAT TT=0.671	UNSAT TT=0.006	? (TO) TT=2520.03	UNSAT TT=1.459	? TT=305.349	UNSAT TT=3.829
Haystacks/ Haystacks-ext-06_c18.xml	UNSAT TT=0.009	? (TO) TT=2519.84	UNSAT TT=12.736	UNSAT TT=0.009	UNSAT TT=0.012	UNSAT TT=14.502	? (TO) TT=2519.93	UNSAT TT=0.01	? (TO) TT=2520.09	UNSAT TT=16.718	? (TO) TT=2520.1	UNSAT TT=686.646
Haystacks/ Haystacks-ext-07_c18.xml	UNSAT TT=0.024	? (TO) TT=2519.98	? (TO) TT=2520.12	UNSAT TT=0.036	UNSAT TT=0.038	? (TO) TT=2520.11	? TT=2520.01	UNSAT TT=0.024	? (TO) TT=2520.04	? (TO) TT=2520.01	? (TO) TT=2520.07	? (TO) TT=2520.04
Haystacks/ Haystacks-ext-08_c18.xml	UNSAT TT=0.125	? (TO) TT=2519.88	? (TO) TT=2520.02	UNSAT TT=0.229	UNSAT TT=0.232	? (TO) TT=2520.04	? (TO) TT=2519.93	UNSAT TT=0.125	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.11
Haystacks/ Haystacks-ext-09_c18.xml	UNSAT TT=1.894	? (TO) TT=2519.95	? (TO) TT=2520.05	UNSAT TT=1.894	UNSAT TT=1.995	? (TO) TT=2520.09	? (TO) TT=2519.93	UNSAT TT=2.808	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.08
Haystacks/ Haystacks-ext-10_c18.xml	UNSAT TT=17.759	? (TO) TT=2520	? (TO) TT=2520.12	UNSAT TT=17.759	UNSAT TT=19.208	? (TO) TT=2520.02	? (TO) TT=2519.97	UNSAT TT=543.859	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.06
Haystacks/ Haystacks-ext-11_c18.xml	UNSAT TT=197.639	? (TO) TT=2519.89	? (TO) TT=2520.12	UNSAT TT=197.639	UNSAT TT=207.7	? (TO) TT=2520.08	? TT=2520.01	? (TO) TT=2519.96	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.05
Haystacks/ Haystacks-ext-12_c18.xml	UNSAT TT=2367.8	? (TO) TT=2519.98	? (TO) TT=2520.11	UNSAT TT=2367.8	? (TO) TT=2519.84	? (TO) TT=2520.03	? TT=2520.02	? (TO) TT=2519.96	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2520.11
Haystacks/ Haystacks-ext-13_c18.xml		? (TO) TT=2519.96	? (TO) TT=2520.09	? TT=2520.01	? (TO) TT=2520.02	? (TO) TT=2520.02	? (TO) TT=2519.8	? (TO) TT=2519.99	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.04	? (TO) TT=2520.09
Langford/ LangfordBin-06_c18.xml	UNSAT TT=0.204	UNSAT TT=1.044	UNSAT TT=0.667	UNSAT TT=0.237	UNSAT TT=0.223	UNSAT TT=0.776	UNSAT TT=0.204	UNSAT TT=0.536	? (TO) TT=2520.09	UNSAT TT=0.536	Wrong Cert. TT=0.607	UNSAT TT=0.663
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Langford/ LangfordBin-07_c18.xml	SAT TT=0.005	SAT TT=0.007	SAT TT=0.595	SAT TT=0.161	SAT TT=0.159	SAT TT=0.604	SAT TT=0.133	SAT TT=0.005	SAT TT=0.625	Wrong UNSAT TT=0.546	SAT TT=0.587	SAT (TO) TT=2520.12
Langford/ LangfordBin-08_c18.xml	SAT TT=0.006	SAT TT=0.008	SAT TT=0.618	SAT TT=3.323	SAT TT=2.589	SAT TT=0.659	SAT TT=2.949	SAT TT=0.006	SAT TT=0.701	Wrong UNSAT TT=0.549	SAT TT=0.609	SAT (TO) TT=2520.1
Langford/ LangfordBin-09_c18.xml	UNSAT TT=0.558	? (TO) TT=2520.07	UNSAT TT=42.091	? (TO) TT=2519.96	? TT=2520	UNSAT TT=6.09	? TT=2520.01	? (TO) TT=2519.8	? (TO) TT=2520.07	UNSAT TT=0.558	Wrong Cert. TT=1.55	UNSAT TT=2.881
Langford/ LangfordBin-10_c18.xml	UNSAT TT=0.575	? (TO) TT=2520.03	? (TO) TT=2520.03	? TT=2520	? TT=2520.01	UNSAT TT=21.088	? (TO) TT=2520	? (TO) TT=2519.84	? (TO) TT=2520.13	UNSAT TT=0.575	Wrong Cert. TT=3.411	UNSAT TT=10.252
Langford/ LangfordBin-11_c18.xml	SAT TT=0.007	SAT TT=0.015	SAT TT=0.642	? TT=2520	? (TO) TT=2519.86	SAT TT=1.447	? (TO) TT=2519.95	SAT TT=0.007	SAT TT=0.995	Wrong UNSAT TT=0.572	SAT TT=0.781	SAT (TO) TT=2520.08
Langford/ LangfordBin-12_c18.xml	SAT TT=0.018	SAT TT=0.018	SAT TT=0.814	? TT=2520	? TT=2520.01	SAT TT=0.913	? (TO) TT=2519.93	? (TO) TT=2519.79	SAT TT=1.009	Wrong UNSAT TT=0.613	SAT TT=0.728	SAT (TO) TT=2520.09
Langford/ LangfordBin-13_c18.xml	UNSAT TT=0.615	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2519.94	? (TO) TT=2519.83	? (TO) TT=2520.07	? (TO) TT=2519.94	? (TO) TT=2519.73	? (TO) TT=2520.05	UNSAT TT=0.615	Wrong Cert. TT=268.549	? (TO) TT=2520.1
Langford/ LangfordBin-14_c18.xml	UNSAT TT=0.624	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2519.92	? (TO) TT=2520.01	? (TO) TT=2520.03	? (TO) TT=2519.98	? (TO) TT=2519.8	? (TO) TT=2520.11	UNSAT TT=0.624	? (TO) TT=2520.07	? (TO) TT=2520.07
Langford/ LangfordBin-15_c18.xml	SAT TT=0.006	SAT TT=0.018	SAT TT=0.664	? TT=2520	? (TO) TT=2519.77	SAT TT=1.027	? (TO) TT=2519.87	SAT TT=0.006	SAT TT=0.901	Wrong UNSAT TT=0.64	SAT TT=0.717	SAT (TO) TT=2520.09
Langford/ LangfordBin-16_c18.xml	SAT TT=0.017	? (TO) TT=2519.75	SAT TT=0.876	? TT=2520.01	? (TO) TT=2519.91	SAT TT=1.713	? (TO) TT=2519.8	SAT TT=0.017	SAT TT=1.187	Wrong UNSAT TT=0.629	SAT TT=0.819	SAT (TO) TT=2520.03
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
MagicHexagon/ MagicHexagon-04-0003_c18.xml	SAT TT=0.337	SAT TT=0.337	SAT TT=1.274	SAT TT=42.998	SAT TT=37.863	SAT TT=16.778	SAT TT=38.234	SAT TT=0.553	SAT TT=3.541	SAT TT=23.018	SAT TT=16.573	? (TO) TT=2520.12
MagicHexagon/ MagicHexagon-04-0017_c18.xml	UNSAT TT=2.749	UNSAT TT=3.399	UNSAT TT=4.278	UNSAT TT=356.449	UNSAT TT=215.334	UNSAT TT=2.749	UNSAT TT=217.454	UNSAT TT=3.944	? (TO) TT=2520.06	UNSAT TT=3.108	? TT=303.005	UNSAT TT=11.875
MagicHexagon/ MagicHexagon-04-m004_c18.xml	SAT TT=0.703	SAT TT=13.062	SAT TT=0.703	SAT TT=274.47	SAT TT=183.111	SAT TT=2.615	SAT TT=183.727	SAT TT=7.694	SAT TT=2.499	SAT TT=3.105	SAT TT=1.895	SAT (TO) TT=2520.02
MagicHexagon/ MagicHexagon-05-0006_c18.xml	SAT TT=3.012	SAT TT=10.174	? (TO) TT=2520.07	? (TO) TT=2520.01	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.01	SAT TT=27.886	SAT TT=3.012	? (TO) TT=2520.04	? (TO) TT=2520.1	? (TO) TT=2520.05
MagicHexagon/ MagicHexagon-05-m003_c18.xml	SAT TT=2.448	? (TO) TT=2519.84	SAT TT=2.448	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2519.91	SAT TT=164.971	SAT TT=13.849	? (TO) TT=2520.02	? (TO) TT=2520.09	? (TO) TT=2520.1
MagicHexagon/ MagicHexagon-06-0010_c18.xml	SAT TT=294.829	? (TO) TT=2520.07	SAT TT=294.829	? (TO) TT=2520.1	? (TO) TT=2519.88	? (TO) TT=2520.1	? (TO) TT=2519.94	? (TO) TT=2519.63	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.11
MagicHexagon/ MagicHexagon-06-m001_c18.xml	SAT TT=810.179	SAT TT=810.179	? (TO) TT=2520.08	? (TO) TT=2520.06	? (TO) TT=2519.94	? (TO) TT=2520.04	? (TO) TT=2519.96	? (TO) TT=2519.61	? (TO) TT=2520.02	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2520.1
MagicHexagon/ MagicHexagon-07-0002_c18.xml		? (TO) TT=2519.73	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.02	? (TO) TT=2519.64	? (TO) TT=2520.02	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.02
MagicHexagon/ MagicHexagon-07-m011_c18.xml		? (TO) TT=2519.97	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2519.94	? (TO) TT=2520.09	? (TO) TT=2519.96	? (TO) TT=2519.71	? (TO) TT=2520.11	? (TO) TT=2520.03	? (TO) TT=2520.03	? (TO) TT=2520.03
MagicHexagon/ MagicHexagon-08-0006_c18.xml		? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2519.81	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2519.72	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.03	? (TO) TT=2520.06
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
MagicHexagon/ MagicHexagon-08-m009_c18.xml		? (TO) TT=2519.97	? (TO) TT=2520.05	? (TO) TT=2520.1	? (TO) TT=2519.98	? (TO) TT=2520.07	? (TO) TT=2520	? (TO) TT=2519.72	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2520.12
MagicSquare/ MagicSquare-04-sum_c18.xml	SAT TT=0.004	SAT TT=0.005	SAT TT=0.551	SAT TT=0.01	SAT TT=0.007	SAT TT=0.512	SAT TT=0.006	SAT TT=0.004	SAT TT=0.558	SAT TT=0.52	SAT TT=0.529	SAT (TO) TT=2520.04
MagicSquare/ MagicSquare-05-sum_c18.xml	SAT TT=0.014	SAT TT=0.014	SAT TT=0.73	SAT TT=0.97	SAT TT=0.296	SAT TT=0.704	SAT TT=0.295	SAT TT=0.015	SAT TT=0.781	SAT TT=0.749	SAT TT=0.73	SAT (TO) TT=2520.08
MagicSquare/ MagicSquare-06-sum_c18.xml	SAT TT=0.01	SAT TT=0.168	SAT TT=10.326	? (TO) TT=2519.89	SAT TT=55.966	SAT TT=2.664	SAT TT=56.064	SAT TT=0.01	SAT TT=0.892	SAT TT=3.374	SAT TT=1.565	SAT (TO) TT=2520.06
MagicSquare/ MagicSquare-07-sum_c18.xml	SAT TT=0.021	SAT TT=0.021	SAT TT=3.473	? (TO) TT=2520.1	SAT TT=2234.36	? (TO) TT=2520.04	SAT TT=2234.84	SAT TT=0.07	SAT TT=1.493	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.08
MagicSquare/ MagicSquare-08-sum_c18.xml	SAT TT=2.457	SAT TT=2.457	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.01	? (TO) TT=2520.05	? (TO) TT=2519.96	? (TO) TT=2519.54	SAT TT=5.822	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.05
MagicSquare/ MagicSquare-09-sum_c18.xml	SAT TT=1038.88	SAT TT=1038.88	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2519.92	? (TO) TT=2520.07	? (TO) TT=2520	? (TO) TT=2519.51	? (TO) TT=2520.03	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2520.03
MagicSquare/ MagicSquare-10-sum_c18.xml	SAT TT=29.52	? (TO) TT=2519.83	? (TO) TT=2520.12	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2519.93	SAT TT=29.52	? (TO) TT=2520.05	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2520.02
MagicSquare/ MagicSquare-11-sum_c18.xml		? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.02	? (TO) TT=2519.7	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.02
MagicSquare/ MagicSquare-12-sum_c18.xml		? (TO) TT=2520	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.01	? (TO) TT=2519.81	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.05
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
MagicSquare/ MagicSquare-13-sum_c18.xml		? (TO) TT=2519.97	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2520.02	? (TO) TT=2520.07	? (TO) TT=2519.96	? (TO) TT=2519.73	? (TO) TT=2520.01	? (TO) TT=2520.02	? (TO) TT=2520.06	? (TO) TT=2520.02
MagicSquare/ MagicSquare-14-sum_c18.xml		? (TO) TT=2519.91	? (TO) TT=2520.09	? (TO) TT=2520.04	? (TO) TT=2519.94	? (TO) TT=2520.11	? (TO) TT=2519.96	? (TO) TT=2519.96	? (TO) TT=2520.07	? (TO) TT=2520.02	? (TO) TT=2520.11	? (TO) TT=2520.07
MagicSquare/ MagicSquare-15-sum_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2519.93	? (TO) TT=2520.08	? (TO) TT=2519.94	? (TO) TT=2519.75	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.04
MagicSquare/ MagicSquare-16-sum_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520.09	? (TO) TT=2519.73	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.02	? (TO) TT=2519.75	? (TO) TT=2520.02	? (TO) TT=2520.03	? (TO) TT=2520.08	? (TO) TT=2520.05
PseudoBoolean-dec/ Pb-chnl10-11_c18.xml	UNSAT TT=187.07	? (TO) TT=2520.07	? (TO) TT=2520.12	? TT=2520	? TT=2520	? (TO) TT=2520.11	? (TO) TT=2519.95	UNSAT TT=187.07	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.06	? (TO) TT=2520.11
PseudoBoolean-dec/ Pb-chnl15-16_c18.xml		? (TO) TT=2520.03	? (TO) TT=2520.04	? TT=2520.01	? (TO) TT=2519.99	? (TO) TT=2520.07	? (TO) TT=2519.93	? (TO) TT=2519.54	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.11	? (TO) TT=2520.02
PseudoBoolean-dec/ Pb-chnl20-21_c18.xml		? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2519.61	? (TO) TT=2519.93	? (TO) TT=2520.06	? (TO) TT=2519.9	? (TO) TT=2519.82	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2520.02	? (TO) TT=2520.08
PseudoBoolean-dec/ Pb-chnl30-31_c18.xml		? (TO) TT=2519.75	? (TO) TT=2520.07	? (TO) TT=2519.94	? (TO) TT=2519.73	? (TO) TT=2520.09	? TT=2520.08	? (TO) TT=2519.62	? (TO) TT=2520.09	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.07
PseudoBoolean-dec/ Pb-chnl50-51_c18.xml		? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2519.92	? (TO) TT=2519.8	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2519.34	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.05
PseudoBoolean-dec/ Pb-fpga25-25-sat_c18.xml	SAT TT=0.235	SAT TT=22.487	SAT TT=1.35	? (TO) TT=2520.09	SAT TT=0.24	SAT TT=1.395	SAT TT=0.235	SAT TT=1.394	SAT TT=1.391	SAT TT=1.362	SAT TT=1.383	SAT (TO) TT=2520.14
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
PseudoBoolean-dec/ Pb-fpga45-45-sat_c18.xml	SAT TT=4.191	SAT TT=1531.73	? (TO) TT=2520.08	? (TO) TT=2520.02	SAT TT=4.205	? (TO) TT=2520.03	SAT TT=4.191	SAT TT=6.825	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.07
PseudoBoolean-dec/ Pb-robin08_c18.xml	SAT TT=0.19	SAT TT=0.433	SAT TT=1.973	SAT TT=2.349	SAT TT=0.359	SAT TT=2.152	SAT TT=0.766	SAT TT=0.19	SAT TT=2.08	SAT TT=2.198	SAT TT=2.004	SAT (TO) TT=2520.06
PseudoBoolean-dec/ Pb-robin10_c18.xml	SAT TT=0.912	SAT TT=43.31	SAT TT=4.819	SAT TT=23.449	SAT TT=96.493	SAT TT=4.544	SAT TT=54.352	SAT TT=0.912	? (TO) TT=2520.06	SAT TT=4.159	SAT TT=5.279	SAT (TO) TT=2520.13
PseudoBoolean-dec/ Pb-robin12_c18.xml	SAT TT=76.943	SAT TT=1261.92	? (TO) TT=2520.07	? (TO) TT=2519.92	? (TO) TT=2520.06	? (TO) TT=2520.09	? (TO) TT=2519.95	SAT TT=76.943	? (TO) TT=2520.03	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.06
PseudoBoolean-dec/ Pb-robin14_c18.xml		? (TO) TT=2519.88	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2519.9	? (TO) TT=2520.05	? (TO) TT=2519.91	? (TO) TT=2519.54	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2520.07
PseudoBoolean-dec/ Pb-robin16_c18.xml		? (TO) TT=2520	? (TO) TT=2520.09	? (TO) TT=2520.03	? (TO) TT=2519.97	? (TO) TT=2520.09	? (TO) TT=2519.98	? (TO) TT=2520.03	? (TO) TT=2520.02	? (TO) TT=2520.05	? (TO) TT=2520.03	? (TO) TT=2520.04
PseudoBoolean-dec/ Pb-robin18_c18.xml		? (TO) TT=2519.89	? (TO) TT=2520.12	? (TO) TT=2519.93	? (TO) TT=2520.05	? (TO) TT=2520.11	? (TO) TT=2519.9	? (TO) TT=2519.82	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.06	? (NS) TT=6.822
Quasigroups_mini/ QuasiGroup-v4-07_mini_c18.xml	UNSAT TT=0.721	UNSAT TT=2.161	UNSAT TT=5.424	UNSAT TT=8.64	UNSAT TT=16.486	UNSAT TT=6.387	UNSAT TT=3.746	UNSAT TT=0.784	? (TO) TT=2520.14	UNSAT TT=0.721	? TT=302.841	UNSAT TT=4.164
Quasigroups_mini/ QuasiGroup-v4-08_mini_c18.xml	UNSAT TT=0.77	UNSAT TT=480.267	? (TO) TT=2520.07	UNSAT TT=1807.25	UNSAT TT=516.879	UNSAT TT=776.537	UNSAT TT=333.62	UNSAT TT=582.11	? (TO) TT=2521.45	UNSAT TT=0.77	? TT=460.512	UNSAT TT=938.28
Quasigroups_mini/ QuasiGroup-v4-09_mini_c18.xml	SAT TT=0.168	SAT TT=0.168	SAT TT=1.897	SAT TT=1.061	SAT TT=0.745	SAT TT=5.623	SAT TT=0.173	SAT TT=0.176	SAT TT=6.98	Wrong UNSAT TT=0.898	SAT TT=2.772	SAT (TO) TT=2520.03
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Quasigroups_mini/ QuasiGroup-v4-10_mini_c18.xml	UNSAT TT=0.953	? (TO) TT=2520.09	? (TO) TT=2520.13	? TT=2520.03	? (TO) TT=2520.03	? (TO) TT=2520.09	? (TO) TT=2519.91	? (TO) TT=2519.5	? (TO) TT=2520.05	UNSAT TT=0.953	? (TO) TT=2520.1	? (TO) TT=2520.04
Quasigroups_mini/ QuasiGroup-v4-12_mini_c18.xml	SAT TT=196.295	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2519.97	? (TO) TT=2520.1	SAT TT=782.479	SAT TT=196.295	? (TO) TT=2528.38	Wrong UNSAT TT=1.12	? (TO) TT=2520.03	? (TO) TT=2520.12
Quasigroups_mini/ QuasiGroup-v4-14_mini_c18.xml	UNSAT TT=1.258	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2519.79	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2519.4	? (TO) TT=2528.37	UNSAT TT=1.258	? (TO) TT=2520.09	? (TO) TT=2520.12
Quasigroups_mini/ QuasiGroup-v4-16_mini_c18.xml	UNSAT TT=1.486	? (TO) TT=2520.1	? (TO) TT=2524.38	? (TO) TT=2520.08	? (TO) TT=2519.99	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2520.03	? TT=1370.94	UNSAT TT=1.486	? (TO) TT=2520.05	? (TO) TT=2520.08
Quasigroups_mini/ QuasiGroup-v4-20_mini_c18.xml	UNSAT TT=2.06	? (TO) TT=2520.07	? (NS) TT=5.698	? (TO) TT=2519.74	? (TO) TT=2519.75	? (TO) TT=2520.11	? (TO) TT=2519.91	? (TO) TT=2519.82	? (TO) TT=2528.27	UNSAT TT=2.06	? (TO) TT=2520.03	? (TO) TT=2520.08
Quasigroups_mini/ QuasiGroup-v7-07_mini_c18.xml	UNSAT TT=0.415	UNSAT TT=0.6	UNSAT TT=2.847	UNSAT TT=0.944	UNSAT TT=3	UNSAT TT=2.131	UNSAT TT=5.318	UNSAT TT=0.415	? (TO) TT=2520.07	UNSAT TT=0.655	? TT=302.529	UNSAT TT=2.457
Quasigroups_mini/ QuasiGroup-v7-08_mini_c18.xml	UNSAT TT=0.682	UNSAT TT=41.23	? (TO) TT=2520.13	UNSAT TT=42.421	UNSAT TT=767.325	UNSAT TT=14.696	UNSAT TT=786.511	UNSAT TT=30.738	? (TO) TT=2520.07	UNSAT TT=0.682	? TT=548.739	UNSAT TT=211.2
Quasigroups_mini/ QuasiGroup-v7-09_mini_c18.xml	SAT TT=0.114	SAT TT=0.29	SAT TT=1.045	SAT TT=12.261	SAT TT=1.478	SAT TT=1.235	SAT TT=5.758	SAT TT=0.114	SAT TT=2.157	Wrong UNSAT TT=0.735	SAT TT=11.795	SAT (TO) TT=2520.06
Quasigroups_mini/ QuasiGroup-v7-10_mini_c18.xml	UNSAT TT=0.783	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2519.92	? TT=2520.01	? (TO) TT=2520.07	? (TO) TT=2519.95	? (TO) TT=2519.79	? (TO) TT=2528.1	UNSAT TT=0.783	? (TO) TT=2520.05	? (TO) TT=2520.09
Quasigroups_mini/ QuasiGroup-v7-12_mini_c18.xml	UNSAT TT=0.88	? (TO) TT=2520.07	? (TO) TT=2520.12	? TT=2520.02	? (TO) TT=2519.68	? (TO) TT=2520.05	? TT=2520.02	? (TO) TT=2519.61	? (TO) TT=2528.22	UNSAT TT=0.88	? (TO) TT=2520.06	? (TO) TT=2520.11
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Quasigroups_mini/ QuasiGroup-v7-14_mini_c18.xml	UNSAT TT=0.977	? (TO) TT=2520.08	? (TO) TT=2520.06	? TT=2520.03	? (TO) TT=2520.01	? (TO) TT=2520.09	? (TO) TT=2520	? (TO) TT=2519.63	? (TO) TT=2527.95	UNSAT TT=0.977	? (TO) TT=2520.04	? (TO) TT=2520.09
Quasigroups_mini/ QuasiGroup-v7-16_mini_c18.xml	UNSAT TT=1.068	? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2519.94	? TT=2520.04	? (TO) TT=2520.1	? (TO) TT=2519.98	? (TO) TT=2519.67	? (TO) TT=2528.03	UNSAT TT=1.068	? (TO) TT=2520.07	? (TO) TT=2520.07
Quasigroups_mini/ QuasiGroup-v7-20_mini_c18.xml	UNSAT TT=1.287	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2519.76	? (TO) TT=2519.91	? (TO) TT=2520.05	? (TO) TT=2519.78	? (TO) TT=2519.83	? (NS) TT=39.986	UNSAT TT=1.287	? (TO) TT=2520.03	? (TO) TT=2520.04
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f01_c18.xml	UNSAT TT=310.657	UNSAT TT=987.226	? (TO) TT=2520.03	? (TO) TT=2519.9	? (TO) TT=2520	? (TO) TT=2520.12	? (TO) TT=2519.99	UNSAT TT=310.657	? (TO) TT=2520.12	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.07
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f02_c18.xml	UNSAT TT=89.784	UNSAT TT=374.531	? (TO) TT=2520.05	UNSAT TT=2473.79	? (TO) TT=2519.9	? (TO) TT=2520.07	? (TO) TT=2519.91	UNSAT TT=89.784	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.06	? (TO) TT=2520.05
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f03_c18.xml	UNSAT TT=21.164	UNSAT TT=106.848	? (TO) TT=2520.06	UNSAT TT=908.285	UNSAT TT=910.067	? (TO) TT=2520.1	UNSAT TT=1326.26	UNSAT TT=21.164	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2520.09	? (TO) TT=2520.1
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f04_c18.xml	UNSAT TT=4.846	UNSAT TT=29.548	? (TO) TT=2520.05	UNSAT TT=230.645	UNSAT TT=223.471	? (TO) TT=2520.09	UNSAT TT=330.11	UNSAT TT=4.846	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520.1	? (TO) TT=2520.09
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f05_c18.xml	UNSAT TT=1.38	UNSAT TT=6.684	? (TO) TT=2520.07	UNSAT TT=56.696	UNSAT TT=49.888	? (TO) TT=2520.11	UNSAT TT=64.384	UNSAT TT=1.38	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.11
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f06_c18.xml	UNSAT TT=0.438	UNSAT TT=1.247	? (TO) TT=2520.08	UNSAT TT=12.25	UNSAT TT=10.775	? (TO) TT=2520.09	UNSAT TT=14.423	UNSAT TT=0.438	? (TO) TT=2520.1	? (TO) TT=2520.02	? (TO) TT=2520.02	? (TO) TT=2520.05
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f07_c18.xml	UNSAT TT=0.381	UNSAT TT=0.83	? (TO) TT=2520.11	UNSAT TT=7.975	UNSAT TT=4.406	? (TO) TT=2520.09	UNSAT TT=7.651	UNSAT TT=0.381	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.09
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f08_c18.xml	UNSAT TT=0.213	UNSAT TT=0.213	? (TO) TT=2520.12	UNSAT TT=2.17	UNSAT TT=1.053	? (TO) TT=2520.1	UNSAT TT=1.495	UNSAT TT=0.305	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.11	UNSAT TT=1263.96
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f09_c18.xml	UNSAT TT=0.187	UNSAT TT=0.187	? (TO) TT=2520.08	UNSAT TT=1.725	UNSAT TT=0.757	? (TO) TT=2520.05	UNSAT TT=1.015	UNSAT TT=0.275	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.11	UNSAT TT=500.233
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f10_c18.xml	UNSAT TT=0.164	UNSAT TT=0.164	? (TO) TT=2520.04	UNSAT TT=0.666	UNSAT TT=0.646	? (TO) TT=2520.12	UNSAT TT=0.637	UNSAT TT=0.249	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2520.03	UNSAT TT=12.889
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f11_c18.xml	UNSAT TT=0.166	UNSAT TT=0.166	? (TO) TT=2520.09	UNSAT TT=0.762	UNSAT TT=0.491	? (TO) TT=2520.05	UNSAT TT=0.62	UNSAT TT=0.253	? (TO) TT=2520.07	? (TO) TT=2520.09	? (TO) TT=2520.07	UNSAT TT=12.002
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f12_c18.xml	UNSAT TT=0.164	UNSAT TT=0.164	? (TO) TT=2520.04	UNSAT TT=0.751	UNSAT TT=0.479	? (TO) TT=2520.1	UNSAT TT=0.612	UNSAT TT=0.243	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.02	UNSAT TT=12.647
Subisomorphism/ Subisomorphism-g08-g43_c18.xml	SAT TT=1.307	SAT TT=16.382	SAT TT=17.276	SAT TT=1.307	SAT TT=3.696	SAT TT=6.064	SAT TT=3.763	SAT TT=28.942	SAT TT=8.414	SAT TT=47.241	SAT TT=63.114	SAT (TO) TT=2520.11
Subisomorphism/ Subisomorphism-g10-g23_c18.xml	UNSAT TT=90.347	? (TO) TT=2519.86	? (TO) TT=2520.12	? TT=2520.02	UNSAT TT=92.459	? (TO) TT=2520.04	UNSAT TT=90.347	? (TO) TT=2519.8	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2520.05	? (TO) TT=2520.06
Subisomorphism/ Subisomorphism-g10-g27_c18.xml	SAT TT=203.898	? (TO) TT=2519.93	? (TO) TT=2520.05	? TT=2520.01	? (TO) TT=2519.91	? (TO) TT=2520.03	? (TO) TT=2519.93	SAT TT=412.542	SAT TT=203.898	? (TO) TT=2520.06	? (TO) TT=2520.03	? (TO) TT=2520.07
Subisomorphism/ Subisomorphism-g11-g30_c18.xml	UNSAT TT=192.732	UNSAT TT=192.732	UNSAT TT=768.339	UNSAT TT=292.707	UNSAT TT=426.292	? (TO) TT=2520.07	UNSAT TT=416.269	UNSAT TT=507.993	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.1
Subisomorphism/ Subisomorphism-g11-g44_c18.xml	UNSAT TT=444.933	UNSAT TT=444.933	UNSAT TT=1884.5	UNSAT TT=1228.13	UNSAT TT=1308.78	? (TO) TT=2520.02	UNSAT TT=1295.26	UNSAT TT=1992.94	? (TO) TT=2520.07	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2520.05
Bench name	Best results	cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Subisomorphism/ Subisomorphism-g17-g37_c18.xml	SAT TT=1784.38	? (TO) TT=2519.86	? (TO) TT=2520.05	SAT TT=2422.5	? (TO) TT=2519.72	SAT TT=1784.38	? TT=2520.04	? (TO) TT=2519.56	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.03
Subisomorphism/ Subisomorphism-g17-g42_c18.xml	UNSAT TT=45.885	? (TO) TT=2519.77	? (TO) TT=2520.01	UNSAT TT=45.885	UNSAT TT=126.159	? (TO) TT=2520.11	UNSAT TT=129.415	? (TO) TT=2519.81	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.09	? (TO) TT=2520.07
Subisomorphism/ Subisomorphism-g19-g43_c18.xml		? (TO) TT=2519.83	? (TO) TT=2520.11	? TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2520.01	? (TO) TT=2519.55	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.07
Subisomorphism/ Subisomorphism-g19-g45_c18.xml		? (TO) TT=2519.98	? (TO) TT=2520.07	? (TO) TT=2520.04	? TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2519.77	? (TO) TT=2519.63	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.09
Subisomorphism/ Subisomorphism-g29-g50_c18.xml		? (TO) TT=2519.94	? (TO) TT=2520.04	? (TO) TT=2519.95	? (TO) TT=2519.76	? (TO) TT=2520.1	? (TO) TT=2519.86	? (TO) TT=2519.63	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.03
Subisomorphism/ Subisomorphism-g34-g36_c18.xml	UNSAT TT=0.571	UNSAT TT=0.571	UNSAT TT=9.389	UNSAT TT=3.501	UNSAT TT=4.422	UNSAT TT=6.659	UNSAT TT=4.535	UNSAT TT=0.852	? (TO) TT=2520.07	UNSAT TT=89.39	? TT=316.615	UNSAT TT=23.398

Statistics		cosoco 1.12 (complete)	GG's minicp 2018-04-29 (complete)	miniBTD 2018.07.27_3 (complete)	miniBTD_12 2018.07.27_12 (complete)	MiniCPFever 2018-04-29 (complete)	minimacht 2018.07.27 (complete)	NACRE 1.0.4 (complete)	slowpoke 2018-04-29 (incomplete)	Solver of Xavier Schul & Yvhan Smal 2018-04-28 (incomplete)	SuperSolver_Macq_Stevenart 2018-04-27 (incomplete)	The dodo solver 2018-04-29 (complete)
Number of times the solver is able to give the best known answer		72	56	77	79	54	69	86	38	54	31	58
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		19	3	20	5	3	11	33	3	16	0	0