2017 XCSP3 competition: mini-solver track (sequential and parallel solvers): results by benchmark

Results by benchmark for category all categories

This page displays the results of the different solvers for each benchmark for category all categories

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
cosoco-mini 1.12 (complete)
cosoco-mini 1.1 (2017-06-27) (complete)
cosoco-mini 1.1 (2017-07-29) (complete)
miniBTD 2017-06-30 (complete)
miniBTD 2017-08-10 (complete)
Naxos 1.1.0 (complete)

Bench name	Best results on this instance	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
AllInterval/ AllInterval-m1-s1/ AllInterval-007.xml	SAT TT=0.014	SAT TT=0.012	SAT TT=0.012	SAT TT=0.012	SAT TT=0.007	SAT TT=0.006	SAT TT=0.014
AllInterval/ AllInterval-m1-s1/ AllInterval-012.xml	SAT TT=0.023	SAT TT=0.023	SAT TT=0.023	SAT TT=0.023	SAT TT=0.011	SAT TT=0.011	SAT TT=0.024
AllInterval/ AllInterval-m1-s1/ AllInterval-016.xml	SAT TT=0.044	SAT TT=0.045	SAT TT=0.045	SAT TT=0.044	SAT TT=0.019	SAT TT=0.019	SAT TT=0.037
AllInterval/ AllInterval-m1-s1/ AllInterval-035.xml	SAT TT=0.534	? (TO) TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.04	SAT TT=0.275	SAT TT=0.27	SAT TT=0.534
AllInterval/ AllInterval-m1-s1/ AllInterval-050.xml	SAT TT=5.593	? (TO) TT=2400.1	? (TO) TT=2400.1	? (TO) TT=2400.02	SAT TT=1.442	SAT TT=1.425	SAT TT=5.593
AllInterval/ AllInterval-m1-s1/ AllInterval-080.xml	SAT TT=12.54	? (TO) TT=2400.11	? (TO) TT=2400.04	? (TO) TT=2400.02	SAT TT=13.497	SAT TT=12.54	SAT TT=244.415
Blackhole/ Blackhole-xcsp2-s04/ Blackhole-4-04-2_X2.xml	UNSAT TT=0.223	UNSAT TT=0.223	UNSAT TT=0.224	UNSAT TT=0.226	UNSAT TT=0.276	UNSAT TT=0.272	? (TO) TT=2400.01
Blackhole/ Blackhole-xcsp2-s04/ Blackhole-4-04-9_X2.xml	UNSAT TT=0.223	UNSAT TT=0.223	UNSAT TT=0.224	UNSAT TT=0.229	UNSAT TT=0.284	UNSAT TT=0.272	? (TO) TT=2400.03
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-6_X2.xml		? (TO) TT=2400.09	? (TO) TT=2400.05	? (TO) TT=2400.03	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.1
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07h-5_X2.xml		? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.05	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.01
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07h-9_X2.xml		? (TO) TT=2400.06	? (TO) TT=2400.1	? (TO) TT=2400.09	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.03
Blackhole/ Blackhole-xcsp2-s13/ Blackhole-4-13-2_X2.xml		? (TO) TT=2400.06	? (TO) TT=2400.05	? (TO) TT=2400.05	? TT=2400.05	? TT=2400.05	? (TO) TT=2400.04
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-03.xml	UNSAT TT=0.003	UNSAT TT=0.004	UNSAT TT=0.004	UNSAT TT=0.004	UNSAT TT=0.003	UNSAT TT=0.004	UNSAT TT=0.003
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-10.xml		? (TO) TT=2400.03	? (TO) TT=2400.09	? (TO) TT=2400.07	? TT=2400	? TT=2400.01	? (TO) TT=2400.07
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-11.xml		? (TO) TT=2400.04	? (TO) TT=2400.05	? (TO) TT=2400.08	? TT=2400	? TT=2400.01	? (TO) TT=2400.08
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-12.xml		? (TO) TT=2400.05	? (TO) TT=2400.06	? (TO) TT=2400.07	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.1
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-14.xml		? (TO) TT=2400.04	? (TO) TT=2400.06	? (TO) TT=2400.02	? TT=2400.01	? TT=2400	? (TO) TT=2400.06
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-19.xml		? (TO) TT=2400.08	? (TO) TT=2400.08	? (TO) TT=2400.03	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.02
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-04-2-08.xml	SAT TT=0.006	SAT TT=0.005	SAT TT=0.005	SAT TT=0.006	Wrong UNSAT TT=0.005	SAT TT=0.006	? (TO) TT=2400.05
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-04-2-09.xml	SAT TT=0.006	SAT TT=0.005	SAT TT=0.005	SAT TT=0.005	SAT TT=0.006	SAT TT=0.006	? (TO) TT=2400.03
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-06-2-12.xml		? (TO) TT=2400.06	? (TO) TT=2400.06	? (TO) TT=2400.05	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.05
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-07-2-12.xml		? (TO) TT=2400.06	? (TO) TT=2400.11	? (TO) TT=2400.1	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.04
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-09-2-12.xml		? (TO) TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.02	? TT=2400.02	? TT=2400.02	? (TO) TT=2400.07
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-10-2-12.xml		? (TO) TT=2400.02	? (TO) TT=2400.08	? (TO) TT=2400.09	? TT=2400.04	? TT=2400.04	? (TO) TT=2400.02
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h1501.xml	SAT TT=0.292	SAT TT=0.292	SAT TT=0.294	SAT TT=0.294	SAT TT=0.495	SAT TT=0.49	? (TO) TT=2400.01
Crossword/Crossword-m1c-ogd2008-vg/ Crossword-m1c-ogd2008-vg-11-12.xml		? (TO) TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.05	? (TO) TT=2400.1	? (TO) TT=2400.02	? (TO) TT=2400.06
Crossword/Crossword-m1c-ogd2008-vg/ Crossword-m1c-ogd2008-vg-13-13.xml		? (TO) TT=2400.1	? (TO) TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.07	? (TO) TT=2400.07	? (TO) TT=2400.08
Crossword/ Crossword-m1c-ogd-herald/ Crossword-m1c-ogd-h2310.xml	SAT TT=45.671	SAT TT=2.802	SAT TT=2.826	SAT TT=2.823	SAT TT=43.91	SAT TT=45.671	SAT TT=33.669
Crossword/Crossword-m1c-uk-vg/ Crossword-m1c-uk-vg-4-8.xml	SAT TT=2.296	SAT TT=0.336	SAT TT=0.337	SAT TT=0.337	SAT TT=2.271	SAT TT=2.296	SAT TT=6.279
Crossword/ Crossword-m1c-words-puzzle/ Crossword-m1c-words-p20.xml	SAT TT=0.1	SAT TT=0.105	SAT TT=0.1	SAT TT=0.1	SAT TT=0.085	SAT TT=0.087	SAT TT=0.203
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Driver/ Driver-m1-s1/ driverlogw-01c.xml	SAT TT=0.008	SAT TT=0.008	SAT TT=0.008	SAT TT=0.007	SAT TT=0.006	SAT TT=0.006	SAT TT=0.007
Driver/ Driver-m1-s1/ driverlogw-02c.xml	SAT TT=0.379	SAT TT=0.086	SAT TT=0.087	SAT TT=0.086	SAT TT=0.371	SAT TT=0.379	SAT TT=50.594
Driver/ Driver-m1-s1/ driverlogw-04c.xml	SAT TT=0.096	SAT TT=0.096	SAT TT=0.095	SAT TT=0.094	SAT TT=0.153	SAT TT=0.158	SAT TT=1.292
Driver/Driver-m1-s1/ driverlogw-08cc.xml	SAT TT=2.703	SAT TT=0.207	SAT TT=0.206	SAT TT=0.208	SAT TT=2.666	SAT TT=2.703	SAT TT=230.262
Driver/ Driver-m1-s1/ driverlogw-08c.xml	SAT TT=2.867	SAT TT=0.203	SAT TT=0.205	SAT TT=0.203	SAT TT=2.839	SAT TT=2.867	SAT TT=234.609
Driver/ Driver-m1-s1/ driverlogw-09.xml	SAT TT=0.581	SAT TT=0.581	SAT TT=0.575	SAT TT=0.575	SAT TT=10.231	SAT TT=11.691	? (TO) TT=2400.04
Dubois/ Dubois-m1-s1/ Dubois-021.xml	UNSAT TT=0.005	UNSAT TT=156.135	UNSAT TT=158.972	UNSAT TT=155.863	UNSAT TT=0.006	UNSAT TT=0.005	UNSAT TT=105.179
Dubois/ Dubois-m1-s1/ Dubois-022.xml	UNSAT TT=0.006	UNSAT TT=340.852	UNSAT TT=347.228	UNSAT TT=341.572	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=190.584
Dubois/ Dubois-m1-s1/ Dubois-025.xml	UNSAT TT=0.006	? (TO) TT=2400.08	? (TO) TT=2400.03	? (TO) TT=2400.1	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=1706.31
Dubois/ Dubois-m1-s1/ Dubois-050.xml	UNSAT TT=0.009	? (TO) TT=2400.06	? (TO) TT=2400.09	? (TO) TT=2400.09	UNSAT TT=0.009	UNSAT TT=0.009	? (TO) TT=2400.04
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Dubois/ Dubois-m1-s1/ Dubois-080.xml	UNSAT TT=0.014	? (TO) TT=2400.03	? (TO) TT=2400.05	? (TO) TT=2400.06	UNSAT TT=0.014	UNSAT TT=0.014	? (TO) TT=2400.1
Dubois/ Dubois-m1-s1/ Dubois-090.xml	UNSAT TT=0.017	? (TO) TT=2400.03	? (TO) TT=2400.05	? (TO) TT=2400.03	UNSAT TT=0.016	UNSAT TT=0.017	? (TO) TT=2400.04
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K03-P04.xml	SAT TT=51.233	SAT TT=51.233	SAT TT=51.165	SAT TT=51.233	SAT TT=8.607	SAT TT=8.695	? (TO) TT=2400.08
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K10-P04.xml		? (TO) TT=2400	? (TO) TT=2400.07	? (TO) TT=2400.08	? (TO) TT=2400.09	? TT=2400.03	? (TO) TT=2400.1
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K10-P09.xml		? (TO) TT=2400.02	? (TO) TT=2400.08	? (TO) TT=2400.07	? (TO) TT=2400.1	? (TO) TT=2400.07	? (TO) TT=2400.01
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K11-P04.xml		? (TO) TT=2400.02	? (TO) TT=2400.07	? (TO) TT=2400.11	? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.05
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K12-P09.xml		? (TO) TT=2400.09	? (TO) TT=2400.05	? (TO) TT=2400.03	? (TO) TT=2400.08	? (TO) TT=2400.02	? (TO) TT=2400.08
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K14-P07.xml		? (TO) TT=2400.07	? (TO) TT=2400.06	? (TO) TT=2400.07	? (TO) TT=2400.11	? (TO) TT=2400.08	? (TO) TT=2400.01
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-2-fullins-3.xml	OPT f=4 TT=0.177 T1=0.14	OPT f=4 TT=0.183 T1=0.24	OPT f=4 TT=0.183 T1=0.18	OPT f=4 TT=0.177 T1=0.22	No data	No data	OPT f=4 TT=9.212 T1=0.14
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-3-fullins-3.xml	OPT f=5 TT=0.303 T1=0.22	OPT f=5 TT=0.327 T1=0.4	OPT f=5 TT=0.306 T1=0.3	OPT f=5 TT=0.303 T1=0.37	No data	No data	SAT (TO) f=5 TT=2400.02 T1=0.22
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-3-fullins-5.xml	SAT f=7 TT=2400.02 T1=21.55	SAT (TO) f=7 TT=2400.02 T1=31.27	No Cert. TT=2400.02	SAT (TO) f=7 TT=2400.08 T1=29.74	No data	No data	SAT (TO) f=7 TT=2400.08 T1=21.55
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-4-insertions-3.xml	SAT f=3 TT=2400.08 T1=0.1	SAT (TO) f=3 TT=2400.08 T1=0.19	? (TO) TT=2400.03	SAT (TO) f=3 TT=2400.08 T1=0.21	No data	No data	SAT (TO) f=3 TT=2400.09 T1=0.1
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-ash331GPIA.xml	OPT f=3 TT=3.704 T1=3.75	OPT f=3 TT=3.788 T1=3.84	OPT f=3 TT=3.837 T1=3.84	OPT f=3 TT=3.704 T1=3.75	No data	No data	OPT f=3 TT=17.53 T1=17.52
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-dsjc-250-1.xml	SAT f=8 TT=2400.02 T1=2.89	SAT (TO) f=8 TT=2400.02 T1=3	? (TO) TT=2400.01	SAT (TO) f=8 TT=2400.1 T1=2.89	No data	No data	SAT (TO) f=8 TT=2400.03 T1=3.1
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-dsjc-250-9.xml	SAT f=87 TT=2400.1 T1=24.73	SAT (TO) f=87 TT=2400.11 T1=26.19	? (TO) TT=2400.06	SAT (TO) f=87 TT=2400.1 T1=24.73	No data	No data	SAT (TO) f=88 TT=2400.03 T1=17.79
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-dsjc-500-5.xml	SAT f=63 TT=2400.02 T1=39.78	SAT (TO) f=63 TT=2400.02 T1=138.33	No Cert. TT=2400.03	SAT (TO) f=63 TT=2400.08 T1=142.38	No data	No data	SAT (TO) f=63 TT=2400.07 T1=39.78
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-games120.xml	OPT f=8 TT=0.95 T1=0.4	OPT f=8 TT=0.976 T1=0.59	OPT f=8 TT=0.969 T1=0.58	OPT f=8 TT=0.95 T1=0.63	No data	No data	SAT (TO) f=8 TT=2400.01 T1=0.4
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-lei450-25d.xml	SAT f=26 TT=2400.07 T1=16.46	SAT (TO) f=26 TT=2400.07 T1=17.25	No Cert. TT=2400.07	SAT (TO) f=26 TT=2400.09 T1=16.46	No data	No data	SAT (TO) f=27 TT=2400.1 T1=11.65
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-mug100-25.xml	SAT f=3 TT=2400.03 T1=0.11	SAT (TO) f=3 TT=2400.04 T1=0.15	? (TO) TT=2400.02	SAT (TO) f=3 TT=2400.03 T1=0.19	No data	No data	SAT (TO) f=3 TT=2400.1 T1=0.11
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-queen8-8.xml	SAT f=8 TT=2400.03 T1=1.14	SAT (TO) f=8 TT=2400.05 T1=3.64	? (TO) TT=2400.01	SAT (TO) f=8 TT=2400.05 T1=3.57	No data	No data	SAT (TO) f=8 TT=2400.03 T1=1.14
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-wap01a.xml	SAT f=42 TT=2400.04 T1=154.37	SAT (TO) f=42 TT=2400.08 T1=156.98	SAT (TO) f=42 TT=2400.07 T1=154.37	SAT (TO) f=42 TT=2400.04 T1=155.16	No data	No data	SAT (TO) f=47 TT=2400.07 T1=71.2
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-wap06a.xml	SAT f=40 TT=2400.05 T1=81.28	SAT (TO) f=40 TT=2400.05 T1=81.28	No Cert. TT=2400.02	SAT (TO) f=40 TT=2400.05 T1=86.49	No data	No data	SAT (TO) f=49 TT=2400.05 T1=27.51
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-zeroin-i-3.xml	SAT f=29 TT=2400.04 T1=2.25	SAT (TO) f=29 TT=2400.08 T1=3.36	? (TO) TT=2400.03	SAT (TO) f=29 TT=2400.04 T1=3.19	No data	No data	SAT (TO) f=29 TT=2400.08 T1=2.25
Kakuro/Kakuro-ext-easy/ Kakuro-easy-084-ext.xml	SAT TT=0.11	SAT TT=0.11	SAT TT=0.11	SAT TT=0.11	SAT TT=0.042	SAT TT=0.044	SAT TT=0.122
Kakuro/Kakuro-ext-easy/ Kakuro-easy-109-ext.xml	SAT TT=0.335	SAT TT=0.335	SAT TT=0.339	SAT TT=0.34	SAT TT=0.137	SAT TT=0.142	SAT TT=0.406
Kakuro/Kakuro-ext-easy/ Kakuro-easy-150-ext.xml	SAT TT=0.295	SAT TT=0.291	SAT TT=0.296	SAT TT=0.295	SAT TT=0.096	SAT TT=0.099	SAT TT=0.337
Kakuro/Kakuro-ext-medium/ Kakuro-medium-016-ext.xml	SAT TT=0.014	SAT TT=0.028	SAT TT=0.029	SAT TT=0.03	SAT TT=0.013	SAT TT=0.014	SAT TT=0.031
Kakuro/Kakuro-ext-medium/ Kakuro-medium-020-ext.xml	SAT TT=0.021	SAT TT=0.019	SAT TT=0.019	SAT TT=0.021	SAT TT=0.01	SAT TT=0.01	SAT TT=0.02
Kakuro/Kakuro-ext-medium/ Kakuro-medium-162-ext.xml	SAT TT=7.103	SAT TT=5.819	SAT TT=5.993	SAT TT=5.933	SAT TT=2.661	SAT TT=2.674	SAT TT=7.103
Kakuro/Kakuro-sumdiff-easy/ Kakuro-easy-015-sumdiff.xml	SAT TT=0.007	SAT TT=0.007	SAT TT=0.007	SAT TT=0.008	SAT TT=0.009	SAT TT=0.01	SAT TT=0.008
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Kakuro/Kakuro-sumdiff-easy/ Kakuro-easy-079-sumdiff.xml	SAT TT=0.011	SAT TT=0.01	SAT TT=0.01	SAT TT=0.011	SAT TT=0.025	SAT TT=0.026	SAT TT=0.014
Kakuro/Kakuro-sumdiff-easy/ Kakuro-easy-164-sumdiff.xml	SAT TT=1.266	SAT TT=0.197	SAT TT=0.199	SAT TT=0.198	SAT TT=1.263	SAT TT=1.266	SAT TT=0.024
Kakuro/Kakuro-sumdiff-hard/ Kakuro-hard-179-sumdiff.xml	SAT TT=0.384	SAT TT=0.163	SAT TT=0.164	SAT TT=0.166	SAT TT=0.384	SAT TT=0.384	SAT TT=0.419
Kakuro/Kakuro-sumdiff-medium/ Kakuro-medium-055-sumdiff.xml	SAT TT=0.01	SAT TT=0.009	SAT TT=0.009	SAT TT=0.009	SAT TT=0.013	SAT TT=0.014	SAT TT=0.01
Kakuro/Kakuro-sumdiff-medium/ Kakuro-medium-191-sumdiff.xml	SAT TT=65.386	SAT TT=64.846	SAT TT=65.027	SAT TT=65.386	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.05
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-04.xml	OPT f=1129 TT=1.191 T1=0.79	OPT f=1129 TT=1.207 T1=0.86	OPT f=1129 TT=1.191 T1=0.79	OPT f=1129 TT=1.201 T1=0.86	No data	No data	OPT f=1129 TT=11.409 T1=10.57
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-06.xml	OPT f=1188 TT=0.968 T1=0.94	OPT f=1188 TT=0.979 T1=1.01	OPT f=1188 TT=0.968 T1=0.94	OPT f=1188 TT=0.969 T1=1.03	No data	No data	OPT f=1188 TT=3.678 T1=3.57
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-16.xml	OPT f=885 TT=2.083 T1=1.94	OPT f=885 TT=2.112 T1=2.06	OPT f=885 TT=2.083 T1=1.94	OPT f=885 TT=2.094 T1=2.01	No data	No data	OPT f=885 TT=15.108 T1=10.61
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-00.xml	OPT f=932 TT=26.094 T1=19.88	OPT f=932 TT=26.37 T1=20.09	OPT f=932 TT=26.11 T1=19.88	OPT f=932 TT=26.094 T1=19.92	No data	No data	OPT f=932 TT=162.447 T1=153.26
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-07.xml	OPT f=1021 TT=38.51 T1=20.43	OPT f=1021 TT=38.908 T1=20.65	OPT f=1021 TT=38.51 T1=20.43	OPT f=1021 TT=38.575 T1=20.53	No data	No data	OPT f=1021 TT=623.446 T1=400.16
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-08.xml	OPT f=887 TT=14.887 T1=12.92	OPT f=887 TT=14.969 T1=12.94	OPT f=887 TT=14.945 T1=12.92	OPT f=887 TT=14.887 T1=12.93	No data	No data	OPT f=887 TT=44.76 T1=39.92
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-12.xml	OPT f=954 TT=45.783 T1=44.67	OPT f=954 TT=46.4 T1=45.18	OPT f=954 TT=46.271 T1=45.05	OPT f=954 TT=45.783 T1=44.67	No data	No data	OPT f=954 TT=351.088 T1=350.57
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-16.xml	OPT f=886 TT=79.505 T1=62.58	OPT f=886 TT=80.564 T1=76.27	OPT f=886 TT=79.72 T1=75.46	OPT f=886 TT=79.505 T1=75.34	No data	No data	OPT f=886 TT=99.531 T1=62.58
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-16.xml	OPT f=1429 TT=1086.89 T1=1062.32	OPT f=1429 TT=1094.23 T1=1069.71	OPT f=1429 TT=1088.48 T1=1063.7	OPT f=1429 TT=1086.89 T1=1062.32	No data	No data	SAT (TO) f=1309 TT=2400.08 T1=2298.09
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-17.xml	OPT f=992 TT=556.538 T1=528.35	OPT f=992 TT=561.291 T1=532.97	OPT f=992 TT=557.622 T1=529.42	OPT f=992 TT=556.538 T1=528.35	No data	No data	SAT (TO) f=992 TT=2400.1 T1=1201.37
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-18.xml	OPT f=937 TT=122.612 T1=79.84	OPT f=937 TT=123.895 T1=80.82	OPT f=937 TT=122.612 T1=79.84	OPT f=937 TT=123.291 T1=80.38	No data	No data	OPT f=937 TT=1012.25 T1=858.25
Knapsack/ Knapsack-m1-s60/ Knapsack-60-250-03.xml	SAT f=1272 TT=2400.02 T1=2096.24	SAT (TO) f=1272 TT=2400.06 T1=2113.65	? (TO) TT=2400.1	SAT (TO) f=1272 TT=2400.02 T1=2096.24	No data	No data	SAT (TO) f=1271 TT=2400.07 T1=872.84
Knapsack/ Knapsack-m1-s60/ Knapsack-60-250-05.xml	OPT f=1333 TT=1009.19 T1=987.05	OPT f=1333 TT=1028.67 T1=1006	OPT f=1333 TT=1017 T1=994.61	OPT f=1333 TT=1009.19 T1=987.05	No data	No data	SAT (TO) f=1165 TT=2400.01 T1=2203.96
Knapsack/ Knapsack-m1-s60/ Knapsack-60-250-06.xml	OPT f=1134 TT=1046.64 T1=966.41	OPT f=1134 TT=1053.31 T1=972.29	OPT f=1134 TT=1053.55 T1=972.37	OPT f=1134 TT=1046.64 T1=966.41	No data	No data	SAT (TO) f=1038 TT=2400.06 T1=1650.46
Knapsack/ Knapsack-m1-s60/ Knapsack-60-250-07.xml	OPT f=1058 TT=1536.49 T1=1335.14	OPT f=1058 TT=1553.58 T1=1350.05	OPT f=1058 TT=1536.49 T1=1335.14	OPT f=1058 TT=1537.42 T1=1336.11	No data	No data	SAT (TO) f=1032 TT=2400.01 T1=463.08
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Knapsack/ Knapsack-m1-s60/ Knapsack-60-250-13.xml	SAT f=1467 TT=2400.01 T1=2322.4	SAT (TO) f=1467 TT=2400.06 T1=2338.49	? (TO) TT=2400.07	SAT (TO) f=1467 TT=2400.01 T1=2322.4	No data	No data	SAT (TO) f=1193 TT=2400.02 T1=2018.77
Knapsack/ Knapsack-m1-s60/ Knapsack-60-250-17.xml	SAT f=1152 TT=2400.03 T1=1923.07	SAT (TO) f=1152 TT=2400.1 T1=1941.83	? (TO) TT=2400.06	SAT (TO) f=1152 TT=2400.03 T1=1923.07	No data	No data	SAT (TO) f=1062 TT=2400.02 T1=1764.42
Langford/ Langford-m1-k2/ Langford-2-18.xml		? (TO) TT=2400.06	? (TO) TT=2400.08	? (TO) TT=2400.1	? TT=2400	? TT=2400.01	? (TO) TT=2400.05
Langford/ Langford-m1-k3/ Langford-3-05.xml	UNSAT TT=0.009	UNSAT TT=0.017	UNSAT TT=0.015	UNSAT TT=0.015	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.022
Langford/ Langford-m1-k3/ Langford-3-70.xml		? (TO) TT=2400.06	? (TO) TT=2400.06	? (TO) TT=2400.04	? TT=2400.08	? (TO) TT=2400.08	? (TO) TT=2400.03
Langford/ Langford-m1-k4/ Langford-4-04.xml	UNSAT TT=0.011	UNSAT TT=0.018	UNSAT TT=0.017	UNSAT TT=0.017	UNSAT TT=0.01	UNSAT TT=0.011	UNSAT TT=0.021
Langford/ Langford-m1-k4/ Langford-4-05.xml	UNSAT TT=0.013	UNSAT TT=0.022	UNSAT TT=0.02	UNSAT TT=0.021	UNSAT TT=0.012	UNSAT TT=0.013	UNSAT TT=0.027
Langford/ Langford-m1-k4/ Langford-4-35.xml		? (TO) TT=2400.03	? (TO) TT=2400.09	? (TO) TT=2400.07	? TT=2400.04	? (TO) TT=2400.02	? (TO) TT=2400.1
Langford/ Langford-m2-s1/ LangfordBin-16.xml		? (TO) TT=2400.07	? (TO) TT=2400.1	? (TO) TT=2400.02	? TT=2400.01	? TT=2400	? (TO) TT=2400.07
Langford/ Langford-m2-s1/ LangfordBin-18.xml		? (TO) TT=2400.09	? (TO) TT=2400.02	? (TO) TT=2400.02	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.05
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Langford/ Langford-m2-s1/ LangfordBin-19.xml		? (TO) TT=2400.02	? (TO) TT=2400.07	? (TO) TT=2400.06	? TT=2400	? TT=2400.01	? (TO) TT=2400.1
Langford/ Langford-m2-s1/ LangfordBin-30.xml		? (TO) TT=2400.03	? (TO) TT=2400.01	? (TO) TT=2400.07	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.03
Langford/ Langford-m2-s1/ LangfordBin-45.xml		? (TO) TT=2400.1	? (TO) TT=2400.06	? (TO) TT=2400.09	? TT=2400.01	? TT=2400.01	? (TO) TT=2400.04
Langford/ Langford-m2-s1/ LangfordBin-90.xml		? (TO) TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.02	? (TO) TT=2400.04	? (TO) TT=2400.03	? (TO) TT=2400.01
LatinSquare/ LatinSquare-xcsp2-bqwh15-106/ bqwh-15-106-35_X2.xml	SAT TT=0.165	SAT TT=0.008	SAT TT=0.008	SAT TT=0.008	SAT TT=0.007	SAT TT=0.008	SAT TT=0.165
LatinSquare/ LatinSquare-xcsp2-bqwh15-106/ bqwh-15-106-36_X2.xml	SAT TT=0.048	SAT TT=0.01	SAT TT=0.009	SAT TT=0.01	SAT TT=0.009	SAT TT=0.008	SAT TT=0.048
LatinSquare/ LatinSquare-xcsp2-bqwh18-141/ bqwh-18-141-09_X2.xml	SAT TT=0.049	SAT TT=0.049	SAT TT=0.049	SAT TT=0.049	SAT TT=0.026	SAT TT=0.025	SAT TT=4.29
LatinSquare/ LatinSquare-xcsp2-bqwh18-141/ bqwh-18-141-31_X2.xml	SAT TT=1.409	SAT TT=1.409	SAT TT=1.415	SAT TT=1.415	SAT TT=0.029	SAT TT=0.029	SAT TT=0.079
LatinSquare/ LatinSquare-xcsp2-bqwh18-141/ bqwh-18-141-83_X2.xml	SAT TT=0.075	SAT TT=0.074	SAT TT=0.076	SAT TT=0.075	SAT TT=0.049	SAT TT=0.049	SAT TT=2.776
LatinSquare/ LatinSquare-xcsp2-qwh10/ qwh-10-57-7_X2.xml	SAT TT=0.028	SAT TT=0.028	SAT TT=0.028	SAT TT=0.028	SAT TT=0.028	SAT TT=0.028	SAT TT=0.029
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-004.xml	OPT f=2 TT=0.014 T1=0.01	OPT f=2 TT=0.014 T1=0.01	OPT f=2 TT=0.014 T1=0.01	OPT f=2 TT=0.014 T1=0.07	No data	No data	OPT f=2 TT=0.019 T1=0.01
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-011.xml	OPT f=5 TT=0.143 T1=0.08	OPT f=5 TT=0.147 T1=0.14	OPT f=5 TT=0.146 T1=0.08	OPT f=5 TT=0.143 T1=0.19	No data	No data	OPT f=5 TT=0.151 T1=0.11
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-013.xml	OPT f=6 TT=0.315 T1=0.16	OPT f=6 TT=0.459 T1=0.46	OPT f=6 TT=0.452 T1=0.39	OPT f=6 TT=0.449 T1=0.49	No data	No data	OPT f=6 TT=0.315 T1=0.16
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-015.xml	OPT f=15 TT=1.093 T1=0.18	OPT f=15 TT=1.888 T1=1.88	OPT f=15 TT=1.888 T1=0.18	OPT f=15 TT=1.88 T1=1.97	No data	No data	OPT f=15 TT=1.093 T1=0.24
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-020.xml	OPT f=26 TT=33.696 T1=1.61	OPT f=26 TT=126.097 T1=126.18	OPT f=26 TT=126.108 T1=65.36	OPT f=26 TT=125.797 T1=125.85	No data	No data	OPT f=26 TT=33.696 T1=1.61
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-033.xml	SAT f=64 TT=2400.09 T1=2123.5	SAT (TO) f=120 TT=2400.09 T1=2400.1	No Cert. TT=2400.06	SAT (TO) f=120 TT=2400.03 T1=2400	No data	No data	SAT (TO) f=64 TT=2400.09 T1=2123.5
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-036.xml	SAT f=126 TT=2400.03 T1=2364.79	SAT (TO) f=126 TT=2400.04 T1=2400.2	No Cert. TT=2400.06	SAT (TO) f=126 TT=2400.09 T1=2400.3	No data	No data	SAT (TO) f=126 TT=2400.03 T1=2364.79
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-038.xml	SAT f=295 TT=2400.02 T1=1933.36	SAT (TO) f=503 TT=2400.01 T1=2399.7	No Cert. TT=2400.1	SAT (TO) f=503 TT=2400.07 T1=2400.3	No data	No data	SAT (TO) f=295 TT=2400.02 T1=1933.36
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-040.xml	SAT f=476 TT=2400.08 T1=2386.19	SAT (TO) f=500 TT=2400.03 T1=2399.8	No Cert. TT=2400.09	SAT (TO) f=500 TT=2400.06 T1=2399.9	No data	No data	SAT (TO) f=476 TT=2400.08 T1=2386.19
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-042.xml	SAT f=269 TT=2400.04 T1=2399.9	SAT (TO) f=269 TT=2400.06 T1=2399.9	No Cert. TT=2400.04	SAT (TO) f=269 TT=2400.04 T1=2399.9	No data	No data	SAT (TO) f=829 TT=2400.02 T1=2392
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-054.xml	SAT f=471 TT=2400.03 T1=2399.9	SAT (TO) f=471 TT=2400.03 T1=2399.9	No Cert. TT=2400.01	SAT (TO) f=471 TT=2400.06 T1=2400.4	No data	No data	SAT (TO) f=5503 TT=2400.02 T1=2382.28
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-055.xml	SAT f=683 TT=2400.08 T1=2400.1	SAT (TO) f=683 TT=2400.08 T1=2400.1	No Cert. TT=2400.05	SAT (TO) f=683 TT=2400.09 T1=2400.1	No data	No data	SAT (TO) f=6211 TT=2400.05 T1=2355.65
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-057.xml	SAT f=772 TT=2400.1 T1=2400	SAT (TO) f=772 TT=2400.1 T1=2400.1	No Cert. TT=2400.1	SAT (TO) f=772 TT=2400.1 T1=2400	No data	No data	SAT (TO) f=7736 TT=2400.1 T1=2315.93
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-069.xml	SAT f=1466 TT=2400.03 T1=1468.36	SAT (TO) f=1466 TT=2400.11 T1=2400.01	SAT (TO) f=1466 TT=2400.06 T1=1468.36	SAT (TO) f=1466 TT=2400.03 T1=2400.2	No data	No data	SAT (TO) f=20618 TT=2400.1 T1=2358.01
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-081.xml	SAT f=1716 TT=2400.02 T1=2400.11	SAT (TO) f=1716 TT=2400.02 T1=2400.11	No Cert. TT=2400.07	SAT (TO) f=1716 TT=2400.07 T1=2400.3	No data	No data	SAT (TO) f=42060 TT=2400.07 T1=2394.1
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-084.xml	SAT f=2798 TT=2400.05 T1=2400	SAT (TO) f=2798 TT=2400.05 T1=2400.2	No Cert. TT=2400.01	SAT (TO) f=2798 TT=2400.08 T1=2400	No data	No data	SAT (TO) f=49370 TT=2400.02 T1=2276.97
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-096.xml	SAT f=3292 TT=2400.07 T1=2400	SAT (TO) f=3292 TT=2400.07 T1=2400.8	No Cert. TT=2400.02	SAT (TO) f=3292 TT=2400.08 T1=2400	No data	No data	SAT (TO) f=86840 TT=2400.1 T1=1868.65
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-02-0000.xml	UNSAT TT=0.004	UNSAT TT=0.005	UNSAT TT=0.006	UNSAT TT=0.005	UNSAT TT=0.004	UNSAT TT=0.004	UNSAT TT=0.007
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-08-0051.xml		? (TO) TT=2400.01	? (TO) TT=2400.06	? (TO) TT=2400.07	? (TO) TT=2400.05	? (TO) TT=2400.05	? (TO) TT=2400.06
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-08-0081.xml		? (TO) TT=2400.03	? (TO) TT=2400.08	? (TO) TT=2400.1	? (TO) TT=2400.09	? (TO) TT=2400.06	? (TO) TT=2400.08
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-10-m021.xml		? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.08	? (TO) TT=2400.04	? (TO) TT=2400.05
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-11-0066.xml		? (TO) TT=2400.09	? (TO) TT=2400.07	? (TO) TT=2400.04	? (TO) TT=2400.05	? (TO) TT=2400.06	? (TO) TT=2400.06
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-11-m186.xml		? (TO) TT=2400.06	? (TO) TT=2400.02	? (TO) TT=2400.08	? (TO) TT=2400.08	? (TO) TT=2400.04	? (TO) TT=2400.08
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-18-sum.xml		? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.09	? (TO) TT=2400.06	? (TO) TT=2400.06	? (TO) TT=2400.1
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-22-sum.xml		? (TO) TT=2400.11	? (TO) TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.08	? (TO) TT=2400.02	? (TO) TT=2400.02
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-23-sum.xml		? (TO) TT=2400.03	? (TO) TT=2400.1	? (TO) TT=2400.07	? (TO) TT=2400.1	? (TO) TT=2400.05	? (TO) TT=2400.05
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-25-sum.xml		? (TO) TT=2400.04	? (TO) TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.08	? (TO) TT=2400.01	? (TO) TT=2400.09
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-26-sum.xml		? (TO) TT=2400.09	? (TO) TT=2400.02	? (TO) TT=2400.05	? (TO) TT=2400.07	? (TO) TT=2400.1	? (TO) TT=2400.02
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-3-sum.xml	SAT TT=0.004	SAT TT=0.004	SAT TT=0.005	SAT TT=0.004	SAT TT=0.004	SAT TT=0.004	SAT TT=0.003
MagicSquare/ MagicSquare-table-s1/ MagicSquare-3-table.xml	SAT TT=0.004	SAT TT=0.005	SAT TT=0.005	SAT TT=0.004	SAT TT=0.004	SAT TT=0.004	SAT TT=0.003
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
MagicSquare/ MagicSquare-table-s1/ MagicSquare-4-table.xml	SAT TT=0.047	SAT TT=0.036	SAT TT=0.035	SAT TT=0.036	SAT TT=0.045	SAT TT=0.04	SAT TT=0.047
MagicSquare/ MagicSquare-table-s1/ MagicSquare-5-table.xml	SAT TT=19.912	SAT TT=2.974	SAT TT=2.985	SAT TT=2.967	SAT TT=20.278	SAT TT=19.912	SAT TT=41.62
MagicSquare/ MagicSquare-table-s1/ MagicSquare-6-table.xml	SAT TT=476.936	SAT TT=476.936	SAT TT=482.482	SAT TT=478.532	? (MO) TT=74.052	? (MO) TT=79.356	? (MO) TT=156.352
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-03.xml	SAT TT=111.776	SAT TT=33.725	SAT TT=33.669	SAT TT=33.633	? TT=2400.01	? TT=2400.02	SAT TT=111.776
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-05.xml	SAT TT=21.761	SAT TT=21.761	SAT TT=21.747	SAT TT=21.842	? TT=2400.01	? (TO) TT=2400.1	SAT TT=140.343
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-07.xml	SAT TT=65.622	SAT TT=94.989	SAT TT=94.864	SAT TT=94.793	? TT=2400.06	? (TO) TT=2400.04	SAT TT=65.622
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-08.xml	SAT TT=33.588	SAT TT=33.588	SAT TT=33.611	SAT TT=33.577	SAT TT=2291.99	SAT TT=2265.65	SAT TT=51.639
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-09.xml	SAT TT=13.881	SAT TT=13.881	SAT TT=14.032	SAT TT=13.936	? TT=2400.01	? TT=2400.04	SAT TT=142.921
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-10.xml	SAT TT=1563.25	SAT TT=273.194	SAT TT=276.528	SAT TT=273.126	SAT TT=1562.3	SAT TT=1563.25	SAT TT=65.168
Mixed/ Mixed-xcsp2-cril/ cril-1_X2.xml	UNSAT TT=0.185	UNSAT TT=0.19	UNSAT TT=0.186	UNSAT TT=0.185	UNSAT TT=1.081	UNSAT TT=1.094	? (TO) TT=2400.03
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Mixed/ Mixed-xcsp2-cril/ cril-3_X2.xml	UNSAT TT=51.831	UNSAT TT=73.008	UNSAT TT=70.463	UNSAT TT=70.396	UNSAT TT=51.089	UNSAT TT=51.831	? (TO) TT=2400.02
Mixed/ Mixed-xcsp2-cril/ cril-5_X2.xml	UNSAT TT=0.111	UNSAT TT=0.113	UNSAT TT=0.111	UNSAT TT=0.111	UNSAT TT=2.501	UNSAT TT=2.363	? (TO) TT=2400.07
Mixed/ Mixed-xcsp2-cril/ cril-7_X2.xml	UNSAT TT=10.149	UNSAT TT=10.155	UNSAT TT=10.121	UNSAT TT=10.149	UNSAT TT=27.303	UNSAT TT=27.317	? (TO) TT=2400.02
Mixed/ Mixed-xcsp2-nengfa/ color_X2.xml	UNSAT TT=12.019	UNSAT TT=12.227	UNSAT TT=12.071	UNSAT TT=12.019	UNSAT TT=21.383	UNSAT TT=21.201	UNSAT TT=24.793
Mixed/ Mixed-xcsp2-nengfa/ sudoku368_X2.xml	SAT TT=0.132	SAT TT=0.132	SAT TT=0.133	SAT TT=0.132	SAT TT=0.557	SAT TT=0.562	? (TO) TT=2400.1
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-1-03.xml	SAT TT=0.02	SAT TT=0.02	SAT TT=0.022	SAT TT=0.02	SAT TT=0.089	SAT TT=0.09	SAT TT=0.176
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-2-16.xml	SAT TT=8.067	SAT TT=8.229	SAT TT=8.129	SAT TT=8.067	? (TO) TT=2400.02	? (TO) TT=2400.03	SAT TT=22.549
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-2-21.xml	SAT TT=9.597	SAT TT=9.854	SAT TT=9.58	SAT TT=9.597	? (TO) TT=2400.1	? (TO) TT=2400.01	SAT TT=12.136
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-2-22.xml	SAT TT=48.057	SAT TT=2.673	SAT TT=2.608	SAT TT=2.614	? (TO) TT=2400	? (TO) TT=2400.09	SAT TT=48.057
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-2-41.xml	SAT TT=0.213	SAT TT=0.215	SAT TT=0.214	SAT TT=0.213	SAT TT=37.559	SAT TT=37.45	SAT TT=1.089
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-2-48.xml	SAT TT=18.31	SAT TT=18.31	SAT TT=17.837	SAT TT=18.037	? (TO) TT=2400.11	? (TO) TT=2400.06	SAT TT=15.827
MultiKnapsack/ MultiKnapsack-xcsp2-s1/ MultiKnapsack-1-5_X2.xml	SAT TT=34.792	SAT TT=0.358	SAT TT=0.348	SAT TT=0.349	? (TO) TT=2400.03	? (TO) TT=2400.09	SAT TT=34.792
MultiKnapsack/ MultiKnapsack-xcsp2-s1/ MultiKnapsack-1-6_X2.xml	SAT TT=271.731	SAT TT=276.112	SAT TT=271.463	SAT TT=271.731	? (TO) TT=2400.1	? (TO) TT=2400.06	? (TO) TT=2400.08
Nonogram/ Nonogram-table-s1/ Nonogram-018-table.xml	SAT TT=1.268	SAT TT=1.227	SAT TT=1.269	SAT TT=1.268	SAT TT=0.513	SAT TT=0.532	SAT TT=1.559
Nonogram/ Nonogram-table-s1/ Nonogram-035-table.xml	SAT TT=0.186	SAT TT=0.182	SAT TT=0.186	SAT TT=0.186	SAT TT=0.337	SAT TT=0.347	SAT TT=1.259
Nonogram/ Nonogram-table-s1/ Nonogram-096-table.xml	SAT TT=0.76	SAT TT=2.66	SAT TT=2.744	SAT TT=2.73	SAT TT=0.735	SAT TT=0.76	SAT TT=3.019
Nonogram/ Nonogram-table-s1/ Nonogram-168-table.xml	SAT TT=0.141	SAT TT=0.412	SAT TT=0.424	SAT TT=0.422	SAT TT=0.132	SAT TT=0.141	SAT TT=0.477
Nonogram/ Nonogram-table-s1/ Nonogram-177-table.xml	SAT TT=1.189	SAT TT=1.156	SAT TT=1.193	SAT TT=1.189	SAT TT=0.346	SAT TT=0.357	SAT TT=1.263
Nonogram/ Nonogram-table-s1/ Nonogram-180-table.xml	SAT TT=3.889	SAT TT=12.33	SAT TT=12.755	SAT TT=12.745	SAT TT=3.634	SAT TT=3.889	SAT TT=16.561
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-07-05.xml	UNSAT TT=0.562	UNSAT TT=0.57	UNSAT TT=0.565	UNSAT TT=0.562	UNSAT TT=0.841	UNSAT TT=0.832	UNSAT TT=4.184
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-07-08.xml	UNSAT TT=175.814	UNSAT TT=175.814	UNSAT TT=175.343	UNSAT TT=176.472	UNSAT TT=461.526	UNSAT TT=464.854	UNSAT TT=1734.38
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-07-09.xml	UNSAT TT=1150.83	UNSAT TT=1152.43	UNSAT TT=1155.14	UNSAT TT=1150.83	? (MO) TT=59.209	? (MO) TT=54.106	? (TO) TT=2400.27
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-08-04.xml	UNSAT TT=1.469	UNSAT TT=1.474	UNSAT TT=1.471	UNSAT TT=1.469	UNSAT TT=1.557	UNSAT TT=1.513	UNSAT TT=17.119
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-09-03.xml	UNSAT TT=1.967	UNSAT TT=1.967	UNSAT TT=1.971	UNSAT TT=1.983	UNSAT TT=2.255	UNSAT TT=2.293	UNSAT TT=92.022
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-11-04.xml		? (TO) TT=2400.1	? (TO) TT=2400.06	? (TO) TT=2400.07	? TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.03
Primes/Primes-m1-p10/ Primes-10-20-3-3.xml	SAT TT=3.132	SAT TT=0.006	SAT TT=0.006	SAT TT=0.006	SAT TT=3.131	SAT TT=3.132	SAT TT=0.007
Primes/Primes-m1-p10/ Primes-10-60-3-3.xml	SAT TT=0.028	SAT TT=0.027	SAT TT=0.028	SAT TT=0.028	SAT TT=6.748	SAT TT=6.744	SAT TT=276.301
Primes/Primes-m1-p15/ Primes-15-20-2-5.xml	SAT TT=0.009	SAT TT=0.008	SAT TT=0.008	SAT TT=0.009	? (TO) TT=2400.08	? (TO) TT=2400.02	SAT TT=0.026
Primes/Primes-m1-p15/ Primes-15-60-3-5.xml		? (TO) TT=2400.06	? (TO) TT=2400.06	? (TO) TT=2400	? (TO) TT=2400.06	? (TO) TT=2400.01	? (TO) TT=2400.04
Primes/Primes-m1-p20/ Primes-20-40-2-1.xml	SAT TT=0.006	SAT TT=0.006	SAT TT=0.006	SAT TT=0.007	SAT TT=0.024	SAT TT=0.024	SAT TT=0.009
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Primes/Primes-m1-p30/ Primes-30-60-3-1.xml	SAT TT=0.01	SAT TT=0.009	SAT TT=0.009	SAT TT=0.01	SAT TT=12.697	SAT TT=12.563	SAT TT=0.1
PropStress/ PropStress-m1-s1/ PropStress-0020.xml	UNSAT TT=0.418	UNSAT TT=2.914	UNSAT TT=2.857	UNSAT TT=2.857	UNSAT TT=1.012	UNSAT TT=1.056	UNSAT TT=0.418
PropStress/ PropStress-m1-s1/ PropStress-0100.xml	UNSAT TT=21.583	UNSAT TT=76.195	UNSAT TT=77.559	UNSAT TT=77.331	? (TO) TT=2400.12	? (TO) TT=2400.13	UNSAT TT=21.583
PropStress/ PropStress-m1-s1/ PropStress-0150.xml	UNSAT TT=128.82	? (TO) TT=2400.26	? (TO) TT=2400.39	? (TO) TT=2400.3	? (TO) TT=2400.1	? (TO) TT=2400.19	UNSAT TT=128.82
PropStress/ PropStress-m1-s1/ PropStress-0250.xml	UNSAT TT=1299.17	? TT=16.338	? TT=15.392	? TT=15.448	? (MO) TT=2175.79	? (MO) TT=2200.46	UNSAT TT=1299.17
PropStress/ PropStress-m1-s1/ PropStress-0450.xml		? (MO) TT=10.097	? TT=10.162	? TT=9.678	? (TO) TT=2400.04	? (TO) TT=2400.07	? (TO) TT=2400.04
PropStress/ PropStress-m1-s1/ PropStress-0600.xml		? TT=7.338	? TT=7.194	? TT=7.126	? (TO) TT=2400.11	? (TO) TT=2400.06	? (TO) TT=2400.05
PseudoBoolean/ PseudoBoolean-dec-fpga/ Pb-chnl15-16.xml		? (TO) TT=2400.02	? (TO) TT=2400.03	? (TO) TT=2400.1	? TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.1
PseudoBoolean/ PseudoBoolean-dec-pigeon/ Pb-pigeon-card-085-080.xml		? (TO) TT=2400.07	? (TO) TT=2400.04	? (TO) TT=2400.1	? (TO) TT=2400.04	? (TO) TT=2400.02	? (TO) TT=2400.07
PseudoBoolean/ PseudoBoolean-dec-queen/ Pb-queen-0974553.xml	SAT TT=0.553	SAT TT=160.034	SAT TT=155.544	SAT TT=155.762	SAT TT=84.538	SAT TT=83.761	SAT TT=0.553
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
PseudoBoolean/ PseudoBoolean-dec-spng/ Pb-NG-K145.xml		? (TO) TT=2400.12	? (TO) TT=2400.07	? (TO) TT=2400.11	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.08
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K097.xml		? (TO) TT=2400.06	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.05	? (TO) TT=2400.08
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K108.xml		? (TO) TT=2400.05	? (TO) TT=2400.02	? (TO) TT=2400.07	? (TO) TT=2400.05	? (TO) TT=2400.01	? (TO) TT=2400.05
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-aim-050-3-4-yes1-1.xml	OPT f=50 TT=0.072 T1=0.02	OPT f=50 TT=0.072 T1=0.07	OPT f=50 TT=0.072 T1=0.02	OPT f=50 TT=0.072 T1=0.12	No data	No data	OPT f=50 TT=0.783 T1=0.43
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-ii08a1.xml	OPT f=54 TT=13.812 T1=0.01	OPT f=54 TT=13.974 T1=13.97	OPT f=54 TT=13.812 T1=0.01	OPT f=54 TT=13.82 T1=13.88	No data	No data	OPT f=54 TT=1046.13 T1=11.37
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-ii8d2.xml	SAT f=627 TT=2400.04 T1=2399.9	SAT (TO) f=627 TT=2400.04 T1=2399.9	No Cert. TT=2400.07	SAT (TO) f=627 TT=2400.09 T1=2400	No data	No data	SAT (TO) f=856 TT=2400.1 T1=2342.38
PseudoBoolean/ PseudoBoolean-opt-domset/ Pb-domset-v500-e2000-w30-mw19-11.xml	SAT f=225 TT=2400.06 T1=2400.1	SAT (TO) f=225 TT=2400.09 T1=2400.1	No Cert. TT=2400.09	SAT (TO) f=225 TT=2400.06 T1=2400.3	No data	No data	SAT (TO) f=230 TT=2400.01 T1=0.09
PseudoBoolean/ PseudoBoolean-opt-factor/ Pb-factor-S9-P071-Q409.xml	OPT f=71 TT=0.067 T1=0.06	OPT f=71 TT=0.067 T1=0.06	OPT f=71 TT=0.069 T1=0.06	OPT f=71 TT=0.068 T1=0.1	No data	No data	OPT f=71 TT=4.383 T1=4.36
PseudoBoolean/ PseudoBoolean-opt-factor/ Pb-factor-S9-P167-Q347.xml	OPT f=167 TT=0.061 T1=0.04	OPT f=167 TT=0.061 T1=0.06	OPT f=167 TT=0.061 T1=0.04	OPT f=167 TT=0.061 T1=0.16	No data	No data	OPT f=167 TT=6.471 T1=6.36
PseudoBoolean/ PseudoBoolean-opt-factor/ Pb-factor-S9-P251-Q379.xml	OPT f=251 TT=0.096 T1=0.07	OPT f=251 TT=0.097 T1=0.09	OPT f=251 TT=0.096 T1=0.07	OPT f=251 TT=0.097 T1=0.2	No data	No data	OPT f=251 TT=7.348 T1=7.15
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
PseudoBoolean/ PseudoBoolean-opt-factor/ Pb-factor-S9-P409-Q499.xml	OPT f=409 TT=0.028 T1=0.02	OPT f=409 TT=0.028 T1=0.02	OPT f=409 TT=0.028 T1=0.02	OPT f=409 TT=0.029 T1=0.11	No data	No data	OPT f=409 TT=1.29 T1=1.26
PseudoBoolean/ PseudoBoolean-opt-fs/ Pb-fs-50-250-0-45-90-4-1000.xml	SAT f=-142 TT=2400.07 T1=2400.1	SAT (TO) f=-142 TT=2400.07 T1=2400.11	No Cert. TT=2400.08	SAT (TO) f=-142 TT=2400.08 T1=2400.1	No data	No data	SAT (TO) f=-132 TT=2400.07 T1=1671.36
PseudoBoolean/ PseudoBoolean-opt-gr/ Pb-gr-05.xml	OPT f=11 TT=0.095 T1=0.06	OPT f=11 TT=0.096 T1=0.16	OPT f=11 TT=0.095 T1=0.06	OPT f=11 TT=0.097 T1=0.19	No data	No data	OPT f=11 TT=0.748 T1=0.73
PseudoBoolean/ PseudoBoolean-opt-logicSynthesis/ Pb-logic-m100-100-30-30-r.xml	OPT f=5 TT=901.153 T1=838.99	SAT (TO) f=7 TT=2400.08 T1=2400.2	? (TO) TT=2400.08	SAT (TO) f=7 TT=2400.05 T1=2400.3	No data	No data	OPT f=5 TT=901.153 T1=838.99
PseudoBoolean/ PseudoBoolean-opt-marketSplit/ Pb-msplit-7-60-4.xml		? (TO) TT=2400.04	? (TO) TT=2400.08	? (TO) TT=2400.01	No data	No data	? (TO) TT=2400.05
PseudoBoolean/ PseudoBoolean-opt-mps/ Pb-mps-v2-20-10-l152lav.xml		? (TO) TT=2400.02	? (TO) TT=2400.09	? (TO) TT=2400.02	No data	No data	? (TO) TT=2400.08
PseudoBoolean/ PseudoBoolean-opt-mps/ Pb-mps-v2-20-10-sentoy.xml	OPT f=-7772 TT=651.715 T1=463.11	OPT f=-7772 TT=668.218 T1=668.24	OPT f=-7772 TT=652.406 T1=463.11	OPT f=-7772 TT=651.715 T1=651.83	No data	No data	SAT (TO) f=-5921 TT=2400.08 T1=1424.54
PseudoBoolean/ PseudoBoolean-opt-radar/ Pb-radar-10-10-45-095-100.xml	SAT f=8 TT=2400.05 T1=400.04	SAT (TO) f=8 TT=2400.05 T1=2399.9	SAT (TO) f=9 TT=2400.07 T1=654.07	SAT (TO) f=9 TT=2400.1 T1=2400.3	No data	No data	SAT (TO) f=8 TT=2400.07 T1=400.04
PseudoBoolean/ PseudoBoolean-opt-routing/ Pb-routing-s3-3-3-1.xml	OPT f=36 TT=0.342 T1=0.13	OPT f=36 TT=0.344 T1=0.34	OPT f=36 TT=0.355 T1=0.13	OPT f=36 TT=0.342 T1=0.44	No data	No data	OPT f=36 TT=18.831 T1=10.89
PseudoBoolean/ PseudoBoolean-opt-vtxcov/ Pb-vtxcov-v2000-e4000-05.xml	SAT f=1180 TT=2400.04 T1=2400.1	SAT (TO) f=1180 TT=2400.05 T1=2400.1	No Cert. TT=2400.01	SAT (TO) f=1180 TT=2400.04 T1=2400.2	No data	No data	SAT (TO) f=1195 TT=2400.02 T1=0.39
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
QRandom/QRandom-bdd-15-21-2/ bdd-15-21-2-2713-79-08.xml	SAT TT=1.985	SAT TT=1.985	SAT TT=1.957	SAT TT=1.955	SAT TT=19.937	SAT TT=20.048	SAT TT=21.023
QRandom/QRandom-bdd-15-21-2/ bdd-15-21-2-2713-79-16.xml	UNSAT TT=193.246	UNSAT TT=193.246	UNSAT TT=192.837	UNSAT TT=193.574	UNSAT TT=866.059	UNSAT TT=751.245	UNSAT TT=1208.25
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-06.xml	UNSAT TT=120.037	UNSAT TT=120.069	UNSAT TT=115.198	UNSAT TT=120.037	? (TO) TT=2400.08	? (TO) TT=2400.09	? (TO) TT=2400.07
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-24.xml	UNSAT TT=117.121	UNSAT TT=117.121	UNSAT TT=118.191	UNSAT TT=120.255	? (TO) TT=2400.08	? (TO) TT=2400.28	? (TO) TT=2400.02
QRandom/ QRandom-composed-s25/ composed-25-01-25-3.xml	UNSAT TT=0.011	UNSAT TT=0.019	UNSAT TT=0.019	UNSAT TT=0.019	UNSAT TT=0.011	UNSAT TT=0.011	UNSAT TT=0.021
QRandom/ QRandom-composed-s25/ composed-25-01-25-4.xml	UNSAT TT=0.011	UNSAT TT=0.019	UNSAT TT=0.019	UNSAT TT=0.02	UNSAT TT=0.012	UNSAT TT=0.011	? (TO) TT=2400.02
QRandom/ QRandom-composed-s25/ composed-25-10-20-5.xml	SAT TT=0.028	SAT TT=0.055	SAT TT=0.055	SAT TT=0.056	SAT TT=0.028	SAT TT=0.028	? (TO) TT=2400.04
QRandom/ QRandom-composed-s75/ composed-75-01-25-6.xml	UNSAT TT=0.025	UNSAT TT=0.044	UNSAT TT=0.043	UNSAT TT=0.043	UNSAT TT=0.025	UNSAT TT=0.025	? (TO) TT=2400.02
QRandom/ QRandom-ehi-s85/ ehi-85-297-30.xml	UNSAT TT=0.078	UNSAT TT=0.101	UNSAT TT=0.103	UNSAT TT=0.1	UNSAT TT=0.75	UNSAT TT=0.763	UNSAT TT=0.078
QRandom/ QRandom-ehi-s85/ ehi-85-297-98.xml	UNSAT TT=0.07	UNSAT TT=0.071	UNSAT TT=0.069	UNSAT TT=0.07	UNSAT TT=0.435	UNSAT TT=0.446	UNSAT TT=0.08
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
QRandom/ QRandom-ehi-s90/ ehi-90-315-13.xml	UNSAT TT=0.075	UNSAT TT=0.076	UNSAT TT=0.073	UNSAT TT=0.075	UNSAT TT=0.4	UNSAT TT=0.408	UNSAT TT=0.079
QRandom/ QRandom-ehi-s90/ ehi-90-315-37.xml	UNSAT TT=0.075	UNSAT TT=0.075	UNSAT TT=0.073	UNSAT TT=0.077	UNSAT TT=0.217	UNSAT TT=0.223	UNSAT TT=0.079
QRandom/QRandom-geometric-rw/ geometric-50-20-d4-75-03.xml	SAT TT=0.114	SAT TT=0.114	SAT TT=0.113	SAT TT=0.113	SAT TT=0.039	SAT TT=0.038	SAT TT=0.136
QRandom/QRandom-geometric-rw/ geometric-50-20-d4-75-14.xml	UNSAT TT=37.266	UNSAT TT=41.211	UNSAT TT=40.87	UNSAT TT=40.914	UNSAT TT=37.839	UNSAT TT=37.266	UNSAT TT=1274.88
QRandom/QRandom-geometric-rw/ geometric-50-20-d4-75-46.xml	SAT TT=0.136	SAT TT=0.136	SAT TT=0.135	SAT TT=0.136	SAT TT=0.055	SAT TT=0.055	SAT TT=5.03
QRandom/QRandom-geometric-rw/ geometric-50-20-d4-75-54.xml	SAT TT=0.12	SAT TT=0.12	SAT TT=0.119	SAT TT=0.119	SAT TT=0.033	SAT TT=0.034	SAT TT=0.153
QRandom/ QRandom-m1-lard/lard-83-83.xml	SAT TT=40.145	SAT TT=30.093	SAT TT=29.863	SAT TT=29.961	SAT TT=6.822	SAT TT=7.008	SAT TT=40.145
QRandom/ QRandom-m1-lard/lard-85-85.xml	SAT TT=33.133	SAT TT=33.133	SAT TT=32.855	SAT TT=32.975	SAT TT=7.663	SAT TT=7.72	SAT TT=43.576
QRandom/ QRandom-m1-lard/lard-90-90.xml	SAT TT=41.127	SAT TT=41.493	SAT TT=41.125	SAT TT=41.127	SAT TT=9.646	SAT TT=9.754	SAT TT=54.838
QRandom/ QRandom-m1-lard/lard-92-92.xml	SAT TT=61.696	SAT TT=46.68	SAT TT=46.453	SAT TT=46.715	SAT TT=10.843	SAT TT=10.976	SAT TT=61.696
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
QRandom/ QRandom-mdd-7-25-5/ mdd-7-25-5-56-02.xml	SAT TT=1295.11	SAT TT=1298.84	SAT TT=1297.8	SAT TT=1295.11	? (TO) TT=2400.09	? (TO) TT=2400.04	? (TO) TT=2400.07
QRandom/ QRandom-mdd-7-25-5/ mdd-7-25-5-56-16.xml		? (TO) TT=2400.01	? (TO) TT=2400.07	? (TO) TT=2400.07	? (TO) TT=2400.06	? (TO) TT=2400.04	? (TO) TT=2400.07
QRandom/QRandom-mdd-7-25-5-p7/ mdd-7-25-5-ps05-psh07-4.xml		? (TO) TT=2400.09	? (TO) TT=2400.02	? (TO) TT=2400.1	? (TO) TT=2400.1	? (TO) TT=2400.08	? (TO) TT=2400.1
QRandom/QRandom-mdd-7-25-5-p9/ mdd-7-25-5-ps05-psh09-3.xml	UNSAT TT=18.712	UNSAT TT=36.815	UNSAT TT=37.039	UNSAT TT=36.738	UNSAT TT=18.908	UNSAT TT=18.712	? (TO) TT=2400.03
QRandom/QRandom-reg2ext-s1/ reg-s20-p03-c20-d10-n10-l5-24.xml	UNSAT TT=10.396	UNSAT TT=12.461	UNSAT TT=12.388	UNSAT TT=12.409	UNSAT TT=10.089	UNSAT TT=10.396	? (TO) TT=2400.01
QRandom/QRandom-reg2ext-s1/ reg-s20-p03-c20-d10-n10-l5-30.xml	UNSAT TT=4.577	UNSAT TT=7.337	UNSAT TT=7.254	UNSAT TT=7.275	UNSAT TT=4.596	UNSAT TT=4.577	? (TO) TT=2400.1
QRandom/QRandom-reg2ext-s1/ reg-s20-p03-c20-d10-n10-l5-40.xml	UNSAT TT=7.356	UNSAT TT=10.278	UNSAT TT=10.295	UNSAT TT=10.353	UNSAT TT=7.14	UNSAT TT=7.356	? (TO) TT=2400.06
QRandom/QRandom-reg2ext-s1/ reg-s20-p03-c20-d10-n10-l5-73.xml	SAT TT=6.969	SAT TT=6.969	SAT TT=6.98	SAT TT=6.995	SAT TT=3.206	SAT TT=3.15	? (TO) TT=2400.02
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-bur26g.xml	SAT f=4370447 TT=2400.09 T1=2400.1	SAT (TO) f=4370447 TT=2400.09 T1=2400.1	No Cert. TT=2400.11	SAT (TO) f=4370447 TT=2400.1 T1=2400.1	No data	No data	SAT (TO) f=5091203 TT=2400.05 T1=1238.25
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-chr15c.xml	OPT f=4752 TT=46.039 T1=23.12	OPT f=4752 TT=46.18 T1=46.17	OPT f=4752 TT=46.263 T1=23.12	OPT f=4752 TT=46.039 T1=46.15	No data	No data	OPT f=4752 TT=61.832 T1=55.47
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-chr20b.xml	OPT f=1149 TT=25.939 T1=16.44	OPT f=1149 TT=25.939 T1=25.94	OPT f=1149 TT=26.028 T1=16.44	OPT f=1149 TT=26.071 T1=26.16	No data	No data	? (TO) TT=2400.1
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-esc16d.xml	OPT f=8 TT=130.158 T1=0.03	OPT f=8 TT=130.346 T1=130.35	OPT f=8 TT=131.775 T1=0.03	OPT f=8 TT=130.158 T1=130.24	No data	No data	? (TO) TT=2400.07
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-esc32a.xml	SAT f=93 TT=2400.02 T1=1893.36	SAT (TO) f=93 TT=2400.02 T1=2399.7	SAT (TO) f=93 TT=2400.03 T1=1893.36	SAT (TO) f=93 TT=2400.08 T1=2400.4	No data	No data	? (TO) TT=2400.1
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-esc32d.xml	SAT f=110 TT=2400.05 T1=22.84	SAT (TO) f=110 TT=2400.05 T1=2399.9	SAT (TO) f=110 TT=2400.08 T1=22.84	SAT (TO) f=110 TT=2400.09 T1=2400.1	No data	No data	? (TO) TT=2400.08
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-lipa20b.xml	SAT f=16264 TT=2400.01 T1=2399.9	SAT (TO) f=16264 TT=2400.06 T1=2399.9	No Cert. TT=2400.06	SAT (TO) f=16264 TT=2400.01 T1=2400	No data	No data	SAT (TO) f=16462 TT=2400.05 T1=1862.81
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-nug20.xml	SAT f=1438 TT=2400.04 T1=2400.1	SAT (TO) f=1438 TT=2400.04 T1=2400.1	No Cert. TT=2400.03	SAT (TO) f=1438 TT=2400.1 T1=2400.4	No data	No data	? (TO) TT=2400.1
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-nug21.xml	SAT f=1448 TT=2400.02 T1=2400	SAT (TO) f=1448 TT=2400.03 T1=2400	No Cert. TT=2400.08	SAT (TO) f=1448 TT=2400.02 T1=2400.2	No data	No data	? (TO) TT=2400.03
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-nug30.xml	SAT f=3716 TT=2400.05 T1=2399.8	SAT (TO) f=3716 TT=2400.05 T1=2399.8	No Cert. TT=2400.08	SAT (TO) f=3716 TT=2400.06 T1=2399.9	No data	No data	? (TO) TT=2400.09
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-sko100a.xml	SAT f=87786 TT=2400.01 T1=2399.9	SAT (TO) f=87786 TT=2400.05 T1=2400.01	No Cert. TT=2400.03	SAT (TO) f=87786 TT=2400.01 T1=2399.9	No data	No data	? (TO) TT=2400.02
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-sko81.xml	SAT f=53487 TT=2400.01 T1=2399.9	SAT (TO) f=53487 TT=2400.01 T1=2399.9	No Cert. TT=2400.1	SAT (TO) f=53487 TT=2400.07 T1=2400.3	No data	No data	? (TO) TT=2400.02
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-ste36c.xml	SAT f=7670520 TT=2400.04 T1=2400.2	SAT (TO) f=7670520 TT=2400.09 T1=2400.2	No Cert. TT=2400.09	SAT (TO) f=7670520 TT=2400.04 T1=2400.3	No data	No data	? (TO) TT=2400.01
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai10b.xml	OPT f=58937 TT=0.759 T1=0.11	OPT f=58937 TT=0.773 T1=0.77	OPT f=58937 TT=0.775 T1=0.65	OPT f=58937 TT=0.759 T1=0.82	No data	No data	OPT f=58937 TT=3.956 T1=0.11
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai12a.xml	SAT f=119956 TT=2400.08 T1=2399.9	SAT (TO) f=119956 TT=2400.09 T1=2399.9	? (TO) TT=2400.1	SAT (TO) f=119956 TT=2400.08 T1=2400.2	No data	No data	SAT (TO) f=121295 TT=2400.04 T1=548.55
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai15a.xml	SAT f=203776 TT=2400.05 T1=684.05	SAT (TO) f=207907 TT=2400.1 T1=2399.8	? (TO) TT=2400.02	SAT (TO) f=207907 TT=2400.04 T1=2399.9	No data	No data	SAT (TO) f=203776 TT=2400.05 T1=684.05
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai80a.xml	SAT f=7687952 TT=2400.02 T1=2400.01	SAT (TO) f=7687952 TT=2400.11 T1=2400.01	No Cert. TT=2400.07	SAT (TO) f=7687952 TT=2400.02 T1=2400.2	No data	No data	SAT (TO) f=7758317 TT=2400.11 T1=1502.78
Random/Random-B-2-23-23/ rand-2-23-23-253-131-0.xml	UNSAT TT=45.399	UNSAT TT=69.485	UNSAT TT=67.291	UNSAT TT=67.534	UNSAT TT=45.633	UNSAT TT=45.399	UNSAT TT=299.598
Random/Random-B-2-23-23/ rand-2-23-23-253-131-1.xml	UNSAT TT=40.973	UNSAT TT=63.142	UNSAT TT=62.704	UNSAT TT=62.605	UNSAT TT=41.233	UNSAT TT=40.973	UNSAT TT=277.43
Random/Random-B-2-25-25/ rand-2-25-25-300-147-6.xml	SAT TT=74.466	SAT TT=74.466	SAT TT=75.328	SAT TT=74.311	SAT TT=205.215	SAT TT=203.694	SAT TT=2076.22
Random/Random-B-2-26-26/ rand-2-26-26-325-155-2.xml	SAT TT=2026.13	SAT TT=52.315	SAT TT=51.405	SAT TT=51.929	SAT TT=94.198	SAT TT=93.496	SAT TT=2026.13
Random/Random-D-t020/ rand-2-40-11-414-020-23.xml	SAT TT=2.767	SAT TT=1.043	SAT TT=1.035	SAT TT=1.031	SAT TT=2.765	SAT TT=2.767	SAT TT=19.252
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Random/Random-D-t020/ rand-2-40-11-414-020-35.xml	SAT TT=1.889	SAT TT=12.429	SAT TT=12.357	SAT TT=12.296	SAT TT=6.852	SAT TT=6.839	SAT TT=1.889
Random/Random-D-t080/ rand-2-40-80-103-080-39.xml	SAT TT=2.096	SAT TT=2.096	SAT TT=2.061	SAT TT=2.066	SAT TT=2.514	SAT TT=2.483	? (TO) TT=2400.02
Random/Random-D-t090/ rand-2-40-180-84-090-51.xml	SAT TT=0.812	SAT TT=0.81	SAT TT=0.809	SAT TT=0.812	SAT TT=1.124	SAT TT=1.118	? (TO) TT=2400.01
Random/Random-m1-15-23-3/ rand-15-23-3-16-150000-18.xml	UNSAT TT=232.251	UNSAT TT=234.045	UNSAT TT=237.308	UNSAT TT=232.251	? (TO) TT=2400.08	? (TO) TT=2400.09	? (TO) TT=2400.09
Random/Random-m1-5-12-12/ rand-5-12-12-200-12442-38.xml	UNSAT TT=21.365	UNSAT TT=53.836	UNSAT TT=54.008	UNSAT TT=53.921	UNSAT TT=21.572	UNSAT TT=21.365	UNSAT TT=283.911
Random/Random-m1-5-12-12t/ rand-5-12-12-200-t95-3.xml	UNSAT TT=42.035	UNSAT TT=54.65	UNSAT TT=54.585	UNSAT TT=54.241	UNSAT TT=42.433	UNSAT TT=42.035	UNSAT TT=284.494
Random/ Random-m1-5-2X-05c/ rand-5-2X-05c-15.xml	UNSAT TT=26.302	UNSAT TT=26.473	UNSAT TT=26.638	UNSAT TT=26.302	UNSAT TT=147.218	UNSAT TT=152.131	UNSAT TT=604.835
Random/Random-RB-2-30-15f/ rand-2-30-15-306-230f-09.xml	SAT TT=0.287	SAT TT=0.287	SAT TT=0.283	SAT TT=0.288	SAT TT=0.433	SAT TT=0.434	SAT TT=4.059
Random/Random-RB-2-40-19/ rand-2-40-19-443-230-30.xml	UNSAT TT=130.201	UNSAT TT=231.551	UNSAT TT=230.326	UNSAT TT=230.856	UNSAT TT=130.209	UNSAT TT=130.201	? (TO) TT=2400.04
Random/Random-RB-3-20-20/ rand-3-20-20-60-632-40.xml	UNSAT TT=471.508	UNSAT TT=471.508	UNSAT TT=473.412	UNSAT TT=473.644	UNSAT TT=546.702	UNSAT TT=513.253	? (TO) TT=2400.04
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Random/Random-RB-3-28-28f/ rand-3-28-28-93-632f-05.xml		? (TO) TT=2400.07	? (TO) TT=2400.09	? (TO) TT=2400.1	? (TO) TT=2400.07	? (TO) TT=2400.02	? (TO) TT=2400.05
RenaultMod/ RenaultMod-m1-s1/ RenaultMod-09.xml	SAT TT=1.374	SAT TT=1.374	SAT TT=1.404	SAT TT=1.389	SAT TT=0.704	SAT TT=0.701	SAT TT=507.484
RenaultMod/ RenaultMod-m1-s1/ RenaultMod-12.xml	UNSAT TT=0.889	UNSAT TT=1.4	UNSAT TT=1.443	UNSAT TT=1.42	UNSAT TT=0.895	UNSAT TT=0.889	? (TO) TT=2400.08
RenaultMod/ RenaultMod-m1-s1/ RenaultMod-18.xml	UNSAT TT=0.511	UNSAT TT=1.353	UNSAT TT=1.398	UNSAT TT=1.376	UNSAT TT=0.507	UNSAT TT=0.511	? (TO) TT=2400.1
RenaultMod/ RenaultMod-m1-s1/ RenaultMod-21.xml	UNSAT TT=0.46	UNSAT TT=1.432	UNSAT TT=1.454	UNSAT TT=1.441	UNSAT TT=0.459	UNSAT TT=0.46	? (TO) TT=2400.02
RenaultMod/ RenaultMod-m1-s1/ RenaultMod-24.xml	UNSAT TT=0.456	UNSAT TT=1.408	UNSAT TT=1.438	UNSAT TT=1.416	UNSAT TT=0.449	UNSAT TT=0.456	? (TO) TT=2400.1
RenaultMod/ RenaultMod-m1-s1/ RenaultMod-40.xml	UNSAT TT=0.899	UNSAT TT=1.424	UNSAT TT=1.459	UNSAT TT=1.432	UNSAT TT=0.909	UNSAT TT=0.899	? (TO) TT=2400.05
Renault/Renault-m1-s1/ Renault-medium-pos.xml	SAT TT=0.056	SAT TT=0.056	SAT TT=0.055	SAT TT=0.056	SAT TT=0.019	SAT TT=0.02	SAT TT=0.063
Renault/Renault-m1-s1/ Renault-megane-pos.xml	SAT TT=1.225	SAT TT=1.225	SAT TT=1.239	SAT TT=1.23	SAT TT=0.372	SAT TT=0.375	SAT TT=1.598
Renault/ Renault-m1-s1/ Renault-mgd.xml	SAT TT=1.231	SAT TT=1.216	SAT TT=1.236	SAT TT=1.231	SAT TT=0.372	SAT TT=0.377	SAT TT=1.82
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Renault/ Renault-m1-s1/ Renault-small.xml	SAT TT=0.017	SAT TT=0.014	SAT TT=0.013	SAT TT=0.014	SAT TT=0.009	SAT TT=0.009	SAT TT=0.017
Renault/Renault-m1-s1/ Renault-souffleuse.xml	SAT TT=0.005	SAT TT=0.005	SAT TT=0.005	SAT TT=0.006	SAT TT=0.004	SAT TT=0.004	SAT TT=0.004
Renault/ Renault-m1-s1/ Renault.xml	SAT TT=1.626	SAT TT=1.223	SAT TT=1.25	SAT TT=1.237	SAT TT=0.371	SAT TT=0.376	SAT TT=1.626
Sat/Sat-dual-flat200-479/ Sat-flat200-06-dual.xml	SAT TT=20.023	SAT TT=2.382	SAT TT=2.365	SAT TT=2.36	SAT TT=22.703	SAT TT=20.023	SAT TT=142.502
Sat/Sat-dual-flat200-479/ Sat-flat200-14-dual.xml	SAT TT=0.64	SAT TT=0.652	SAT TT=0.648	SAT TT=0.64	SAT TT=13.921	SAT TT=15.582	SAT TT=101.192
Sat/Sat-dual-flat200-479/ Sat-flat200-32-dual.xml	SAT TT=9.694	SAT TT=0.636	SAT TT=0.631	SAT TT=0.629	SAT TT=11.409	SAT TT=9.694	SAT TT=7.025
Sat/Sat-dual-flat200-479/ Sat-flat200-55-dual.xml	SAT TT=3.585	SAT TT=3.585	SAT TT=3.54	SAT TT=3.505	SAT TT=20.423	SAT TT=22.879	SAT TT=755.419
Sat/Sat-dual-flat200-479/ Sat-flat200-67-dual.xml	SAT TT=1.904	SAT TT=1.961	SAT TT=1.915	SAT TT=1.904	SAT TT=14.795	SAT TT=16.207	SAT TT=434.027
Sat/Sat-dual-flat200-479/ Sat-flat200-80-dual.xml	SAT TT=0.556	SAT TT=0.573	SAT TT=0.562	SAT TT=0.556	SAT TT=14.492	SAT TT=14.643	SAT TT=921.376
Sat/Sat-m1-aim050/ aim-50-2-0-unsat-2.xml	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=0.007	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.305
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Sat/Sat-m1-aim100/ aim-100-2-0-unsat-2.xml	UNSAT TT=0.016	UNSAT TT=0.016	UNSAT TT=0.012	UNSAT TT=0.017	UNSAT TT=0.02	UNSAT TT=0.02	? (TO) TT=2400.08
Sat/Sat-m1-aim200/ aim-200-1-6-sat-3.xml	SAT TT=0.038	SAT TT=0.022	Wrong UNSAT TT=0.039	SAT TT=0.023	SAT TT=0.038	SAT TT=0.038	? (TO) TT=2400.01
Sat/ Sat-m1-jnh/ jnh-012.xml	SAT TT=0.033	SAT TT=0.034	SAT TT=0.033	SAT TT=0.033	SAT TT=0.071	SAT TT=0.072	SAT TT=0.041
Sat/ Sat-m1-jnh/ jnh-213.xml	SAT TT=0.062	SAT TT=0.037	SAT TT=0.036	SAT TT=0.036	SAT TT=0.062	SAT TT=0.062	SAT TT=0.052
Sat/ Sat-m1-jnh/ jnh-302.xml	UNSAT TT=0.023	UNSAT TT=0.033	UNSAT TT=0.032	UNSAT TT=0.033	UNSAT TT=0.022	UNSAT TT=0.023	UNSAT TT=0.031
Sat/Sat-sum-flat200-479/ Sat-flat200-13-sum.xml	SAT TT=1.156	SAT TT=0.22	SAT TT=0.22	SAT TT=0.221	SAT TT=1.146	SAT TT=1.156	? (TO) TT=2400.05
Sat/Sat-sum-flat200-479/ Sat-flat200-23-sum.xml	SAT TT=2.201	SAT TT=0.486	SAT TT=0.489	SAT TT=0.486	SAT TT=2.184	SAT TT=2.201	? (TO) TT=2400.02
Sat/Sat-sum-flat200-479/ Sat-flat200-24-sum.xml	SAT TT=0.132	SAT TT=0.132	SAT TT=0.133	SAT TT=0.133	SAT TT=1.056	SAT TT=1.06	? (TO) TT=2400.06
Sat/Sat-sum-flat200-479/ Sat-flat200-37-sum.xml	SAT TT=0.806	SAT TT=0.106	SAT TT=0.105	SAT TT=0.107	SAT TT=0.799	SAT TT=0.806	SAT TT=2073.34
Sat/Sat-sum-flat200-479/ Sat-flat200-65-sum.xml	SAT TT=0.514	SAT TT=0.514	SAT TT=0.514	SAT TT=0.512	SAT TT=2.583	SAT TT=2.594	SAT TT=42.496
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Sat/Sat-sum-flat200-479/ Sat-flat200-67-sum.xml	SAT TT=0.288	SAT TT=0.288	SAT TT=0.287	SAT TT=0.287	SAT TT=0.503	SAT TT=0.506	? (TO) TT=2400.02
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-012-9.xml	UNSAT TT=3.676	UNSAT TT=14.128	UNSAT TT=14.302	UNSAT TT=14.211	UNSAT TT=6.496	UNSAT TT=6.314	UNSAT TT=3.676
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-015-9.xml	UNSAT TT=4.736	UNSAT TT=18.251	UNSAT TT=18.46	UNSAT TT=18.218	UNSAT TT=7.794	UNSAT TT=7.715	UNSAT TT=4.736
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-020-9.xml	UNSAT TT=6.354	UNSAT TT=28.317	UNSAT TT=28.772	UNSAT TT=28.323	UNSAT TT=14.008	UNSAT TT=13.769	UNSAT TT=6.354
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-030-9.xml	UNSAT TT=8.449	UNSAT TT=50.381	UNSAT TT=51.052	UNSAT TT=50.066	UNSAT TT=31.994	UNSAT TT=33.173	UNSAT TT=8.449
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-050-9.xml	UNSAT TT=14.912	UNSAT TT=124.483	UNSAT TT=126.1	UNSAT TT=124.218	UNSAT TT=85.725	UNSAT TT=87.187	UNSAT TT=14.912
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-100-9.xml	UNSAT TT=40.214	UNSAT TT=384.148	UNSAT TT=390.169	UNSAT TT=384.094	UNSAT TT=317.905	UNSAT TT=316.333	UNSAT TT=40.214
StillLife/ StillLife-m1-s1/ StillLife-03-05.xml	OPT f=8 TT=0.007 T1=0	OPT f=8 TT=0.007 T1=0	OPT f=8 TT=0.007 T1=0	OPT f=8 TT=0.009 T1=0.11	No data	No data	OPT f=8 TT=0.008 T1=0
StillLife/ StillLife-m1-s1/ StillLife-03-06.xml	OPT f=10 TT=0.012 T1=0	OPT f=10 TT=0.013 T1=0.07	OPT f=10 TT=0.012 T1=0.01	OPT f=10 TT=0.012 T1=0.05	No data	No data	OPT f=10 TT=0.013 T1=0
StillLife/ StillLife-m1-s1/ StillLife-03-10.xml	OPT f=16 TT=0.276 T1=0.07	OPT f=16 TT=0.299 T1=0.17	OPT f=16 TT=0.285 T1=0.07	OPT f=16 TT=0.287 T1=0.15	No data	No data	OPT f=16 TT=0.276 T1=0.1
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
StillLife/ StillLife-m1-s1/ StillLife-03-11.xml	OPT f=18 TT=0.651 T1=0.51	OPT f=18 TT=0.999 T1=0.89	OPT f=18 TT=0.983 T1=0.87	OPT f=18 TT=0.995 T1=0.94	No data	No data	OPT f=18 TT=0.651 T1=0.51
StillLife/ StillLife-m1-s1/ StillLife-04-05.xml	OPT f=12 TT=0.013 T1=0	OPT f=12 TT=0.014 T1=0.09	OPT f=12 TT=0.013 T1=0	OPT f=12 TT=0.014 T1=0.06	No data	No data	OPT f=12 TT=0.018 T1=0.01
StillLife/ StillLife-m1-s1/ StillLife-05-06.xml	OPT f=16 TT=0.144 T1=0.01	OPT f=16 TT=0.373 T1=0.01	OPT f=16 TT=0.361 T1=0.01	OPT f=16 TT=0.365 T1=0.05	No data	No data	OPT f=16 TT=0.144 T1=0.09
StillLife/ StillLife-m1-s1/ StillLife-06-13.xml	SAT f=40 TT=2400 T1=376.38	SAT (TO) f=40 TT=2400 T1=376.38	? (TO) TT=2400.1	SAT (TO) f=40 TT=2400.1 T1=378.49	No data	No data	SAT (TO) f=39 TT=2400.01 T1=1373.25
StillLife/ StillLife-m1-s1/ StillLife-08-10.xml	SAT f=43 TT=2400.02 T1=20.86	SAT (TO) f=43 TT=2400.02 T1=20.86	? (TO) TT=2400.07	SAT (TO) f=43 TT=2400.06 T1=21.49	No data	No data	SAT (TO) f=41 TT=2400.05 T1=1994.92
StillLife/ StillLife-m1-s1/ StillLife-08-13.xml	SAT f=51 TT=2400.01 T1=220.16	SAT (TO) f=51 TT=2400.05 T1=220.5	? (TO) TT=2400.09	SAT (TO) f=51 TT=2400.01 T1=220.16	No data	No data	SAT (TO) f=39 TT=2400.11 T1=1267.77
StillLife/ StillLife-m1-s1/ StillLife-09-12.xml	SAT f=53 TT=2400.08 T1=1664.1	SAT (TO) f=53 TT=2400.08 T1=1664.1	? (TO) TT=2400.09	SAT (TO) f=53 TT=2400.11 T1=1664.14	No data	No data	SAT (TO) f=38 TT=2400.09 T1=1747.37
StillLife/ StillLife-m1-s1/ StillLife-10-11.xml	SAT f=57 TT=2400.04 T1=92.83	SAT (TO) f=57 TT=2400.04 T1=92.83	? (TO) TT=2400.1	SAT (TO) f=57 TT=2400.06 T1=92.84	No data	No data	SAT (TO) f=40 TT=2400.03 T1=2304.07
StillLife/ StillLife-m1-s1/ StillLife-10-12.xml	SAT f=60 TT=2400.05 T1=405.67	SAT (TO) f=60 TT=2400.05 T1=406.13	? (TO) TT=2400.06	SAT (TO) f=60 TT=2400.05 T1=405.67	No data	No data	SAT (TO) f=38 TT=2400.02 T1=1766.63
StillLife/ StillLife-m1-s1/ StillLife-10-14.xml	SAT f=65 TT=2400.04 T1=1206.7	SAT (TO) f=65 TT=2400.04 T1=1206.7	? (TO) TT=2400.04	SAT (TO) f=65 TT=2400.07 T1=1216.08	No data	No data	SAT (TO) f=40 TT=2400.05 T1=1954.7
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
StillLife/ StillLife-m1-s1/ StillLife-11-12.xml	SAT f=62 TT=2400.01 T1=1029.24	SAT (TO) f=62 TT=2400.02 T1=1029.24	? (TO) TT=2400.01	SAT (TO) f=62 TT=2400.01 T1=1030.12	No data	No data	SAT (TO) f=38 TT=2400.03 T1=1792.59
StillLife/ StillLife-m1-s1/ StillLife-11-14.xml	SAT f=67 TT=2400.09 T1=863.43	SAT (TO) f=67 TT=2400.09 T1=863.43	? (TO) TT=2400.07	SAT (TO) f=67 TT=2400.1 T1=868.35	No data	No data	SAT (TO) f=40 TT=2400.09 T1=2069.34
StillLife/ StillLife-m1-s1/ StillLife-11-15.xml	SAT f=70 TT=2400.02 T1=1955.34	SAT (TO) f=70 TT=2400.02 T1=1969.09	? (TO) TT=2400.08	SAT (TO) f=70 TT=2400.05 T1=1955.34	No data	No data	SAT (TO) f=40 TT=2400.03 T1=2247.37
StillLife/ StillLife-m1-s1/ StillLife-12-12.xml	SAT f=67 TT=2400.04 T1=2368.59	SAT (TO) f=67 TT=2400.05 T1=2368.59	? (TO) TT=2400.05	SAT (TO) f=67 TT=2400.04 T1=2370.61	No data	No data	SAT (TO) f=39 TT=2400.05 T1=2142.31
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g07-g39.xml	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.008	Signal TT=0.02	Signal TT=0.019	UNSAT TT=0.011
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g08-g31.xml	SAT TT=0.018	SAT TT=0.017	SAT TT=0.018	SAT TT=0.018	SAT TT=0.261	SAT TT=0.258	SAT TT=1.208
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g10-g35.xml	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.007	UNSAT TT=0.008	UNSAT TT=0.027	UNSAT TT=0.026	UNSAT TT=0.01
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-A-15.xml	SAT TT=1.497	SAT TT=0.062	SAT TT=0.063	SAT TT=0.062	SAT TT=17.163	SAT TT=16.069	SAT TT=1.497
Subisomorphism/ Subisomorphism-m1-si2-bvg/ Subisomorphism-si2-b09-m200-02.xml	SAT TT=0.014	SAT TT=0.014	SAT TT=0.014	SAT TT=0.014	SAT TT=0.124	SAT TT=0.124	SAT TT=0.099
Subisomorphism/ Subisomorphism-m1-si6-bvg/ Subisomorphism-si6-b03-m800-07.xml	SAT TT=0.47	SAT TT=0.615	SAT TT=0.611	SAT TT=0.612	SAT TT=1275.1	SAT TT=1217.68	SAT TT=0.47
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
TravellingSalesman/ TravellingSalesman-m1-n100/ TravellingSalesman-100-50-00.xml	SAT f=451 TT=2400.01 T1=2400.1	SAT (TO) f=451 TT=2400.03 T1=2400.1	? (TO) TT=2400.08	SAT (TO) f=451 TT=2400.01 T1=2400.1	No data	No data	? (TO) TT=2400.09
TravellingSalesman/ TravellingSalesman-m1-n100/ TravellingSalesman-100-50-05.xml	SAT f=424 TT=2400.03 T1=2399.9	SAT (TO) f=424 TT=2400.03 T1=2400.1	? (TO) TT=2400.05	SAT (TO) f=424 TT=2400.04 T1=2399.9	No data	No data	? (TO) TT=2400.02
TravellingSalesman/ TravellingSalesman-m1-n100/ TravellingSalesman-100-50-08.xml	SAT f=443 TT=2400.07 T1=2399.8	SAT (TO) f=443 TT=2400.09 T1=2399.8	? (TO) TT=2400.01	SAT (TO) f=443 TT=2400.07 T1=2399.9	No data	No data	? (TO) TT=2400.09
TravellingSalesman/ TravellingSalesman-m1-n150/ TravellingSalesman-150-50-14.xml	SAT f=551 TT=2400.07 T1=2400.1	SAT (TO) f=551 TT=2400.07 T1=2400.2	? (TO) TT=2400.01	SAT (TO) f=551 TT=2400.1 T1=2400.1	No data	No data	? (TO) TT=2400.03
TravellingSalesman/ TravellingSalesman-m1-n150/ TravellingSalesman-150-50-15.xml	SAT f=585 TT=2400.06 T1=2400.1	SAT (TO) f=585 TT=2400.06 T1=2400.2	? (TO) TT=2400.02	SAT (TO) f=585 TT=2400.06 T1=2400.1	No data	No data	? (TO) TT=2400.09
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-13.xml	SAT f=104 TT=2400.02 T1=119.14	SAT (TO) f=104 TT=2400.04 T1=2400	? (TO) TT=2400.02	SAT (TO) f=104 TT=2400.1 T1=2400	No data	No data	SAT (TO) f=104 TT=2400.02 T1=119.14
TravellingSalesman/ TravellingSalesman-m1-n20/ TravellingSalesman-20-30-02.xml	SAT f=119 TT=2400.02 T1=575.64	SAT (TO) f=119 TT=2400.03 T1=2399.8	? (TO) TT=2400.08	SAT (TO) f=119 TT=2400.06 T1=2400.2	No data	No data	SAT (TO) f=119 TT=2400.02 T1=575.64
TravellingSalesman/ TravellingSalesman-m1-n20/ TravellingSalesman-20-30-09.xml	SAT f=123 TT=2400.05 T1=2399.9	SAT (TO) f=123 TT=2400.07 T1=2400.2	? (TO) TT=2400.06	SAT (TO) f=123 TT=2400.05 T1=2399.9	No data	No data	SAT (TO) f=125 TT=2400.04 T1=1860.65
TravellingSalesman/ TravellingSalesman-m1-n20/ TravellingSalesman-20-30-18.xml	SAT f=106 TT=2400.01 T1=515.07	SAT (TO) f=106 TT=2400.01 T1=2400	? (TO) TT=2400.06	SAT (TO) f=106 TT=2400.04 T1=2400	No data	No data	SAT (TO) f=106 TT=2400.11 T1=515.07
TravellingSalesman/ TravellingSalesman-m1-n30/ TravellingSalesman-30-30-18.xml	SAT f=137 TT=2400.02 T1=2399.9	SAT (TO) f=137 TT=2400.06 T1=2399.9	? (TO) TT=2400.09	SAT (TO) f=137 TT=2400.02 T1=2400.2	No data	No data	SAT (TO) f=154 TT=2400.01 T1=2371.02
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
TravellingSalesman/ TravellingSalesman-m1-n40/ TravellingSalesman-40-30-19.xml	SAT f=155 TT=2400.09 T1=2400.3	SAT (TO) f=155 TT=2400.09 T1=2400.3	? (TO) TT=2400.1	SAT (TO) f=155 TT=2400.09 T1=2400.4	No data	No data	? (TO) TT=2400.08
TravellingSalesman/ TravellingSalesman-m1-n45/ TravellingSalesman-45-30-04.xml	SAT f=171 TT=2400.09 T1=2400.2	SAT (TO) f=171 TT=2400.09 T1=2400.2	? (TO) TT=2400.07	SAT (TO) f=171 TT=2400.09 T1=2400.3	No data	No data	? (TO) TT=2400.09
TravellingSalesman/ TravellingSalesman-m1-n45/ TravellingSalesman-45-30-09.xml	SAT f=185 TT=2400.05 T1=2399.9	SAT (TO) f=185 TT=2400.1 T1=2399.9	? (TO) TT=2400.03	SAT (TO) f=185 TT=2400.05 T1=2400.3	No data	No data	? (TO) TT=2400.02
TravellingSalesman/ TravellingSalesman-m1-n50/ TravellingSalesman-50-30-01.xml	SAT f=203 TT=2400.03 T1=2399.9	SAT (TO) f=203 TT=2400.04 T1=2399.9	? (TO) TT=2400.09	SAT (TO) f=203 TT=2400.03 T1=2399.9	No data	No data	? (TO) TT=2400.03
TravellingSalesman/ TravellingSalesman-m1-n50/ TravellingSalesman-50-30-05.xml	SAT f=185 TT=2400.03 T1=2399.8	SAT (TO) f=185 TT=2400.03 T1=2399.8	? (TO) TT=2400.08	SAT (TO) f=185 TT=2400.09 T1=2399.9	No data	No data	? (TO) TT=2400.03
TravellingSalesman/ TravellingSalesman-m1-n75/ TravellingSalesman-75-50-02.xml	SAT f=401 TT=2400.07 T1=2400	SAT (TO) f=401 TT=2400.07 T1=2400	? (TO) TT=2400.06	SAT (TO) f=401 TT=2400.1 T1=2400.3	No data	No data	? (TO) TT=2400.05
TravellingSalesman/ TravellingSalesman-m1-n75/ TravellingSalesman-75-50-15.xml	SAT f=375 TT=2400.03 T1=2400	SAT (TO) f=375 TT=2400.03 T1=2400	? (TO) TT=2400.03	SAT (TO) f=375 TT=2400.09 T1=2400	No data	No data	? (TO) TT=2400.02
TravellingSalesman/ TravellingSalesman-xcsp2-s20a4/ TravellingSalesman-4-20-001-a4_X2.xml	SAT TT=0.766	SAT TT=0.767	SAT TT=0.762	SAT TT=0.766	SAT TT=3.541	SAT TT=3.606	SAT TT=194.982
TravellingSalesman/ TravellingSalesman-xcsp2-s20a4/ TravellingSalesman-4-20-727-a4_X2.xml	SAT TT=18.609	SAT TT=26.094	SAT TT=26.466	SAT TT=26.848	SAT TT=18.594	SAT TT=18.609	SAT TT=1833.55
TravellingSalesman/ TravellingSalesman-xcsp2-s20/ TravellingSalesman-20-076_X2.xml	SAT TT=20.472	SAT TT=20.478	SAT TT=20.455	SAT TT=20.472	SAT TT=4.183	SAT TT=4.078	SAT TT=126.457
Bench name	Best results	cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
TravellingSalesman/ TravellingSalesman-xcsp2-s20/ TravellingSalesman-20-142_X2.xml	SAT TT=1325.71	SAT TT=18.752	SAT TT=18.765	SAT TT=18.325	SAT TT=79.712	SAT TT=81.483	SAT TT=1325.71
TravellingSalesman/ TravellingSalesman-xcsp2-s25/ TravellingSalesman-25-003_X2.xml	SAT TT=2.18	SAT TT=2.18	SAT TT=2.199	SAT TT=2.189	SAT TT=5.612	SAT TT=5.638	SAT TT=325.61
TravellingSalesman/ TravellingSalesman-xcsp2-s25/ TravellingSalesman-25-066_X2.xml	SAT TT=2.227	SAT TT=0.088	SAT TT=0.088	SAT TT=0.088	SAT TT=2.17	SAT TT=2.227	SAT TT=33.95
Wwtpp/Wwtpp-ord-s1/ Wwtpp-ord-ex02280.xml	UNSAT TT=0.307	UNSAT TT=0.309	UNSAT TT=0.303	UNSAT TT=0.307	? (TO) TT=2400.1	? (TO) TT=2400.12	UNSAT TT=0.461
Wwtpp/Wwtpp-ord-s1/ Wwtpp-ord-ex03380.xml	UNSAT TT=0.397	UNSAT TT=0.4	UNSAT TT=0.409	UNSAT TT=0.397	? (TO) TT=2400.1	? (TO) TT=2400.11	UNSAT TT=0.546
Wwtpp/Wwtpp-ord-s1/ Wwtpp-ord-ex05480.xml		? (TO) TT=2400.1	? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.06
Wwtpp/Wwtpp-ord-s1/ Wwtpp-ord-ex06460.xml	SAT TT=38.253	SAT TT=38.038	SAT TT=38.524	SAT TT=38.253	? (TO) TT=2400.1	? (TO) TT=2400.05	? (TO) TT=2400.1
Wwtpp/Wwtpp-ord-s1/ Wwtpp-ord-ex07180.xml	SAT TT=10.853	SAT TT=10.851	SAT TT=10.911	SAT TT=10.853	? (TO) TT=2400.18	? (TO) TT=2400.05	SAT TT=74.686
Wwtpp/Wwtpp-ord-s1/ Wwtpp-ord-ex08540.xml	SAT TT=1.15	SAT TT=1.571	SAT TT=1.58	SAT TT=1.568	? (TO) TT=2400.11	? (TO) TT=2400.12	SAT TT=1.15

Statistics		cosoco-mini 1.12 (complete)	cosoco-mini 1.1 (2017-06-27) (complete)	cosoco-mini 1.1 (2017-07-29) (complete)	miniBTD 2017-06-30 (complete)	miniBTD 2017-08-10 (complete)	Naxos 1.1.0 (complete)
Number of times the solver is able to give the best known answer		291	46	290	0	163	192
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		119	17	133	0	61	58