2019 XCSP3 competition: mini-solver track (sequential and parallel solvers): results by benchmark

Results by benchmark for category all categories

This page displays the results of the different solvers for each benchmark for category all categories

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
cosoco 2 (complete)
cosoco 2.0 (complete)
cosoco 2.0 (complete)
miniBTD 19.06.16 (complete)
NACRE 1.0.5 (complete)
NACRE 1.0.5-Hybrid (complete)
(reference) PicatSAT 2019-09-12 (complete)

Bench name	Best results on this instance	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
AllInterval/ AllInterval-m1-s1/ AllInterval-040.xml	SAT TT=0.048	SAT TT=0.348	SAT TT=0.347	SAT TT=0.349	SAT TT=0.048	SAT TT=0.073	SAT TT=0.07	SAT TT=291.992
AllInterval/ AllInterval-m1-s1/ AllInterval-075.xml	SAT TT=0.179	? (TO) TT=2400.01	? (TO) TT=2400.04	? (TO) TT=2400.01	SAT TT=0.501	SAT TT=0.181	SAT TT=0.179	? (TO) TT=2400.08
Auction/Auction-sum/ Auction-sum-decay200_c18.xml	SAT TO f=1164883 TT=2400.02 T1=111.29	SAT (TO) f=1164883 TT=2400.06 T1=111.29	SAT (TO) f=1164883 TT=2400.02 T1=113.28	SAT (TO) f=1164883 TT=2400.1 T1=112.67	No data	No data	No data	? (TO) TT=2400.07
Auction/Auction-sum/ Auction-sum-decay500_c18.xml	SAT TO f=2539538 TT=2400.01 T1=186.97	SAT (TO) f=2539538 TT=2400.11 T1=186.97	SAT (TO) f=2539538 TT=2400.1 T1=202.17	SAT (TO) f=2539538 TT=2400.01 T1=187.97	No data	No data	No data	? (TO) TT=2400.1
Bacp/Bacp-m2/ Bacp-m2-12b_c18.xml	SAT TO f=17 TT=2400.03 T1=0	SAT (TO) f=17 TT=2400.05 T1=0	SAT (TO) f=17 TT=2400.03 T1=0	SAT (TO) f=17 TT=2400.07 T1=0	No data	No data	No data	? (TO) TT=2400.06
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-2_X2.xml	UNSAT TT=524.722	? (TO) TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.06	? TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.01	UNSAT TT=524.722
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-3_X2.xml	UNSAT TT=523.635	? (TO) TT=2400.04	? (TO) TT=2400.01	? (TO) TT=2400.01	? TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.06	UNSAT TT=523.635
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-4_X2.xml	UNSAT TT=565.312	? (TO) TT=2400.08	? (TO) TT=2400.03	? (TO) TT=2400.03	? TT=2400.03	? (TO) TT=2400.08	? (TO) TT=2400.09	UNSAT TT=565.312
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-7_X2.xml	UNSAT TT=515.662	? (TO) TT=2400.09	? (TO) TT=2400.02	? (TO) TT=2400.01	? TT=2400.03	? (TO) TT=2400.08	? (TO) TT=2400.03	UNSAT TT=515.662
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-8_X2.xml	UNSAT TT=654.324	? (TO) TT=2400.06	? (TO) TT=2400.02	? (TO) TT=2400.05	? TT=2400.03	? (TO) TT=2400.07	? (TO) TT=2400.01	UNSAT TT=654.324
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07-9_X2.xml	UNSAT TT=573.798	? (TO) TT=2400.02	? (TO) TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.07	UNSAT TT=573.798
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07h-0_X2.xml	UNSAT TT=704.404	? (TO) TT=2400.08	? (TO) TT=2400.06	? (TO) TT=2400.1	? TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.03	UNSAT TT=704.404
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07h-3_X2.xml	UNSAT TT=404.557	? (TO) TT=2400.08	? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.05	? (TO) TT=2400.04	UNSAT TT=404.557
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07h-6_X2.xml	UNSAT TT=538.997	? (TO) TT=2400.11	? (TO) TT=2400.04	? (TO) TT=2400.09	? (TO) TT=2400.03	? (TO) TT=2400.02	? (TO) TT=2400.03	UNSAT TT=538.997
Blackhole/ Blackhole-xcsp2-s07/ Blackhole-4-07h-8_X2.xml	UNSAT TT=537.808	? (TO) TT=2400.03	? (TO) TT=2400.08	? (TO) TT=2400.06	? TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.02	UNSAT TT=537.808
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-08.xml	UNSAT TT=8.526	? (TO) TT=2400.07	? (TO) TT=2400.08	? (TO) TT=2400.05	? TT=2400	? (TO) TT=2400.02	? (TO) TT=2400.1	UNSAT TT=8.526
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-13.xml		? (TO) TT=2400.02	? (TO) TT=2400.04	? (TO) TT=2400.02	? TT=2400	? (TO) TT=2400.05	? (TO) TT=2400.02	? (TO) TT=2400.01
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-15.xml		? (TO) TT=2400.07	? (TO) TT=2400.09	? (TO) TT=2400.01	? TT=2400.01	? (TO) TT=2400.05	? (TO) TT=2400.09	? (TO) TT=2400.05
ColouredQueens/ ColouredQueens-m1-s1/ ColouredQueens-16.xml		? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.07	? TT=2400	? (TO) TT=2400.05	? (TO) TT=2400.11	? (TO) TT=2400.03
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-08-2-11.xml		? (TO) TT=2400.03	? (TO) TT=2400.02	? (TO) TT=2400.05	? (TO) TT=2400.05	? (TO) TT=2400.1	? (TO) TT=2400.1	? (TO) TT=2400.02
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-09-2-12.xml	SAT TT=805.886	SAT TT=999.958	SAT TT=1004.68	SAT TT=1008.33	? (TO) TT=2400.06	? (TO) TT=2400.07	? (TO) TT=2400.08	SAT TT=805.886
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-10-2-11.xml		? (TO) TT=2400.05	? (TO) TT=2400.02	? (TO) TT=2400.05	? (TO) TT=2400.1	? (TO) TT=2400.06	? (TO) TT=2400.01	? (TO) TT=2400.07
CoveringArray/ CoveringArray-elt-s1/ CoveringArray-elt-3-11-2-11.xml		? (TO) TT=2400.06	? (TO) TT=2400.1	? (TO) TT=2400.05	? (TO) TT=2400.02	? (TO) TT=2400.06	? (TO) TT=2400.09	? (TO) TT=2400.09
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h1502.xml	SAT TT=0.318	SAT TT=0.653	SAT TT=0.653	SAT TT=0.74	SAT TT=0.318	SAT TT=1.115	SAT TT=1.807	SAT TT=16.942
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h1503.xml	SAT TT=0.099	SAT TT=0.129	SAT TT=0.128	SAT TT=0.148	SAT TT=0.099	SAT TT=0.285	SAT TT=0.316	SAT TT=24.376
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h1901.xml	SAT TT=0.134	SAT TT=0.208	SAT TT=0.209	SAT TT=0.237	SAT TT=0.134	SAT TT=0.318	SAT TT=0.324	SAT TT=28.364
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h1906.xml	SAT TT=0.107	SAT TT=0.205	SAT TT=0.205	SAT TT=0.235	SAT TT=0.107	SAT TT=0.27	SAT TT=0.262	SAT TT=15.467
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h2102.xml	SAT TT=0.243	SAT TT=0.249	SAT TT=0.251	SAT TT=0.285	SAT TT=0.243	SAT TT=1.275	SAT TT=0.413	SAT TT=93.951
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h2304.xml	SAT TT=5.616	SAT TT=33.277	SAT TT=33.566	SAT TT=37.782	? (TO) TT=2400.04	SAT TT=108.105	SAT TT=5.616	? (TO) TT=2400.04
Crossword/ Crossword-m1c-lex-herald/ Crossword-m1c-lex-h2307.xml	SAT TT=0.984	SAT TT=0.984	SAT TT=0.996	SAT TT=1.125	SAT TT=2.812	SAT TT=1.878	SAT TT=1.002	SAT TT=809.533
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Crossword/ Crossword-m1c-lex-vg/ Crossword-m1c-lex-vg-4-8.xml	SAT TT=0.314	SAT TT=8.612	SAT TT=8.735	SAT TT=9.884	SAT TT=0.314	SAT TT=27.921	SAT TT=25.918	SAT TT=66.27
Crossword/ Crossword-m1c-lex-vg/ Crossword-m1c-lex-vg-5-6.xml	SAT TT=0.477	SAT TT=0.477	SAT TT=0.477	SAT TT=0.545	SAT TT=3.475	SAT TT=11.849	SAT TT=1.135	SAT TT=16.094
Crossword/Crossword-m1c-lex-vg/ Crossword-m1c-lex-vg-7-10.xml	UNSAT TT=3.397	UNSAT TT=9.331	UNSAT TT=9.355	UNSAT TT=10.691	UNSAT TT=3.397	UNSAT TT=19.408	UNSAT TT=4.422	UNSAT TT=43.044
Crossword/ Crossword-m1c-ogd2008-herald/ Crossword-m1c-ogd2008-h1905.xml	SAT TT=0.567	SAT TT=1.396	SAT TT=1.601	SAT TT=1.401	SAT TT=0.567	SAT TT=1.688	SAT TT=1.689	SAT TT=161.526
Crossword/ Crossword-m1c-ogd2008-herald/ Crossword-m1c-ogd2008-h1907.xml	SAT TT=0.904	SAT TT=3.112	SAT TT=3.547	SAT TT=3.125	SAT TT=0.904	SAT TT=2.193	SAT TT=2.044	SAT TT=530.579
Crossword/ Crossword-m1c-ogd2008-puzzle/ Crossword-m1c-ogd2008-p16.xml	SAT TT=0.263	SAT TT=0.849	SAT TT=0.97	SAT TT=0.851	SAT TT=0.263	SAT TT=0.681	SAT TT=0.672	SAT TT=18.2
Crossword/ Crossword-m1c-ogd2008-puzzle/ Crossword-m1c-ogd2008-p19.xml	SAT TT=0.732	SAT TT=2.58	SAT TT=2.977	SAT TT=2.595	SAT TT=0.732	SAT TT=1.704	SAT TT=1.685	SAT TT=267.234
Crossword/Crossword-m1c-ogd2008-vg/ Crossword-m1c-ogd2008-vg-7-7.xml	SAT TT=3.18	SAT TT=5.404	SAT TT=6.14	SAT TT=5.512	SAT TT=3.18	SAT TT=12.974	SAT TT=66.101	SAT TT=144.224
Crossword/ Crossword-m1c-ogd-puzzle/ Crossword-m1c-ogd-p14.xml	SAT TT=0.097	SAT TT=0.315	SAT TT=0.317	SAT TT=0.316	SAT TT=0.097	SAT TT=0.242	SAT TT=0.241	SAT TT=6.334
Crossword/Crossword-m1c-ogd-vg/ Crossword-m1c-ogd-vg-14-15.xml	UNSAT TT=4.037	UNSAT TT=18.138	UNSAT TT=17.899	UNSAT TT=18.245	UNSAT TT=4.037	UNSAT TT=167.224	UNSAT TT=44.451	UNSAT TT=1251.18
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Crossword/Crossword-m1c-ogd-vg/ Crossword-m1c-ogd-vg-14-18.xml	UNSAT TT=0.382	UNSAT TT=0.816	UNSAT TT=0.814	UNSAT TT=0.922	UNSAT TT=0.382	UNSAT TT=1.442	UNSAT TT=1.45	UNSAT TT=363.298
Crossword/Crossword-m1c-ogd-vg/ Crossword-m1c-ogd-vg-15-15.xml	UNSAT TT=1.609	UNSAT TT=5.992	UNSAT TT=6.012	UNSAT TT=6.787	UNSAT TT=1.609	UNSAT TT=17.947	UNSAT TT=12.71	UNSAT TT=644.141
Crossword/Crossword-m1c-ogd-vg/ Crossword-m1c-ogd-vg-9-12.xml		? (TO) TT=2400.04	? (TO) TT=2400.08	? (TO) TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.1
Crossword/ Crossword-m1c-uk-herald/ Crossword-m1c-uk-h1506.xml	SAT TT=0.467	SAT TT=1.09	SAT TT=1.255	SAT TT=1.096	SAT TT=0.467	SAT TT=1.269	SAT TT=1.212	SAT TT=226.984
Crossword/ Crossword-m1c-uk-herald/ Crossword-m1c-uk-h1903.xml	SAT TT=0.529	SAT TT=1.157	SAT TT=1.334	SAT TT=1.164	SAT TT=0.529	SAT TT=1.637	SAT TT=1.661	SAT TT=214.864
Crossword/ Crossword-m1c-uk-herald/ Crossword-m1c-uk-h1908.xml	SAT TT=0.428	SAT TT=1.267	SAT TT=1.456	SAT TT=1.276	SAT TT=0.428	SAT TT=1.063	SAT TT=1.067	SAT TT=97.286
Crossword/ Crossword-m1c-uk-herald/ Crossword-m1c-uk-h2302.xml	SAT TT=0.68	SAT TT=1.923	SAT TT=2.219	SAT TT=1.932	SAT TT=0.68	SAT TT=1.787	SAT TT=1.799	SAT TT=190.26
Crossword/ Crossword-m1c-uk-herald/ Crossword-m1c-uk-h2304.xml	SAT TT=0.813	SAT TT=1.307	SAT TT=1.507	SAT TT=1.314	SAT TT=0.813	SAT TT=2.485	SAT TT=2.437	SAT TT=480.048
Crossword/ Crossword-m1c-uk-herald/ Crossword-m1c-uk-h2308.xml	SAT TT=1.061	SAT TT=1.287	SAT TT=1.478	SAT TT=1.294	SAT TT=1.061	SAT TT=2.389	SAT TT=2.381	SAT TT=260.024
Crossword/Crossword-m1c-uk-vg/ Crossword-m1c-uk-vg-14-14.xml	UNSAT TT=52.534	UNSAT TT=108.061	UNSAT TT=122.351	UNSAT TT=106.873	UNSAT TT=52.534	UNSAT TT=251.856	UNSAT TT=83.411	UNSAT TT=426.994
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Crossword/Crossword-m1c-uk-vg/ Crossword-m1c-uk-vg-16-20.xml	UNSAT TT=0.071	UNSAT TT=0.093	UNSAT TT=0.093	UNSAT TT=0.093	UNSAT TT=0.071	UNSAT TT=0.132	UNSAT TT=0.138	UNSAT TT=16.344
Crossword/Crossword-m1c-uk-vg/ Crossword-m1c-uk-vg-5-7.xml	SAT TT=0.098	SAT TT=0.353	SAT TT=0.355	SAT TT=0.355	SAT TT=0.098	SAT TT=0.777	SAT TT=0.845	SAT TT=21.535
Crossword/Crossword-m1c-uk-vg/ Crossword-m1c-uk-vg-8-9.xml		? (TO) TT=2400.02	? (TO) TT=2400.04	? (TO) TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.02	? (TO) TT=2400.09
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h0509.xml	SAT TT=0.013	SAT TT=0.04	SAT TT=0.046	SAT TT=0.04	SAT TT=0.013	SAT TT=0.023	SAT TT=0.022	SAT TT=0.145
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h1502.xml	SAT TT=0.365	SAT TT=0.412	SAT TT=0.472	SAT TT=0.418	SAT TT=0.365	SAT TT=1.203	SAT TT=1.538	SAT TT=18.14
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h1510.xml	SAT TT=3.756	SAT TT=9.36	SAT TT=10.687	SAT TT=9.404	SAT TT=3.756	SAT TT=24.707	SAT TT=9.573	SAT TT=1022.31
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h1901.xml	SAT TT=0.158	SAT TT=0.319	SAT TT=0.365	SAT TT=0.324	SAT TT=0.158	SAT TT=0.418	SAT TT=0.423	SAT TT=32.266
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h1903.xml	SAT TT=0.688	SAT TT=3.762	SAT TT=4.301	SAT TT=3.778	SAT TT=0.688	SAT TT=9.716	SAT TT=5.523	SAT TT=307.325
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h1908.xml	SAT TT=0.157	SAT TT=0.316	SAT TT=0.363	SAT TT=0.317	SAT TT=0.157	SAT TT=0.347	SAT TT=0.362	SAT TT=29.316
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h2104.xml	SAT TT=1.421	SAT TT=1.438	SAT TT=1.615	SAT TT=1.421	SAT TT=14.639	SAT TT=2.114	SAT TT=2.643	SAT TT=615.879
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Crossword/ Crossword-m1c-words-herald/ Crossword-m1c-words-h2307.xml	SAT TT=0.571	SAT TT=0.632	SAT TT=0.725	SAT TT=0.64	SAT TT=0.571	SAT TT=1.905	SAT TT=1.184	SAT TT=311.945
Crossword/Crossword-m1c-words-vg/ Crossword-m1c-words-vg-10-10.xml	UNSAT TT=7.471	UNSAT TT=11.4	UNSAT TT=12.881	UNSAT TT=11.326	UNSAT TT=7.471	UNSAT TT=10.994	UNSAT TT=9.119	UNSAT TT=100.7
Crossword/Crossword-m1c-words-vg/ Crossword-m1c-words-vg-6-10.xml	UNSAT TT=24.632	UNSAT TT=126.471	UNSAT TT=143.554	UNSAT TT=127.073	UNSAT TT=24.632	UNSAT TT=251.911	UNSAT TT=45.166	UNSAT TT=264.028
Dubois/ Dubois-m1-s1/ Dubois-028.xml	UNSAT TT=0.007	? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.03	UNSAT TT=0.007	? (TO) TT=2400.02	UNSAT TT=0.009	UNSAT TT=0.007
GolombRuler/ GolombRuler-a3v18-08_c18.xml	OPT f=34 TT=0.402 T1=0	OPT f=34 TT=0.405 T1=0	OPT f=34 TT=0.403 T1=0	OPT f=34 TT=0.402 T1=0	No data	No data	No data	OPT f=34 TT=33.041 T1=33
GolombRuler/ GolombRuler-a3v18-09_c18.xml	OPT f=44 TT=5.281 T1=1.79	OPT f=44 TT=5.306 T1=1.79	OPT f=44 TT=5.304 T1=1.79	OPT f=44 TT=5.281 T1=1.79	No data	No data	No data	OPT f=44 TT=93.628 T1=93.59
GolombRuler/ GolombRuler-a3v18-10_c18.xml	OPT f=55 TT=62.662 T1=46.59	OPT f=55 TT=63.147 T1=46.89	OPT f=55 TT=62.662 T1=46.59	OPT f=55 TT=62.754 T1=46.59	No data	No data	No data	OPT f=55 TT=296.652 T1=296.58
GolombRuler/ GolombRuler-a3v18-14_c18.xml	SAT TO f=140 TT=2400.01 T1=881.8	SAT (TO) f=140 TT=2400.01 T1=888.27	SAT (TO) f=140 TT=2400.08 T1=885.83	SAT (TO) f=140 TT=2400.09 T1=881.8	No data	No data	No data	? (TO) TT=2400.07
GolombRuler/ GolombRuler-a3v18-nodv-10_c18.xml	OPT f=55 TT=306.942 T1=306.87	OPT f=55 TT=868.748 T1=804.55	OPT f=55 TT=989.471 T1=916.46	OPT f=55 TT=860.549 T1=796.95	No data	No data	No data	OPT f=55 TT=306.942 T1=306.87
GracefulGraph/ GracefulGraph-K04-P04_c18.xml	SAT TT=2.511	SAT TT=2.511	SAT TT=2.515	SAT TT=2.527	SAT TT=175.743	? (TO) TT=2400.06	? (TO) TT=2400.07	SAT TT=82.523
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
GracefulGraph/ GracefulGraph-K05-P02_c18.xml	SAT TT=15.723	SAT TT=18.504	SAT TT=18.537	SAT TT=18.615	SAT TT=32.889	? (TO) TT=2400.05	? (TO) TT=2400.05	SAT TT=15.723
GracefulGraph/ GracefulGraph-K07-P03_c18.xml		? (TO) TT=2400.09	? (TO) TT=2400.05	? (TO) TT=2400.04	? TT=2400.02	? (TO) TT=2400.05	? (TO) TT=2400.06	? (TO) TT=2400.11
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K05-P02.xml	SAT TT=15.5	SAT TT=18.688	SAT TT=18.555	SAT TT=18.647	SAT TT=33.213	? (TO) TT=2400.06	? (TO) TT=2400.08	SAT TT=15.5
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K08-P03.xml		? (TO) TT=2400.07	? (TO) TT=2400.02	? (TO) TT=2400.03	? TT=2400.03	? (TO) TT=2400.05	? (TO) TT=2400.05	? (TO) TT=2400.1
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K12-P08.xml		? (TO) TT=2400.08	? (TO) TT=2400.02	? (TO) TT=2400.04	? (TO) TT=2400.01	? (TO) TT=2400.02	? (TO) TT=2400.05	? (TO) TT=2400.1
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K13-P04.xml		? (TO) TT=2400.1	? (TO) TT=2400.06	? (TO) TT=2400.1	? (TO) TT=2400.04	? (TO) TT=2400	? (TO) TT=2400.07	? (TO) TT=2400.08
GracefulGraph/ GracefulGraph-m1-s1/ GracefulGraph-K14-P05.xml		? (TO) TT=2400.02	? (TO) TT=2400.04	? (TO) TT=2400.01	? (TO) TT=2400.05	? (TO) TT=2400.01	? (TO) TT=2400.07	? (TO) TT=2400.05
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-1-fullins-5.xml	OPT f=5 TT=7.633 T1=0.41	OPT f=5 TT=19.522 T1=0.41	OPT f=5 TT=19.992 T1=0.52	OPT f=5 TT=19.488 T1=0.46	No data	No data	No data	OPT f=5 TT=7.633 T1=7.58
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-2-fullins-5.xml	OPT f=6 TT=57.738 T1=1.88	OPT f=6 TT=373.004 T1=1.89	OPT f=6 TT=332.521 T1=1.88	OPT f=6 TT=301.293 T1=1.88	No data	No data	No data	OPT f=6 TT=57.738 T1=57.67
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-3-fullins-3.xml	OPT f=5 TT=0.059 T1=0	OPT f=5 TT=0.059 T1=0	OPT f=5 TT=0.061 T1=0	OPT f=5 TT=0.069 T1=0	No data	No data	No data	OPT f=5 TT=1.586 T1=1.48
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-4-insertions-3.xml	OPT f=3 TT=4.94 T1=4.88	SAT (TO) f=3 TT=2400.01 T1=0	SAT (TO) f=3 TT=2400.04 T1=0	SAT (TO) f=3 TT=2400.05 T1=0	No data	No data	No data	OPT f=3 TT=4.94 T1=4.88
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-5-fullins-3.xml	OPT f=7 TT=0.316 T1=0.1	OPT f=7 TT=0.317 T1=0.1	OPT f=7 TT=0.316 T1=0.1	OPT f=7 TT=0.368 T1=0.1	No data	No data	No data	OPT f=7 TT=4.901 T1=4.88
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-5-fullins-4.xml	OPT f=8 TT=75.362 T1=1.69	OPT f=8 TT=138.77 T1=1.79	OPT f=8 TT=171.478 T1=1.69	OPT f=8 TT=162.982 T1=2.1	No data	No data	No data	OPT f=8 TT=75.362 T1=75.26
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-ash331GPIA.xml	OPT f=3 TT=0.688 T1=0.59	OPT f=3 TT=0.688 T1=0.59	OPT f=3 TT=0.694 T1=0.6	OPT f=3 TT=0.797 T1=0.69	No data	No data	No data	OPT f=3 TT=11.814 T1=11.77
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-david.xml	OPT f=10 TT=8.67 T1=0	OPT f=10 TT=84.179 T1=0	OPT f=10 TT=85.452 T1=0	OPT f=10 TT=84.836 T1=0	No data	No data	No data	OPT f=10 TT=8.67 T1=8.59
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-dsjc-1000-9.xml	SAT TO f=298 TT=2400.05 T1=84.49	SAT (TO) f=298 TT=2400.05 T1=84.49	SAT (TO) f=298 TT=2400.07 T1=85.23	SAT (TO) f=298 TT=2400.1 T1=85.35	No data	No data	No data	? (TO) TT=2400.06
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-dsjc-250-5.xml	SAT TO f=33 TT=2400.05 T1=526.1	SAT (TO) f=33 TT=2400.09 T1=526.1	SAT (TO) f=33 TT=2400.05 T1=539	SAT (TO) f=33 TT=2400.06 T1=580.38	No data	No data	No data	? (TO) TT=2400.05
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-dsjr-500-1.xml	OPT f=11 TT=29.781 T1=0.5	OPT f=11 TT=1014.26 T1=0.5	OPT f=11 TT=1096.03 T1=0.6	OPT f=11 TT=1401.47 T1=0.51	No data	No data	No data	OPT f=11 TT=29.781 T1=29.67
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-fpsol2-i-1.xml	SAT TO f=64 TT=2400.02 T1=1.77	SAT (TO) f=64 TT=2400.08 T1=1.79	SAT (TO) f=64 TT=2400.02 T1=1.79	SAT (TO) f=64 TT=2400.09 T1=1.77	No data	No data	No data	? (TO) TT=2400.03
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-fpsol2-i-2.xml	SAT TO f=29 TT=2400.03 T1=1.28	SAT (TO) f=29 TT=2400.07 T1=1.28	SAT (TO) f=29 TT=2400.03 T1=1.52	SAT (TO) f=29 TT=2400.04 T1=1.33	No data	No data	No data	? (TO) TT=2400.09
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-huck.xml	OPT f=10 TT=7.337 T1=0	OPT f=10 TT=39.779 T1=0	OPT f=10 TT=39.663 T1=0	OPT f=10 TT=39.421 T1=0	No data	No data	No data	OPT f=10 TT=7.337 T1=7.28
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-lei450-05b.xml	OPT f=4 TT=227.213 T1=227.16	OPT f=4 TT=227.213 T1=227.16	OPT f=4 TT=234.661 T1=234.61	OPT f=4 TT=237.224 T1=237.16	No data	No data	No data	OPT f=4 TT=733.332 T1=733.23
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-lei450-05c.xml	OPT f=4 TT=1.959 T1=1.89	OPT f=4 TT=1.959 T1=1.89	OPT f=4 TT=2.044 T1=1.94	OPT f=4 TT=2.227 T1=2.12	No data	No data	No data	OPT f=4 TT=23.918 T1=23.88
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-lei450-05d.xml	OPT f=4 TT=2.201 T1=2.15	OPT f=4 TT=2.201 T1=2.2	OPT f=4 TT=2.222 T1=2.16	OPT f=4 TT=2.22 T1=2.15	No data	No data	No data	OPT f=4 TT=23.148 T1=23.09
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-queen8-12.xml	OPT f=11 TT=29.416 T1=0.12	OPT f=11 TT=1306.23 T1=0.2	OPT f=11 TT=1317.4 T1=0.12	OPT f=11 TT=1310.37 T1=0.15	No data	No data	No data	OPT f=11 TT=29.416 T1=29.32
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-wap05a.xml	SAT TO f=49 TT=2400.03 T1=7.06	SAT (TO) f=49 TT=2400.08 T1=7.09	SAT (TO) f=49 TT=2400.03 T1=7.1	SAT (TO) f=49 TT=2400.11 T1=7.06	No data	No data	No data	? (TO) TT=2400.05
GraphColoring/ GraphColoring-m1-mono/ GraphColoring-zeroin-i-2.xml	SAT TO f=29 TT=2400.03 T1=0.5	SAT (TO) f=29 TT=2400.05 T1=0.5	SAT (TO) f=29 TT=2400.03 T1=0.5	SAT (TO) f=29 TT=2400.08 T1=0.52	No data	No data	No data	? (TO) TT=2400.05
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-02.xml	OPT f=820 TT=0.48 T1=0	OPT f=820 TT=0.48 T1=0	OPT f=820 TT=0.49 T1=0	OPT f=820 TT=0.48 T1=0	No data	No data	No data	OPT f=820 TT=29.427 T1=29.38
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-03.xml	OPT f=781 TT=1.714 T1=0.1	OPT f=781 TT=1.714 T1=0.1	OPT f=781 TT=1.727 T1=0.1	OPT f=781 TT=1.724 T1=0.1	No data	No data	No data	OPT f=781 TT=48.454 T1=48.38
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-08.xml	OPT f=1059 TT=1.69 T1=0	OPT f=1059 TT=1.703 T1=0	OPT f=1059 TT=1.706 T1=0	OPT f=1059 TT=1.69 T1=0	No data	No data	No data	OPT f=1059 TT=54.563 T1=54.49
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Knapsack/ Knapsack-m1-s30/ Knapsack-30-100-09.xml	OPT f=569 TT=0.478 T1=0	OPT f=569 TT=0.48 T1=0	OPT f=569 TT=0.479 T1=0	OPT f=569 TT=0.478 T1=0	No data	No data	No data	OPT f=569 TT=48.687 T1=48.59
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-00.xml	OPT f=932 TT=19.125 T1=1	OPT f=932 TT=19.261 T1=1	OPT f=932 TT=19.148 T1=1	OPT f=932 TT=19.125 T1=1	No data	No data	No data	OPT f=932 TT=124.267 T1=124.18
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-01.xml	OPT f=1128 TT=10.015 T1=1.69	OPT f=1128 TT=10.015 T1=1.69	OPT f=1128 TT=10.046 T1=1.69	OPT f=1128 TT=10.038 T1=1.69	No data	No data	No data	OPT f=1128 TT=83.127 T1=83.09
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-05.xml	OPT f=1072 TT=16.831 T1=1.89	OPT f=1072 TT=16.976 T1=1.99	OPT f=1072 TT=16.831 T1=1.89	OPT f=1072 TT=16.888 T1=1.89	No data	No data	No data	OPT f=1072 TT=70.502 T1=70.4
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-06.xml	OPT f=741 TT=20.16 T1=0	OPT f=741 TT=20.192 T1=0	OPT f=741 TT=20.16 T1=0	OPT f=741 TT=20.236 T1=0	No data	No data	No data	OPT f=741 TT=143.653 T1=143.58
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-07.xml	OPT f=1021 TT=23.56 T1=0	OPT f=1021 TT=23.56 T1=0	OPT f=1021 TT=23.915 T1=0	OPT f=1021 TT=23.637 T1=0	No data	No data	No data	OPT f=1021 TT=136.185 T1=136.11
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-08.xml	OPT f=887 TT=6.928 T1=0.31	OPT f=887 TT=6.938 T1=0.31	OPT f=887 TT=6.981 T1=0.31	OPT f=887 TT=6.928 T1=0.31	No data	No data	No data	OPT f=887 TT=122.808 T1=122.79
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-09.xml	OPT f=1102 TT=26.295 T1=3.59	OPT f=1102 TT=26.331 T1=3.59	OPT f=1102 TT=26.558 T1=3.69	OPT f=1102 TT=26.295 T1=3.59	No data	No data	No data	OPT f=1102 TT=91.847 T1=91.8
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-10.xml	OPT f=973 TT=12.136 T1=1.89	OPT f=973 TT=12.158 T1=1.89	OPT f=973 TT=12.258 T1=1.89	OPT f=973 TT=12.136 T1=1.89	No data	No data	No data	OPT f=973 TT=129.081 T1=128.99
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-12.xml	OPT f=954 TT=21.712 T1=3.49	OPT f=954 TT=21.712 T1=3.49	OPT f=954 TT=21.945 T1=3.59	OPT f=954 TT=21.722 T1=3.49	No data	No data	No data	OPT f=954 TT=152.826 T1=152.78
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-13.xml	OPT f=1242 TT=16.121 T1=0.4	OPT f=1242 TT=16.121 T1=0.4	OPT f=1242 TT=16.176 T1=0.41	OPT f=1242 TT=16.176 T1=0.41	No data	No data	No data	OPT f=1242 TT=99.756 T1=99.68
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-14.xml	OPT f=941 TT=4.942 T1=1.39	OPT f=941 TT=4.953 T1=1.39	OPT f=941 TT=4.942 T1=1.39	OPT f=941 TT=4.957 T1=1.4	No data	No data	No data	OPT f=941 TT=129.978 T1=129.88
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-15.xml	OPT f=867 TT=12.988 T1=1.89	OPT f=867 TT=13.025 T1=1.89	OPT f=867 TT=12.988 T1=1.89	OPT f=867 TT=13.009 T1=1.89	No data	No data	No data	OPT f=867 TT=150.679 T1=150.66
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-16.xml	OPT f=886 TT=29.829 T1=4.39	OPT f=886 TT=30.172 T1=4.49	OPT f=886 TT=29.829 T1=4.39	OPT f=886 TT=30.029 T1=4.39	No data	No data	No data	OPT f=886 TT=189.816 T1=189.77
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-17.xml	OPT f=1030 TT=28.987 T1=2.39	OPT f=1030 TT=28.987 T1=2.39	OPT f=1030 TT=29.047 T1=2.39	OPT f=1030 TT=29.201 T1=2.39	No data	No data	No data	OPT f=1030 TT=104.542 T1=104.49
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-18.xml	OPT f=863 TT=10.475 T1=1.19	OPT f=863 TT=10.531 T1=1.19	OPT f=863 TT=10.54 T1=1.19	OPT f=863 TT=10.475 T1=1.19	No data	No data	No data	OPT f=863 TT=123.288 T1=123.27
Knapsack/ Knapsack-m1-s40/ Knapsack-40-150-19.xml	OPT f=975 TT=7.296 T1=0.21	OPT f=975 TT=7.296 T1=0.21	OPT f=975 TT=7.318 T1=0.21	OPT f=975 TT=7.334 T1=0.21	No data	No data	No data	OPT f=975 TT=96.884 T1=96.78
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-00.xml	OPT f=960 TT=29.277 T1=6.29	OPT f=960 TT=29.353 T1=6.29	OPT f=960 TT=29.277 T1=6.29	OPT f=960 TT=29.468 T1=6.39	No data	No data	No data	OPT f=960 TT=215.72 T1=215.66
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-01.xml	OPT f=1297 TT=47.488 T1=0	OPT f=1297 TT=47.488 T1=0	OPT f=1297 TT=47.644 T1=0	OPT f=1297 TT=47.51 T1=0	No data	No data	No data	OPT f=1297 TT=216.556 T1=216.49
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-02.xml	OPT f=1003 TT=244.357 T1=8.29	OPT f=1003 TT=245.075 T1=8.29	OPT f=1003 TT=244.357 T1=8.29	OPT f=1003 TT=244.943 T1=8.29	No data	No data	No data	OPT f=1003 TT=621.711 T1=181.27
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-03.xml	OPT f=1109 TT=123.668 T1=12.59	OPT f=1109 TT=123.668 T1=12.59	OPT f=1109 TT=124.103 T1=12.59	OPT f=1109 TT=123.989 T1=12.59	No data	No data	No data	OPT f=1109 TT=433.921 T1=373.62
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-04.xml	OPT f=1289 TT=107.777 T1=4.49	OPT f=1289 TT=108.748 T1=4.59	OPT f=1289 TT=108.267 T1=4.59	OPT f=1289 TT=107.777 T1=4.49	No data	No data	No data	OPT f=1289 TT=491.882 T1=491.84
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-05.xml	OPT f=1105 TT=37.593 T1=22	OPT f=1105 TT=37.694 T1=22	OPT f=1105 TT=37.593 T1=22	OPT f=1105 TT=38.47 T1=22.49	No data	No data	No data	OPT f=1105 TT=198.684 T1=198.62
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-06.xml	OPT f=1079 TT=74.423 T1=0	OPT f=1079 TT=74.59 T1=0	OPT f=1079 TT=74.423 T1=0	OPT f=1079 TT=74.938 T1=0	No data	No data	No data	OPT f=1079 TT=426.806 T1=166.9
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-07.xml	OPT f=1070 TT=47.812 T1=9.69	OPT f=1070 TT=48.06 T1=9.69	OPT f=1070 TT=47.812 T1=9.69	OPT f=1070 TT=48.122 T1=9.69	No data	No data	No data	OPT f=1070 TT=142.181 T1=142.1
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-08.xml	OPT f=1169 TT=41.198 T1=21.49	OPT f=1169 TT=41.43 T1=21.69	OPT f=1169 TT=41.39 T1=21.59	OPT f=1169 TT=41.198 T1=21.49	No data	No data	No data	OPT f=1169 TT=125.924 T1=125.87
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-09.xml	OPT f=1241 TT=86.406 T1=14	OPT f=1241 TT=86.525 T1=14	OPT f=1241 TT=86.859 T1=14.1	OPT f=1241 TT=86.406 T1=14	No data	No data	No data	OPT f=1241 TT=204.978 T1=204.9
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-10.xml	OPT f=1148 TT=33.05 T1=4.89	OPT f=1148 TT=33.548 T1=5	OPT f=1148 TT=33.301 T1=5	OPT f=1148 TT=33.05 T1=4.89	No data	No data	No data	OPT f=1148 TT=229.322 T1=229.31
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-11.xml	OPT f=1049 TT=86.956 T1=17.99	OPT f=1049 TT=86.956 T1=17.99	OPT f=1049 TT=87.728 T1=18.1	OPT f=1049 TT=87.416 T1=18.1	No data	No data	No data	OPT f=1049 TT=273.396 T1=132.48
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-15.xml	OPT f=986 TT=64.971 T1=4.79	OPT f=986 TT=65.643 T1=4.89	OPT f=986 TT=65.056 T1=4.79	OPT f=986 TT=64.971 T1=4.79	No data	No data	No data	OPT f=986 TT=180.209 T1=180.11
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-16.xml	OPT f=1429 TT=309.279 T1=12.79	OPT f=1429 TT=310.27 T1=12.89	OPT f=1429 TT=309.97 T1=12.79	OPT f=1429 TT=309.279 T1=12.79	No data	No data	No data	OPT f=1429 TT=699.051 T1=641.56
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-17.xml	OPT f=992 TT=221.484 T1=1.39	OPT f=992 TT=222.499 T1=1.39	OPT f=992 TT=221.484 T1=1.39	OPT f=992 TT=222.23 T1=1.39	No data	No data	No data	OPT f=992 TT=409.737 T1=409.69
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-18.xml	OPT f=937 TT=65.045 T1=6.1	OPT f=937 TT=65.045 T1=6.1	OPT f=937 TT=65.09 T1=6.1	OPT f=937 TT=65.49 T1=6.19	No data	No data	No data	OPT f=937 TT=175.161 T1=115.99
Knapsack/ Knapsack-m1-s50/ Knapsack-50-200-19.xml	OPT f=1025 TT=206.976 T1=1.1	OPT f=1025 TT=206.976 T1=1.1	OPT f=1025 TT=208.001 T1=1.1	OPT f=1025 TT=207.116 T1=1.1	No data	No data	No data	OPT f=1025 TT=635.746 T1=355.79
Langford/ Langford-m1-k3/ Langford-3-45.xml	SAT TT=65.653	? (TO) TT=2400.03	? (TO) TT=2400.03	? (TO) TT=2400.11	? (TO) TT=2400.01	SAT TT=440.419	SAT TT=65.653	? (TO) TT=2400.09
Langford/ Langford-m1-k4/ Langford-4-16.xml	UNSAT TT=84.703	UNSAT TT=1528.47	UNSAT TT=1714.87	UNSAT TT=1492.98	? TT=2400	UNSAT TT=1188.5	? (TO) TT=2400.03	UNSAT TT=84.703
Langford/ Langford-m2-s1/ LangfordBin-20.xml	SAT TT=0.008	SAT TT=0.03	SAT TT=0.029	SAT TT=0.029	? TT=2400.01	SAT TT=0.008	SAT TT=0.009	SAT TT=4.725
Langford/ Langford-m2-s1/ LangfordBin-24.xml	SAT TT=5.862	? (TO) TT=2400.07	? (TO) TT=2400.01	? (TO) TT=2400.01	? TT=2400.01	? (TO) TT=2400.06	? (TO) TT=2400.07	SAT TT=5.862
Langford/ Langford-m2-s1/ LangfordBin-70.xml		? (TO) TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.02	? (TO) TT=2400.06	? (TO) TT=2400.05	? (TO) TT=2400.14	? (TO) TT=2400.06
LatinSquare/ LatinSquare-xcsp2-qcp20/ qcp-20-187-03_X2.xml	SAT TT=4.701	SAT TT=311.564	SAT TT=322.85	SAT TT=315.684	SAT TT=429.87	? (TO) TT=2400.03	SAT TT=4.701	SAT TT=5.626
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
LatinSquare/ LatinSquare-xcsp2-qcp25/ qcp-25-264-09_X2.xml	SAT TT=14.06	? (TO) TT=2400.03	? (TO) TT=2400.02	? (TO) TT=2400.01	? (TO) TT=2400.08	? (TO) TT=2400.04	SAT TT=191.236	SAT TT=14.06
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-006.xml	OPT f=7 TT=0.012 T1=0	OPT f=7 TT=0.013 T1=0	OPT f=7 TT=0.014 T1=0	OPT f=7 TT=0.012 T1=0	No data	No data	No data	OPT f=7 TT=0.099 T1=0
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-012.xml	OPT f=10 TT=0.214 T1=0	OPT f=10 TT=0.214 T1=0	OPT f=10 TT=0.216 T1=0	OPT f=10 TT=0.215 T1=0	No data	No data	No data	OPT f=10 TT=0.801 T1=0.69
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-013.xml	OPT f=6 TT=0.096 T1=0	OPT f=6 TT=0.096 T1=0	OPT f=6 TT=0.097 T1=0	OPT f=6 TT=0.096 T1=0	No data	No data	No data	OPT f=6 TT=0.192 T1=0.09
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-017.xml	OPT f=32 TT=26.99 T1=0.2	OPT f=32 TT=36.585 T1=0.21	OPT f=32 TT=35.725 T1=0.2	OPT f=32 TT=36.016 T1=0.2	No data	No data	No data	OPT f=32 TT=26.99 T1=26.89
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-018.xml	OPT f=25 TT=17.611 T1=1	OPT f=25 TT=60.955 T1=1	OPT f=25 TT=60.715 T1=1	OPT f=25 TT=60.39 T1=1	No data	No data	No data	OPT f=25 TT=17.611 T1=17.59
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-019.xml	OPT f=29 TT=32.232 T1=1.88	OPT f=29 TT=133.222 T1=1.89	OPT f=29 TT=133.545 T1=1.88	OPT f=29 TT=133.396 T1=1.89	No data	No data	No data	OPT f=29 TT=32.232 T1=32.18
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-020.xml	OPT f=26 TT=29.899 T1=3.19	OPT f=26 TT=331.978 T1=3.19	OPT f=26 TT=331.118 T1=3.19	OPT f=26 TT=356.782 T1=3.19	No data	No data	No data	OPT f=26 TT=29.899 T1=29.79
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-021.xml	OPT f=26 TT=31.785 T1=31.68	OPT f=26 TT=365.275 T1=168.2	OPT f=26 TT=361.351 T1=167	OPT f=26 TT=381.21 T1=172.79	No data	No data	No data	OPT f=26 TT=31.785 T1=31.68
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-029.xml	SAT TO f=62 TT=2400.05 T1=1710.11	SAT (TO) f=62 TT=2400.09 T1=1745.14	SAT (TO) f=62 TT=2400.05 T1=1710.11	SAT (TO) f=62 TT=2400.07 T1=1784.55	No data	No data	No data	? (TO) TT=2400.01
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-033.xml	SAT TO f=96 TT=2400.05 T1=956.56	SAT (TO) f=96 TT=2400.09 T1=956.56	SAT (TO) f=96 TT=2400.09 T1=959.22	SAT (TO) f=96 TT=2400.05 T1=962.44	No data	No data	No data	? (TO) TT=2400.02
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-038.xml	SAT TO f=143 TT=2400.05 T1=1954.77	SAT (TO) f=143 TT=2400.07 T1=2015.12	SAT (TO) f=159 TT=2400.09 T1=62.48	SAT (TO) f=143 TT=2400.05 T1=1954.77	No data	No data	No data	? (TO) TT=2400.1
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-062.xml	SAT TO f=595 TT=2400.08 T1=2264	SAT (TO) f=595 TT=2400.09 T1=2321.39	SAT (TO) f=595 TT=2400.08 T1=2288.27	SAT (TO) f=595 TT=2400.1 T1=2264	No data	No data	No data	? (TO) TT=2400.02
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-067.xml	SAT TO f=881 TT=2400.05 T1=264.64	SAT (TO) f=881 TT=2400.05 T1=264.64	SAT (TO) f=881 TT=2400.11 T1=289.07	SAT (TO) f=881 TT=2400.1 T1=293.01	No data	No data	No data	? (TO) TT=2400.09
LowAutocorrelation/ LowAutocorrelation-m1-s1/ LowAutocorrelation-089.xml	SAT TO f=1620 TT=2400.03 T1=170.03	SAT (TO) f=1620 TT=2400.1 T1=170.03	SAT (TO) f=1620 TT=2400.03 T1=175.2	SAT (TO) f=1620 TT=2400.08 T1=170.83	No data	No data	No data	? (TO) TT=2400.06
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-04-0003.xml	SAT TT=0.556	SAT TT=1.546	SAT TT=1.546	SAT TT=1.544	SAT TT=3.167	SAT TT=2.039	SAT TT=0.556	SAT TT=119.281
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-05-0033.xml		? (TO) TT=2400.08	? (TO) TT=2400.04	? (TO) TT=2400.1	? TT=2400.01	? (TO) TT=2400.06	? (TO) TT=2400.1	? (TO) TT=2400.01
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-05-m012.xml	SAT TT=626.863	SAT TT=636.031	SAT TT=627.174	SAT TT=626.863	? TT=2400	? (TO) TT=2400.06	? (TO) TT=2400.1	? (TO) TT=2400.03
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-06-m111.xml		? (TO) TT=2400.11	? (TO) TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.01	? (TO) TT=2400.04	? (TO) TT=2400.09	? (TO) TT=2400.08
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-07-0080.xml	UNSAT TT=1154.64	? (TO) TT=2400.01	? (TO) TT=2400.07	? (TO) TT=2400.07	? TT=2400.01	? (TO) TT=2400.02	? (TO) TT=2400.04	UNSAT TT=1154.64
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-07-m102.xml		? (TO) TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.09	? TT=2400.01	? (TO) TT=2400.04	? (TO) TT=2400.06	? (TO) TT=2400.03
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-07-m167.xml		? (TO) TT=2400.05	? (TO) TT=2400.1	? (TO) TT=2400.05	? TT=2400.01	? (TO) TT=2400.05	? (TO) TT=2400.08	? (TO) TT=2400.03
MagicHexagon/ MagicHexagon-m1-s1/ MagicHexagon-11-m018.xml		? (TO) TT=2400.07	? (TO) TT=2400.01	? (TO) TT=2400.07	? TT=2400.03	? (TO) TT=2400.07	? (TO) TT=2400.03	? (TO) TT=2400.07
MagicSquare/ MagicSquare-sum-s1/ MagicSquare-27-sum.xml		? (TO) TT=2400.09	? (TO) TT=2400.07	? (TO) TT=2400.01	? (TO) TT=2400.06	? (TO) TT=2400.03	? (TO) TT=2400.07	? (TO) TT=2400.03
MarketSplit/ MarketSplit-m1-gp/ MarketSplit-08.xml	SAT TT=7.19	SAT TT=68.844	SAT TT=80.33	SAT TT=69.228	SAT TT=7.19	SAT TT=466.269	SAT TT=226.792	SAT TT=52.747
MaxCSP/MaxCSP-maxclique-s1/ MaxCSP-p-hat1000-2.xml	UNSAT TT=1.198	UNSAT TT=4.506	UNSAT TT=4.399	UNSAT TT=4.471	UNSAT TT=901.776	UNSAT TT=1.223	UNSAT TT=1.198	UNSAT TT=2.833
MaxCSP/ MaxCSP-maxclique-s1/MaxCSP-san-1000.xml	UNSAT TT=1.18	UNSAT TT=4.33	UNSAT TT=4.225	UNSAT TT=4.313	UNSAT TT=865.364	UNSAT TT=1.18	UNSAT TT=1.182	UNSAT TT=2.746
MaxCSP/ MaxCSP-random-denseLoose/rand-30-10-25-48-32.xml	UNSAT TT=0.011	UNSAT TT=0.02	UNSAT TT=0.023	UNSAT TT=0.022	UNSAT TT=0.022	UNSAT TT=0.012	UNSAT TT=0.011	UNSAT TT=0.189
Mixed/ Mixed-xcsp2-cril/ cril-2_X2.xml	UNSAT TT=0.037	UNSAT TT=0.104	UNSAT TT=0.104	UNSAT TT=0.104	UNSAT TT=198.029	UNSAT TT=0.038	UNSAT TT=0.037	UNSAT TT=0.142
MultiKnapsack/ MultiKnapsack-m1-gp/ MultiKnapsack-1-06.xml	SAT TT=18.947	SAT TT=30.038	SAT TT=29.928	SAT TT=34.39	SAT TT=18.947	SAT TT=1567.38	SAT TT=221.406	SAT TT=123.101
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Nonogram/ Nonogram-table-s1/ Nonogram-019-table.xml	SAT TT=18.078	SAT TT=110.277	SAT TT=110.059	SAT TT=109.539	SAT TT=18.078	SAT TT=30.598	SAT TT=32.18	SAT TT=78.424
Nonogram/ Nonogram-table-s1/ Nonogram-122-table.xml	SAT TT=28.527	SAT TT=173.555	SAT TT=173.028	SAT TT=173.128	SAT TT=28.527	SAT TT=48.48	SAT TT=47.186	SAT TT=163.414
Nonogram/ Nonogram-table-s1/ Nonogram-151-table.xml	SAT TT=3.436	SAT TT=20.891	SAT TT=20.978	SAT TT=20.852	SAT TT=3.436	SAT TT=5.829	SAT TT=5.917	SAT TT=16.571
OpenStacks/ OpenStacks-m1c-s1/ OpenStacks-m1c-gp-050-1.xml	OPT f=45 TT=624.527 T1=624.47	SAT (TO) f=47 TT=2400.1 T1=320.23	SAT (TO) f=47 TT=2400.03 T1=308.09	SAT (TO) f=47 TT=2400.07 T1=321.92	No data	No data	No data	OPT f=45 TT=624.527 T1=624.47
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-pb-20-10-1.xml	OPT f=9 TT=20.109 T1=0.09	OPT f=9 TT=1708.88 T1=0.09	OPT f=9 TT=1937.96 T1=0.1	OPT f=9 TT=1731.21 T1=0.09	No data	No data	No data	OPT f=9 TT=20.109 T1=20.09
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbo-10-20-1.xml	OPT f=5 TT=37.457 T1=37.4	SAT (TO) f=6 TT=2400.1 T1=0.69	SAT (TO) f=6 TT=2400.08 T1=0.69	SAT (TO) f=6 TT=2400.04 T1=0.69	No data	No data	No data	OPT f=5 TT=37.457 T1=37.4
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbo-15-30-1.xml	OPT f=4 TT=44.082 T1=43.99	SAT (TO) f=9 TT=2400.05 T1=641.3	SAT (TO) f=9 TT=2400.1 T1=615.06	SAT (TO) f=9 TT=2400.06 T1=624.26	No data	No data	No data	OPT f=4 TT=44.082 T1=43.99
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbo-20-20-1.xml	OPT f=3 TT=27.614 T1=27.58	SAT (TO) f=5 TT=2400.07 T1=6.38	SAT (TO) f=5 TT=2400.05 T1=6.29	SAT (TO) f=5 TT=2400.05 T1=6.29	No data	No data	No data	OPT f=3 TT=27.614 T1=27.58
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbo-30-10-1.xml	OPT f=10 TT=37.36 T1=24.18	OPT f=10 TT=1084.01 T1=24.18	OPT f=10 TT=1107.92 T1=24.69	OPT f=10 TT=1095.76 T1=24.38	No data	No data	No data	OPT f=10 TT=37.36 T1=37.28
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbo-30-15-1.xml	OPT f=7 TT=48.588 T1=48.49	SAT (TO) f=8 TT=2400.01 T1=249.96	SAT (TO) f=8 TT=2400 T1=266.5	SAT (TO) f=8 TT=2400.1 T1=260.67	No data	No data	No data	OPT f=7 TT=48.588 T1=48.49
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbop-10-20-1.xml	OPT f=5 TT=27.263 T1=27.18	SAT (TO) f=6 TT=2400.09 T1=24.99	SAT (TO) f=6 TT=2400.01 T1=26.69	SAT (TO) f=6 TT=2400.01 T1=30	No data	No data	No data	OPT f=5 TT=27.263 T1=27.18
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbop-20-20-1.xml	OPT f=3 TT=29.582 T1=29.49	SAT (TO) f=6 TT=2400.08 T1=1882.97	SAT (TO) f=6 TT=2400.01 T1=1854.51	SAT (TO) f=6 TT=2400.07 T1=1937.47	No data	No data	No data	OPT f=3 TT=29.582 T1=29.49
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbop-30-15-1.xml	OPT f=10 TT=48.681 T1=48.58	SAT (TO) f=11 TT=2400.08 T1=8.99	SAT (TO) f=11 TT=2400.06 T1=8.89	SAT (TO) f=11 TT=2400.08 T1=8.88	No data	No data	No data	OPT f=10 TT=48.681 T1=48.58
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbop-30-30-1.xml	OPT f=3 TT=44.111 T1=44.09	SAT (TO) f=13 TT=2400.03 T1=6.39	SAT (TO) f=13 TT=2400.01 T1=6.19	SAT (TO) f=13 TT=2400 T1=6.1	No data	No data	No data	OPT f=3 TT=44.111 T1=44.09
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbp-15-15-1.xml	OPT f=4 TT=15.718 T1=15.69	SAT (TO) f=4 TT=2400.01 T1=560.86	SAT (TO) f=4 TT=2400.02 T1=570.26	SAT (TO) f=4 TT=2400.02 T1=558.07	No data	No data	No data	OPT f=4 TT=15.718 T1=15.69
OpenStacks/OpenStacks-m1c-s1/ OpenStacks-m1c-wbp-20-10-1.xml	OPT f=8 TT=14.98 T1=0.1	OPT f=8 TT=1643.84 T1=0.1	OPT f=8 TT=1634.97 T1=0.1	OPT f=8 TT=1666.67 T1=0.1	No data	No data	No data	OPT f=8 TT=14.98 T1=14.89
OpenStacks/ OpenStacks-m1-s1/ OpenStacks-m1-gp-050-1.xml	OPT f=45 TT=705.708 T1=705.63	SAT (TO) f=47 TT=2400.08 T1=386.6	SAT (TO) f=47 TT=2400.07 T1=302.51	SAT (TO) f=47 TT=2400.08 T1=339.28	No data	No data	No data	OPT f=45 TT=705.708 T1=705.63
OpenStacks/ OpenStacks-m1-s1/ OpenStacks-m1-Miller-19.xml	SAT TO f=17 TT=2400.02 T1=19.59	SAT (TO) f=17 TT=2400.02 T1=19.59	SAT (TO) f=17 TT=2400.03 T1=19.59	SAT (TO) f=17 TT=2400.09 T1=19.68	No data	No data	No data	? (TO) TT=2400.02
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-nrwsLarge-4-1.xml	OPT f=12 TT=899.249 T1=899.16	SAT (TO) f=19 TT=2400.09 T1=3.38	SAT (TO) f=19 TT=2400.11 T1=4.29	SAT (TO) f=19 TT=2400.05 T1=4.09	No data	No data	No data	OPT f=12 TT=899.249 T1=899.16
OpenStacks/ OpenStacks-m1-s1/ OpenStacks-m1-pb-20-10-1.xml	OPT f=9 TT=18.659 T1=0.09	OPT f=9 TT=1684.23 T1=0.09	OPT f=9 TT=1762.11 T1=0.1	OPT f=9 TT=1707.29 T1=0.1	No data	No data	No data	OPT f=9 TT=18.659 T1=18.59
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
OpenStacks/ OpenStacks-m1-s1/ OpenStacks-m1-pb-30-10-1.xml	UNSAT TT=37.743	UNSAT TT=1582.87	UNSAT TT=1586.66	UNSAT TT=1559.84	No data	No data	No data	UNSAT TT=37.743
OpenStacks/ OpenStacks-m1-s1/ OpenStacks-m1-pb-30-15-1.xml	OPT f=14 TT=78.244 T1=78.19	? (TO) TT=2400.1	? (TO) TT=2400.02	? (TO) TT=2400.04	No data	No data	No data	OPT f=14 TT=78.244 T1=78.19
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbo-10-20-1.xml	OPT f=5 TT=29.521 T1=29.48	SAT (TO) f=6 TT=2400.08 T1=0.69	SAT (TO) f=6 TT=2400.08 T1=0.69	SAT (TO) f=6 TT=2400.1 T1=0.69	No data	No data	No data	OPT f=5 TT=29.521 T1=29.48
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbo-10-30-1.xml	OPT f=6 TT=82.691 T1=34.3	SAT (TO) f=7 TT=2400.06 T1=40	SAT (TO) f=7 TT=2400.01 T1=47	SAT (TO) f=7 TT=2400.08 T1=46.88	No data	No data	No data	OPT f=6 TT=82.691 T1=34.3
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbo-20-20-1.xml	OPT f=3 TT=32.502 T1=32.49	SAT (TO) f=5 TT=2400.04 T1=6.29	SAT (TO) f=5 TT=2400.1 T1=6.29	SAT (TO) f=5 TT=2400.09 T1=6.1	No data	No data	No data	OPT f=3 TT=32.502 T1=32.49
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbo-30-15-1.xml	OPT f=7 TT=42.815 T1=42.79	SAT (TO) f=8 TT=2400.02 T1=269.12	SAT (TO) f=8 TT=2400.03 T1=260.28	SAT (TO) f=8 TT=2400.06 T1=251.87	No data	No data	No data	OPT f=7 TT=42.815 T1=42.79
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbop-10-20-1.xml	OPT f=5 TT=26.764 T1=26.7	SAT (TO) f=6 TT=2400.07 T1=24.49	SAT (TO) f=6 TT=2400.01 T1=29.6	SAT (TO) f=6 TT=2400.02 T1=26	No data	No data	No data	OPT f=5 TT=26.764 T1=26.7
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbop-30-15-1.xml	OPT f=10 TT=49.552 T1=49.48	SAT (TO) f=11 TT=2400.1 T1=9.28	SAT (TO) f=11 TT=2400.1 T1=9.59	SAT (TO) f=11 TT=2400.03 T1=9.38	No data	No data	No data	OPT f=10 TT=49.552 T1=49.48
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbop-30-30-1.xml	OPT f=3 TT=61.183 T1=61.11	SAT (TO) f=13 TT=2400.11 T1=6.19	SAT (TO) f=13 TT=2400.02 T1=6	SAT (TO) f=13 TT=2400.09 T1=6.59	No data	No data	No data	OPT f=3 TT=61.183 T1=61.11
OpenStacks/OpenStacks-m1-s1/ OpenStacks-m1-wbp-15-15-1.xml	OPT f=4 TT=15.657 T1=15.59	SAT (TO) f=4 TT=2400.05 T1=558.73	SAT (TO) f=4 TT=2400.07 T1=563.04	SAT (TO) f=4 TT=2400.03 T1=566.36	No data	No data	No data	OPT f=4 TT=15.657 T1=15.59
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-06-10.xml	UNSAT TT=25.708	UNSAT TT=209.597	UNSAT TT=209.061	UNSAT TT=209.079	UNSAT TT=25.708	UNSAT TT=235.2	UNSAT TT=218.817	Signal TT=631.463
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-10-07.xml	UNSAT TT=703.608	? (TO) TT=2400.03	? (TO) TT=2400.02	? (TO) TT=2400.11	UNSAT TT=703.608	? (TO) TT=2400.07	? (TO) TT=2400.12	? (TO) TT=2400.1
PigeonsPlus/ PigeonsPlus-m1-s1/ pigeonsPlus-11-07.xml		? (TO) TT=2400.07	? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.05	? (TO) TT=2400.22	? (TO) TT=2400.3	? (MO) TT=876.365
Primes/Primes-m1-p10/ Primes-10-40-3-5.xml	SAT TT=261.548	? (TO) TT=2400.02	? (TO) TT=2400.05	? (TO) TT=2400.09	SAT TT=261.548	? (TO) TT=2400.07	? (TO) TT=2400.06	SAT TT=2266.26
Primes/Primes-m1-p10/ Primes-10-60-3-5.xml	SAT TT=0.312	SAT TT=0.331	SAT TT=0.335	SAT TT=0.331	SAT TT=0.312	SAT TT=2.206	SAT TT=16.821	SAT TT=38.764
Primes/Primes-m1-p20/ Primes-20-20-3-3.xml	SAT TT=0.007	SAT TT=0.008	SAT TT=0.007	SAT TT=0.008	SAT TT=0.008	SAT TT=0.007	SAT TT=0.013	SAT TT=5.436
Primes/Primes-m1-p20/ Primes-20-40-2-7.xml	SAT TT=1.826	? (TO) TT=2400.08	? (TO) TT=2400.02	? (TO) TT=2400.03	SAT TT=1.826	SAT TT=15.722	SAT TT=81.722	SAT TT=1520.8
Primes/Primes-m1-p20/ Primes-20-60-3-5.xml	SAT TT=517.414	? (TO) TT=2400.1	? (TO) TT=2400.1	? (TO) TT=2400.1	SAT TT=517.414	? (TO) TT=2400.08	SAT TT=1703.2	? (TO) TT=2400.1
Primes/Primes-m1-p25/ Primes-25-60-3-3.xml	SAT TT=0.046	SAT TT=9.436	SAT TT=9.321	SAT TT=9.285	SAT TT=0.345	SAT TT=0.046	SAT TT=0.088	SAT TT=28.875
Primes/Primes-m1-p25/ Primes-25-60-3-7.xml		? (TO) TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.06	? TT=2400	? (TO) TT=2400.04	? (TO) TT=2400.03	? (TO) TT=2400.07
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Primes/Primes-m1-p25/ Primes-25-80-3-5.xml	SAT TT=0.42	SAT TT=101.484	SAT TT=101.021	SAT TT=100.777	SAT TT=0.42	SAT TT=25.401	SAT TT=0.681	SAT TT=896.754
Primes/Primes-m1-p30/ Primes-30-80-3-7.xml	SAT TT=0.237	SAT TT=1.25	SAT TT=1.239	SAT TT=1.24	SAT TT=0.353	SAT TT=0.298	SAT TT=0.237	SAT TT=114.646
PropStress/ PropStress-m1-s1/ PropStress-0200.xml	UNSAT TT=72.383	? (TO) TT=2400.06	? (TO) TT=2400.02	? (TO) TT=2400.04	UNSAT TT=588.109	? (TO) TT=2400.04	? (TO) TT=2400.02	UNSAT TT=72.383
PropStress/ PropStress-m1-s1/ PropStress-0450.xml	UNSAT TT=503.862	? (TO) TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.08	? (MO) TT=342.388	? (exit code) TT=16.113	? (exit code) TT=15.8	UNSAT TT=503.862
PropStress/ PropStress-m1-s1/ PropStress-0650.xml	UNSAT TT=1671.21	? (TO) TT=2400.36	? (TO) TT=2400.28	? (TO) TT=2400.36	? (MO) TT=66.566	? (exit code) TT=15.725	? (exit code) TT=15.575	UNSAT TT=1671.21
PseudoBoolean/ PseudoBoolean-dec-fpga/ Pb-chnl10-11.xml	UNSAT TT=56.641	UNSAT TT=2113.98	UNSAT TT=2128.69	UNSAT TT=2136.77	? TT=2400.01	? (TO) TT=2400.01	UNSAT TT=213.336	UNSAT TT=56.641
PseudoBoolean/ PseudoBoolean-dec-fpga/ Pb-chnl40-50.xml		? (TO) TT=2400.01	? (TO) TT=2400.01	? (TO) TT=2400.09	? (TO) TT=2400.07	? (TO) TT=2400.08	? (TO) TT=2400.05	? (TO) TT=2400.1
PseudoBoolean/ PseudoBoolean-dec-fpga/ Pb-fpga45-44-sat.xml	SAT TT=16.378	? (TO) TT=2400.06	? (TO) TT=2400.09	? (TO) TT=2400.02	? (TO) TT=2400.08	? (TO) TT=2400.08	SAT TT=16.378	SAT TT=135.487
PseudoBoolean/ PseudoBoolean-dec-pigeon/ Pb-pigeon-card-011-010.xml	UNSAT TT=29.588	UNSAT TT=1690.99	UNSAT TT=1706.72	UNSAT TT=1706.11	? TT=2400.01	? (TO) TT=2400.06	UNSAT TT=223.709	UNSAT TT=29.588
PseudoBoolean/ PseudoBoolean-dec-pigeon/ Pb-pigeon-card-015-010.xml	UNSAT TT=106.65	? (TO) TT=2400.06	? (TO) TT=2400.08	? (TO) TT=2400.1	? TT=2400	? (TO) TT=2400.03	UNSAT TT=121.72	UNSAT TT=106.65
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
PseudoBoolean/ PseudoBoolean-dec-pigeon/ Pb-pigeon-clau-011-010.xml	UNSAT TT=6.61	? (TO) TT=2400.05	? (TO) TT=2400.04	? (TO) TT=2400.05	UNSAT TT=1511.17	? (TO) TT=2400.06	UNSAT TT=6.61	UNSAT TT=63.564
PseudoBoolean/ PseudoBoolean-dec-spba/ Pb-BeauxArts-K67.xml	SAT TT=165.471	? (TO) TT=2400.07	? (TO) TT=2400.08	? (TO) TT=2400.03	? (TO) TT=2400.04	? (TO) TT=2400.06	? (TO) TT=2400.15	SAT TT=165.471
PseudoBoolean/ PseudoBoolean-dec-spba/ Pb-BeauxArts-K75.xml	SAT TT=16.783	? (TO) TT=2400.06	? (TO) TT=2400.11	? (TO) TT=2400.07	? (TO) TT=2400.1	? (TO) TT=2400.04	SAT TT=99.403	SAT TT=16.783
PseudoBoolean/ PseudoBoolean-dec-spba/ Pb-BeauxArts-K80.xml	SAT TT=15.801	? (TO) TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.02	? (TO) TT=2400.05	SAT TT=20.912	SAT TT=26.556	SAT TT=15.801
PseudoBoolean/ PseudoBoolean-dec-spng/ Pb-NG-K138.xml		? (TO) TT=2400.03	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.03	? (TO) TT=2400.06	? (TO) TT=2400.22	? (TO) TT=2400.01
PseudoBoolean/ PseudoBoolean-dec-spng/ Pb-NG-K151.xml		? (TO) TT=2400.08	? (TO) TT=2400.08	? (TO) TT=2400.09	? (TO) TT=2400.08	? (TO) TT=2400.06	? (TO) TT=2400.12	? (TO) TT=2400.05
PseudoBoolean/ PseudoBoolean-dec-spng/ Pb-NG-K152.xml		? (TO) TT=2400.07	? (TO) TT=2400.07	? (TO) TT=2400.12	? (TO) TT=2400.01	? (TO) TT=2400.1	? (TO) TT=2400.11	? (TO) TT=2400.06
PseudoBoolean/ PseudoBoolean-dec-spng/ Pb-NG-K171.xml		? (TO) TT=2400.13	? (TO) TT=2400.06	? (TO) TT=2400.03	? (TO) TT=2400.12	? (TO) TT=2400.02	? (TO) TT=2400.15	? (TO) TT=2400.03
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K090.xml	SAT TT=606.023	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.09	? (TO) TT=2400.03	? (TO) TT=2400.02	? (TO) TT=2400.15	SAT TT=606.023
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K097.xml	SAT TT=120.497	? (TO) TT=2400.04	? (TO) TT=2400.07	? (TO) TT=2400.01	? (TO) TT=2400.04	? (TO) TT=2400.06	? (TO) TT=2400.18	SAT TT=120.497
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K100.xml	SAT TT=119.847	? (TO) TT=2400.08	? (TO) TT=2400.05	? (TO) TT=2400.06	? (TO) TT=2400.09	? (TO) TT=2400.07	? (TO) TT=2400.11	SAT TT=119.847
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K104.xml	SAT TT=110.658	? (TO) TT=2400.04	? (TO) TT=2400.01	? (TO) TT=2400.05	? (TO) TT=2400.02	? (TO) TT=2400.07	SAT TT=485.692	SAT TT=110.658
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K105.xml	SAT TT=107.681	? (TO) TT=2400.04	? (TO) TT=2400.04	? (TO) TT=2400.01	? (TO) TT=2400.03	? (TO) TT=2400.05	? (TO) TT=2400.31	SAT TT=107.681
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K106.xml	SAT TT=102.276	? (TO) TT=2400.01	? (TO) TT=2400.04	? (TO) TT=2400.03	? (TO) TT=2400.1	? (TO) TT=2400.06	? (TO) TT=2400.18	SAT TT=102.276
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K108.xml	SAT TT=78.684	? (TO) TT=2400.09	? (TO) TT=2400.07	? (TO) TT=2400.09	? (TO) TT=2400.05	? (TO) TT=2400.04	? (TO) TT=2400.15	SAT TT=78.684
PseudoBoolean/ PseudoBoolean-dec-sptate/ Pb-Tate-K109.xml	SAT TT=80.14	? (TO) TT=2400.1	? (TO) TT=2400.07	? (TO) TT=2400.07	? (TO) TT=2400.06	? (TO) TT=2400.04	? (TO) TT=2400.21	SAT TT=80.14
PseudoBoolean/ PseudoBoolean-opt-bgr/ Pb-bgr-06.xml	OPT f=17 TT=0.294 T1=0.1	OPT f=17 TT=0.294 T1=0.1	OPT f=17 TT=0.299 T1=0.1	OPT f=17 TT=0.297 T1=0.1	No data	No data	No data	OPT f=17 TT=2.85 T1=2.79
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-aim-200-2-0-yes1-1.xml	OPT f=200 TT=5.35 T1=5.29	OPT f=200 TT=2173.24 T1=171.29	OPT f=200 TT=2189.79 T1=172.27	OPT f=200 TT=2223.63 T1=175.57	No data	No data	No data	OPT f=200 TT=5.35 T1=5.29
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-hanoi4.xml	OPT f=718 TT=109.55 T1=109.49	? (TO) TT=2400.09	? (TO) TT=2400.08	? (TO) TT=2400.03	No data	No data	No data	OPT f=718 TT=109.55 T1=109.49
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-ii16c1.xml	SAT TO f=1420 TT=2400.02 T1=439.13	SAT (TO) f=1420 TT=2400.11 T1=440.21	SAT (TO) f=1420 TT=2400.02 T1=446.26	SAT (TO) f=1420 TT=2400.02 T1=439.13	No data	No data	No data	? (TO) TT=2400.03
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-ii32a1.xml	OPT f=444 TT=76.037 T1=76	SAT (TO) f=444 TT=2400.02 T1=8.59	SAT (TO) f=444 TT=2400.04 T1=8.79	SAT (TO) f=444 TT=2400.09 T1=8.69	No data	No data	No data	OPT f=444 TT=76.037 T1=76
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-ii32e3.xml	OPT f=310 TT=92.068 T1=92	SAT (TO) f=310 TT=2400.06 T1=25.78	SAT (TO) f=310 TT=2400.03 T1=25.68	SAT (TO) f=310 TT=2400.05 T1=25.68	No data	No data	No data	OPT f=310 TT=92.068 T1=92
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-ii32e5.xml	OPT f=503 TT=226.188 T1=218.29	SAT (TO) f=503 TT=2400.06 T1=1573.68	SAT (TO) f=503 TT=2400.03 T1=1572.7	SAT (TO) f=503 TT=2400.04 T1=1587.11	No data	No data	No data	OPT f=503 TT=226.188 T1=218.29
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-jnh217.xml	OPT f=89 TT=28.83 T1=28.79	OPT f=89 TT=55.836 T1=34	OPT f=89 TT=55.991 T1=34.09	OPT f=89 TT=56.884 T1=34.59	No data	No data	No data	OPT f=89 TT=28.83 T1=28.79
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-jnh220.xml	OPT f=94 TT=4.843 T1=0.79	OPT f=94 TT=4.843 T1=0.79	OPT f=94 TT=4.873 T1=0.79	OPT f=94 TT=5.591 T1=0.89	No data	No data	No data	OPT f=94 TT=18.476 T1=18.39
PseudoBoolean/ PseudoBoolean-opt-dimacs/ Pb-par16-3.xml	OPT f=1015 TT=53.826 T1=53.79	SAT (TO) f=1015 TT=2400.01 T1=123.68	SAT (TO) f=1015 TT=2400.1 T1=125.59	SAT (TO) f=1015 TT=2400.1 T1=137.01	No data	No data	No data	OPT f=1015 TT=53.826 T1=53.79
PseudoBoolean/ PseudoBoolean-opt-factor/ Pb-factor-S9-P293-Q307.xml	OPT f=293 TT=0.143 T1=0.1	OPT f=293 TT=0.158 T1=0.1	OPT f=293 TT=0.159 T1=0.1	OPT f=293 TT=0.158 T1=0.1	No data	No data	No data	OPT f=293 TT=0.143 T1=0.1
PseudoBoolean/ PseudoBoolean-opt-fx/ Pb-fx84.xml	OPT f=8 TT=66.982 T1=66.87	OPT f=8 TT=70.473 T1=70.38	OPT f=8 TT=71.214 T1=71.11	OPT f=8 TT=66.982 T1=66.87	No data	No data	No data	OPT f=8 TT=138.509 T1=138.4
PseudoBoolean/ PseudoBoolean-opt-logicSynthesis/ Pb-logic-ex6inp-r.xml	OPT f=4 TT=0.596 T1=0	OPT f=4 TT=0.596 T1=0	OPT f=4 TT=0.602 T1=0	OPT f=4 TT=0.687 T1=0	No data	No data	No data	OPT f=4 TT=0.672 T1=0.59
PseudoBoolean/ PseudoBoolean-opt-logicSynthesis/ Pb-logic-m050-100-30-30-r.xml	OPT f=4 TT=7.978 T1=7.89	OPT f=4 TT=50.115 T1=46.9	OPT f=4 TT=50.283 T1=47.1	OPT f=4 TT=50.191 T1=47	No data	No data	No data	OPT f=4 TT=7.978 T1=7.89
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
PseudoBoolean/ PseudoBoolean-opt-logicSynthesis/ Pb-logic-m100-100-30-30-r.xml	OPT f=5 TT=29.283 T1=29.19	OPT f=5 TT=1163.6 T1=1045.25	OPT f=5 TT=1166.96 T1=1048.28	OPT f=5 TT=1328.51 T1=1193.11	No data	No data	No data	OPT f=5 TT=29.283 T1=29.19
PseudoBoolean/ PseudoBoolean-opt-logicSynthesis/ Pb-logic-prom2-pi.xml	SAT TO f=445 TT=2400.04 T1=0.5	SAT (TO) f=445 TT=2400.07 T1=0.5	SAT (TO) f=445 TT=2400.04 T1=0.51	SAT (TO) f=445 TT=2400.04 T1=0.51	No data	No data	No data	? (TO) TT=2400.03
PseudoBoolean/ PseudoBoolean-opt-marketSplit/ Pb-msplit-opt-4-30-1.xml	OPT f=2 TT=858.942 T1=35.39	OPT f=2 TT=858.942 T1=35.39	OPT f=2 TT=860.212 T1=35.59	OPT f=2 TT=860.587 T1=35.49	No data	No data	No data	OPT f=2 TT=1248.49 T1=1248.45
PseudoBoolean/ PseudoBoolean-opt-mps/ Pb-mps-v2-20-10-p0040.xml	OPT f=62027 TT=11.214 T1=2.49	OPT f=62027 TT=11.214 T1=2.49	OPT f=62027 TT=11.275 T1=2.49	OPT f=62027 TT=11.269 T1=2.5	No data	No data	No data	OPT f=62027 TT=137.878 T1=137.8
PseudoBoolean/ PseudoBoolean-opt-routing/ Pb-routing-s4-4-3-1.xml	OPT f=62 TT=58.085 T1=57.99	SAT (TO) f=66 TT=2400.08 T1=1101.13	SAT (TO) f=66 TT=2400.04 T1=1135.4	SAT (TO) f=66 TT=2400.05 T1=1135.51	No data	No data	No data	OPT f=62 TT=58.085 T1=57.99
PseudoBoolean/ PseudoBoolean-opt-ttp/ Pb-circ8-3.xml	SAT TO f=182 TT=2400.03 T1=4.89	SAT (TO) f=182 TT=2400.03 T1=4.89	SAT (TO) f=182 TT=2400.1 T1=4.99	SAT (TO) f=182 TT=2400.09 T1=4.89	No data	No data	No data	? (TO) TT=2400.06
QRandom/QRandom-bdd-15-21-2/ bdd-15-21-2-2713-79-01.xml	UNSAT TT=19.555	UNSAT TT=183.041	UNSAT TT=182.357	UNSAT TT=184.388	UNSAT TT=19.555	UNSAT TT=671.514	UNSAT TT=201.124	? (TO) TT=2400.11
QRandom/QRandom-bdd-15-21-2/ bdd-15-21-2-2713-79-19.xml	SAT TT=21.154	SAT TT=21.154	SAT TT=21.409	SAT TT=21.755	SAT TT=22.42	SAT TT=493.326	SAT TT=162.56	? (TO) TT=2400.04
QRandom/QRandom-bdd-15-21-2/ bdd-15-21-2-2713-79-21.xml	UNSAT TT=20.978	UNSAT TT=191.603	UNSAT TT=191.901	UNSAT TT=195.946	UNSAT TT=20.978	UNSAT TT=487.414	UNSAT TT=177.9	? (TO) TT=2400.03
QRandom/QRandom-bdd-15-21-2/ bdd-15-21-2-2713-79-30.xml	UNSAT TT=19.106	UNSAT TT=163.898	UNSAT TT=166.326	UNSAT TT=169.772	UNSAT TT=19.106	UNSAT TT=531.404	UNSAT TT=191.743	? (TO) TT=2400.09
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-04.xml	SAT TT=7.753	SAT TT=66.693	SAT TT=71.024	SAT TT=67.689	SAT TT=7.753	SAT TT=600.365	SAT TT=129.862	? (TO) TT=2400.06
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-06.xml	UNSAT TT=8.68	UNSAT TT=129.97	UNSAT TT=128.771	UNSAT TT=131.044	UNSAT TT=8.68	UNSAT TT=497.131	UNSAT TT=110.398	? (TO) TT=2400.09
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-11.xml	SAT TT=7.29	SAT TT=43.933	SAT TT=41.289	SAT TT=45.715	SAT TT=7.29	SAT TT=670.531	SAT TT=127.583	? (TO) TT=2400.09
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-15.xml	UNSAT TT=8.394	UNSAT TT=132.877	UNSAT TT=132.053	UNSAT TT=128.152	UNSAT TT=8.394	UNSAT TT=532.053	UNSAT TT=107.705	? (TO) TT=2400.07
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-16.xml	UNSAT TT=8.691	UNSAT TT=122.879	UNSAT TT=124.662	UNSAT TT=125.327	UNSAT TT=8.691	UNSAT TT=542.236	UNSAT TT=106.5	? (TO) TT=2400.11
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-18.xml	SAT TT=1.664	SAT TT=1.666	SAT TT=1.664	SAT TT=1.669	SAT TT=1.958	SAT TT=13.584	SAT TT=13.847	? (TO) TT=2400.08
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-23.xml	SAT TT=1.668	SAT TT=1.668	SAT TT=1.903	SAT TT=1.684	SAT TT=1.959	SAT TT=14.1	SAT TT=13.684	? (TO) TT=2400.09
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-24.xml	UNSAT TT=7.972	UNSAT TT=121.572	UNSAT TT=120.053	UNSAT TT=123.522	UNSAT TT=7.972	UNSAT TT=508.636	UNSAT TT=97.315	? (TO) TT=2400.02
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-28.xml	UNSAT TT=8.565	UNSAT TT=135.607	UNSAT TT=137.575	UNSAT TT=133.704	UNSAT TT=8.565	UNSAT TT=756.382	UNSAT TT=123.504	? (TO) TT=2400.01
QRandom/QRandom-bdd-18-21-2/ bdd-18-21-2-133-78-30.xml	UNSAT TT=8.845	UNSAT TT=137.302	UNSAT TT=136.081	UNSAT TT=135.809	UNSAT TT=8.845	UNSAT TT=607.642	UNSAT TT=128.242	? (TO) TT=2400.09
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
QRandom/QRandom-geometric-rw/ geometric-50-20-d4-75-31.xml	SAT TT=0.035	SAT TT=0.108	SAT TT=0.108	SAT TT=0.107	SAT TT=0.045	SAT TT=0.035	SAT TT=0.035	SAT TT=0.309
QRandom/QRandom-geometric-rw/ geometric-50-20-d4-75-70.xml	UNSAT TT=29.397	UNSAT TT=29.397	UNSAT TT=29.771	UNSAT TT=29.491	UNSAT TT=690.9	UNSAT TT=37.999	UNSAT TT=69.722	UNSAT TT=246.683
QRandom/ QRandom-mdd-7-25-5/ mdd-7-25-5-56-01.xml	SAT TT=95.049	SAT TT=1913.61	SAT TT=2057.74	SAT TT=1930.23	SAT TT=95.049	? (TO) TT=2400.11	SAT TT=1845.57	SAT TT=864.608
QRandom/ QRandom-mdd-7-25-5/ mdd-7-25-5-56-02.xml	SAT TT=92.114	? (TO) TT=2400.07	? (TO) TT=2400.1	? (TO) TT=2400.06	SAT TT=92.114	? (TO) TT=2400.07	SAT TT=1286.69	SAT TT=390.716
QRandom/ QRandom-mdd-7-25-5/ mdd-7-25-5-56-06.xml	SAT TT=22.492	? (TO) TT=2400.03	? (TO) TT=2400.09	? (TO) TT=2400.02	SAT TT=22.492	? (TO) TT=2400.01	? (TO) TT=2400.09	SAT TT=1021.45
QRandom/ QRandom-mdd-7-25-5/ mdd-7-25-5-56-17.xml	SAT TT=253.884	? (TO) TT=2400.06	? (TO) TT=2400.05	? (TO) TT=2400.08	SAT TT=253.884	? (TO) TT=2400.04	SAT TT=1919.53	SAT TT=384.883
QRandom/QRandom-mdd-7-25-5-p7/ mdd-7-25-5-ps05-psh07-2.xml	UNSAT TT=20.864	UNSAT TT=1494.06	UNSAT TT=1486.06	UNSAT TT=1562.06	UNSAT TT=20.864	? (TO) TT=2400.03	UNSAT TT=487.812	UNSAT TT=1035.87
QRandom/QRandom-mdd-7-25-5-p7/ mdd-7-25-5-ps05-psh07-3.xml	UNSAT TT=56.078	? (TO) TT=2400.06	? (TO) TT=2400.07	? (TO) TT=2400.09	UNSAT TT=56.078	? (TO) TT=2400.1	UNSAT TT=1754.39	UNSAT TT=1066.75
QRandom/QRandom-mdd-7-25-5-p7/ mdd-7-25-5-ps05-psh07-9.xml	UNSAT TT=42.968	? (TO) TT=2400.1	? (TO) TT=2400.09	? (TO) TT=2400.08	UNSAT TT=42.968	? (TO) TT=2400.1	UNSAT TT=916.896	UNSAT TT=1121.99
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-bur26d.xml	SAT TO f=1718419 TT=2400.04 T1=22.47	SAT (TO) f=1718419 TT=2400.1 T1=24.39	SAT (TO) f=1718419 TT=2400.05 T1=22.47	SAT (TO) f=1718419 TT=2400.04 T1=22.48	No data	No data	No data	? (TO) TT=2400.09
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-chr15b.xml	OPT f=3995 TT=1.146 T1=0.41	OPT f=3995 TT=1.149 T1=0.41	OPT f=3995 TT=1.157 T1=0.41	OPT f=3995 TT=1.146 T1=0.41	No data	No data	No data	OPT f=3995 TT=165.341 T1=165.3
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-chr15c.xml	OPT f=4752 TT=49.23 T1=22.69	OPT f=4752 TT=49.46 T1=22.79	OPT f=4752 TT=49.553 T1=22.79	OPT f=4752 TT=49.23 T1=22.69	No data	No data	No data	OPT f=4752 TT=191.692 T1=191.58
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-chr18a.xml	OPT f=5549 TT=65.618 T1=27.59	OPT f=5549 TT=65.81 T1=27.69	OPT f=5549 TT=65.979 T1=27.79	OPT f=5549 TT=65.618 T1=27.59	No data	No data	No data	OPT f=5549 TT=405.166 T1=141.57
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-chr20b.xml	OPT f=1149 TT=27.782 T1=21.79	OPT f=1149 TT=31 T1=21.79	OPT f=1149 TT=30.995 T1=21.79	OPT f=1149 TT=31.029 T1=21.87	No data	No data	No data	OPT f=1149 TT=27.782 T1=27.69
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-esc16d.xml	OPT f=8 TT=23.782 T1=0	OPT f=8 TT=143.145 T1=0	OPT f=8 TT=143.08 T1=0	OPT f=8 TT=164.121 T1=0.1	No data	No data	No data	OPT f=8 TT=23.782 T1=23.69
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-esc16e.xml	OPT f=14 TT=30.699 T1=0	OPT f=14 TT=30.772 T1=0	OPT f=14 TT=30.747 T1=0	OPT f=14 TT=30.699 T1=0	No data	No data	No data	OPT f=14 TT=42.44 T1=42.38
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-kra32.xml	SAT TO f=57895 TT=2400.05 T1=1385.22	SAT (TO) f=57895 TT=2400.09 T1=1385.22	SAT (TO) f=57895 TT=2400.1 T1=1403.11	SAT (TO) f=57895 TT=2400.05 T1=1394.8	No data	No data	No data	? (TO) TT=2400.02
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-lipa30b.xml	SAT TO f=93923 TT=2400.03 T1=19.1	SAT (TO) f=93923 TT=2400.1 T1=19.28	SAT (TO) f=93923 TT=2400.03 T1=19.29	SAT (TO) f=93923 TT=2400.06 T1=19.1	No data	No data	No data	? (TO) TT=2400.08
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-lipa50a.xml	SAT TO f=31710 TT=2400.04 T1=1292.79	SAT (TO) f=31710 TT=2400.06 T1=1302.2	SAT (TO) f=31710 TT=2400.11 T1=1298.26	SAT (TO) f=31710 TT=2400.04 T1=1292.79	No data	No data	No data	? (TO) TT=2400.03
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-lipa60b.xml	SAT TO f=1615360 TT=2400.05 T1=2105.33	SAT (TO) f=1615360 TT=2400.11 T1=2105.33	SAT (TO) f=1615360 TT=2400.09 T1=2150.18	SAT (TO) f=1615360 TT=2400.05 T1=2136.83	No data	No data	No data	? (TO) TT=2400.03
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-nug16a.xml	SAT TO f=875 TT=2400.07 T1=2271.78	SAT (TO) f=875 TT=2400.11 T1=2298.5	SAT (TO) f=875 TT=2400.07 T1=2271.78	SAT (TO) f=875 TT=2400.1 T1=2278.77	No data	No data	No data	? (TO) TT=2400.06
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-nug17.xml	SAT TO f=960 TT=2400.03 T1=915.91	SAT (TO) f=960 TT=2400.03 T1=930.48	SAT (TO) f=960 TT=2400.05 T1=915.91	SAT (TO) f=960 TT=2400.03 T1=918.93	No data	No data	No data	? (TO) TT=2400.09
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-scr12.xml	OPT f=15705 TT=3.854 T1=0.51	OPT f=15705 TT=3.864 T1=0.51	OPT f=15705 TT=3.854 T1=0.51	OPT f=15705 TT=3.865 T1=0.51	No data	No data	No data	OPT f=15705 TT=147.308 T1=147.29
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-scr15.xml	OPT f=25570 TT=726.59 T1=145.8	OPT f=25570 TT=728.107 T1=503.47	OPT f=25570 TT=729.672 T1=504.27	OPT f=25570 TT=726.59 T1=502.26	No data	No data	No data	OPT f=25570 TT=1426.76 T1=145.8
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-sko42.xml	SAT TO f=9639 TT=2400.02 T1=1824.83	SAT (TO) f=9639 TT=2400.04 T1=1825.06	SAT (TO) f=9639 TT=2400.02 T1=1834.42	SAT (TO) f=9639 TT=2400.03 T1=1824.83	No data	No data	No data	? (TO) TT=2400.05
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai10a.xml	OPT f=67514 TT=305.003 T1=137.89	OPT f=67514 TT=306.653 T1=138.66	OPT f=67514 TT=313.893 T1=141.89	OPT f=67514 TT=305.003 T1=137.89	No data	No data	No data	? (TO) TT=2400.07
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai10b.xml	OPT f=58937 TT=0.549 T1=0.31	OPT f=58937 TT=0.555 T1=0.31	OPT f=58937 TT=0.555 T1=0.31	OPT f=58937 TT=0.549 T1=0.31	No data	No data	No data	OPT f=58937 TT=1947.18 T1=1947.14
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-tai17a.xml	SAT TO f=273667 TT=2400.02 T1=183.89	SAT (TO) f=273667 TT=2400.02 T1=191.64	SAT (TO) f=273667 TT=2400.04 T1=183.89	SAT (TO) f=273667 TT=2400.07 T1=188.09	No data	No data	No data	? (TO) TT=2400.02
QuadraticAssignment/ QuadraticAssignment-m1-s1/ QuadraticAssignment-wil100.xml	SAT TO f=148097 TT=2400.05 T1=163.85	SAT (TO) f=148097 TT=2400.05 T1=167.26	SAT (TO) f=148097 TT=2400.1 T1=163.85	SAT (TO) f=148097 TT=2400.1 T1=173.66	No data	No data	No data	? (TO) TT=2400.13
Random/Random-D-t050/ rand-2-40-25-180-050-28.xml	SAT TT=1.87	SAT TT=1.88	SAT TT=1.882	SAT TT=1.87	SAT TT=36.301	SAT TT=2.623	SAT TT=12.837	SAT TT=11.594
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Random/Random-D-t065/ rand-2-40-40-135-065-20.xml	SAT TT=0.293	SAT TT=0.293	SAT TT=0.296	SAT TT=0.294	SAT TT=0.392	SAT TT=0.453	SAT TT=0.706	SAT TT=28.783
Random/Random-m1-10-60-20/ rand-10-60-20-30-51200-43.xml	UNSAT TT=13.594	UNSAT TT=520.06	UNSAT TT=520.343	UNSAT TT=554.703	UNSAT TT=13.594	UNSAT TT=586.304	UNSAT TT=314.037	? (TO) TT=2400.1
Random/Random-m1-5-12-12/ rand-5-12-12-200-12442-18.xml	UNSAT TT=3.354	UNSAT TT=59.498	UNSAT TT=59.376	UNSAT TT=59.658	UNSAT TT=3.354	UNSAT TT=153.877	UNSAT TT=85.434	UNSAT TT=1337.44
Random/Random-m1-7-40-8t/ rand-7-40-8-40-20000-2.xml	UNSAT TT=32.706	UNSAT TT=1527.79	UNSAT TT=1521.6	UNSAT TT=1546.96	UNSAT TT=32.706	? (TO) TT=2400.03	UNSAT TT=1599.55	UNSAT TT=1275.55
Random/Random-RB-2-40-19f/ rand-2-40-19-443-230f-30.xml	SAT TT=44.079	SAT TT=99.845	SAT TT=102.84	SAT TT=114.1	SAT TT=1733.97	SAT TT=44.079	SAT TT=975.988	SAT TT=60.222
Random/Random-RB-2-50-23/ rand-2-50-23-587-230-39.xml	SAT TT=2044.77	? (TO) TT=2400.04	? (TO) TT=2400.06	? (TO) TT=2400.01	SAT TT=2253.38	? (TO) TT=2400.05	? (TO) TT=2400.02	SAT TT=2044.77
Random/Random-RB-3-20-20f/ rand-3-20-20-60-632f-11.xml	SAT TT=33.067	SAT TT=613.434	SAT TT=615.08	SAT TT=610.67	SAT TT=33.067	SAT TT=1414.87	SAT TT=398.745	SAT TT=104.938
Random/Random-RB-3-24-24f/ rand-3-24-24-76-632f-04.xml	SAT TT=74.75	SAT TT=1063.69	SAT TT=1069.73	SAT TT=1072.32	SAT TT=112.408	? (TO) TT=2400.07	SAT TT=1685.52	SAT TT=74.75
Random/Random-RB-3-28-28f/ rand-3-28-28-93-632f-36.xml		? (TO) TT=2400.08	? (TO) TT=2400.1	? (TO) TT=2400.1	? (TO) TT=2400.04	? (TO) TT=2400.05	? (TO) TT=2400.07	? (TO) TT=2400.05
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f02_c18.xml	UNSAT TT=95.944	UNSAT TT=300.456	UNSAT TT=298.414	UNSAT TT=299.338	? (TO) TT=2400.07	UNSAT TT=1877.41	UNSAT TT=95.944	UNSAT TT=181.22
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Rlfap_mini/ Rlfap-dec-scens11_mini/ Rlfap-ext-scen-11-f04_c18.xml	UNSAT TT=4.986	UNSAT TT=25.654	UNSAT TT=27.325	UNSAT TT=25.636	UNSAT TT=289.421	UNSAT TT=110.35	UNSAT TT=4.986	UNSAT TT=37.918
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-graph-03-opt_c18.xml	OPT f=380 TT=0.346 T1=0.31	OPT f=380 TT=0.346 T1=0.31	OPT f=380 TT=0.4 T1=0.31	OPT f=380 TT=0.346 T1=0.31	No data	No data	No data	OPT f=380 TT=33.878 T1=33.79
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-graph-04-opt_c18.xml	OPT f=394 TT=0.601 T1=0.5	OPT f=394 TT=0.602 T1=0.51	OPT f=394 TT=0.699 T1=0.68	OPT f=394 TT=0.601 T1=0.5	No data	No data	No data	OPT f=394 TT=61.571 T1=61.48
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-graph-07-opt_c18.xml	OPT f=4324 TT=1359.07 T1=1359.02	? (exit code) TT=20.972	? (exit code) TT=23.648	? (exit code) TT=21.065	No data	No data	No data	OPT f=4324 TT=1359.07 T1=1359.02
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-graph-10-opt_c18.xml	OPT f=394 TT=0.88 T1=0.79	OPT f=394 TT=0.881 T1=0.79	OPT f=394 TT=1.015 T1=0.99	OPT f=394 TT=0.88 T1=0.79	No data	No data	No data	OPT f=394 TT=123.297 T1=123.2
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-graph-11-opt_c18.xml		? (exit code) TT=23.186	? (exit code) TT=26.211	? (exit code) TT=22.728	No data	No data	No data	? (TO) TT=2400.05
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-scen-04-opt_c18.xml	OPT f=46 TT=0.599 T1=0.5	OPT f=46 TT=0.599 T1=0.5	OPT f=46 TT=0.696 T1=0.59	OPT f=46 TT=0.601 T1=0.5	No data	No data	No data	OPT f=46 TT=39.659 T1=39.59
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-scen-05-opt_c18.xml	OPT f=792 TT=0.472 T1=0.4	OPT f=792 TT=0.472 T1=0.41	OPT f=792 TT=0.549 T1=0.51	OPT f=792 TT=0.472 T1=0.4	No data	No data	No data	OPT f=792 TT=104.534 T1=104.47
Rlfap_mini/Rlfap-opt_mini/ Rlfap-ext-scen-10-opt_c18.xml		? (exit code) TT=292.796	? (exit code) TT=312.56	? (exit code) TT=278.065	No data	No data	No data	? (TO) TT=2400.08
Sat/ Sat-m1-various/ hole-10.xml	UNSAT TT=20.505	UNSAT TT=1385.04	UNSAT TT=1439.43	UNSAT TT=1389.57	UNSAT TT=1842.91	? (TO) TT=2400.06	UNSAT TT=20.505	UNSAT TT=63.383
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-015-9.xml	UNSAT TT=7.195	UNSAT TT=22.425	UNSAT TT=22.523	UNSAT TT=22.777	UNSAT TT=7.195	UNSAT TT=14.232	UNSAT TT=113.973	UNSAT TT=25.574
SchurrLemma/ SchurrLemma-mod-s1/ SchurrLemma-mod-100-9.xml	UNSAT TT=35.313	UNSAT TT=530.765	UNSAT TT=530.667	UNSAT TT=528.464	UNSAT TT=288.742	UNSAT TT=323.946	UNSAT TT=141.179	UNSAT TT=35.313
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g04-g43.xml	SAT TT=0.029	SAT TT=0.723	SAT TT=0.831	SAT TT=0.723	SAT TT=0.029	SAT TT=0.701	SAT TT=12.85	SAT TT=0.102
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g07-g50.xml	SAT TT=0.024	SAT TT=0.024	SAT TT=0.027	SAT TT=0.024	SAT TT=0.126	SAT TT=0.232	SAT TT=0.229	SAT TT=1.086
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g10-g15.xml	UNSAT TT=0.01	UNSAT TT=1406.37	UNSAT TT=1405.14	UNSAT TT=1409.54	UNSAT TT=0.01	? (TO) TT=2400.11	? (TO) TT=2400.09	UNSAT TT=699.086
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g10-g22.xml	UNSAT TT=0.03	UNSAT TT=210.658	UNSAT TT=210.696	UNSAT TT=212.231	UNSAT TT=0.03	UNSAT TT=1075.87	UNSAT TT=855.805	UNSAT TT=269
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g10-g47.xml	SAT TT=0.096	SAT TT=0.22	SAT TT=0.252	SAT TT=0.222	SAT TT=2.212	SAT TT=0.096	SAT TT=0.109	SAT TT=16.792
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g13-g27.xml	UNSAT TT=2.654	UNSAT TT=388.116	UNSAT TT=384.926	UNSAT TT=384.612	UNSAT TT=2.654	UNSAT TT=1270.82	UNSAT TT=483.059	UNSAT TT=72.33
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g19-g25.xml	UNSAT TT=0.015	UNSAT TT=1961.87	UNSAT TT=2237.25	UNSAT TT=2238.06	UNSAT TT=0.015	? (TO) TT=2400.05	? (TO) TT=2400.09	UNSAT TT=110.884
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g29-g30.xml	UNSAT TT=0.029	UNSAT TT=0.03	UNSAT TT=0.03	UNSAT TT=0.029	UNSAT TT=0.313	UNSAT TT=0.633	UNSAT TT=0.626	UNSAT TT=1.22
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g34-g41.xml	UNSAT TT=2.905	UNSAT TT=3.623	UNSAT TT=3.643	UNSAT TT=3.611	UNSAT TT=11.394	UNSAT TT=2.905	UNSAT TT=3.074	UNSAT TT=41.601
Subisomorphism/ Subisomorphism-m1-LV/ Subisomorphism-g41-g47.xml	UNSAT TT=0.475	UNSAT TT=0.476	UNSAT TT=0.542	UNSAT TT=0.475	UNSAT TT=10.928	UNSAT TT=1.244	UNSAT TT=1.358	UNSAT TT=22.344
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-B-07.xml	SAT TT=0.719	SAT TT=0.722	SAT TT=0.72	SAT TT=0.719	SAT TT=10.564	SAT TT=2.652	SAT TT=2.661	SAT TT=24.062
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-04.xml	SAT TT=1.8	SAT TT=1.807	SAT TT=1.8	SAT TT=1.819	SAT TT=34.227	SAT TT=9.185	SAT TT=9.233	SAT TT=160.832
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-05.xml	SAT TT=3.102	SAT TT=3.109	SAT TT=3.108	SAT TT=3.102	SAT TT=48.611	SAT TT=8.364	SAT TT=8.383	SAT TT=275.677
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-07.xml	SAT TT=1.623	SAT TT=1.628	SAT TT=1.623	SAT TT=1.629	SAT TT=117.279	SAT TT=8.459	SAT TT=8.521	SAT TT=264.112
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-08.xml	SAT TT=2.656	SAT TT=2.656	SAT TT=2.656	SAT TT=2.663	SAT TT=60.674	SAT TT=9.614	SAT TT=9.599	SAT TT=159.531
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-09.xml	SAT TT=1.583	SAT TT=1.586	SAT TT=1.583	SAT TT=1.588	SAT TT=36.136	SAT TT=8.797	SAT TT=8.801	SAT TT=227.58
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-11.xml	SAT TT=1.579	SAT TT=1.582	SAT TT=1.579	SAT TT=1.583	SAT TT=43.342	SAT TT=8.611	SAT TT=8.561	SAT TT=278.211
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-15.xml	SAT TT=2.015	SAT TT=2.019	SAT TT=2.015	SAT TT=2.021	SAT TT=34.369	SAT TT=9.724	SAT TT=9.871	SAT TT=134.611
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-C-17.xml	SAT TT=2.1	SAT TT=2.103	SAT TT=2.1	SAT TT=2.103	SAT TT=52.388	SAT TT=8.4	SAT TT=8.458	SAT TT=228.914
Subisomorphism/ Subisomorphism-m1-SF/ Subisomorphism-F-20.xml	SAT TT=233.201	? (TO) TT=2400.02	? (TO) TT=2400.02	? (TO) TT=2400.01	SAT TT=233.201	? (TO) TT=2400.09	? (TO) TT=2400.13	? (TO) TT=2400.12
Subisomorphism/ Subisomorphism-m1-si2-bvg/ Subisomorphism-si2-b03m-m800-09.xml	SAT TT=0.042	SAT TT=0.042	SAT TT=0.047	SAT TT=0.042	SAT TT=3.61	SAT TT=0.227	SAT TT=0.232	SAT TT=3.628
Subisomorphism/ Subisomorphism-m1-si2-bvg/ Subisomorphism-si2-b06m-m400-03.xml	SAT TT=0.019	SAT TT=0.019	SAT TT=0.021	SAT TT=0.019	SAT TT=0.297	SAT TT=0.131	SAT TT=0.132	SAT TT=1.957
Subisomorphism/ Subisomorphism-m1-si2-m4D/ Subisomorphism-si2-m4Dr6-m1296-03.xml	SAT TT=0.176	SAT TT=0.176	SAT TT=0.176	SAT TT=0.178	SAT TT=13.669	SAT TT=2.015	SAT TT=1.973	SAT TT=12.673
Subisomorphism/ Subisomorphism-m1-si4-rand/ Subisomorphism-si4-r005-m600-07.xml	SAT TT=1144.16	SAT TT=1144.97	SAT TT=1144.45	SAT TT=1144.16	? (MO) TT=30.265	SAT TT=1662.46	SAT TT=1381.41	? (TO) TT=2400.12
Subisomorphism/ Subisomorphism-m1-si6-bvg/ Subisomorphism-si6-b09m-m200-07.xml	SAT TT=0.022	SAT TT=0.022	SAT TT=0.022	SAT TT=0.022	SAT TT=0.286	SAT TT=0.286	SAT TT=0.283	SAT TT=1.234
Subisomorphism/ Subisomorphism-m1-si6-m4D/ Subisomorphism-si6-m4D-m1296-05.xml	SAT TT=2.21	SAT TT=2.21	SAT TT=2.233	SAT TT=2.224	SAT TT=1270.65	SAT TT=10.879	SAT TT=10.856	SAT TT=311.003
Subisomorphism/ Subisomorphism-m1-si6-m4D/ Subisomorphism-si6-m4D-m256-08.xml	SAT TT=0.031	SAT TT=0.031	SAT TT=0.031	SAT TT=0.031	SAT TT=0.489	SAT TT=0.236	SAT TT=0.233	SAT TT=6.71
Subisomorphism/ Subisomorphism-m1-si6-m4D/ Subisomorphism-si6-m4Dr2-m1296-00.xml	SAT TT=0.896	SAT TT=0.899	SAT TT=0.899	SAT TT=0.896	SAT TT=153.597	SAT TT=7.211	SAT TT=7.204	SAT TT=48.077
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Subisomorphism/ Subisomorphism-m1-si6-m4D/ Subisomorphism-si6-m4Dr4-m1296-00.xml	SAT TT=0.404	SAT TT=0.404	SAT TT=0.404	SAT TT=0.404	SAT TT=94.255	SAT TT=7.882	SAT TT=7.863	SAT TT=32.337
Subisomorphism/ Subisomorphism-m1-si6-m4D/ Subisomorphism-si6-m4Dr4-m1296-05.xml	SAT TT=1.112	SAT TT=1.112	SAT TT=1.113	SAT TT=1.114	SAT TT=100.664	SAT TT=7.888	SAT TT=8.038	SAT TT=218.584
SumColoring/ SumColoring-dsjc-250-1_c18.xml	SAT TO f=941 TT=2400.04 T1=0.4	SAT (TO) f=941 TT=2400.07 T1=0.4	SAT (TO) f=941 TT=2400.05 T1=0.41	SAT (TO) f=941 TT=2400.04 T1=0.4	No data	No data	No data	? (TO) TT=2400.03
test/cop/ OpenStacks-m1-pb-10-10-1.xml	OPT f=5 TT=5.451 T1=0	OPT f=5 TT=52.654 T1=0	OPT f=5 TT=52.92 T1=0.05	OPT f=5 TT=52.52 T1=0.01	No data	No data	No data	OPT f=5 TT=5.451 T1=5.37
TravellingSalesman/ TravellingSalesman-m1-n100/ TravellingSalesman-100-50-01.xml	SAT TO f=1006 TT=2400.01 T1=306.82	SAT (TO) f=1006 TT=2400.08 T1=308.78	SAT (TO) f=1006 TT=2400.01 T1=306.82	SAT (TO) f=1006 TT=2400.09 T1=313.63	No data	No data	No data	? (TO) TT=2400.05
TravellingSalesman/ TravellingSalesman-m1-n100/ TravellingSalesman-100-50-13.xml	SAT TO f=1066 TT=2400.04 T1=286.28	SAT (TO) f=1066 TT=2400.09 T1=289.41	SAT (TO) f=1066 TT=2400.05 T1=286.28	SAT (TO) f=1066 TT=2400.04 T1=287.68	No data	No data	No data	? (TO) TT=2400.02
TravellingSalesman/ TravellingSalesman-m1-n150/ TravellingSalesman-150-50-00.xml	SAT TO f=1424 TT=2400.03 T1=779.39	SAT (TO) f=1424 TT=2400.03 T1=779.39	SAT (TO) f=1424 TT=2400.06 T1=783.47	SAT (TO) f=1424 TT=2400.08 T1=784.63	No data	No data	No data	? (TO) TT=2400.08
TravellingSalesman/ TravellingSalesman-m1-n150/ TravellingSalesman-150-50-07.xml	SAT TO f=1190 TT=2400.01 T1=1412.31	SAT (TO) f=1190 TT=2400.01 T1=1412.67	SAT (TO) f=1190 TT=2400.04 T1=1412.31	SAT (TO) f=1190 TT=2400.09 T1=1418.77	No data	No data	No data	? (TO) TT=2400.06
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-00.xml	OPT f=93 TT=106.436 T1=106.39	SAT (TO) f=93 TT=2400.08 T1=91.56	SAT (TO) f=93 TT=2400.03 T1=91.79	SAT (TO) f=93 TT=2400.04 T1=90.69	No data	No data	No data	OPT f=93 TT=106.436 T1=106.39
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-01.xml	OPT f=106 TT=81.079 T1=8.29	OPT f=106 TT=462.975 T1=8.39	OPT f=106 TT=458.206 T1=8.29	OPT f=106 TT=456.583 T1=8.29	No data	No data	No data	OPT f=106 TT=81.079 T1=80.98
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-02.xml	OPT f=114 TT=51.752 T1=16.59	OPT f=114 TT=604.761 T1=16.79	OPT f=114 TT=602.187 T1=16.69	OPT f=114 TT=596.432 T1=16.59	No data	No data	No data	OPT f=114 TT=51.752 T1=51.69
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-03.xml	OPT f=98 TT=65.725 T1=7.79	OPT f=98 TT=320.868 T1=7.89	OPT f=98 TT=318.301 T1=7.89	OPT f=98 TT=317.207 T1=7.79	No data	No data	No data	OPT f=98 TT=65.725 T1=65.7
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-04.xml	OPT f=116 TT=84.682 T1=84.6	SAT (TO) f=116 TT=2400.01 T1=56.79	SAT (TO) f=116 TT=2400.06 T1=56.49	SAT (TO) f=116 TT=2400.01 T1=65.01	No data	No data	No data	OPT f=116 TT=84.682 T1=84.6
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-06.xml	OPT f=99 TT=53.951 T1=5.48	OPT f=99 TT=432.255 T1=5.48	OPT f=99 TT=433.272 T1=5.49	OPT f=99 TT=428.297 T1=5.49	No data	No data	No data	OPT f=99 TT=53.951 T1=53.88
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-10.xml	OPT f=104 TT=105.886 T1=105.79	SAT (TO) f=104 TT=2400.03 T1=12.6	SAT (TO) f=104 TT=2400.01 T1=14.39	SAT (TO) f=104 TT=2400.08 T1=12.58	No data	No data	No data	OPT f=104 TT=105.886 T1=105.79
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-11.xml	OPT f=76 TT=96.059 T1=26	OPT f=76 TT=1279.19 T1=26.1	OPT f=76 TT=1282.21 T1=26.1	OPT f=76 TT=1278.22 T1=26	No data	No data	No data	OPT f=76 TT=96.059 T1=96.01
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-12.xml	OPT f=105 TT=43.099 T1=42.99	SAT (TO) f=105 TT=2400.02 T1=35.29	SAT (TO) f=105 TT=2400.01 T1=40.6	SAT (TO) f=105 TT=2400.06 T1=34.79	No data	No data	No data	OPT f=105 TT=43.099 T1=42.99
TravellingSalesman/ TravellingSalesman-m1-n15/ TravellingSalesman-15-30-17.xml	OPT f=85 TT=113.495 T1=66.1	OPT f=85 TT=1951.09 T1=66.48	OPT f=85 TT=1952.15 T1=66.48	OPT f=85 TT=1941.17 T1=66.1	No data	No data	No data	OPT f=85 TT=113.495 T1=113.39
TravellingSalesman/ TravellingSalesman-m1-n20/ TravellingSalesman-20-30-04.xml	OPT f=122 TT=389.928 T1=389.88	SAT (TO) f=150 TT=2400.08 T1=2195.56	SAT (TO) f=150 TT=2400.02 T1=2204.39	SAT (TO) f=150 TT=2400.1 T1=2189.01	No data	No data	No data	OPT f=122 TT=389.928 T1=389.88
TravellingSalesman/ TravellingSalesman-m1-n20/ TravellingSalesman-20-30-05.xml	OPT f=114 TT=740.811 T1=96.59	SAT (TO) f=151 TT=2400.12 T1=2392.39	SAT (TO) f=151 TT=2400.04 T1=2363.43	SAT (TO) f=151 TT=2400.06 T1=2373.65	No data	No data	No data	OPT f=114 TT=740.811 T1=96.59
Bench name	Best results	cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
TravellingSalesman/ TravellingSalesman-m1-n20/ TravellingSalesman-20-30-14.xml	OPT f=97 TT=258.185 T1=258.1	SAT (TO) f=116 TT=2400.04 T1=2125.04	SAT (TO) f=116 TT=2400.04 T1=2129.92	SAT (TO) f=116 TT=2400.02 T1=2104.71	No data	No data	No data	OPT f=97 TT=258.185 T1=258.1
TravellingSalesman/ TravellingSalesman-m1-n25/ TravellingSalesman-25-30-01.xml	OPT f=111 TT=478.215 T1=478.14	SAT (TO) f=169 TT=2400.09 T1=2364.48	SAT (TO) f=169 TT=2400.02 T1=2354.11	SAT (TO) f=169 TT=2400.08 T1=2349.37	No data	No data	No data	OPT f=111 TT=478.215 T1=478.14
TravellingSalesman/ TravellingSalesman-m1-n30/ TravellingSalesman-30-30-10.xml	SAT TO f=225 TT=2400.01 T1=2348.61	SAT (TO) f=225 TT=2400.08 T1=2354.46	SAT (TO) f=225 TT=2400.01 T1=2352.6	SAT (TO) f=225 TT=2400.03 T1=2348.61	No data	No data	No data	? (TO) TT=2400.02
TravellingSalesman/ TravellingSalesman-m1-n40/ TravellingSalesman-40-30-06.xml	SAT TO f=373 TT=2400.01 T1=1383.06	SAT (TO) f=373 TT=2400.08 T1=1391.27	SAT (TO) f=373 TT=2400.02 T1=1393.12	SAT (TO) f=373 TT=2400.01 T1=1383.06	No data	No data	No data	? (TO) TT=2400.04
TravellingSalesman/ TravellingSalesman-m1-n40/ TravellingSalesman-40-30-12.xml	SAT TO f=315 TT=2400.01 T1=22.59	SAT (TO) f=315 TT=2400.03 T1=22.59	SAT (TO) f=315 TT=2400.01 T1=22.69	SAT (TO) f=315 TT=2400.01 T1=25.79	No data	No data	No data	? (TO) TT=2400.01
TravellingSalesman/ TravellingSalesman-m1-n45/ TravellingSalesman-45-30-01.xml	SAT TO f=330 TT=2400.02 T1=1726.08	SAT (TO) f=330 TT=2400.03 T1=1734.65	SAT (TO) f=330 TT=2400.02 T1=1728.62	SAT (TO) f=330 TT=2400.09 T1=1726.08	No data	No data	No data	? (TO) TT=2400.05
TravellingSalesman/ TravellingSalesman-m1-n45/ TravellingSalesman-45-30-17.xml	SAT TO f=298 TT=2400.03 T1=2328.41	SAT (TO) f=298 TT=2400.07 T1=2349.96	SAT (TO) f=298 TT=2400.03 T1=2328.41	SAT (TO) f=298 TT=2400.03 T1=2340.73	No data	No data	No data	? (TO) TT=2400.02
TravellingSalesman/ TravellingSalesman-m1-n50/ TravellingSalesman-50-30-04.xml	SAT TO f=344 TT=2400.04 T1=874.97	SAT (TO) f=344 TT=2400.1 T1=883.51	SAT (TO) f=344 TT=2400.05 T1=878.53	SAT (TO) f=344 TT=2400.04 T1=874.97	No data	No data	No data	? (TO) TT=2400.08
TravellingSalesman/ TravellingSalesman-xcsp2-s20a4/ TravellingSalesman-4-20-075-a4_X2.xml	SAT TT=9.148	SAT TT=175.219	SAT TT=174.218	SAT TT=176.071	SAT TT=9.148	SAT TT=158.664	SAT TT=39.359	SAT TT=24.598
TravellingSalesman/ TravellingSalesman-xcsp2-s20/ TravellingSalesman-20-453_X2.xml	SAT TT=1.132	SAT TT=1.132	SAT TT=1.134	SAT TT=1.147	SAT TT=20.448	SAT TT=23.544	SAT TT=25.033	SAT TT=28.818

Statistics		cosoco 2 (complete)	cosoco 2.0 (complete)	cosoco 2.0 (complete)	miniBTD 19.06.16 (complete)	NACRE 1.0.5 (complete)	NACRE 1.0.5-Hybrid (complete)	(reference) PicatSAT 2019-09-12 (complete)
Number of times the solver is able to give the best known answer		268	267	268	133	116	135	284
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		78	64	82	78	7	15	107